Marqua

Augmented Markdown Compiler

Introduction

Marqua is an enhanced markdown compiler with code syntax highlighting and built-in front matter parser that splits your markdown into two parts, `content` and `metadata`. The generated output is highly adaptable to be used with any framework and designs of your choice as it is just JSON. The markdown compiler is powered by [markdown-it](https://github.com/markdown-it/markdown-it) and code syntax highlighter is powered by [Shikiji](https://github.com/antfu/shikiji). The front matter parser for the `metadata` is powered by a lightweight implementation in-house, which supports a minimal subset of [YAML](https://yaml.org/) syntax and can be used as a standalone module. ### Quick Start ``` pnpm install marqua ``` Use the functions from the FileSystem module to `compile` a file or `traverse` directories. ```javascript import { compile, traverse } from 'marqua/fs'; compile(/* string */, /* optional hydrate callback */); traverse(/* options */, /* optional hydrate callback */); ``` Add interactivity to the code blocks with `hydrate` from `/browser` module. ```svelte
```

Getting Started

``` pnpm install marqua ``` ### Include base styles Make sure to include the stylesheets from `/styles` to your app ```svelte ``` The following CSS variables are made available and can be modified as needed ```css :root { --font-default: 'Rubik', 'Ubuntu', 'Helvetica Neue', sans-serif; --font-heading: 'Karla', sans-serif; --font-monospace: 'Fira Code', 'Inconsolata', 'Consolas', monospace; --mrq-rounding: 0.3rem; --mrq-tab-size: 2; --mrq-primary: #0070bb; --mrq-bg-dark: #2d2d2d; --mrq-bg-light: #f7f7f7; --mrq-cl-dark: #242424; --mrq-cl-light: #dadada; } .mrq[data-mrq='block'], .mrq[data-mrq='header'], .mrq[data-mrq='pre'] { --mrq-pre-bg: #525252; --mrq-bounce: 10rem; --mrq-tms: 100ms; --mrq-tfn: cubic-bezier(0.6, -0.28, 0.735, 0.045); } .mrq[data-mrq='header'] { --mrq-hbg-dark: #323330; --mrq-hbg-light: #feefe8; } ```

Semantics

### Front Matter Marqua supports a minimal subset of [YAML](https://yaml.org/) syntax for the front matter, which is semantically placed at the start of the file between two `---` lines, and it will be parsed as a JSON object. All values will be attempted to be parsed into the supported types, which are `null`, `true`, and `false`. Any other values will go through the following checks and the first one to pass will be used. - Comments, `#`; indicated by a hash followed by the value, will be omitted from the output - Literal Block, `|`; indicated by a pipe followed by a newline and the value, will be parsed as multi-line string - Inline Array, `[x, y, 2]`; indicated by comma-separated values surrounded by square brackets, can only be primitives - Sequence, `- x`; indicated by a dash followed by a space and the value, this can contain nested maps and sequences To have a line be parsed as-is, simply wrap the value with single or double quotes. ```yaml --- title: My First Blog Post, Hello World! description: Welcome to my first post. tags: [blog, life, coding] date:published: 2021-04-01 date:updated: 2021-04-13 # do not assign top-level data when using compressed nested properties syntax # because this will overwrite previous 'date:published' and 'date:updated' # date: ... --- ``` The above front matter will output the following JSON object... ```json { "title": "My First Blog Post, Hello World!", "description": "Welcome to my first post.", "tags": ["blog", "life", "coding"], "date": { "published": "2021-04-01", "updated": "2021-04-03" } } ``` Where we usually use indentation to represent the start of a nested maps, we can additionally denote them using a compressed syntax by combining the properties into one key separated by a colon without space, such as `key:x: value`. This should only be declared at the top-level and not inside nested maps. ### Content Everything after front matter will be considered as content and will be parsed as markdown. You can use the `!{}` syntax to access the metadata from the front matter. ```yaml --- title: "My Amazing Series: Second Coming" tags: [blog, life, coding] date: published: 2021-04-01 updated: 2021-04-13 --- # the properties above will result to # # title = 'My Amazing Series: Second Coming' # tags = ['blog', 'life', 'coding'] # date = { # published: '2021-04-01', # updated: '2021-04-13', # } # # these can be accessed with !{} # !{tags:0} - accessing tags array at index 0 This article's main topic will be about !{tags:0} # !{date:property} - accessing property of date This article was originally published on !{date:published} Thoroughly updated through this website on !{date:updated} ``` There should only be one `

` heading per page, and it's usually declared in the front matter as `title`, which is why headings in the content starts at 2 `##` (equivalent to `

`) with the lowest one being 4 `####` (equivalent to `

`) and should conform with the [rules of markdownlint](https://github.com/DavidAnson/markdownlint#rules--aliases), with some essential ones to follow are - MD001: Heading levels should only increment by one level at a time - MD003: Heading style; only ATX style - MD018: No space after hash on atx style heading - MD023: Headings must start at the beginning of the line - MD024: Multiple headings with the same content; siblings only - MD042: No empty links Generated ids can be specified from the text by wrapping them in `$(...)` as the delimiter. The text inside will be converted to kebab-case and will be used as the id. If no delimiter is detected, the whole text will be used. If you're using VSCode, you can install the [markdownlint extension](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint) to help you catch these lint errors / warnings and write better markdown. These rules can be configured, see the [.jsonc template](https://github.com/DavidAnson/markdownlint/blob/main/schema/.markdownlint.jsonc) and [.yaml template](https://github.com/DavidAnson/markdownlint/blob/main/schema/.markdownlint.yaml) with an [example here](https://github.com/ignatiusmb/mauss.dev/blob/master/.markdownlint.yaml). ### Code Blocks Code blocks are fenced with 3 backticks and can optionally be assigned a language for syntax highlighting. The language must be a valid [shikiji supported language](https://github.com/antfu/shikiji/blob/main/docs/languages.md) and is case-insensitive. ````markdown ```language // code ``` ```` Additional information can be added to the code block through data attributes, accessible via `data-[key]="[value]"`. The dataset can be specified from any line within the code block using `#$ key: value` syntax, and it will be omitted from the output. The key-value pair should roughly conform to the [`data-*` rules](https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/data-*), meaning `key` can only contain alphanumeric characters and hyphens, while `value` can be any string that fits in the data attribute value. There are some special keys that will be used to modify the code block itself, and they are - `#$ file: string` | add a filename to the code block that will be shown above the output - `#$ line-start: number` | define the starting line number of the code block

Module / Core

Marqua provides a lightweight core module with minimal features and dependencies that does not rely on platform-specific modules so that it could be used anywhere safely. ### parse Where the parsing happens, it accepts a source string and returns a `{ content, metadata }` structure. This function is mainly used to separate the front matter from the content. ```typescript export function parse(source: string): { content: string; metadata: Record & { readonly estimate: number; readonly table: MarquaTable[]; }; }; ``` If you need to read from a file or folder, use the `compile` and `traverse` functions from the [FileSystem module](#module-fs). ### construct Where the `metadata` or front matter index gets constructed, it is used in the `parse` function. ```typescript type Primitives = null | boolean | string; type ValueIndex = Primitives | Primitives[]; type FrontMatter = { [key: string]: ValueIndex | FrontMatter }; export function construct(raw: string): ValueIndex | FrontMatter; ```

Module / Artisan

### transform This isn't usually necessary, but in case you want to handle the markdown parsing and rendering by yourself, here's how you can tap into the `transform` function provided by the module. ```typescript export interface Dataset { lang?: string; file?: string; [data: string]: string | undefined; } export function transform(source: string, dataset: Dataset): string; ``` A simple example would be passing a raw source code as a string. ```javascript import { transform } from 'marqua/artisan'; const source = ` interface User { id: number; name: string; } const user: User = { id: 0, name: 'User' } `; transform(source, { lang: 'typescript' }); ``` Another one would be to use as a highlighter function. ```javascript import MarkdownIt from 'markdown-it'; import { transform } from 'marqua/artisan'; // passing as a 'markdown-it' options const marker = MarkdownIt({ highlight: (source, lang) => transform(source, { lang }); }); ``` ### marker The artisan module also exposes the `marker` import that is a markdown-it object. ```javascript import { marker } from 'marqua/artisan'; import plugin from 'markdown-it-plugin'; // some markdown-it plugin marker.use(plugin); // add this before calling 'compile' or 'traverse' ``` Importing `marker` to extend with plugins is optional, it is usually used to enable you to write [LaTeX](https://www.latex-project.org/) in your markdown for example, which is useful for math typesetting and writing abstract symbols using TeX functions. Here's a working example with a plugin that uses [KaTeX](https://katex.org/). ```javascript import { marker } from 'marqua/artisan'; import { compile } from 'marqua/fs'; import TexMath from 'markdown-it-texmath'; import KaTeX from 'katex'; marker.use(TexMath, { engine: KaTeX, delimiters: 'dollars', }); const data = compile(/* source path */); ```

Module / Browser

### hydrate This is the browser module to hydrate and give interactivity to your HTML. ```typescript import type { ActionReturn } from 'svelte/action'; export function hydrate(node: HTMLElement, key: any): ActionReturn; ``` The `hydrate` function can be used to make the rendered code blocks from your markdown interactive, some of which are - toggle code line numbers - copy block to clipboard Usage using [SvelteKit](https://kit.svelte.dev/) would simply be ```svelte
``` Passing in the `navigating` store into the `key` parameter is used to trigger the update inside `hydrate` function and re-hydrate the DOM when the page changes but is not remounted.

Module / FileSystem

Marqua provides a couple of functions coupled with the FileSystem module to `compile` or `traverse` a directory, given an entry point. Using a folder structure shown below as a reference for the next examples, the usage will be as follows ``` content ├── posts │ ├── draft.my-amazing-two-part-series-part-1.md │ ├── draft.my-amazing-two-part-series-part-2.md │ ├── 2021-04-01.my-first-post.md │ └── 2021-04-13.marqua-is-the-best.md └── reviews ├── game │ └── doki-doki-literature-club.md ├── book │ ├── amazing-book-one.md │ └── manga-is-literature.md └── movie ├── spirited-away.md └── your-name.md ``` ### compile ```typescript interface HydrateChunk { breadcrumb: string[]; buffer: Buffer; parse: typeof parse; } export function compile( entry: string, hydrate?: (chunk: HydrateChunk) => undefined | Output, ): undefined | Output; ``` The first argument of `compile` is the source entry point. ### traverse ```typescript export function traverse( options: { entry: string; compile?(path: string): boolean; depth?: number; }, hydrate?: (chunk: HydrateChunk) => undefined | Output, transform?: (items: Output[]) => Transformed, ): Transformed; ``` The first argument of `traverse` is its `typeof options` and the second argument is an optional `hydrate` callback function. The third argument is an optional `transform` callback function. The `compile` property of the `options` object is an optional function that takes the full path of a file from the `entry` point and returns a boolean. If the function returns `true`, the file will be processed by the `compile` function, else it will be passed over to the `hydrate` function if it exists. An example usage from the _hypothetical_ content folder structure above should look like ```javascript import { compile, traverse } from 'marqua/fs'; /* compile - parse a single source file */ const body = compile( 'content/posts/2021-04-01.my-first-post.md', ({ breadcrumb: [filename], buffer, parse }) => { const [date, slug] = filename.split('.'); const { content, metadata } = parse(buffer.toString('utf-8')); return { ...metadata, slug, date, content }; }, ); // {'posts/2021-04-01.my-first-post.md'} /* traverse - scans a directory for sources */ const data = traverse({ entry: 'content/posts' }, ({ breadcrumb: [filename], buffer, parse }) => { if (filename.startsWith('draft')) return; const [date, slug] = filename.split('.'); const { content, metadata } = parse(buffer.toString('utf-8')); return { ...metadata, slug, date, content }; }); // [{'posts/3'}, {'posts/4'}] /* traverse - nested directories infinite recursive traversal */ const data = traverse( { entry: 'content/reviews', depth: -1 }, ({ breadcrumb: [slug, category], buffer, parse }) => { const { content, metadata } = parse(buffer.toString('utf-8')); return { ...metadata, slug, category, content }; }, ); // [{'game/0'}, {'book/0'}, {'book/1'}, {'movie/0'}, {'movie/1'}] ```

Module / Transform

This module provides a set of transformer functions for the [`traverse.transform`](#traverse) parameter. These functions can be used in conjunction with each other, by utilizing the `pipe` function provided from the `'mauss'` package and re-exported by this module, you can do the following ```typescript import { traverse } from 'marqua/fs'; import { pipe } from 'marqua/transform'; traverse({ entry: 'content' }, () => {}, pipe(/* ... */)); ``` ### chain The `chain` transformer is used to add a `flank` property to each items and attaches the previous (`idx - 1`) and the item after (`idx + 1`) as `flank: { back, next }`, be sure to sort it the way you intend it to be before running this transformer. ```typescript export function chain(options: { base?: string; breakpoint?: (next: T) => boolean; sort?: (x: T, y: T) => number; }): (items: T[]) => Array; ``` - A `base` string can be passed as a prefix in the `slug` property of each items. - A `breakpoint` function can be passed to stop the chain on a certain condition. ```typescript traverse( { entry: 'content' }, ({}) => {}, chain({ breakpoint(item) { return; // ... }, }), ); ``` - A `sort` function can be passed to sort the items before chaining them.