Marqua is an enhanced markdown compiler with code syntax highlighting and built-in front matter parser that splits your markdown into two parts, `content` and `metadata`. The generated output is highly adaptable to be used with any framework and designs of your choice as it is just JSON.
The markdown compiler is powered by [markdown-it](https://github.com/markdown-it/markdown-it) and code syntax highlighter is powered by [Shikiji](https://github.com/antfu/shikiji).
The front matter parser for the `metadata` is powered by a lightweight implementation in-house, which supports a minimal subset of [YAML](https://yaml.org/) syntax and can be used as a standalone module.
### Quick Start
```
pnpm install marqua
```
Use the functions from the FileSystem module to `compile` a file or `traverse` directories.
```javascript
import { compile, traverse } from 'marqua/fs';
compile(/* string */, /* optional hydrate callback */);
traverse(/* options */, /* optional hydrate callback */);
```
Add interactivity to the code blocks with `hydrate` from `/browser` module.
```svelte
```
Getting Started
```
pnpm install marqua
```
### Include base styles
Make sure to include the stylesheets from `/styles` to your app
```svelte
```
The following CSS variables are made available and can be modified as needed
```css
:root {
--font-default: 'Rubik', 'Ubuntu', 'Helvetica Neue', sans-serif;
--font-heading: 'Karla', sans-serif;
--font-monospace: 'Fira Code', 'Inconsolata', 'Consolas', monospace;
--mrq-rounding: 0.3rem;
--mrq-tab-size: 2;
--mrq-primary: #0070bb;
--mrq-bg-dark: #2d2d2d;
--mrq-bg-light: #f7f7f7;
--mrq-cl-dark: #242424;
--mrq-cl-light: #dadada;
}
.mrq[data-mrq='block'],
.mrq[data-mrq='header'],
.mrq[data-mrq='pre'] {
--mrq-pre-bg: #525252;
--mrq-bounce: 10rem;
--mrq-tms: 100ms;
--mrq-tfn: cubic-bezier(0.6, -0.28, 0.735, 0.045);
}
.mrq[data-mrq='header'] {
--mrq-hbg-dark: #323330;
--mrq-hbg-light: #feefe8;
}
```
Semantics
### Front Matter
Marqua supports a minimal subset of [YAML](https://yaml.org/) syntax for the front matter, which is semantically placed at the start of the file between two `---` lines, and it will be parsed as a JSON object.
All values will be attempted to be parsed into the supported types, which are `null`, `true`, and `false`. Any other values will go through the following checks and the first one to pass will be used.
- Comments, `#`; indicated by a hash followed by the value, will be omitted from the output
- Literal Block, `|`; indicated by a pipe followed by a newline and the value, will be parsed as multi-line string
- Inline Array, `[x, y, 2]`; indicated by comma-separated values surrounded by square brackets, can only be primitives
- Sequence, `- x`; indicated by a dash followed by a space and the value, this can contain nested maps and sequences
To have a line be parsed as-is, simply wrap the value with single or double quotes.
```yaml
---
title: My First Blog Post, Hello World!
description: Welcome to my first post.
tags: [blog, life, coding]
date:published: 2021-04-01
date:updated: 2021-04-13
# do not assign top-level data when using compressed nested properties syntax
# because this will overwrite previous 'date:published' and 'date:updated'
# date: ...
---
```
The above front matter will output the following JSON object...
```json
{
"title": "My First Blog Post, Hello World!",
"description": "Welcome to my first post.",
"tags": ["blog", "life", "coding"],
"date": {
"published": "2021-04-01",
"updated": "2021-04-03"
}
}
```
Where we usually use indentation to represent the start of a nested maps, we can additionally denote them using a compressed syntax by combining the properties into one key separated by a colon without space, such as `key:x: value`. This should only be declared at the top-level and not inside nested maps.
### Content
Everything after front matter will be considered as content and will be parsed as markdown. You can use the `!{}` syntax to access the metadata from the front matter.
```yaml
---
title: "My Amazing Series: Second Coming"
tags: [blog, life, coding]
date:
published: 2021-04-01
updated: 2021-04-13
---
# the properties above will result to
#
# title = 'My Amazing Series: Second Coming'
# tags = ['blog', 'life', 'coding']
# date = {
# published: '2021-04-01',
# updated: '2021-04-13',
# }
#
# these can be accessed with !{}
# !{tags:0} - accessing tags array at index 0
This article's main topic will be about !{tags:0}
# !{date:property} - accessing property of date
This article was originally published on !{date:published}
Thoroughly updated through this website on !{date:updated}
```
There should only be one `
` heading per page, and it's usually declared in the front matter as `title`, which is why headings in the content starts at 2 `##` (equivalent to `
`) with the lowest one being 4 `####` (equivalent to `
`) and should conform with the [rules of markdownlint](https://github.com/DavidAnson/markdownlint#rules--aliases), with some essential ones to follow are
- MD001: Heading levels should only increment by one level at a time
- MD003: Heading style; only ATX style
- MD018: No space after hash on atx style heading
- MD023: Headings must start at the beginning of the line
- MD024: Multiple headings with the same content; siblings only
- MD042: No empty links
Generated ids can be specified from the text by wrapping them in `$(...)` as the delimiter. The text inside will be converted to kebab-case and will be used as the id. If no delimiter is detected, the whole text will be used.
If you're using VSCode, you can install the [markdownlint extension](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint) to help you catch these lint errors / warnings and write better markdown. These rules can be configured, see the [.jsonc template](https://github.com/DavidAnson/markdownlint/blob/main/schema/.markdownlint.jsonc) and [.yaml template](https://github.com/DavidAnson/markdownlint/blob/main/schema/.markdownlint.yaml) with an [example here](https://github.com/ignatiusmb/mauss.dev/blob/master/.markdownlint.yaml).
### Code Blocks
Code blocks are fenced with 3 backticks and can optionally be assigned a language for syntax highlighting. The language must be a valid [shikiji supported language](https://github.com/antfu/shikiji/blob/main/docs/languages.md) and is case-insensitive.
````markdown
```language
// code
```
````
Additional information can be added to the code block through data attributes, accessible via `data-[key]="[value]"`. The dataset can be specified from any line within the code block using `#$ key: value` syntax, and it will be omitted from the output. The key-value pair should roughly conform to the [`data-*` rules](https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/data-*), meaning `key` can only contain alphanumeric characters and hyphens, while `value` can be any string that fits in the data attribute value.
There are some special keys that will be used to modify the code block itself, and they are
- `#$ file: string` | add a filename to the code block that will be shown above the output
- `#$ line-start: number` | define the starting line number of the code block
Module / Core
Marqua provides a lightweight core module with minimal features and dependencies that does not rely on platform-specific modules so that it could be used anywhere safely.
### parse
Where the parsing happens, it accepts a source string and returns a `{ content, metadata }` structure. This function is mainly used to separate the front matter from the content.
```typescript
export function parse(source: string): {
content: string;
metadata: Record & {
readonly estimate: number;
readonly table: MarquaTable[];
};
};
```
If you need to read from a file or folder, use the `compile` and `traverse` functions from the [FileSystem module](#module-fs).
### construct
Where the `metadata` or front matter index gets constructed, it is used in the `parse` function.
```typescript
type Primitives = null | boolean | string;
type ValueIndex = Primitives | Primitives[];
type FrontMatter = { [key: string]: ValueIndex | FrontMatter };
export function construct(raw: string): ValueIndex | FrontMatter;
```
Module / Artisan
### transform
This isn't usually necessary, but in case you want to handle the markdown parsing and rendering by yourself, here's how you can tap into the `transform` function provided by the module.
```typescript
export interface Dataset {
lang?: string;
file?: string;
[data: string]: string | undefined;
}
export function transform(source: string, dataset: Dataset): string;
```
A simple example would be passing a raw source code as a string.
```javascript
import { transform } from 'marqua/artisan';
const source = `
interface User {
id: number;
name: string;
}
const user: User = {
id: 0,
name: 'User'
}
`;
transform(source, { lang: 'typescript' });
```
Another one would be to use as a highlighter function.
```javascript
import MarkdownIt from 'markdown-it';
import { transform } from 'marqua/artisan';
// passing as a 'markdown-it' options
const marker = MarkdownIt({
highlight: (source, lang) => transform(source, { lang });
});
```
### marker
The artisan module also exposes the `marker` import that is a markdown-it object.
```javascript
import { marker } from 'marqua/artisan';
import plugin from 'markdown-it-plugin'; // some markdown-it plugin
marker.use(plugin); // add this before calling 'compile' or 'traverse'
```
Importing `marker` to extend with plugins is optional, it is usually used to enable you to write [LaTeX](https://www.latex-project.org/) in your markdown for example, which is useful for math typesetting and writing abstract symbols using TeX functions. Here's a working example with a plugin that uses [KaTeX](https://katex.org/).
```javascript
import { marker } from 'marqua/artisan';
import { compile } from 'marqua/fs';
import TexMath from 'markdown-it-texmath';
import KaTeX from 'katex';
marker.use(TexMath, {
engine: KaTeX,
delimiters: 'dollars',
});
const data = compile(/* source path */);
```
Module / Browser
### hydrate
This is the browser module to hydrate and give interactivity to your HTML.
```typescript
import type { ActionReturn } from 'svelte/action';
export function hydrate(node: HTMLElement, key: any): ActionReturn;
```
The `hydrate` function can be used to make the rendered code blocks from your markdown interactive, some of which are
- toggle code line numbers
- copy block to clipboard
Usage using [SvelteKit](https://kit.svelte.dev/) would simply be
```svelte
```
Passing in the `navigating` store into the `key` parameter is used to trigger the update inside `hydrate` function and re-hydrate the DOM when the page changes but is not remounted.
Module / FileSystem
Marqua provides a couple of functions coupled with the FileSystem module to `compile` or `traverse` a directory, given an entry point.
Using a folder structure shown below as a reference for the next examples, the usage will be as follows
```
content
├── posts
│ ├── draft.my-amazing-two-part-series-part-1.md
│ ├── draft.my-amazing-two-part-series-part-2.md
│ ├── 2021-04-01.my-first-post.md
│ └── 2021-04-13.marqua-is-the-best.md
└── reviews
├── game
│ └── doki-doki-literature-club.md
├── book
│ ├── amazing-book-one.md
│ └── manga-is-literature.md
└── movie
├── spirited-away.md
└── your-name.md
```
### compile
```typescript
interface HydrateChunk {
breadcrumb: string[];
buffer: Buffer;
parse: typeof parse;
}
export function compile(
entry: string,
hydrate?: (chunk: HydrateChunk) => undefined | Output,
): undefined | Output;
```
The first argument of `compile` is the source entry point.
### traverse
```typescript
export function traverse(
options: {
entry: string;
compile?(path: string): boolean;
depth?: number;
},
hydrate?: (chunk: HydrateChunk) => undefined | Output,
transform?: (items: Output[]) => Transformed,
): Transformed;
```
The first argument of `traverse` is its `typeof options` and the second argument is an optional `hydrate` callback function. The third argument is an optional `transform` callback function.
The `compile` property of the `options` object is an optional function that takes the full path of a file from the `entry` point and returns a boolean. If the function returns `true`, the file will be processed by the `compile` function, else it will be passed over to the `hydrate` function if it exists.
An example usage from the _hypothetical_ content folder structure above should look like
```javascript
import { compile, traverse } from 'marqua/fs';
/* compile - parse a single source file */
const body = compile(
'content/posts/2021-04-01.my-first-post.md',
({ breadcrumb: [filename], buffer, parse }) => {
const [date, slug] = filename.split('.');
const { content, metadata } = parse(buffer.toString('utf-8'));
return { ...metadata, slug, date, content };
},
); // {'posts/2021-04-01.my-first-post.md'}
/* traverse - scans a directory for sources */
const data = traverse({ entry: 'content/posts' }, ({ breadcrumb: [filename], buffer, parse }) => {
if (filename.startsWith('draft')) return;
const [date, slug] = filename.split('.');
const { content, metadata } = parse(buffer.toString('utf-8'));
return { ...metadata, slug, date, content };
}); // [{'posts/3'}, {'posts/4'}]
/* traverse - nested directories infinite recursive traversal */
const data = traverse(
{ entry: 'content/reviews', depth: -1 },
({ breadcrumb: [slug, category], buffer, parse }) => {
const { content, metadata } = parse(buffer.toString('utf-8'));
return { ...metadata, slug, category, content };
},
); // [{'game/0'}, {'book/0'}, {'book/1'}, {'movie/0'}, {'movie/1'}]
```
Module / Transform
This module provides a set of transformer functions for the [`traverse.transform`](#traverse) parameter. These functions can be used in conjunction with each other, by utilizing the `pipe` function provided from the `'mauss'` package and re-exported by this module, you can do the following
```typescript
import { traverse } from 'marqua/fs';
import { pipe } from 'marqua/transform';
traverse({ entry: 'content' }, () => {}, pipe(/* ... */));
```
### chain
The `chain` transformer is used to add a `flank` property to each items and attaches the previous (`idx - 1`) and the item after (`idx + 1`) as `flank: { back, next }`, be sure to sort it the way you intend it to be before running this transformer.
```typescript
export function chain(options: {
base?: string;
breakpoint?: (next: T) => boolean;
sort?: (x: T, y: T) => number;
}): (items: T[]) => Array;
```
- A `base` string can be passed as a prefix in the `slug` property of each items.
- A `breakpoint` function can be passed to stop the chain on a certain condition.
```typescript
traverse(
{ entry: 'content' },
({}) => {},
chain({
breakpoint(item) {
return; // ...
},
}),
);
```
- A `sort` function can be passed to sort the items before chaining them.