Markdown for agents

Every page available as raw Markdown via `.md` URL suffix or `Accept: text/markdown`. ~10× token reduction for AI coding assistants.

~ 2 min read

Markdown for agents

When a coding assistant fetches a docs page, it doesn’t want HTML. It wants the source — the same Markdown the page was written in, without the chrome, navigation, or rendered components getting in the way.

Tangly serves it. Two ways, both wired up by default:

.md URL suffix. Append .md to any page URL. Works on every static host, no config required.
Accept: text/markdown header. Same URL, content-negotiated. Works in tangly dev; production needs a one-line CDN config (recipes below).

The .md files are emitted at build time alongside the HTML, so deploys to Vercel, Cloudflare Pages, Netlify, S3, or GitHub Pages get this for free.

Why it matters

A typical docs page renders to ~16k tokens of HTML (nav, sidebar, footer, embedded JS). The same page as raw Markdown is ~3k tokens — an 80% reduction. Bun reported a 10× drop in their own measurements.

Coding assistants like Claude Code, OpenCode, Cursor, and Aider already send Accept: text/markdown on docs requests. Tangly responds in kind.

How to use it

From a coding assistant

Read https://docs.example.com/installation.md

Or — for any agent that respects Accept headers:

bash

curl -H "Accept: text/markdown" https://docs.example.com/installation

Both return the raw .mdx source with frontmatter intact and a URL: preamble identifying the page.

Discovery

Every HTML page advertises its Markdown twin in the <head>:

html

<link rel="alternate" type="text/markdown" href="/installation.md">

RFC 8288-aware crawlers find the .md without needing to send Accept headers. The HTML response also includes:

Vary: Accept
Link: </llms.txt>; rel="llms-txt", </llms-full.txt>; rel="llms-full-txt"
X-Llms-Txt: /llms.txt

Same wire format as Mintlify and Cloudflare’s “Markdown for Agents.”

What gets emitted

For each page, Tangly writes <slug>.md to dist/:

dist/
  installation/
    index.html
  installation.md          ← agent-facing source
  guides/
    deploying/
      index/
        index.html
      index.md

The .md file contains the raw on-disk source plus a one-line URL preamble:

markdown

URL: /installation

---
title: Installation
description: Install Tangly into a new or existing project.
---

# Installation

...

Frontmatter is preserved (agents use title and description as context). MDX components stay in their JSX form — agents handle that fine, and keeping the file byte-identical to source means no surprises.

What’s excluded

Same rules as llms.txt and sitemap.xml:

Pages with noindex: true in frontmatter.
Drafts (draft: true) when not building with --include-drafts.

The .md variant also gets X-Robots-Tag: noindex on dev responses so it never competes with the HTML page in search results.

Production content negotiation

The static .md files work everywhere. If you also want Accept: text/markdown on the same URL (so agents don’t have to know about the .md suffix), add a one-line rewrite at the CDN.

Cloudflare Pages

Add to _headers:

/*
  Vary: Accept
  Link: </llms.txt>; rel="llms-txt", </llms-full.txt>; rel="llms-full-txt"
  X-Llms-Txt: /llms.txt

And in _redirects:

/:path  /:path.md  200  Accept=*text/markdown*

Vercel

In vercel.json:

json

{
  "rewrites": [
    {
      "source": "/:path*",
      "has": [
        { "type": "header", "key": "accept", "value": ".*text/markdown.*" }
      ],
      "destination": "/:path*.md"
    }
  ],
  "headers": [
    {
      "source": "/(.*)",
      "headers": [
        { "key": "Vary", "value": "Accept" },
        { "key": "Link", "value": "</llms.txt>; rel=\"llms-txt\", </llms-full.txt>; rel=\"llms-full-txt\"" },
        { "key": "X-Llms-Txt", "value": "/llms.txt" }
      ]
    }
  ]
}

Netlify

In _redirects:

/*  /:splat.md  200!  Accept=*text/markdown*

Relation to llms.txt

The two work together:

/llms.txt — index. “Here are all the pages.”
/llms-full.txt — bulk. “Here is every page concatenated.”
/<slug>.md — per-page. “Here is one specific page.”

An agent typically fetches /llms.txt once to discover the IA, then pulls individual .md files as the conversation needs them.

See llms.txt for the index format.

Disabling

There’s no opt-out flag. The .md files are byte-cheap and harmless — the same content already in /llms-full.txt just split per page. If a specific page shouldn’t ship, mark it noindex: true and it’s excluded from all four (sitemap, llms.txt, llms-full.txt, <slug>.md).

Last updated 2 h ago Edit this page