Docs for agents

Nimbus serves documentation through endpoints that AI tools can read without parsing the HTML site.

Release-matched guide

Every framework package includes this guide at node_modules/@cloudflare/nimbus-docs/dist/docs-for-agents.md. For a preview release, use that installed copy: it matches the package you are testing even when the public website describes an older stable release. The starter’s AGENT.md points coding agents to it.

What ships

• Alternate Markdown/MDX versions for every discoverable page — /<slug>/index.md is clean Markdown with components converted to text; /<slug>/index.mdx keeps imports, JSX and directives, and the .md links to it. API reference pages have only the .md, and hidden versions have neither.
• A site llms.txt index — names top-level pages and section indexes of the current docs. Each section index links its discoverable pages to their alternate Markdown versions. Older docs versions aren’t listed; each has its own index at /<version>/llms.txt.
• llms-full.txt — the current site’s pages in one Markdown document, for tools that fetch everything in one request. Over 25 MiB, it becomes an index of one file per section, so no file passes the limit. See Add llms-full.txt.
• Per-section indexes at /<section>/llms.txt — each top-level group and collection (e.g. a blog or a docs version) gets its own index, and so does each folder below it (/reference/functions/llms.txt), linked from its parent.
• Version labels — on versioned sites, every alternate Markdown version’s frontmatter carries a version: key (resolved from your versions manifest) so agents can pin a version. Unversioned sites emit nothing.
• JSON-LD in every page head — structured data so search and agent tools recognize what they’re looking at.
• /robots.txt and a sitemap — wired automatically when site is set.
• A remote MCP server, when agents: { mcp: true } is set on a site with a server adapter — /mcp serves list_pages and get_page over the published Markdown pages, with a Server Card at /mcp/server-card. See Remote MCP server.

# Nimbus

A way to build documentation sites on top of Astro.

## Pages
- [Quickstart](https://nimbus-docs.com/quickstart/index.md)
- [Why Nimbus](https://nimbus-docs.com/why-nimbus/index.md)

Agent discovery

Every site serves /.well-known/ard.json and /.well-known/ai-catalog.json, with the same content. They list /llms.txt, the homepage Markdown at /index.md, and the remote MCP server’s card when the site runs one. A site on a domain another site owns can turn them off (Shared domains).

The homepage advertises these files and each visible API reference in HTTP Link headers. Request-rendered homepages send them directly. Static builds write _headers rules, which Cloudflare and some other hosts read; on other hosts, set the same headers in the host’s config.

Your own _headers wins. If your rule already sets a header for a path, exactly or through a wildcard such as /*, Nimbus leaves that header out, because hosts join repeated values and browsers refuse Access-Control-Allow-Origin: *, *.

Link is the exception, since joined Link values are valid. If your rule covers only part of a generated wildcard, the build warns and names the paths to cover.

/index.md is the root content entry when there is one, otherwise the llms.txt content. A root entry with noindex: true is served but left out of discovery.

If your site already has a file or route at either .well-known path, the build fails so you can move your entries first, unless discovery is off.

Discovery files belong at the origin root. With a base, the Cloudflare adapter puts them there for you; on other hosts, deploy the emitted .well-known directory at the root alongside the site. Static output with the Cloudflare adapter and a base needs wrangler deploy --assets ./dist/client.

For a static host serving a site under /docs, the public layout is:

host-root/
├── .well-known/        # emitted discovery and Skills files
├── docs/               # the site's HTML, Markdown, indexes, and Pagefind files
└── _headers            # when the host supports this format

Use your host’s mounting or deployment configuration to produce that layout. Serve discovery JSON as application/json with Access-Control-Allow-Origin: *, and apply the same CORS header to Skills artifacts. Homepage Markdown uses text/markdown; charset=utf-8; Skills archives use application/gzip. Verify the actual deployed responses, rather than only checking the emitted file:

curl -I https://docs.example.com/.well-known/ard.json
curl -I https://docs.example.com/docs/
curl -I https://docs.example.com/docs/index.md

Shared domains

When another site owns the domain, such as docs at example.com/docs beside a main site at example.com, the root .well-known files belong to that site. Turn Nimbus’s off:

const nimbusConfig = defineNimbusConfig({
  // …
  agents: { discovery: false },
});

Nimbus then publishes no ard.json, ai-catalog.json or api-catalog, and no homepage Link or _headers rules for them. Your own files in public/.well-known/ are kept; under a base they’re served there, such as /docs/.well-known/ard.json.

After switching, purge the cached homepage, whose Link header still lists the old files. On hosts that keep files from earlier deploys, such as an S3 sync without --delete, remove them too.

Everything else stays on, including llms.txt, Markdown versions, Markdown negotiation and the remote MCP server, which agents still find through llms.txt.

Agent Skills still publish. With the Cloudflare adapter, or without a base, they land at the origin root’s /.well-known/agent-skills/, so leave out the skills/ folder unless this site owns that path. A site serving its own api-catalog sets its Content-Type in its own _headers, since the file has no extension.

Content Signals

The starter’s src/pages/robots.txt.ts declares Content-Signal: ai-train=yes, search=yes, ai-input=yes. Change the values to match your policy; if you set one to no, add the policy text from contentsignals.org above it. Existing sites keep their robots.txt; nimbus-docs migrate describes the change. Content Signals and robots.txt are requests to crawlers, not access control. Private docs need authentication.

API catalog and published specs

Sites with API collections serve an RFC 9727 catalog at /.well-known/api-catalog, unless agents.discovery is false. It has one entry per visible collection and version, linking the reference page, its Markdown and the published spec. A version selected by query is listed at its query URL, such as /pets/?version=v1, and has no Markdown link when it isn’t the default.

Each spec is published as one self-contained JSON file at its mount path, such as /pets/openapi.json, with referenced files embedded under x-ext. The discovery manifest lists every visible spec. The homepage Link header lists only each family’s default reference and spec, plus the catalog.

Set publishSpec: false on a collection, or on one version, to keep its spec unpublished; the catalog still links its docs. A spec that can’t be bundled is skipped with a warning. Hidden versions publish their file but aren’t listed anywhere.

The catalog file has no extension, so a host that ignores _headers needs its own rule for its media type and its Link header on HEAD.

Markdown negotiation

On server output, a request-rendered page answers Accept: text/markdown with the same Markdown as its index.md, at the same URL. Both forms send Vary: Accept. Markdown wins only when it outranks every HTML and wildcard type in the header, so a browser’s */* and text/plain still get HTML. Once a collection renders on request, the homepage negotiates too.

Cloudflare reads the published Markdown through its assets binding. Other adapters fetch it from the origin in Nimbus’s site, never from the request’s Host, so deploy the static Markdown with the server and set site to the deployment’s origin, previews included.

API collections rendered on request need Cloudflare. On other adapters, render API collections at build and prose collections on request.

For example, a Node deployment uses the official Astro Node adapter:

import node from "@astrojs/node";
import { defineConfig } from "astro/config";
import nimbus, { defineConfig as defineNimbusConfig } from "@cloudflare/nimbus-docs";

export default defineConfig({
  output: "server",
  adapter: node({ mode: "standalone" }),
  integrations: [nimbus(defineNimbusConfig({
    site: "https://docs.example.com",
    title: "Example",
    rendering: { collections: { docs: "request" } },
  }))],
});

Use the same Nimbus rendering policy with the official Netlify or Vercel adapter, following that adapter’s deployment instructions. Static output, discovery assets, Skills, and browser search do not require a server adapter.

curl -i -H 'Accept: text/markdown, text/html;q=0.9' https://docs.example.com/guide/
curl -I -H 'Accept: text/markdown' https://docs.example.com/guide/

Prerendered pages and static sites return HTML: the host serves the file before any middleware runs. For those, Cloudflare’s Markdown for Agents negotiates at the platform level and converts the HTML. Assets, API endpoints, and unknown routes pass through unchanged.

Agent Skills

Add skills/<name>/SKILL.md folders at the project root, in the Agent Skills format, and Nimbus publishes them at /.well-known/agent-skills/. The index follows the discovery RFC v0.2.0 and gives each entry a SHA-256 digest.

A single-file skill is published as its SKILL.md; a multi-file skill as a .tar.gz with SKILL.md at the root. The discovery manifest lists the index, unless agents.discovery is false.

A skill whose frontmatter breaks the spec is skipped with a build warning that names the folder and the rule. Dotfiles are left out of archives and symlinks are skipped. Without a skills/ folder nothing is published.

For example, create skills/getting-started/SKILL.md:

---
name: getting-started
description: Help a user install and configure the example project.
---

# Getting started

Read the installation guide before changing project configuration.

Run the production build, then inspect /.well-known/agent-skills/index.json and fetch each advertised artifact. Removing a skill folder removes its publication on the next build. Existing public files or routes at the generated discovery and Skills paths cause a build error with move/remove guidance.

Browser-agent search

The starter registers a search_documentation tool with document.modelContext (WebMCP, an experimental browser API). It searches the site’s Pagefind index and returns section titles, anchored URLs, and plain-text excerpts, with the same visibility rules as site search. Nothing loads until an agent calls the tool, and browsers without the API do nothing.

Sites with search: false or a custom provider register no tool. The index comes from a production build, so in development the tool returns index_unavailable. Pagefind 1.5.2 or newer is required. Existing sites copy WebMCP.astro from the starter and render it inside <head> in BaseLayout.astro.

The browser tool is independent of the deployment provider. It uses the experimental Document.modelContext API, not Navigator.modelContext, and is not a remote MCP server or a browser polyfill. A browser without that API keeps ordinary search and registers no agent tool.

An agent invokes the registered tool with JSON such as:

{ "query": "installation", "limit": 5 }

Drafts, hidden versions and noindex pages aren’t searched, unless a page sets searchable: true. Each result has a title, heading, anchored url and a plain-text excerpt of at most 1,000 characters.

Search failures and recovery

How pages are indexed

Every page collection is indexed: those made with docsCollection(), componentsCollection() or a withNimbusMarkdown() loader, and API collections. Nimbus reads each page’s title and description when present. Plain data collections aren’t indexed, and hidden versions are left out.

Alternate Markdown/MDX versions and llms.txt indexes use public agent-endpoint assets. Request-time authorization cannot make private material safe here. Keep private material outside public content collections. Nimbus omits excluded entries and fails the build when it cannot determine whether content is public.

To keep a page out of the llms.txt indexes, set draft: true (excluded entirely) or noindex: true. A page that should never be agent-readable belongs outside the content collection altogether.

Custom content loaders

Nimbus’s collection factories prepare Markdown automatically. If a custom loader retains Markdown bodies for alternate Markdown versions or llms-full.txt, wrap it with withNimbusMarkdown() from @cloudflare/nimbus-docs/content. See Other collections for a complete example.

Route setup

New projects include these routes. To add or update them in an existing project, follow Markdown and llms.txt endpoints.

AGENT.md

The scaffolder writes an AGENT.md at the project root — the same context a coding agent reads when working in your repo. It’s a real file you maintain; keep it in sync with your project’s conventions.

Why it matters

The cost of being a dead end on the agent web is real and growing. Because these files and metadata are defaults, a Nimbus site is legible to agents the day it ships — see Metadata and SEO for the human-search side.