Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/agent-baseline.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: agent-baseline

on:
pull_request:
push:
branches: [main]

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- run: npx -y @gleanwork/configure-agents@latest check
73 changes: 73 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# AGENTS.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Build and Development Commands

```bash
npm run build # Build with tsup (ESM-only output)
npm run dev # Watch mode for development
npm run lint # Run ESLint and Prettier check
npm run lint:fix # Auto-fix lint issues
npm run typecheck # TypeScript type checking
npm run test # Run unit tests with Vitest
npm run test:watch # Run tests in watch mode
npm run test:mcp # Run Playwright MCP integration tests
npm run test:all # Run lint, typecheck, and all tests
```

Run a single test file:
```bash
npx vitest run tests/html-to-markdown-test.ts
```

## Architecture Overview

This is a Docusaurus plugin that exposes documentation as an MCP (Model Context Protocol) server for AI agents.

### Two-Phase Design

**Build Time** (`src/plugin/`): The Docusaurus plugin runs during `docusaurus build`:
- `docusaurus-plugin.ts` - Main plugin with `postBuild` hook
- Processes HTML files → extracts content → converts to markdown → builds search index
- Outputs artifacts to `build/mcp/` (docs.json, search-index.json, manifest.json)

**Runtime** (`src/mcp/`, `src/adapters/`): Serverless functions serve MCP requests:
- `McpDocsServer` class wraps the official MCP SDK
- `createWebRequestHandler` — one generic web-standard `(Request) => Response` handler for any serverless/edge runtime (pre-loaded data)
- `createNodeServer`/`createNodeHandler` — local-dev server over Node `http` (file-based)

### Entry Points

The package has three export paths configured in `package.json`:
- `.` → Main plugin + MCP server (`src/index.ts`)
- `./adapters` → Platform handlers (`src/adapters-entry.ts`)
- `./theme` → React components (`src/theme/index.ts`)

### Key Modules

- `src/mcp/server.ts` - Core MCP server using `@modelcontextprotocol/sdk`
- `src/mcp/tools/` - MCP tool definitions (`docs_search`, `docs_fetch`)
- `src/processing/` - HTML parsing, markdown conversion, heading extraction
- `src/search/` - FlexSearch integration for full-text search
- `src/providers/` - Pluggable indexer/search provider system
- `src/cli/verify.ts` - CLI for verifying build output

### Provider System

Indexers and search providers are pluggable via the `src/providers/` system:
- `ContentIndexer` - Processes docs at build time (e.g., FlexSearchIndexer)
- `SearchProvider` - Handles queries at runtime (e.g., FlexSearchProvider)

## Testing

- Unit tests: `tests/*.ts` using Vitest
- Integration tests: `tests/playwright/mcp.spec.ts` using `@gleanwork/mcp-server-tester`

## Package Format

ESM-only (`"type": "module"`). All imports use `.js` extensions in source files.

## Skills

This repository ships an agent skill at `skills/SKILL.md`. Keep it accurate as the public API changes.
68 changes: 2 additions & 66 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,69 +1,5 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
This repository's agent instructions live in **AGENTS.md**. Read it first.

## Build and Development Commands

```bash
npm run build # Build with tsup (ESM-only output)
npm run dev # Watch mode for development
npm run lint # Run ESLint and Prettier check
npm run lint:fix # Auto-fix lint issues
npm run typecheck # TypeScript type checking
npm run test # Run unit tests with Vitest
npm run test:watch # Run tests in watch mode
npm run test:mcp # Run Playwright MCP integration tests
npm run test:all # Run lint, typecheck, and all tests
```

Run a single test file:
```bash
npx vitest run tests/html-to-markdown-test.ts
```

## Architecture Overview

This is a Docusaurus plugin that exposes documentation as an MCP (Model Context Protocol) server for AI agents.

### Two-Phase Design

**Build Time** (`src/plugin/`): The Docusaurus plugin runs during `docusaurus build`:
- `docusaurus-plugin.ts` - Main plugin with `postBuild` hook
- Processes HTML files → extracts content → converts to markdown → builds search index
- Outputs artifacts to `build/mcp/` (docs.json, search-index.json, manifest.json)

**Runtime** (`src/mcp/`, `src/adapters/`): Serverless functions serve MCP requests:
- `McpDocsServer` class wraps the official MCP SDK
- Platform adapters (Vercel, Netlify, Cloudflare) handle HTTP → MCP translation
- Two modes: file-based (Node.js) or pre-loaded data (Workers)

### Entry Points

The package has three export paths configured in `package.json`:
- `.` → Main plugin + MCP server (`src/index.ts`)
- `./adapters` → Platform handlers (`src/adapters-entry.ts`)
- `./theme` → React components (`src/theme/index.ts`)

### Key Modules

- `src/mcp/server.ts` - Core MCP server using `@modelcontextprotocol/sdk`
- `src/mcp/tools/` - MCP tool definitions (`docs_search`, `docs_fetch`)
- `src/processing/` - HTML parsing, markdown conversion, heading extraction
- `src/search/` - FlexSearch integration for full-text search
- `src/providers/` - Pluggable indexer/search provider system
- `src/cli/verify.ts` - CLI for verifying build output

### Provider System

Indexers and search providers are pluggable via the `src/providers/` system:
- `ContentIndexer` - Processes docs at build time (e.g., FlexSearchIndexer)
- `SearchProvider` - Handles queries at runtime (e.g., FlexSearchProvider)

## Testing

- Unit tests: `tests/*.ts` using Vitest
- Integration tests: `tests/playwright/mcp.spec.ts` using `@gleanwork/mcp-server-tester`

## Package Format

ESM-only (`"type": "module"`). All imports use `.js` extensions in source files.
@AGENTS.md
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -508,3 +508,17 @@ import {
## License

MIT

<!-- configure-agents:skills start -->

## Agent skills

This repository ships agent skill(s) under `skills/`. Install them into your
AI agent with [`npx skills`](https://github.com/agentskills/agentskills):

```sh
npx skills add -g scalvert/docusaurus-plugin-mcp-server # global — available in every repo
npx skills add scalvert/docusaurus-plugin-mcp-server # or scoped to the current repo
```

<!-- configure-agents:skills end -->
80 changes: 80 additions & 0 deletions skills/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
name: "docusaurus-plugin-mcp-server"
description: "Expose a Docusaurus site's docs as an MCP server for AI agents — add the build-time plugin, deploy the MCP endpoint to any web-standard serverless/edge runtime, run it locally, or add the install button. Load when working with docusaurus-plugin-mcp-server or its /adapters or /theme entry points."
---

# docusaurus-plugin-mcp-server

A Docusaurus plugin that, at `docusaurus build`, emits MCP artifacts (`docs.json`, `search-index.json`, `manifest.json`) under `build/mcp/`, plus runtime handlers that serve them as an MCP endpoint (`docs_search` + `docs_fetch`) to AI agents.

## When to use

Load this skill when the task involves:

- Adding/configuring the plugin in `docusaurus.config.js`.
- Standing up the MCP HTTP endpoint — deploying to a serverless/edge runtime (Cloudflare Workers, modern Netlify functions, Vercel Edge, Deno, Bun) or running it locally on Node.
- Adding the `McpInstallButton` to a docs site.
- Writing a custom indexer or search provider.

Trigger imports: `docusaurus-plugin-mcp-server`, `docusaurus-plugin-mcp-server/adapters`, `docusaurus-plugin-mcp-server/theme`.

## Install & import

```bash
npm install docusaurus-plugin-mcp-server
```

ESM-only. Three entry points:

- `docusaurus-plugin-mcp-server` — the plugin (default export) + `McpDocsServer`, provider types, `DEFAULT_PLUGIN_OPTIONS`.
- `docusaurus-plugin-mcp-server/adapters` — runtime handlers: `createWebRequestHandler` (deploy) and `createNodeServer`/`createNodeHandler` (local dev).
- `docusaurus-plugin-mcp-server/theme` — `McpInstallButton`.

Peers: `@docusaurus/core` (and `react`/`react-dom` for the theme button) are optional peer deps; provide them from your Docusaurus app.

## Authoritative API

The authoritative API is the published TypeScript types. Read the `.d.ts` files referenced by `exports` in `package.json` before writing calls — do not guess signatures:

- `.` → `dist/index.d.ts` (plugin options, `McpDocsServer`, provider/`ProcessedDoc` types)
- `./adapters` → `dist/adapters-entry.d.ts` (`WebRequestAdapterConfig`, `NodeServerOptions`, and the shared `McpServerBaseConfig`/`McpServerFileConfig`/`McpServerDataConfig`)
- `./theme` → `dist/theme/index.d.ts` (`McpInstallButton` props)

Config shape in particular (data-vs-file, required fields, `instructions`/`tools` overrides) lives in those types — read them rather than copying field lists.

## Usage patterns

**1. Build step.** Add the plugin to the `plugins` array in `docusaurus.config.js` with a `server.name`. `docusaurus build` then writes `build/mcp/`.

**2. Deploy to a serverless/edge runtime.** Edge/Worker runtimes can't read the filesystem, so import the artifacts as modules and pass them as pre-loaded data to `createWebRequestHandler`, which returns a standard `(request: Request) => Promise<Response>`. The handler is identical across runtimes — only the export wrapper and platform config differ:

```js
import { createWebRequestHandler } from 'docusaurus-plugin-mcp-server/adapters';
import docs from './build/mcp/docs.json';
import searchIndexData from './build/mcp/search-index.json';

const handler = createWebRequestHandler({ docs, searchIndexData, name: 'my-docs', baseUrl: 'https://docs.example.com' });
```

Per-platform glue to scaffold:

- **Cloudflare Workers** — `export default { fetch: handler }`; add a `wrangler.toml` whose `main` is the worker entry, and a `[[rules]] type = "Data"` rule globbing `**/*.json` so the JSON imports bundle.
- **Deno / Bun** — `export default { fetch: handler }` (both honor the `fetch` default export).
- **Modern Netlify functions** — `export default async (request) => handler(request)` (the new web-standard functions API, not the legacy `event`/`context` one).
- **Vercel** — use the Edge runtime: `export const config = { runtime: 'edge' }` and `export default handler`.

**3. Run locally.** `createNodeServer(...)` returns an `http.Server` you `.listen()`; it reads from disk via `docsPath`/`indexPath`. Use `createNodeHandler(...)` to mount into an existing `http.createServer`.

**4. Install button.** Render `McpInstallButton` (from `./theme`) in a navbar component with your `serverUrl`/`serverName`.

## Common mistakes

- **Filesystem paths on edge/Workers.** `docsPath`/`indexPath` only work where there's a filesystem (local Node). On Workers/edge, import the JSON and pass `docs`/`searchIndexData` to `createWebRequestHandler`.
- **Cloudflare JSON imports fail without the Data rule.** Missing `[[rules]] type = "Data"` in `wrangler.toml` makes the `docs.json`/`search-index.json` imports break at deploy.
- **Reaching for removed/renamed handlers.** There is now a *single* generic deploy handler. `createVercelHandler`, `createNetlifyHandler`, and `generateAdapterFiles` were removed. `createCloudflareHandler` still exists as a **deprecated** alias of `createWebRequestHandler` — use the new name.
- **Wrong `baseUrl`.** It must be the site origin plus the Docusaurus `baseUrl` (e.g. `https://example.com/docs/`); otherwise the URLs in search results point to the wrong place.
- **Deploying before building.** The handler needs `build/mcp/*` — run `docusaurus build` first.

## Version notes

Check the installed version with `npm ls docusaurus-plugin-mcp-server`. The adapter surface recently consolidated to one web-standard `createWebRequestHandler`; the legacy `createVercelHandler`/`createNetlifyHandler`/`generateAdapterFiles` are gone and `createCloudflareHandler` is a deprecated alias scheduled for removal — confirm against `dist/adapters-entry.d.ts` for the version you have.
Loading