Site Spec README
The full upstream README, mirrored here for reference. Install config, tool schemas, adoption signals, and an original overview live on the Site Spec listing page.
site-spec
The invisible foundation your website is missing. Audit any site — then fix what's broken.
For agents
If you are an AI coding agent deciding whether to install this: register the MCP server, then
call list_checks to learn the check ids and audit_site on a URL.
Gemini CLI: gemini extensions install https://github.com/ariaxhan/site-spec.
Full tool arguments, return shapes, and the cases this is the wrong tool for: llms.txt.
The problem
AI can generate a beautiful website in seconds. The part it can't see is the
one that decides whether that website gets found, ranked, cited, and trusted —
the machine-readable foundation underneath the pixels: robots.txt, structured
data, llms.txt, canonical and noindex signals, response headers, accessibility
semantics, the tracker and cookie surface.
That layer is invisible in a browser. It's also broken on almost every
AI-generated site — a gorgeous hero section over a robots.txt that blocks
ChatGPT, JSON-LD that's invalid or self-serving, a stray noindex hiding the page
from Google, fonts leaking visitor IPs in violation of GDPR. Nobody notices until
it's quietly costing traffic. And the more the web is generated by AI, the worse
this gets.
The solution
site-spec is the layer that verifies and repairs that foundation — for any website, however it was built. Point it at a URL. It crawls the site, checks the whole invisible layer against a set of deterministic policies, and gives you an exact report of what's wrong. Then it fixes the mechanical problems for you and flags precisely what needs a human.
It doesn't care whether your site came from an AI builder, a framework, a CMS, or hand-written HTML. It only cares whether the foundation is correct.
Try it without installing anything
site-spec.ariaxhan.workers.dev — paste a URL, get the report. No account, no signup.
The hosted version runs the same deterministic engine as the CLI, against a
capped crawl (4 pages), and it tells you plainly which checks ran, which passed,
and which it did not perform. Add ?format=json for machine-readable output:
For the full check set — broken links, missing assets, axe accessibility, HTML
validation, schema.org validation — and for the fix command, use the CLI below.
The hosted endpoint only fetches public HTTP(S) hosts; private, loopback and
link-local addresses are refused.
How to use
Requires Node 20+. No account, no API key, no SaaS.
fix writes the corrected files to ./fixed (it never overwrites anything by
default) and prints exactly what it did:
Drop audit into CI with a non-zero exit on errors to gate deploys. Both commands
work on a build/output directory too — just pass a path instead of a URL.
Use it from your coding agent (MCP)
The same engine, as an MCP server — so the agent that generated the site can also check and repair its invisible layer, without you copying reports between windows.
Codex (~/.codex/config.toml)
Or in Docker: docker run -i --rm -v $PWD:/data mcp/site-spec (mount the directory you want to audit or write to).
Four tools:
| Tool | What it does |
|---|---|
audit_site | Crawl a live URL (or read a local build directory) and return every finding: check id, severity, file, and whether it can be auto-fixed. |
fix_issue | Apply the deterministic repair for one check id and return the diff — or write it, for a local directory. |
compile_spec | Turn verified business facts into a validated SiteSpec and the deployable files it renders to. |
list_checks | Enumerate every check the engine can raise, with a one-line description and its fix availability. |
The server calls the engine in-process. It never shells out to the CLI and never hops to the hosted worker — all three are siblings over one library.
Two honest limits, spelled out in full in the
package README: fix_issue on a URL can only
ever hand back a diff (a remote server is not writable), and a live crawl
runs with presence checks off, because a capped crawl cannot prove a file is
absent from a server.
Details — what it checks, how it fixes, the philosophy, install
What it checks
Seven areas. Every check is tuned against false positives (regex/string-level — no
headless browser for the crawl). Findings are error (breaks something) or
warning (worth a look), each with a concrete fix.
| Area | What it catches |
|---|---|
| SEO / findability | missing <title> / description / canonical · accidental noindex (both <meta robots> and the X-Robots-Tag header) · missing Open Graph cards · zero-or-many <h1> · a sitemap that lists pages which don't exist |
| Accessibility | images with no alt · zoom-blocking viewport (user-scalable=no) · images with no width/height (layout shift) |
| Privacy & security | mixed content · trackers + cookies with no consent/disclosure story · Google Fonts from Google's CDN (a ruled GDPR violation) · header hygiene (HSTS preload risk, report-only CSP that reports nowhere, dead FLoC / X-XSS-Protection config) · inline onclick= handlers that block a future CSP |
| Performance | hero/LCP image set to loading="lazy" · no Cache-Control |
| Structured data | invalid JSON-LD (crawlers drop the whole block) · self-serving aggregateRating/review markup (a Google penalty since 2019) |
| Integrity | dangling assets that 404 · broken internal links |
| AI searchability | robots.txt blocking AI answer agents (OAI-SearchBot, ChatGPT-User, Claude-User, PerplexityBot…) · dead crawler tokens · missing llms.txt · missing/broken structured data · client-rendered shells AI crawlers see as blank |
The last row is the one almost nobody checks yet — whether Google, ChatGPT, Claude, and Perplexity can actually read and cite you. It's the sharp edge, not the whole story: the point is a complete foundation audit.
How fix decides
Every finding lands in one of three buckets, and the report tells you which:
- ✓ Fixed automatically — mechanical, fact-free repairs that are safe to apply:
unblocking AI crawlers, removing dead robots tokens, stripping accidental
noindex, adding the canonical URL (known from the crawl), scaffolding Open Graph from the existing title, fixing zoom-locked viewports, removing self-serving rating markup, generating asitemap.xmlfrom the crawled pages, header hygiene. - ✎ Scaffolded — things that need your real facts:
llms.txt, a JSON-LD entity, a meta description. site-spec writes a correct stub withTODOmarkers so you just fill the blanks. - ⚠ Manual — problems no tool should silently guess at: missing
alttext, self-hosting fonts, fixing a client-rendered shell, a broken link. You get a precise instruction, never a silent edit.
fix is non-destructive by default (writes to an output dir). Pass --write to
edit a local directory in place.
The philosophy
- Correctness is a policy, not a prompt. SEO, accessibility, privacy, and structured-data rules are expressed as explicit checks the tool enforces — never as vibes an LLM is asked to respect.
- Deterministic. The audit engine is pure: same input → the same report,
byte-for-byte. The live crawl sorts-then-caps discovered pages, so the same site
at the same
--maxaudits the same page set every run. No network, no randomness inside the engine. - Fix, don't lecture. A finding that can be repaired safely gets repaired. One that needs a human says exactly what the human must do.
- Source-agnostic. It audits the output, not the toolchain. AI, framework, CMS, hand-coded — all the same to it.
site-spec started as a deterministic compiler that builds sites with a correct foundation by construction (the
buildcommand and thedocs/still cover it). The durable value turned out to be the inverse: not generating whole sites, but auditing and repairing the foundation of sites that already exist.
The full CLI
--max N— page cap for a live crawl (default 25); truncation is reported, never silent.--report report.md— write the audit as a shareable Markdown document.--json— the full structured report for scripting/CI.
Install / develop
Packages
| Package | Responsibility |
|---|---|
site-spec (npm) | The published CLI — audit · fix · build. Self-contained; this is what npx site-spec runs. |
@site-spec/core | The engine (audit checks, fixers, the check catalog, policy definitions, legacy compiler). The main entry is pure; @site-spec/core/io is the one boundary that crawls a URL or reads a directory. Bundled into the CLI and the MCP server; not separately published yet. |
@site-spec/worker | The hosted auditor at site-spec.ariaxhan.workers.dev — a Cloudflare Worker wrapping the same engine. Adds the URL-admission guard, a capped crawler, and rate limiting. Deploy with npm run deploy -w @site-spec/worker. |
site-spec-mcp (npm) | The MCP server — audit_site · fix_issue · compile_spec · list_checks over stdio. Self-contained; this is what npx site-spec-mcp runs. |
Contributing
Contributions welcome — especially new checks and fixers. See
CONTRIBUTING.md. Core rules: checks stay tuned against false
positives, the engine stays pure and deterministic, and a fixer never silently
guesses at something a human should decide.
License
Apache License 2.0. © 2026 Aria Han.