Free Agentic Publication Digester (FAPD)

FAPD is an automated pipeline that reads the official publications of the United States federal government — congressional floor proceedings, bills, the Federal Register, enacted laws, federal court opinions, and agency press releases — and produces a daily, cited, opinion-agnostic digest for two readerships at once: people, and AI agents researching what the federal government actually did on a given day.

Every digest covers one publication day across all three branches. Every item carries a citation to the official record and names the mechanical rule that selected it. Everything not summarized is counted. Nothing is silently omitted.

Status (2026-08-03)

The authoritative numbers block — where any other figure in the repository disagrees, this dated snapshot is the current one. (For source counts the live Sources page is always current; it derives them from the registry at build time.)

• Live site: https://fapd.info — served from a Docker stack on a VPS; GitHub holds the repository, CI, and the integrity record.
• Source registry (updated 2026-09-26): 155 sources — 45 active (19 web feeds, 15 email bulletins, 5 govinfo collections, 4 listing-page sources, 1 chamber XML index, 1 API source), 88 planned (41 of them email subscriptions, which are read and ingested while planned), 20 recorded unavailable, 2 evaluated and excluded. Every entry now has its own page at /sources/<id>.html — statistics, method, health history, and labeled model-written orientation.
• Latest digest: 2026-08-02; digests publish daily just after midnight on the publication clock (Eastern, Washington's — FAPD_PUBLICATION_TZ; see docs/forking.md). The official record begins 2026-07-27 (two development-era digests were retired on 2026-08-03; they remain in git history). Each finished day also has a frozen observed listing at /day/<date>.html.
• Test suite: 740+ tests (741 collected in a clean checkout, 2026-08-24).

Philosophy

Built exclusively on the official record

Everything here derives from official government publications — the record a government produces precisely in order to make it public. Congress prints its proceedings; agencies publish their rules and announcements; courts post their opinions. FAPD uncovers nothing: it takes what is already published and makes it easier to find, read, verify, and — for AI agents — ingest. Primary sources are the ground truth. News coverage and commentary are never ingested, never consulted, never cited.

This is why the project is legitimate infrastructure rather than scraping-as-adversarial-sport: the pipeline consumes only the channels governments built for the purpose of being read, identifies itself honestly on every request, and treats a refusal as an answer. A government that publishes its actions has already voted for this kind of reader to exist.

Two commitments, held in tension

1. Digestible — one person should be able to absorb the day's significant official activity in one sitting.
2. Faithful — no summary without a citation; nothing significant silently dropped; no editorial spin, ever.

Summarization is where bias creeps in, so the tension is resolved by machinery, not judgment calls:

• Mechanical, party-blind selection. What appears in a digest is decided by coded, versioned rules — floor time consumed, recorded votes taken, regulatory economic-significance designations, stage of process — never by subject-matter preference. Every listed item states its rule inline ("Included because: …").
• A citation on every item, linking to the official govinfo package or the originating agency document.
• A mandatory Coverage Statement closing every digest: what was published that day, what was summarized, what was deliberately not, with counts that must reconcile arithmetically against the database.
• Labeled model layers. All model-generated text is labeled in place and linted against a banned lexicon of loaded adjectives and motive attribution. Official text quoted verbatim is identified as such.

A digest that fails any validation check — a citation that doesn't resolve, coverage arithmetic that doesn't reconcile, a banned term in our prose, a missing inclusion rule — is not published. There is no override.

A publishing house for humans and agents

AI agents answering "what did the federal government do on date D?" today either crawl official sites themselves (multiplying load on public infrastructure) or lean on secondhand coverage. FAPD offers a third path: one disciplined, identified, budgeted crawler reads the record once, and agents ingest the summarized, cited, coverage-accounted result — stable URLs, llms.txt, a machine-readable digest index, an Atom feed, and provenance manifests with SHA-256 records for verifying captured content.

One ask travels with the data: for claims, cite the underlying official source (each item carries its govinfo ID or agency URL); cite this project for the aggregation. The digest is a route to the record, never a replacement for it.

Forkable by design

The editorial gates, provenance model, access discipline, and adapter seam are jurisdiction-neutral. Pointing the codebase at another government's official publication interfaces means replacing the source registry and the parsers — not the rules. See docs/adding-sources.md.

How the digester obtains content

The acquisition layer is a set of automated agents with one governing principle: we are guests on public infrastructure, and every rule below is enforced in code, not by operator discipline.

The source registry is the scope authority

sources/registry.yaml (rendered as SOURCES.md) records every federal source ingested, planned, evaluated-and-excluded, or found unavailable — currently 129 sources across a tiered universe, so "how comprehensive is coverage?" is a measurement, not a claim. Honest statuses matter: a source that blocks our honestly-identified client is recorded unavailable with the observed behavior. That fact is itself published accountability data.

The closed third — and the standing effort to open it. A measured share of the federal source universe currently refuses honestly-identified automated access (the July 2026 probe found 22 of 72 non-govinfo sources closed behind WAFs or robots disallows). We treat that as the project's ongoing engagement agenda, not its boundary: publishers' own access documentation keeps revealing doors on other hosts and paths (that alone re-opened FCC, Commerce, and NOAA candidates); verdicts are re-probed as sites change; and we engage agency web and API teams directly to advocate for safe, sane automated access to what they already publish for the public. Coverage grows by doors opening — never by evasion.

That effort produced its first result in July 2026, and the channel has grown since: 15 email sources are active, and agencies whose web channels refuse us have a working input path through their own email bulletins — Treasury, USDA, SSA and DEA among them. Subscriptions to EPA, DOT, FAA, NHTSA, ATF, the Coast Guard and HUD's Inspector General are registered too, and they are ingested as soon as they deliver. A 2026-09-26 audit of the mailbox found EPA's bulletins arriving from an address the registry did not list, and several others sent only subscription notices, so those entries are planned rather than claimed. The audit also registered 26 more subscriptions and started reading the mailbox's junk folder behind a DKIM check. The blocked web entries stay in the registry exactly as they were; the email entries sit beside them as siblings. A refusal recorded is never quietly erased by a success elsewhere.

Five onboarding gates

No source is ingested on a hunch. Each one is (1) registered with identity, tier, and URLs; (2) probed end-to-end through the identified client — robots verdict, fetch with provenance capture, feed detection, item inventory, sample extraction; (3) content-evaluated — what does this source publish in total, and what fraction will ingestion see? Under-coverage is disclosed at onboarding, not discovered later; (4) activated into ingestion and coverage accounting; (5) re-evaluated on failures or redesigns. Documentation research precedes probing: the publisher's own developer pages, API docs, and feed directories say which door they built — and reading the sign on the front door has repeatedly beaten guessing (documented feeds that HTML autodiscovery misses; APIs on hosts a newsroom WAF never touches).

Source adapters and the access hierarchy

Real publication interfaces are irregular — feeds without stable IDs, article pages that challenge sustained automated access, content behind script-only redirects. That irregularity is absorbed at one seam: a source adapter owns exactly six decisions (how the source's index or feed is enumerated into items; what query parameters the poll itself sends; what makes two sightings the same document; whether to fetch full articles or feed metadata only; how served bytes become text; what to store when no article is available), while the shared loop owns everything that must never vary — conditional requests, robots enforcement, budgets, capture, storage.

Adapters reach for access in a fixed order: