The full upstream README, mirrored here for reference. Install config, tool schemas, adoption signals, and an original overview live on the Rangefind listing page.
Rangefind is a static search engine for large sites that cannot or should not run a search server.
It builds a sparse inverted index into static files, then lets the browser query
that index with HTTP Range requests. The core idea is simple: keep the logical
search shards small, but pack them into a small number of static files so deploys
and cold starts stay practical.
This repository is the standalone extraction of a production prototype built for a large thesis corpus.
The list below is the compact overview. The complete feature guide explains every supported search lane, build mode, runtime, integration, feature combination, fallback, and deliberate boundary with runnable examples.
rangefind build ./dist indexes built HTML directly,
with data-rangefind-* attributes for body scoping, metadata, facets, and
sorts.terms/packs/*.bin.gz.dist/runtime.browser.js.rangefind/node) for MCP servers, CLIs, and SSR: same query
engine over local index directories (positional file reads) or remote
HTTP(S) indexes, with browser-equivalent caching — immutable
content-addressed objects cached on disk and in a memory LRU, manifests
revalidated with ETags.rangefind/mobile) for React Native/Hermes and other
embedded JS hosts: fully offline search over a bundled or downloaded index
via positional file reads, or remote HTTP(S) indexes with memory + persistent
caching — see docs/mobile.md (covers WebView, Flutter, and native apps
too).terms/directory-root.<hash>.bin.gz
and terms/directory-pages/*.bin.gz.Range coalescing and bounded overfetch.rangefind/route): a CRP/MLD
route graph built from OSM road networks into range-addressed cell and
overlay objects — point-to-point driving routes with geometry and named
steps, travel-time matrices, and multi-stop itinerary optimization, all
exact versus a full-graph Dijkstra and served from static files with a
bounded fetch set per query; sharded builds share one top overlay and
route identically to monolithic ones — see docs/route-graph.md.analysis config): per-document language detection
or an explicit language field, per-language light stemmers and stopwords
for 20+ languages, script-aware Unicode folding (Latin, Greek, Cyrillic,
Arabic, Hebrew, Devanagari), dictionary-free CJK bigram tokenization that
is deterministic across Node and every browser, cross-language query
expansion with index-evidence base-plan selection, and the whole profile
frozen into the manifest so query analysis always matches the index.rangefind build --update adds small delta
generations over an existing index — unchanged pack bytes keep their
content-addressed names (and CDN cache entries), replaced documents
tombstone their old version, and the runtime merges every lane (text,
sorted browse, geo, vector, hybrid, suggest, facets) across generations
with scores that stay exactly comparable. rangefind build --compact
folds generations back into a single index when they pile up.addr:interpolation ranges, structured forward and reverse
geocoding, bounded multi-provider civic/postal augmentation (including the
Québec RQA adapter), reusable
rangefind/osm and rangefind/osm/node integration APIs, and exhaustive geo
oracles.opening_hours evaluation; route-aware place search; compact place details;
and simplified encoded area geometry for real polygon/line map rendering.scoringStats config + rangefind/scoring-stats), a tiny sharded root
manifest (rangefind/shards), lazy shard engines with bbox geo routing and
expanding nearest-first search, and merged lanes whose rankings match a
monolithic build of the same corpus exactly — see docs/sharded-osm.md.linkRank): the static-site crawler resolves internal
<a href> links into a document graph and folds a build-time PageRank prior
into a sortable numeric doc-value. Well-linked pages win near-ties in ranking
(an opt-out multiplicative boost, score *= 1 + boost*linkRank, applied as a
bounded window rerank with no change to the scoring hot loop), you can sort by
authority, and there is zero query-time graph traversal. Any corpus that knows
its own edges can opt in with the turnkey examples/link-graph-enrich.mjs
enricher (or the exported rangefind/link-graph helper); see
docs/link-graph.md.Most browser search libraries assume the browser downloads a whole index, or they optimize for smaller sites. Rangefind is for the case where you want:
Demos: https://rangefind.dev/ — an OpenStreetMap map search backed by the rolling public sharded index (autocomplete, viewport geo queries, and nearest-neighbor) and the minimal example, both served as pure static files.
The already-generated planet-scale OSM index is free to query directly from browsers or Node—no account or API key required:
See its live coverage and freshness at https://osm.rangefind.dev/. It is a
best-effort public service; self-host the index when you need guaranteed
capacity, version pinning, custom fields, or an operational SLA. OSM result
UIs must display
© OpenStreetMap contributors.
Open http://localhost:5178/.
Start with the guide that matches the job:
| Guide | Use it for |
|---|---|
| Feature guide | Complete capability catalog, examples, supported combinations, compatibility, and boundaries. |
| Reference | Configuration properties, builder/runtime APIs, tuning knobs, UI component, and deployment requirements. |
| Architecture | Static file layout, range directories, posting retrieval, geo, authority, and vector internals. |
| Autocomplete guide | Search-as-you-type patterns, measured per-keystroke costs, hydrated previews, instant selection, and the anti-patterns. |
| OSM example | Map search, autocomplete, geocoding, constraints, route corridors, geometry, extraction, and RQA. |
| Replace Google Maps APIs | Free public OSM index, migration adapter, API mapping, complete use cases, production checklist, attribution, and parity boundaries. |
| Node / mobile | Local, remote, cached, embedded, and offline runtimes. |
| Sharded OSM | Planet-scale regional builds, root routing, and exact score comparability. |
| Route graph | Static routing, travel-time matrices, and itinerary planning over range-addressed road-network cells and overlays. |
| Production maps benchmark | Google Maps-style workload coverage, quality assertions, and phone budgets. |
If you already have a built static site (from any generator), point Rangefind at the output directory and it indexes the HTML directly — no JSONL to author:
This crawls every .html/.htm file under ./dist, extracts the title,
headings, and main body text, and writes the index to ./dist/rangefind (change
it with --output). Result URLs default to site-root paths (--base-url sets a
prefix or origin, e.g. --base-url https://example.com/). Deploy the site and
load the index with createSearch({ baseUrl: "/rangefind/" }).
Content selection follows the document: the crawler indexes <main>, else
<article>, else <body>, and always drops <script>, <style>, <nav>, and
<aside>. <header>/<footer> are dropped only when the fallback <body>
region is used (no <main>/<article>), so an in-article post header and its
<h1> are still indexed. Opt-in data-rangefind-* attributes
(mirroring Pagefind's vocabulary) tune it:
| Attribute | Effect |
|---|---|
data-rangefind-body | Index only the text inside marked elements. |
data-rangefind-ignore | Drop this subtree (on <html>/<body> skips the page). |
data-rangefind-meta="name" | Capture element text as metadata name ("name:attr" reads an attribute). |
data-rangefind-filter="key" | Add the element text as a facet value under key. |
data-rangefind-sort="key" | Use the element text as a sortable value key. |
Per-document language comes from <html lang="…"> and the page description from
<meta name="description">. See the
reference guide for the full rules.
Pair the crawler with a drop-in search box. <rangefind-search> is a
framework-agnostic Web Component: two lines of HTML give any static site a
polished, accessible search box (WAI-ARIA combobox, keyboard navigation,
instant results, autocomplete).
It renders into its light DOM and ships no styling of its own, so it
composes with whatever CSS you already use. Style it three ways: the optional
theme stylesheet, your own CSS via the rf-search* hook classes, or utility
classes passed per part. Here it is styled purely with Tailwind — no theme CSS:
Prefer the batteries-included look? Link the opt-in theme (light + dark) and
skip the *-class attributes:
Works in plain HTML, React, Vue, Svelte, and Angular — Web Components are
universal. It emits rangefind:search, rangefind:select, and
rangefind:error events, and takes .classNames / .searchOptions JS
properties for programmatic use. Build the bundle with npm run build:element;
try it against the basic example (npm run build:example && npm run serve:example) at /component.html (theme) and /component-tailwind.html
(Tailwind). See the
reference guide for every attribute,
the full class-hook list, events, and accessibility notes.
The crawler and the search component compose naturally with any static site generator's own build lifecycle — an adapter runs the crawl right after the generator writes its output and drops the client assets in for you, so there's nothing to wire up by hand beyond installing it.
| Generator | Package | How it hooks in |
|---|---|---|
| Astro | rangefind-astro | Real integration on astro:build:done, plus a <RangefindSearch /> component. |
| Eleventy | eleventy-plugin-rangefind | Real plugin on eleventy.after, plus a universal {% rangefindSearch %} shortcode. |
| Docusaurus | docusaurus-plugin-rangefind | Real plugin on postBuild + injectHtmlTags; drop <rangefind-search> into a navbar item or MDX. |
| Hugo | integrations/hugo | Hugo has no plugin loader, so this is a hugo && rangefind build public recipe plus a copy-paste partial. |
| MkDocs | mkdocs-rangefind (PyPI) | A real Python plugin (on_post_build / on_post_page) that shells out to the Node CLI and auto-injects the widget into every page. |
Astro, Eleventy, and Docusaurus are each real, independently installable
packages under packages/ (built with zero dependencies beyond the target
generator itself); Hugo and MkDocs live under integrations/ since neither
has an npm-based plugin system of its own. Every one of them is verified
end to end against the real tool — a real Astro/Eleventy/Docusaurus build, a
Homebrew-installed Hugo binary, and a pip-installed MkDocs — crawling a
fixture site and confirming the resulting index actually answers a search
query. See each package's own README for install instructions and options.
The examples/wiki-search project is a fuller static search application for
Wikimedia article dumps. It defaults to the latest English Wikipedia articles
dump and can run on a bounded sample or the full dump:
Use npm run build:wiki-site:full to build the full default dump with the
bounded body cap, or npm run build:wiki-site:fr:full for the full French
Wikipedia dump. The generated site lives at examples/wiki-search/public/ and
keeps the same static hosting requirement as every Rangefind index: the host
must support HTTP Range requests for .bin.gz files.
Rangefind includes a reproducible French Wikipedia fixture that streams the official Wikimedia article dump, builds a static site, and benchmarks query latency, request count, and transfer size:
For runtime-only changes, reuse the existing generated index and rerun just the request/transfer benchmark:
Use --limit=0 to run against the full dump. The generated site lives at
examples/frwiki/public/. The fixture validates text query top-k against the
exact retrieval path by default and records cold request counts, transfer bytes,
runtime posting-block stats, typed filter/sort validation, and scale reports
across multiple Wikipedia sample sizes.
Extracted JSONL is cached under examples/frwiki/data/, so later runs with the
same dump/body cap and an equal or smaller limit do not stream the dump again.
Each generated index includes manifest.min.json, manifest.full.json, and
debug/build-telemetry.json. The full diagnostic manifest.json is still
written for local inspection and records phase timings, sampled memory peaks,
CPU time, disk byte deltas, and segment counters.
For the complete configuration schema, every tuning knob, the full runtime API, and deployment requirements, see the reference guide. The walkthrough below is the quick version.
Create newline-delimited JSON:
Create rangefind.config.json:
display controls only the payload returned with search results. Indexed fields
can stay long while returned fields are capped, for example:
For large static corpora, targetPostingsPerDoc is the body-term budget.
bodyIndexChars caps only the text considered by the indexer, while display
payload size stays controlled by display entries. Terms from
alwaysIndexFields are indexed before the body budget is applied.
typoMode: "main-index" uses the normal term vocabulary for typo candidates
instead of building a separate typo sidecar. Use typoMode: "off" to disable
correction.
authority fields build a packed lexicon for canonical labels such as
titles, entity names, product names, slugs, and aliases. The runtime first tries
a diacritic-preserving surface-exact key, then falls back to folded exact and
token keys only when needed, so common title rescue stays precise and cheap
without forcing all label logic into the BM25 posting lists. suggest fields
stream into the same external-run reducer under an autocomplete namespace, so
multi-million-document builds never create a duplicate in-memory title map or
a separate suggest/ pack family.
Build:
Query in the browser:
Filters and sort use range-addressed doc-value columns:
Geo fields index one point per document into a range-addressed static KD tree
(Lucene LatLonPoint-style, E7 fixed precision):
Results include distanceMeters whenever geo.near is present. Radius and
box filters are exact (bounding-box prune plus Haversine verification), and
stats reports the tree traversal (geoLane, geoLeavesVisited,
geoPointsScanned, ...). See examples/osm-geo/ for an OpenStreetMap-scale
example and benchmark.
Weighted and mid-label search-as-you-type suggestions use the authority lexicon:
Matching is prefix-based over diacritic-folded keys ("montre" finds
"Montréal") and covers mid-label tokens ("eiffel" finds "Tour Eiffel").
Ranking uses an optional weightPath (for example population or importance)
and falls back to direct-prefix priority plus popularity — how many documents
share the surface. Each keystroke costs at most a few small range requests;
repeat keystrokes in a session are usually served entirely from cache.
Suggestions also carry the best documents behind each surface (doc/docs),
and hydrate: true resolves them into real search hits — the same display
payloads search() returns — so a dropdown can show result cards and a
selected suggestion renders instantly instead of re-running the query as a
search (measured 0.5 KB and 4 ms, versus 215 KB and 32 ms re-searching its
text). The autocomplete guide covers the patterns,
what each costs per keystroke, and when to prefer focused previews over
hydrating every row.
For title-prefix completion on an older index, a single
authority: [{ "name": "title", "path": "title" }] field is also sufficient.
The runtime walks the first matching authority shards in key order and hydrates
only the requested titles. New builds should configure suggest to get custom
weights, popularity fallback, mid-label completion, and hot-prefix lists in the
same authority packs.
Facet counts for filter UIs come back with the search response:
Hybrid semantic search takes a query embedding from the host (for example transformers.js in the browser) and fuses it with the text lane:
Typo fallback is automatic. For example, if statik search has no exact
first-page hits but static search does, the response includes:
The runtime expects the host to support HTTP Range requests for .bin.gz files.
GitHub Pages supports this. The included local server also supports it.
The unit tests cover analyzer normalization, binary varint/fixed-width codecs,
config resolution, shard/range planning, term/code binary round-trips, and an
end-to-end build plus browser-runtime query against a local HTTP Range server.
The end-to-end build test also verifies emitted build telemetry and file-backed
spool counters in the manifest.
The benchmark scripts are dependency-free and run against the example static
site. bench:quality reports known-item and typo-recovery Hit@k/MRR plus
structured filter/sort checks for facets, dates, booleans, and signed numbers.
bench:performance reports query latency, HTTP request count, and transfer size.
bench:directories compares global, naive prefix, and paged range-directory
layouts against an existing built index.
bench:osm-maps runs the weighted common map-search workload documented in
docs/osm-maps-benchmark.md, including locality,
nearest-first, viewport, shard fan-out, and phone-budget checks.
docs/performance-research.md tracks the top-k retrieval papers currently
guiding format decisions.
Planned next milestones:
Contributions are welcome — see CONTRIBUTING.md.
MIT © Xavier Jodoin