The full upstream README, mirrored here for reference. Install config, tool schemas, adoption signals, and an original overview live on the RemoveDuplicates.org listing page.
A focused static tool for removing duplicates from lines, lists, pasted spreadsheet data, CSV files, and local XLSX workbooks, deployed as a Cloudflare Worker with Static Assets.
Links:
License: MIT
A hosted MCP server is available at https://removeduplicates.org/mcp (Streamable HTTP). There is nothing to install and no API key or account. It is listed in the Official MCP Registry as org.removeduplicates/remove-duplicates.
Claude Code:
Cline (cline_mcp_settings.json):
Cursor (mcp.json) takes the same mcpServers entry without type. VS Code (.vscode/mcp.json):
Gemini CLI can install the repository as an extension (gemini extensions install https://github.com/suvadadepolo-blip/remove-duplicates); gemini-extension.json points it at the same endpoint.
Clients that only speak stdio can bridge with npx mcp-remote https://removeduplicates.org/mcp.
The server has one read-only tool, remove_duplicates. It takes text (up to 128 KiB) plus optional format (auto, lines, table), ignoreCase, trim, removeEmpty, keep (first, last), order (preserve, sort), compare (row, a column index, or a list of indices) and header, and returns the cleaned text with counts. Each call is processed in memory and nothing is stored. For .xlsx workbooks, use the Excel tool, which runs only in the browser.
.xlsx files, compares an entire row or a stable combination of selected columns, and creates the cleaned workbook in a dedicated Module Worker.public/vendor/sheetjs-0.20.3/ with its official source URLs, license, and SHA-256 receipt. It is lazy-loaded only after a valid XLSX passes local ZIP preflight and is never requested by the homepage./mcp endpoint, which handles text an AI agent sends for that one request and stores nothing.robots.txt names the major AI crawlers and declares Content-Signal: search=yes, ai-input=yes, ai-train=yes./llms.txt and /llms-full.txt (with /llm.txt redirects) describe the site for language models./index.md, /excel.md, /privacy.md, /terms.md), also served for Accept: text/markdown. HTML routes send Vary: Accept and RFC 8288 Link headers for the API catalog, llms.txt, the agent-skills index, and the Markdown alternate./.well-known/api-catalog (RFC 9727), /.well-known/mcp/server-card.json, /.well-known/agent-skills/index.json with remove-duplicates/SKILL.md, and /.well-known/ai-catalog.json./mcp is a stateless Streamable HTTP MCP server with one tool, remove_duplicates, capped at 128 KiB of text per call. The homepage registers the same tool through WebMCP when the browser exposes navigator.modelContext; that path runs locally in the tab. Both share public/js/agent-tool.js and the site's dedupe.js engine.server.json is the Official MCP Registry entry org.removeduplicates/remove-duplicates. Namespace ownership is proven over HTTP by /.well-known/mcp-registry-auth (Ed25519 public key); the private half lives only in the gitignored local.env.txt.node scripts/build-agent-files.mjs regenerates the server card, the skill digest, and llms-full.txt; npm run check fails when they are stale.Runtime preview, the XLSX benchmark, browser QA, downloaded-file readback, and Lighthouse belong on OpenClaw. Port 3630 is reserved for the managed preview. Deployment uses the project-scoped Cloudflare wrapper one directory above; credentials never live in this project.
output/, .codex-results/, .wrangler/, logs, and .DS_Store..env*, local.env.txt, and *.env.txt; only placeholder templates such as .env.example may be committed.EXCEL_PLAN.md, IMPROVEMENT_PLAN.md, PROJECT_STATE.md, UTILITY_BASELINE.md, and ops/.public/vendor/sheetjs-0.20.3/ is intentionally vendored with its upstream license, source receipt, and SHA-256 checks.removeduplicates-orgpublic/src/worker.jsASSETS, with run_worker_first: truehttps://removeduplicates-org.whatmyname.workers.dev/https://removeduplicates.org/https://www.removeduplicates.org/The Worker owns /excel, direct legal routes, public HTTP-to-HTTPS and canonical-host redirects, method handling, branded 404s, cache policy, HSTS, a hash-pinned CSP, and Workers.dev noindex protection. Excel path case and trailing-slash variants canonicalize to /excel in the same one-hop redirect as scheme or host changes. The Worker accepts only GET and HEAD.
The active Cloudflare Zone and wrangler.jsonc contain exactly these two custom_domain routes:
removeduplicates.orgwww.removeduplicates.orgThe Worker returns a 308 from www to the apex while preserving path and query. A partial route state is invalid, and no Zone Redirect Rule is used. The production apex, redirect alias, TLS, HTTP matrix, five-viewport browser suite, and Lighthouse are recorded as passing in the project state and production deployment ledger.