# rlhf-feedback-loop **Category:** ๐Ÿ’ป Developer Tools **Repository:** https://github.com/IgorGanapolsky/rlhf-feedback-loop **Views:** 0 **Installs:** 0 **Upvotes:** 0 **Directory Page:** https://allmcps.com/mcp/rlhf-feedback-loop ## Description RLHF feedback loop for AI agents. Capture feedback, block mistakes, export DPO data. ## Claude Desktop Quick Installation Heuristic fallback โ€” verify the package name and runner against the repository README before running it. Uses `npx` (confidence: low): ```json "mcpServers": { "rlhf-feedback-loop": { "command": "npx", "args": ["-y","rlhf-feedback-loop"] } } ``` ## Documentation & README # ThumbGate ๐Ÿ‘ ๐Ÿ‘Ž

ThumbGate Infrastructure Firewall with Thumbs Up and Thumbs Down

Self-improving pre-action firewall for AI coding agents
AI coding agents repeat mistakes โ€” and one wrong tool call can wipe a directory, leak a key, or push broken code.

MCP Toplist CI npm License: MIT

Quick Start Watch Demo Try GPT Pro Tier

--- ## What it does ThumbGate is the local-first **Pre-Action Checks** engine for AI coding agents. It runs in the PreToolUse hook to evaluate the proposed tool call before execution โ€” so costly mistakes can be caught before they happen. ### Tech memes (shareable) Lightweight visuals for how agents fail without a pre-action gate: | Meme | Meaning | |------|---------| | ![Agent destroys prod without a gate](docs/media/meme-agent-destroys-prod.svg) | Unchecked tool calls ship destructive commands. | | ![Prompt vs PreToolUse hook](docs/media/meme-prompt-vs-hook.svg) | A prompt is advice; a PreToolUse hook is enforcement. | It **hard-blocks detected secret leaks and two direct self-disable command classes by default** โ€” commands that terminate the ThumbGate gate process or enable its bypass environment override. Other high-risk classes (`rm -rf`, force-push, fetch-and-run, direct guardrail edits) **warn and log by default**. Set `THUMBGATE_STRICT_ENFORCEMENT=1` for strict enforcement (warnings become hard denies). | Verdict | Default behavior | |---------|------------------| | โ›” **Hard-block** | Detected secret leaks; process-kill/environment-override self-disable | | ๐Ÿ‘Ž **Warn + log** | `rm -rf`, `git push --force`, fetch-and-run, direct guardrail edits โ€” **warn by default** | | ๐Ÿ‘ **Allow** | Everything else | **Accepted feedback is stored as local lessons.** Repeated concrete failures can become prevention rules that promote from warnings to blocking gates. The firewall improves from operations without retraining the model. Prompt evaluation (`npx thumbgate eval`) turns accepted feedback into reusable eval cases and local proof reports. **Honest disclaimer:** ThumbGate does not update model weights. It intercepts tool calls at runtime. Local-first โ€” no cloud required for the enforcement path. Works with **Claude Code, Cursor, Codex, Gemini CLI, Amp, Cline, OpenCode**, and other MCP agents. [![AI Agent without ThumbGate vs Agent guarded by ThumbGate](public/assets/diagrams/thumbgate-agent-meme.jpg)](https://thumbgate.ai/#demo?utm_source=github&utm_medium=readme_meme) ``` Agent tries: rm -rf tests/ ThumbGate: ๐Ÿ‘Ž WARN + LOG โ€” "Never delete test directories" Pattern matched: rm.*-rf.*tests Source: your thumbs-down from last Tuesday Strict mode: โ›” DENY before tool execution ``` ### Agentic development cycle fit Agentic development is becoming a loop: **Guide โ†’ Generate โ†’ Verify โ†’ Solve**. ThumbGate is the pre-action gate / pre-action boundary between generated intent and executed action. --- ## Quick Start ```bash npx thumbgate init # wire PreToolUse hooks npx thumbgate capture down "Never run DROP on production tables" # ๐Ÿ‘Ž lesson npx thumbgate doctor # health check ``` Later `DROP` attempts in the same scope surface the check: ``` โš ๏ธ Check fired: "Never run DROP on production tables" Pattern: DROP.*production Verdict: ๐Ÿ‘Ž WARN + LOG (โ›” BLOCK when THUMBGATE_STRICT_ENFORCEMENT=1) ``` ### MCP / Glama / registry install (stdio) Directories and clients that install ThumbGate as an MCP server must start **stdio MCP**, not the HTTP API: ```bash npx -y thumbgate serve ``` - Equivalent: `npx -y thumbgate mcp` - Do **not** use `npm start` for MCP โ€” that launches the hosted HTTP API (`src/api/server.js`), not the agent-facing stdio server. [**โ–ถ 90-second demo**](https://thumbgate.ai/#demo?utm_source=github&utm_medium=readme&utm_campaign=demo_video) ยท [GIF walkthrough](docs/media/thumbgate-demo.gif) --- ## Install for your agent | Agent | Command | Enforcement | |-------|---------|-------------| | **Claude Code** | `npx thumbgate init --agent claude-code` | ๐Ÿ›ก๏ธ Hard โ€” PreToolUse | | **Codex** | `npx thumbgate init --agent codex` | ๐Ÿ›ก๏ธ Hard โ€” `pre_tool_use` | | **Gemini CLI** | `npx thumbgate init --agent gemini` | ๐Ÿ›ก๏ธ Hard โ€” PreToolUse | | **ForgeCode** | `npx thumbgate init --agent forge` | ๐Ÿ›ก๏ธ Hard โ€” `pre_tool_use` | | **Cursor** | `npx thumbgate init --agent cursor` | ๐Ÿ’ฌ Advisory โ€” MCP `gate_check` | | **Cline** | `npx thumbgate init --agent cline` | ๐Ÿ’ฌ Advisory โ€” MCP + `.clinerules` | | **OpenCode** | `npx thumbgate init --agent opencode` | ๐Ÿ’ฌ Advisory โ€” MCP `gate_check` | | **Any MCP agent** | `npx thumbgate serve` | ๐Ÿ’ฌ Advisory โ€” MCP `gate_check` | | **Amp** | `npx thumbgate init --agent amp` | ๐Ÿ“ Feedback capture | Per-agent guides: [Claude/Codex bridge](plugins/claude-codex-bridge/README.md) ยท [Codex profile](plugins/codex-profile/README.md) ยท [Cursor](docs/CURSOR_PLUGIN_OPERATIONS.md) ยท [MCP setup](docs/MCP_AUTONOMOUS_SETUP.md) ### Install scope: machine-wide vs per-project | Scope | Command | Settings | Lessons | Best for | |-------|---------|----------|---------|----------| | **Machine-wide** (default) | `npx thumbgate init` | `~/.claude/settings.json` | `~/.claude/memory/feedback/` | Solo operators โ€” **same machine-local feedback store** across repos | | **Per-project** | `npx thumbgate init --project` | `/.claude/settings.json` | `/.claude/memory/feedback/` | Client / compliance โ€” **separate dashboard** / isolated lessons per repo | Both scopes write `mcpServers.thumbgate` plus PreToolUse / UserPromptSubmit / PostToolUse / SessionStart hooks. Machine-wide is the right default for most developers. Cross-repo blocking is not automatic: a lesson learned in one project only applies elsewhere when you share the store (machine-wide) or export/import lessons. **MCP tools (surface):** `gate_check` (read/evaluate proposed tool call), feedback capture + session tools (write), dashboard/stats (read). Destructive agent actions stay blocked/warned by PreToolUse โ€” ThumbGate does not execute user shell commands for you. --- ## Discoverable slash-commands โ€” the guardrail layer for spec-driven agents Spec-driven agent frameworks like **GSD** (get-shit-done) and **GitHub Spec Kit** plan and generate work. ThumbGate is the **guardrail layer for spec-driven agents**: it sits *after* the plan, on the boundary between a generated tool call and its execution โ€” **alongside GSD / Spec-Kit, not instead of them**. `npx thumbgate init` installs these into your agent palette: | Command | What it does | |---------|--------------| | `/thumbgate-dashboard` | Open local project dashboard | | `/thumbgate-guard` | Turn last mistake into a hard prevention rule | | `/thumbgate-rules` | List active rules & lessons | | `/thumbgate-blocked` | Gate stats + enforcement matrix | | `/thumbgate-protect` | Branch governance + scoped approval | | `/thumbgate-doctor` | Health-check hooks, MCP, readiness | --- ## Pricing & buyer paths Free tier: **2 feedback captures/day (10 total)** and **up to 3 active auto-promoted prevention rules**. Pro ($19/mo or $149/yr) is the individual tier for unlimited rules, history-aware lessons, linked feedback session flow, personal dashboard, and DPO export. **Enterprise is custom and scoped after intake**; hosted team lesson sync and a hosted org dashboard are not general availability. | | Free | Pro ($19/mo or $149/yr) | Enterprise | |---|---|---|---| | Local CLI + PreToolUse | โœ… | โœ… | Scoped after intake | | Feedback captures | 2 feedback captures/day (10 total) | Unlimited | Scoped after intake | | Active auto-promoted rules | up to 3 active auto-promoted prevention rules | Unlimited | Scoped after intake | | Personal dashboard + DPO export | โ€” | โœ… | Reviewed during intake | | Hosted team lesson sync | โ€” | โ€” | Not general availability | | Hosted org dashboard | โ€” | โ€” | Not general availability | **Enterprise intake path:** the **Workflow Hardening Sprint** scopes one repeated failure before any broader rollout commitment. **[Start intake โ†’](https://thumbgate.ai/?utm_source=github&utm_medium=readme&utm_campaign=team_rollout#workflow-sprint-intake)** **Local technical path:** install the CLI and use `init` plus the documented setup so Pre-Action Checks evaluate tool calls where the agent actually runs. **First-dollar activation path:** open the [ThumbGate GPT](https://thumbgate.ai/go/gpt?utm_source=github&utm_medium=readme), paste the risky action, capture typed feedback (`thumbs down:` / `thumbs up:`). **Native ChatGPT rating buttons are not the ThumbGate capture path.** Ask: **what repeated AI mistake would be worth catching before the tool executes?** **Paid path for individual operators:** [ThumbGate Pro](https://thumbgate.ai/checkout/pro?utm_source=github&utm_medium=readme&utm_campaign=pro_page) is the self-serve side lane for a personal dashboard and export-ready evidence. [**Start free**](https://thumbgate.ai/?utm_source=github&utm_medium=readme) ยท [**Pro $19/mo**](https://thumbgate.ai/checkout/pro?utm_source=github&utm_medium=readme) ยท [**Live Dashboard**](https://thumbgate.ai/dashboard?utm_source=github&utm_medium=readme) ยท [**Team Sprint intake**](https://thumbgate.ai/?utm_source=github&utm_medium=readme#workflow-sprint-intake) ยท [**Workflow Hardening Sprint**](https://thumbgate.ai/?utm_source=github&utm_medium=readme&utm_campaign=top_cta#workflow-sprint-intake) ยท [**First Dollar Playbook**](docs/FIRST_DOLLAR_PLAYBOOK.md) **Popular buyer questions:** **[AI search topical presence](https://thumbgate.ai/guides/ai-search-topical-presence?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Relational knowledge and AI recommendations](https://thumbgate.ai/guides/relational-knowledge-ai-recommendations?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[AI Mode ads for agent governance](https://thumbgate.ai/guides/ai-mode-ads-agent-governance?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[MCP tool governance](https://thumbgate.ai/guides/mcp-tool-governance?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[AI agent pre-action approval gates](https://thumbgate.ai/guides/ai-agent-pre-action-approval-gates?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Background agent governance](https://thumbgate.ai/guides/background-agent-governance?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[GPT-5.5 model evaluation](https://thumbgate.ai/guides/gpt-5-5-model-evaluation?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Stop repeated AI agent mistakes](https://thumbgate.ai/guides/stop-repeated-ai-agent-mistakes?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Browser automation safety](https://thumbgate.ai/guides/browser-automation-safety?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Native messaging host security](https://thumbgate.ai/guides/native-messaging-host-security?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Autoresearch agent safety](https://thumbgate.ai/guides/autoresearch-agent-safety?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Cursor guardrails](https://thumbgate.ai/guides/cursor-agent-guardrails?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Codex CLI guardrails](https://thumbgate.ai/guides/codex-cli-guardrails?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Gemini CLI memory + enforcement](https://thumbgate.ai/guides/gemini-cli-feedback-memory?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Google Cloud MCP guardrails](https://thumbgate.ai/guides/gcp-mcp-guardrails?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** ยท **[Roo Code alternative: migrate to Cline](https://thumbgate.ai/guides/roo-code-alternative-cline?utm_source=github&utm_medium=readme&utm_campaign=buyer_questions)** --- ## How it works (short) 1. **Capture** ๐Ÿ‘/๐Ÿ‘Ž feedback (CLI, MCP, linked feedback session flow / `open_feedback_session`, or [ThumbGate GPT](https://thumbgate.ai/go/gpt?utm_source=github&utm_medium=readme)) 2. **Promote** concrete lessons via history-aware lesson distillation into prevention rules 3. **Evaluate** the next proposed tool call against active rules (literal/AST + local vectors) 4. **Allow / warn / deny** before the tool runs ```bash npx thumbgate brain --write # โ†’ .thumbgate/BRAIN.md (lessons + gates in one artifact) ``` Pro operators can invoke `search_lessons` through MCP and use `npx thumbgate lessons` from the CLI. History-aware feedback sessions and lesson search are Pro capabilities; Free does not include recall or search.
Architecture diagram & stack [![ThumbGate Architecture](docs/diagrams/thumbgate_architecture.png)](https://thumbgate.ai/#how-it-works) ```mermaid flowchart LR A["Agent tool call"] --> B{"Rule match?"} B -- exact --> D["On-device gate"] B -- semantic --> C["Local LanceDB"] C --> D D -- secret/kill --> E["โ›” Hard-block"] D -- known-bad --> G["๐Ÿ‘Ž Warn + log"] D -- safe --> F["๐Ÿ‘ Allow"] ```
Built-in checks ``` โ›” secret-exfiltration โ†’ hard-block (default) โ›” self-protect-kill โ†’ hard-block (default) โ›” self-protect-env โ†’ hard-block (default) โš ๏ธ force-push โ†’ warn; hard-block under strict โš ๏ธ protected-branch โ†’ warn; hard-block under strict โš ๏ธ unresolved-threads โ†’ warn; hard-block under strict โš ๏ธ package-lock-reset โ†’ warn; hard-block under strict ```
CLI cheatsheet ```bash npx thumbgate init npx thumbgate doctor npx thumbgate capture up|down "" npx thumbgate lessons npx thumbgate brain --write npx thumbgate dashboard --open npx thumbgate break-glass --reason="ThumbGate over-fired" # 5-min recovery ```
Pro: lesson + DPO export ```bash # Portable lessons curl -X POST http://localhost:3456/v1/lessons/export \ -H "Authorization: Bearer $THUMBGATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"outputPath": "./lessons-export.json"}' # DPO pairs for fine-tuning curl -X POST http://localhost:3456/v1/dpo/export \ -H "Authorization: Bearer $THUMBGATE_API_KEY" \ -o dpo-pairs.jsonl ```
--- ## Tech Stack | Layer | Tech | |-------|------| | **Runtime** | Node.js โ‰ฅ18 | | **Interfaces** | MCP stdio, HTTP API, CLI | | **Storage** | SQLite + FTS5, LanceDB vectors, JSONL logs | | **Intelligence** | MemAlign dual recall, Thompson Sampling, local embeddings | | **Billing / host** | Stripe, Railway | | **Execution** | Railway, Cloudflare Workers, Docker Sandboxes | | **Governance** | Workflow Sentinel, control plane, Docker Sandboxes | Every Changeset is tied to the exact `main` merge commit and generates Verification Evidence for Release Confidence. --- ## Integrations (compact) | Surface | Start here | |---------|------------| | **Open ThumbGate GPT** | [thumbgate.ai/go/gpt](https://thumbgate.ai/go/gpt?utm_source=github&utm_medium=readme&utm_campaign=readme_gpt) โ€” **ThumbGate GPT: start here.** Paste agent actions, get advice + checkpointing. **No, users do not have to keep chatting inside the ThumbGate GPT to use ThumbGate** โ€” the **hard enforcement layer still runs where the work happens**. | | **Install Codex Plugin** | Open the Codex plugin install page: [thumbgate.ai/codex-plugin](https://thumbgate.ai/codex-plugin) ยท zip: [thumbgate-codex-plugin.zip](https://github.com/IgorGanapolsky/ThumbGate/releases/latest/download/thumbgate-codex-plugin.zip) ยท [plugins/codex-profile/INSTALL.md](plugins/codex-profile/INSTALL.md) | | Claude Desktop `.mcpb` | [latest release](https://github.com/IgorGanapolsky/ThumbGate/releases/latest/download/thumbgate-claude-desktop.mcpb) | | **VS Code / Open VSX** | [plugins/vscode-extension/README.md](plugins/vscode-extension/README.md) | | **Antigravity-compatible** | [plugins/antigravity-extension/INSTALL.md](plugins/antigravity-extension/INSTALL.md) | | **JetBrains** | [plugins/jetbrains-plugin/README.md](plugins/jetbrains-plugin/README.md) ยท JetBrains Marketplace path for the same runtime | | ChatGPT App / GPT Action | [thumbgate.ai/chatgpt-app](https://thumbgate.ai/chatgpt-app) | | ThumbGate-Core (staging) | [https://github.com/IgorGanapolsky/ThumbGate-Core](https://github.com/IgorGanapolsky/ThumbGate-Core) โ€” pre-release staging + a few internal cache scripts; **not** the product moat | ## Docs Full index: **[docs/INDEX.md](docs/INDEX.md)** | Need | Link | |------|------| | Agent workflow contract | [WORKFLOW.md](WORKFLOW.md) | | Ready-for-agent intake | [.github/ISSUE_TEMPLATE/ready-for-agent.yml](.github/ISSUE_TEMPLATE/ready-for-agent.yml) | | Verification Evidence | [docs/VERIFICATION_EVIDENCE.md](docs/VERIFICATION_EVIDENCE.md) | | Release Confidence | [docs/RELEASE_CONFIDENCE.md](docs/RELEASE_CONFIDENCE.md) | | Changeset strategy | [docs/CHANGESET_STRATEGY.md](docs/CHANGESET_STRATEGY.md) | | First Dollar Playbook | [docs/FIRST_DOLLAR_PLAYBOOK.md](docs/FIRST_DOLLAR_PLAYBOOK.md) | | Security policy | [SECURITY.md](SECURITY.md) | | Threat model | [THREAT_MODEL.md](THREAT_MODEL.md) | | Federal / regulated | [docs/FEDERAL.md](docs/FEDERAL.md) | | Commercial Truth | [docs/COMMERCIAL_TRUTH.md](docs/COMMERCIAL_TRUTH.md) | | Issues / PRs | [GitHub Issues](https://github.com/IgorGanapolsky/ThumbGate/issues) ยท [PR template](.github/pull_request_template.md) | **FAQ (one-liners):** Not a fine-tuner (runtime intercept only). Different from `CLAUDE.md` / `.cursorrules` (those are context; ThumbGate is an external allow/warn/deny before tools run). --- ## Who builds this **Igor Ganapolsky** โ€” payments (Stripe/Connect), AI agent guardrails/MCP, Android + backends. Small number of contract slots: **$120โ€“150/hr, 1099, remote US**. [LinkedIn](https://www.linkedin.com/in/igor-ganapolsky-859317343/) ยท [thumbgate.ai](https://thumbgate.ai) ## License MIT โ€” see [LICENSE](LICENSE). Project policy: [SECURITY.md](SECURITY.md) ยท [THREAT_MODEL.md](THREAT_MODEL.md).