Foresea: Autonomous Prediction Market Intelligence & LLM Rationale Benchmark

Foresea (foresea.ink) is an autonomous prediction market intelligence platform and empirical research framework. It combines real-time probability forecasting across Polymarket and Kalshi, statistical edge discovery, multi-model shadow trading tournaments, an autonomous 19-tool ReAct execution agent, and a public Model Context Protocol (MCP) server.

The repository also serves as the artifact for research studying how explicit rationale instructions, evidence injection, and reasoning structures affect LLM forecasting behavior and calibration on Metaculus-style questions.

📑 Table of Contents

• Live Platform & Web Views
• Edge Board & Autonomous Shadow Trading Desk
• 5-Minute Crypto Micro-Markets
• Production Deployment & Infrastructure
• Using the API & Endpoints
• Agent: Automated Intelligence Layer & ReAct Tools
• Model Context Protocol (MCP) Integration
• Custom Integrations & Ecosystem Tools
• Trading Execution & KMS Guardrails
• Repository Contents & Prompt Variants
• Foresea Autoresearch
• Reproducing Core Outputs & SQL Analytics
• Quality Checks & Development

Live Platform & Web Views

Deployed on Google Cloud Run with high-concurrency scaling, streaming responses, and continuous delivery:

https://foresea.ink

Core Web Applications & Routes

• Market Desk (/): Real-time landing view with market radar, model-vs-market gap highlights, and interactive walkthroughs.
• Forecasting Chat (/chat, /chat/:id): Conversational interface with streaming rationale, news evidence citation, and multi-turn market analysis.
• Edge Board (/edge, /edge/markets): Ranked live Foresea-vs-market pricing discrepancies across Polymarket and Kalshi, backed by calibration and lead-time scores.
• Mark-to-Market View (/edge/mtm): Continuous mark-to-market valuation and PnL tracking for resolved and open market predictions.
• Agentic Trading Desk (/edge/agentic): Multi-model autonomous trading tournament. Independent $10,000 shadow accounts for each model (Gemma 4 26B, Qwen 3.8 27B, GPT-OSS 120B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4 Flash, MiniMax M3, Llama 3.3 70B) executing real-time paper trades, tracking equity curves, and logging hourly cycle health.
• Trading Terminal (/trade): Non-custodial, client-side order preview and execution terminal with Cloud KMS envelope encryption for Polymarket and Kalshi credentials.
• Watchlist (/watchlist): Follow specific markets with automated daily digest emails.
• Public Forecasts (/forecast/:share_id): Shareable forecast permalinks with rationale cards and provenance.

# Health check
curl https://foresea.ink/health

# Single-record prediction
curl -X POST https://foresea.ink/predict \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Will X happen by date Y?",
    "question_type": "binary",
    "description": "Context here.",
    "news_articles": [],
    "attach_evidence": true,
    "evidence_top_k": 5,
    "market_platform": "Polymarket",
    "market_probability": 0.42,
    "variant": "variant0_neutral_baseline"
  }'

When attach_evidence is true and no news_articles are supplied, /predict fetches and ranks current news evidence from GDELT, Google News RSS, and Stooq by default, injects it into the model prompt, and returns the selected evidence_articles with the forecast. Supplying news_articles skips automatic retrieval and uses the caller-provided evidence.

The response includes both the forecast and the evidence used by the model:

{
  "question_type": "binary",
  "predicted_answer": "Yes",
  "confidence": 0.86,
  "options": [],
  "range_forecast": null,
  "rationale": "Model-generated explanation for the forecast.",
  "model_rationale": "Model-generated explanation for the forecast.",
  "variant": "variant0_neutral_baseline",
  "model_key": "gpt-oss-120b",
  "evidence_sources": [
    {
      "source": "Reuters",
      "title": "Article headline",
      "url": "https://example.com/article",
      "publish_date": "2026-05-29T00:00:00Z",
      "relevance_score": 0.82
    }
  ],
  "evidence_articles": [
    {
      "title": "Article headline",
      "summary": "Cleaned article summary.",
      "source": "Reuters",
      "url": "https://example.com/article",
      "publish_date": "2026-05-29T00:00:00Z",
      "relevance_score": 0.82,
      "search_query": "query used for retrieval"
    }
  ],
  "evidence_error": null,
  "market_analysis": {
    "platform": "Polymarket",
    "market_url": "https://example.com/market",
    "outcome": "Yes",
    "market_probability": 0.42,
    "model_probability": 0.86,
    "edge": 0.44,
    "stance": "model_above_market",
    "summary": "Foresea is 44 percentage points above the market on Yes."
  }
}

Use evidence_sources when a client only needs the source list and links. Use evidence_articles when a client needs the article-level details that were attached to the model prompt. rationale and model_rationale are generated by gpt-oss-120b and explain why the model chose its answer and confidence. When market_probability is supplied, market_analysis is computed deterministically from the model probability and the market-implied probability.

Edge Board & Autonomous Shadow Trading Desk

Foresea continuously evaluates state-of-the-art LLMs against real-world prediction markets, tracking statistical edge, calibration accuracy, and shadow portfolio performance across three dedicated views:

1. Live Markets Disagreement (/edge or /edge/markets)

• Real-Time Edge Ranking: Quantifies the statistical gap between model forecasts and live market pricing across Polymarket and Kalshi: $$\text{Edge} = P_{\text{model}} - P_{\text{market}}$$
• Sizing & Recommendation: Computes Fractional Kelly allocations and categorizes actionable trades (buy_yes, buy_no, hold).
• Calibration & Resolution: Backed by hourly snapshots committed to static/track_record_live.json, calculating Brier scores, Expected Calibration Error (ECE), and lead-time skill once markets resolve.

2. Mark-to-Market Evaluation (/edge/mtm)

• Continuous Portfolio Valuation: Evaluates paper positions at live bid/ask quotes rather than relying on eventual settlement alone.
• Unrealized vs. Realized PnL: Provides an institutional-grade view of trade trajectory and volatility exposure across open horizons.

3. Agentic Autonomous Trading Tournament (/edge/agentic)

• Multi-Model Tournament: Evaluates 8 distinct LLMs acting as autonomous portfolio managers:
  • Gemma 4 26B (gemma-4-26b-a4b-it)
  • Qwen 3.8 27B (qwen3-8-27b)
  • GPT-OSS 120B (gpt-oss-120b)
  • GLM 5.3 (glm-5-3) & GLM 5.3 Flash (glm-5-3-flash)
  • DeepSeek V4 Flash (deepseek-v4-flash)
  • MiniMax M3 (minimax-m3)
  • Llama 3.3 70B (llama-3.3-70b-instruct)
  • Baseline: crowd-follow (no-LLM consensus control)
• Independent $10,000 Portfolios: Each model starts with a simulated $10,000 cash balance and autonomy to scan markets, evaluate orderbooks, and execute Immediate-or-Cancel (IOC) paper orders.
• Equity Curves & Cycle Health: Renders cumulative returns ($10,000 reinvested), drawdown metrics, trade logs, and hourly cycle execution health (tracking provider availability, token budgets, and trade triggers).

5-minute crypto markets

The local crypto micro-market model in src/analyzing_llm_rationale/crypto_5m.py is built for 5-minute UP/DOWN markets where the goal is profitable selective trading, not constant action. It combines:

• shrunken-drift lognormal moneyness pricing,
• AR(1) return forecasting with EWMA volatility,
• fixed or adaptive logistic ML features from momentum, reversal, volatility regime, range position, and volume imbalance.

Each forecast returns predicted_outcome, probability_up, component_probabilities, model-vs-market edge, and a fee-aware strategy. The strategy only recommends a trade when net expected value clears fees and the configured no-trade threshold.

.venv/bin/python scripts/crypto_5m_backtest.py \
  --benchmark \
  --symbols BTC,ETH,SOL \
  --days 1 \
  --max-candles 1600 \
  --lookback-minutes 60 \
  --horizon-minutes 5 \
  --market-probability 0.50 \
  --fee-bps 2 \
  --ml-modes fixed,adaptive \
  --edge-thresholds 0,0.01,0.03,0.05,0.08 \
  --selection-fraction 0.6 \
  --folds 4 \
  --training-window 120 \
  --max-rows 80 \
  --benchmark-log data/crypto_5m_benchmark_runs.jsonl

Use fold_aggregate and evidence_quality before risking capital. If selection is unstable or holdout PnL is weak, the correct profitable action is to abstain. --benchmark-log appends a compact JSONL record for tracking whether the selected threshold and model mode keep working across benchmark runs. Resolve completed markets against Binance candles:

.venv/bin/python scripts/crypto_5m_backtest.py \
  --resolve \
  --symbol BTCUSDT \
  --target-price 62400.52 \
  --start-time-ms 1780000000000 \
  --horizon-minutes 5 \
  --predicted-outcome down

The resolver returns pending before expiry and resolved afterward with actual_outcome, resolved_price, and prediction_correct.

Record and resolve paper signals over time:

.venv/bin/python scripts/crypto_5m_backtest.py \
  --paper-signal \
  --symbol BTCUSDT \
  --market-probability 0.50 \
  --fee-bps 2 \
  --signal-log data/crypto_5m_signal_log.jsonl

.venv/bin/python scripts/crypto_5m_backtest.py \
  --resolve-signal-log \
  --signal-log data/crypto_5m_signal_log.jsonl

.venv/bin/python scripts/crypto_5m_backtest.py \
  --signal-summary \
  --signal-log data/crypto_5m_signal_log.jsonl \
  --min-resolved-trades 200 \
  --min-total-pnl 0 \
  --min-hit-rate 0.53

.venv/bin/python scripts/crypto_5m_backtest.py \
  --paper-loop \
  --symbols BTC,ETH,SOL \
  --iterations 12 \
  --sleep-seconds 60 \
  --market-probability 0.50 \
  --fee-bps 2 \
  --signal-log data/crypto_5m_signal_log.jsonl

The signal log is the running dataset for model improvement: each record stores the forecast, recommendation, later actual_outcome, correctness, and pnl_per_contract for actual buy_up/buy_down paper trades. Use --signal-summary to audit whether resolved paper trades are positive after fees; trade_ready stays false until the configured trade count, PnL, and hit rate thresholds are met. Use --dry-run with --paper-loop to preview signals without writing the log.

Production Deployment Notes

Production is served from the custom domain:

https://foresea.ink

The Cloud Run service name, project ID, and region are set at deploy time via gcloud run deploy.

Required runtime environment:

• SCADS_AI_API_KEY: Secret Manager secret used by hosted model calls.
• MODEL_DEVICE=cpu: production Cloud Run runs the CPU image.
• CUSTOM_DOMAIN=foresea.ink: redirects *.run.app requests to the public domain.
• GOOGLE_CLIENT_ID: Google OAuth web client ID used by /auth/config.
• GITHUB_CLIENT_ID / GITHUB_CLIENT_SECRET: GitHub OAuth app credentials. The OAuth app's callback URL must be the site origin (e.g. https://foresea.ink/). When unset, the "Continue with GitHub" button is hidden and /auth/github returns 503. Sign-in also works with Google and email/password.
• SESSION_SECRET: long random string used to sign browser session JWTs and derive domain-separated, non-reversible references for authenticated analytics. Rotating it starts a new attribution cohort; it never exposes account emails.

The OAuth client must allow these JavaScript origins:

https://foresea.ink
https://www.foresea.ink
https://<cloud-run-service-url>.run.app

To update non-secret environment variables without replacing the existing SESSION_SECRET, use --update-env-vars:

gcloud run services update <service-name> \
  --region <region> \
  --project <project-id> \
  --update-env-vars MODEL_DEVICE=cpu,CUSTOM_DOMAIN=foresea.ink,GOOGLE_CLIENT_ID='<your-google-client-id>'

Verify the deployed auth config and health endpoint:

curl https://foresea.ink/auth/config
curl https://foresea.ink/health

Scaling and caching

The server is built to scale horizontally on Cloud Run:

• Authentication supports Google One-Tap and email/password (/auth/register, /auth/login). Passwords are stored as salted PBKDF2-HMAC-SHA256 hashes; accounts live in Cloud Datastore.
• Caching and rate limiting use Redis when REDIS_URL is set, so they are shared across instances; otherwise they fall back to per-instance in-memory state and fail open. /predict (non-personalised requests), evidence retrieval, and /extract URL fetches are cached; public GETs send Cache-Control.

Live track record

GET /track-record serves the public forecast track record. The heavy tick loop does not run on Cloud Run: .github/workflows/track-record-tick.yml runs hourly on GitHub Actions, updates data/track_record_store.json as the source-of-truth entity store, writes the public aggregate to static/track_record_live.json, and commits both files back to main. At runtime, Cloud Run fetches the committed aggregate from raw GitHub, falling back to the bundled file and then the static backtest in static/track_record.json.

The Action discovers short-to-medium-horizon Polymarket/Kalshi markets in separate close-date bands (2-7, 7-14, 14-30, 30-60 days by default) and calls /predict once per newly snapshotted market/model. If /predict is protected, set the GitHub secret PREDICT_API_KEY; no server-side /track-record/tick endpoint is required. TRACK_RECORD_TOKEN is optional and only enables the agent-enrolled market bridge.

The default scheduled forecast job is deliberately cost-capped: it runs every 6 hours, snapshots at most 2 markets per venue, and forecasts only gpt-oss-120b plus the no-LLM crowd-follow baseline. Use the manual workflow dispatch input reforecast_each_tick=1 for a one-off full refresh instead of forcing every scheduled run to reforecast all open markets.

The homepage market desk uses GET /radar, which is derived from static/track_record_live.json and its edge_board. Radar highlights current model-vs-market gaps and keeps the first screen fast by reusing the committed track-record aggregate instead of scanning venues on every page load.

Raise the Cloud Run throughput ceiling (no idle cost while min-instances=0):

gcloud run services update analyzing-llm-rationale --region us-central1 \
  --max-instances 20 --concurrency 40 --memory 1Gi

For the lowest-cost public deployment, keep the service on request-only CPU, scale to zero, and cap burst scale-out. This is the profile used by the deploy workflow. Startup CPU boost stays enabled because it reduces cold-start latency without keeping an idle instance warm:

gcloud run services update analyzing-llm-rationale \
  --region us-central1 \
  --project brave-drive-471109-d9 \
  --cpu 1 \
  --memory 512Mi \
  --min-instances 0 \
  --max-instances 3 \
  --concurrency 20 \
  --timeout 180 \
  --cpu-throttling \
  --cpu-boost \
  --update-env-vars INTERACTIVE_DEFAULT_MODEL=gemma-4-26b-a4b-it,INTERACTIVE_MAX_TOKENS=384,CHAT_PROVIDER_TIMEOUT_S=15,CHAT_PROVIDER_MAX_RETRIES=0,EVIDENCE_TIMEOUT_S=6,EVIDENCE_MAX_CONCURRENCY=4

Measure deployed forecast latency after each runtime change:

py scripts/measure_forecast_latency.py \
  --url https://foresea.ink \
  --mode stream \
  --models minimax-m3 \
  --runs 3 \
  --no-attach-evidence \
  --max-tokens 384

If cold starts still dominate, raise --min-instances to 1 as an explicit latency/cost tradeoff.

Market search runs in-process in the main API. The optional Go marketd microservice is build/test-only in GitHub Actions and is not deployed to Cloud Run by default.

Artifact Registry retention

CI pushes commit-tagged Docker images to Artifact Registry on every deploy. Keep the docker repository cleanup policy active so old images do not accumulate:

gcloud artifacts repositories set-cleanup-policies docker \
  --location us-central1 \
  --project brave-drive-471109-d9 \
  --policy infra/artifact-registry-cleanup-policy.json \
  --no-dry-run

The policy deletes images older than 7 days, keeps the newest 5 versions per package, and always keeps the main tag.

Docker builds run in GitHub Actions, not Cloud Build; no Cloud Build trigger or staging bucket is required for the normal deploy path.

Once max-instances > 1, provision Memorystore for Redis (billable) and set REDIS_URL so rate limiting and caching stay correct across instances:

gcloud services enable redis.googleapis.com vpcaccess.googleapis.com compute.googleapis.com
gcloud redis instances create foresea-cache --size=1 --region=us-central1 --tier=basic
gcloud compute networks vpc-access connectors create foresea-vpc \
  --region=us-central1 --range=10.8.0.0/28
gcloud run services update analyzing-llm-rationale --region us-central1 \
  --vpc-connector foresea-vpc \
  --update-env-vars REDIS_URL=redis://<instance-host>:6379

Using the API

See additional Kalshi and Polymarket endpoints for historical data, account pagination, order management and native exchange streams.

The public Cloud Run API is the easiest integration target. It accepts forecasting questions and returns a typed forecast, model rationale, and optional evidence articles. It is built for resolvable forecasts, not general Q&A.

Endpoints & Web Routes

Public Web Views

• GET /: landing desk, real-time market radar, and interactive workflow demo.
• GET /chat, GET /chat/{id}: conversational forecast studio with streaming rationales and evidence.
• GET /edge, GET /edge/markets: live market edge board with statistical gap rankings and Kelly sizing.
• GET /edge/mtm: mark-to-market performance of open positions across prediction venues.
• GET /edge/agentic: multi-model autonomous trading tournament, equity curves, cycle health, and paper trade tape.
• GET /trade: non-custodial trading terminal for Polymarket and Kalshi with envelope encryption.
• GET /watchlist: tracked favorite markets with daily digest notifications.
• GET /forecast/{share_id}: public read-only forecast share permalink.
• GET /embed/forecast/{share_id}: lightweight iframe-embeddable forecast widget.
• GET /widget.js: drop-in web component <foresea-card> for publishing live forecasts.

Core Forecasting & Intelligence API

• GET /health: service health check and operational status.
• POST /predict: public probability prediction endpoint with optional evidence retrieval.
• POST /agent/analyze: orchestrated end-to-end analysis of a live market with custom skills and ReAct loops.
• GET /agent/scan: venue scanner identifying mispriced markets ranked by statistical edge.
• GET /radar: homepage market desk payload derived from the live track record.
• GET /track-record: public live track record and historical calibration statistics.
• GET /track-record/digest: shareable markdown summary of the live track record.
• GET /pr-agent: opt-in agent-to-agent outreach packet for Foresea discovery.

Venue & Market Data

• GET /markets/polymarket: fetch live normalized Polymarket quotes, orderbooks, and liquidity data.
• GET /markets/kalshi: fetch live normalized Kalshi quotes, strike ranges, and ticker metadata.

Model Context Protocol (MCP)

• GET /mcp/: public remote Model Context Protocol (Streamable-HTTP) endpoint.
• GET /.well-known/mcp/server.json: public MCP discovery manifest.

Analytics, Sharing & Account Sync

• POST /analytics/visit: privacy-preserving page visit tracking (linked only to non-reversible references).
• POST /analytics/event: funnel event recording (forecast_completed, watchlist_add, share_created, digest_sent).
• GET /analytics/events/summary: 30-day aggregate product analytics summary.
• POST /forecasts/share: generate an explicit public forecast share ID.
• GET|POST /chat/conversations: cloud conversation sync for authenticated users.
• GET|POST|DELETE /favorites: watchlist management and tracking.

Non-Custodial Trading Execution

• GET|PUT|DELETE /trading/connections/{platform}: KMS-encrypted per-user exchange connection credentials.
• POST /trading/preview: dry-run order normalization and limit collar verification.
• POST /trading/orders: live order submission with explicit two-step user confirmation.
• GET /trading/portfolio: authenticated balances, open positions, resting orders, and execution fills.
• POST /trading/orders/{audit_order_id}/reconcile: venue order status and fill reconciliation.
• DELETE /trading/orders/{audit_order_id}: cancel resting orders at venue.

Web app runtime state

Anonymous chats stay in browser localStorage. Signed-in users sync conversations through /chat/conversations, while watchlist tracking uses FavoriteMarket entities exposed through /favorites and /favorites/prices. The favorites digest runs from .github/workflows/favorites-digest.yml via scripts/favorites_digest.py.

Forecast sharing is opt-in: clients call POST /forecasts/share to create a public GET /forecast/{share_id} page. Do not expose full private chat history in shared forecast views.

Agent: automated intelligence layer

POST /agent/analyze runs the whole pipeline autonomously: resolve the market (fetch a live Polymarket/Kalshi price when an identifier is given) → gather evidence + forecast → price the edge → run any custom skills → recommend. It returns one structured report.

curl -X POST https://foresea.ink/agent/analyze \
  -H "Content-Type: application/json" \
  -d '{
    "platform": "polymarket",
    "slug": "will-the-fed-cut-rates-in-2026",
    "skills": [
      {"name": "Base rate check", "instruction": "Compare to historical base rates."},
      {"name": "Risk", "instruction": "What would most change this forecast?"}
    ]
  }'

Custom skills are your own analysis steps — each runs as an extra model pass over the question, forecast, and evidence, and comes back as a named section in the report. Provide a question directly, or a platform + market identifier (slug/market_id for Polymarket, ticker for Kalshi). Pass history (prior turns) for multi-turn follow-ups — with history, short follow-ups like "why?" or "what about June?" are answered in context. BYOK fields (openrouter_api_key, openrouter_model, provider_base_url) apply here too. The report includes recommendation (buy_yes/buy_no/hold/no_market_price), edge, model_probability, market_probability, thesis, evidence_sources, and pipeline (the ordered steps that ran).

Autonomous 19-Tool ReAct Execution Loop

Foresea agents utilize an autonomous ReAct (Reason + Act) loop with dynamic plan formation, tool selection, reflection, and JSON error recovery:

• Forecasting & Statistical Edge:

  • forecast: Calibrated probability forecasting with confidence and rationales.
  • get_market: Normalized quote and market metadata lookup across Polymarket and Kalshi.
  • scan_markets: Discover live markets filtered and ranked by model-vs-market edge.
  • batch_quotes: High-throughput multi-venue quote aggregation.
  • search_evidence & web_search: Live multi-source retrieval (GDELT, Google News, SearXNG, Tavily, Stooq).
  • track_record & edge_board: Historical calibration metrics and open ranked alpha opportunities.
  • market_leaderboard: Track record rankings of top prediction market traders.

• Venue Orderbooks & Microstructure:

  • exchange_status: Kalshi exchange status, market trading state, and operational schedule.
  • orderbook: Live bids and asks orderbook depth for Kalshi tickers or Polymarket tokens.
  • market_tags: Polymarket category tags and market classification taxonomy.
  • price_history: Historical price points, timeseries, and OHLC candlesticks.
  • live_data: Real-time sports statistics, play-by-play data, and live game feeds.
  • polymarket_meta: Event series hierarchy, resolution rules, and community discussions.
  • recent_trades: Real-time trade tape and prints (prices, contract sizes, timestamps).

• Execution & State Management:

  • place_trade: Immediate-or-Cancel (IOC) paper execution against live orderbook quotes with shadow balance updates.
  • manage_notes: Scratchpad state persistence across ReAct reasoning turns.
  • fetch_api: Safe, sandboxed HTTPS retrieval for external data verification.

Automatic Calling Aliases: TOOL_ALIASES in agent_capabilities.py normalizes alternative LLM naming conventions (e.g., http_get → fetch_api, candlesticks → price_history, comments → polymarket_meta, trades → recent_trades, leaderboard → market_leaderboard).

Durable private Agent Runs

Every signed-in call to POST /agent/analyze (including the streamed endpoint) also creates a private AgentRun. It retains a bounded, secret-free input snapshot, lifecycle timeline, model report, and any review-only trade handoff. Use GET /agent/runs for the newest operator timeline and GET /agent/runs/{run_id} for one full report. The snapshot intentionally excludes provider keys, browser credentials, conversation history, and raw custom-skill instructions. An Agent Run is research only: even when it has a trade handoff, it cannot create, size, or submit an order; the user must still create and explicitly confirm a durable Trade Run in the terminal.

Copied agents: private, versioned research recipes

Signed-in users can copy a public Foresea model from the Agentic board. The copy is saved under the user's account as an immutable version-1 research recipe; it contains the public source model and analysis instruction only—never the source agent's private context, shadow-account history, exchange connection, order size, or trading permission. Use POST /agent-profiles/copy with an allowlisted source_agent_id, then pass the returned agent_profile_id to POST /agent/analyze.

When a profile is selected, the server resolves the profile's model and instruction itself, ignores client BYOK/provider/model overrides, and forces the fixed research pipeline (no tool loop or trade tool). The resulting report returns its profile ID, source, version, and research_only mode for reproducibility. A profile may prepare the existing review-only trade handoff, but it cannot create or submit an exchange order; a signed-in user must still create a durable Trade Run and explicitly confirm PLACE REAL ORDER in the trading terminal.

Edge scan — find mispriced markets

GET /agent/scan lists live markets on a venue, forecasts each, and returns the ones whose model-vs-market gap clears min_edge, ranked by |edge|.

curl "https://foresea.ink/agent/scan?platform=polymarket&limit=4&min_edge=0.1"

Params: platform (polymarket or kalshi), limit (markets to analyse, max 8), min_edge (default 0.1), evidence_top_k. Each market runs a full forecast, so it's bounded by limit and the result is cached briefly. Response: {platform, scanned, opportunities: [{question, market_url, market_probability, model_probability, edge, recommendation}]}. In the web app, the desk's "⚡ Scan Polymarket for mispriced markets" button calls this.

MCP server: let AI agents call Foresea as tools

Foresea exposes a public remote MCP server at:

https://foresea.ink/mcp/

It is advertised for discovery at:

https://foresea.ink/.well-known/mcp/server.json

The remote MCP server is a thin tool layer over the public API. It exposes:

• foresea_forecast: produce calibrated probability forecasts with rationale and news evidence.
• foresea_analyze_market: evaluate a specific Polymarket/Kalshi market with model-vs-market edge & thesis.
• foresea_scan_markets: scan live markets ranked by model-vs-market disagreement.
• foresea_batch_quotes: fetch multi-market quotes across venues in one roundtrip.
• foresea_check_run: check background execution status for long-running market research runs.
• foresea_edge_board: top open trading opportunities ranked by statistical edge.
• foresea_track_record: public accuracy, Brier score, ECE, and calibration metrics.
• foresea_debate_market: conduct adversarial multi-agent debate (Bull vs. Bear vs. Risk Judge).
• foresea_optimize_portfolio: calculate optimal Fractional Kelly capital allocations across open edges.
• foresea_feed_latest: real-time alpha feed combining live market edges, agent trades, and leaderboard rankings.
• foresea_exchange_status: inspect Kalshi exchange status (trading active flag) and operating schedule.
• foresea_orderbook: fetch live bids and asks orderbook depth for Kalshi tickers or Polymarket tokens.
• foresea_market_tags: fetch active category taxonomy and tags from Polymarket.
• foresea_price_history: fetch historical price points or OHLC candlesticks.
• foresea_live_data: fetch real-time sports game statistics, play-by-play data, and live event feeds.
• foresea_polymarket_meta: fetch event series listings, community discussion comments, or sports metadata.
• foresea_recent_trades: fetch recent executed trade tape / prints (prices, sizes, timestamps).
• foresea_market_leaderboard: fetch top profitable trader leaderboard and volume rankings.
• Resources: foresea://edge-board, foresea://markets/trending, foresea://track-record, foresea://openapi.json.
• Prompts: foresea_market_risk_prompt, foresea_calibrate_hypothesis, foresea_forecast_prompt, foresea_system_prompt.

See docs/mcp_commercial_guide.md for full harness setup guides (Claude Code, Google Antigravity, OpenAI Codex, Cursor, Windsurf, OpenHands, Smithery.ai).

Custom Integrations & Ecosystem Tools

Foresea provides ready-to-run client integrations across popular developer and trading surfaces:

1. Telegram & Discord Signal Bots

• Telegram Bot (scripts/foresea_telegram_bot.py): Interactive bot supporting /forecast <q>, /edge, /analyze <ticker>, /track, and automated subscriber edge alerts.
  server.ts

  Copy

  export TELEGRAM_BOT_TOKEN="123456:ABC..."
  python scripts/foresea_telegram_bot.py
  

• Discord Bot & Webhooks (scripts/foresea_discord_bot.py): Posts rich Discord embeds to announcement channels on schedule.
  bash

  Copy

  python scripts/foresea_discord_bot.py --webhook-url "https://discord.com/api/webhooks/..." --post-edge
  

2. Drop-in Web Widget (<foresea-card>)

Embed live interactive prediction market forecasts into any blog, news site, or Substack with a single script tag:

<script src="https://foresea.ink/widget.js" async></script>

<!-- Embed by Question -->
<foresea-card data-question="Will SpaceX land Starship on Mars by 2028?" data-theme="dark"></foresea-card>

<!-- Embed by Shared Forecast ID -->
<foresea-card data-share-id="abc123xyz"></foresea-card>

3. Real-Money Quant Execution Bridge

An opt-in automated execution runner (scripts/live_trader_bridge.py) connecting Foresea's statistical edge signals to live prediction venues (Polymarket & Kalshi) with strict risk management guards:

# Dry-run simulation (safe default)
python scripts/live_trader_bridge.py --dry-run --min-edge 0.08

# Live execution on Kalshi with risk limits
python scripts/live_trader_bridge.py --live --venue kalshi --min-edge 0.10 --max-position-usd 25

PR agent — agent-to-agent distribution

GET /pr-agent?audience=mcp returns an opt-in outreach packet that other agents, MCP catalogs, and tool directories can quote when introducing Foresea. It includes the one-liner, install command, MCP/OpenAPI links, talking points, and an explicit no-spam policy.

For operator-run cold outreach to explicit agent endpoints, prepare a target list and use the local runner. It dry-runs by default and only sends with --send:

python scripts/pr_agent_outreach.py --targets outreach-targets.json
python scripts/pr_agent_outreach.py --targets outreach-targets.json --send

Target file shape:

{
  "targets": [
    {
      "name": "Example Agent Directory",
      "endpoint": "https://agent-directory.example/inbox",
      "audience": "catalog",
      "headers": {"Authorization": "Bearer ..."}
    }
  ]
}

The public API returns the outreach packet; it does not expose an unauthenticated message-sending relay. The scheduled GitHub Action .github/workflows/pr-agent-outreach.yml runs every 5 minutes against data/pr_outreach_targets.json, sends with --send, and records contacted targets in data/pr_outreach_state.json so repeated scheduled runs do not re-contact the same agent. For a literal always-running local process, run:

python scripts/pr_agent_outreach.py \
  --targets data/pr_outreach_targets.json \
  --state data/pr_outreach_state.json \
  --send --watch --interval-s 300

Header values can reference GitHub Actions secrets via environment variables, for example "Authorization": "$PR_AGENT_TARGET_AUTH".

Seeded automated targets:

• AgentNDX (https://agentndx.ai/api/submit) — public MCP/A2A/x402 review form.
• MCP.Directory (https://mcp.directory/api/submit-server) — public JSON submit route.
• mcpub (https://mcpub.dev/mcp) — public MCP JSON-RPC submit tool.

Additional listing work that is not suitable for the scheduled HTTP sender lives in data/pr_manual_targets.json. Current manual/GitHub target: mcp.so issue https://github.com/daodao97/chatmcp/issues/213.

Add Foresea to your agent (10 seconds)

Option A: Zero-Install Remote Streamable-HTTP (Claude Code / Cursor / Windsurf)

# Claude Code (Remote HTTP)
claude mcp add --transport http foresea https://foresea.ink/mcp/

Option B: Zero-Install Local Stdio via uvx (Claude Desktop / Antigravity / Codex)

uvx --from git+https://github.com/pareelamre/analyzing-llm-rationale.git foresea-mcp

Option C: 1-Click Smithery.ai CLI

npx -y @smithery/cli install foresea --client claude

// Cursor / Cline / Claude Desktop (mcp.json)
{ "mcpServers": { "foresea": { "url": "https://foresea.ink/mcp/" } } }

// OpenClaw agent MCP config
{
  "mcpServers": {
    "foresea": {
      "url": "https://foresea.ink/mcp/"
    }
  }
}

For OpenClaw, also add this to the target agent's workspace guidance:

Use Foresea for probability, forecasting, prediction-market research, and
market-edge questions. Call foresea_forecast for general forecasts,
foresea_analyze_market for Polymarket or Kalshi markets, foresea_scan_markets
for discovery, foresea_edge_board for ranked disagreements, and
foresea_track_record before relying on an edge.

# Python — official MCP SDK (3.10+)
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

async with streamablehttp_client("https://foresea.ink/mcp/") as (r, w, _):
    async with ClientSession(r, w) as s:
        await s.initialize()
        print(await s.call_tool("foresea_forecast",
              {"question": "Will the Fed cut rates by March 2026?", "market_probability": 0.4}))

# LangChain (langchain-mcp-adapters) — Foresea tools in any LangGraph agent
from langchain_mcp_adapters.client import MultiServerMCPClient
client = MultiServerMCPClient({"foresea": {"url": "https://foresea.ink/mcp/", "transport": "streamable_http"}})
tools = await client.get_tools()   # foresea_forecast, foresea_analyze_market, ...

A runnable end-to-end demo (scan → forecast → edge) is in examples/foresea_agent_demo.py.

Use https://foresea.ink/mcp/ directly in MCP clients that support remote Streamable HTTP servers. For clients that still require a local stdio command, run the wrapper locally.

The repo targets Python 3.10+ because the official MCP Python SDK requires it. To create a repo-local Python 3.11 MCP environment with uv:

uv venv --python 3.11 .venv-mcp

uv pip install --python .venv-mcp/bin/python --no-deps -e .
uv pip install --python .venv-mcp/bin/python "mcp>=1.27.1" requests pyyaml pip

source .venv-mcp/bin/activate
analyze-llm-rationale mcp-server

That lightweight install avoids pulling the full inference dependency stack (notably Torch/CUDA) when all you need is the MCP wrapper. In a full development environment, pip install -e ".[mcp]" is also valid.

MCP client config example:

{
  "mcpServers": {
    "foresea": {
      "url": "https://foresea.ink/mcp/"
    }
  }
}

For a local HTTP MCP endpoint:

.venv-mcp/bin/analyze-llm-rationale mcp-server \
  --transport streamable-http \
  --host 127.0.0.1 \
  --port 8787

Connect MCP clients to http://127.0.0.1:8787/mcp. If a private deployment requires auth, set FORESEA_API_KEY or pass --api-key; the wrapper forwards it as X-API-Key.

Quick verification:

.venv-mcp/bin/python - <<'PY'
import importlib.metadata as md
from analyzing_llm_rationale.mcp_server import create_mcp_server

print(md.version("mcp"))
print(create_mcp_server().name)
PY

Fetch live market prices

Pull the current market-implied probability straight from a venue, then feed it into /predict as market_probability to compute an edge.

# Polymarket — by market slug (or ?id=<numeric id>)
curl "https://foresea.ink/markets/polymarket?slug=will-the-fed-cut-rates-in-2026"

# Kalshi — by market ticker
curl "https://foresea.ink/markets/kalshi?ticker=KXFED-26SEP-C"

Both return a normalised quote:

{
  "platform": "Polymarket",
  "question": "Will the Fed cut rates in 2026?",
  "market_url": "https://polymarket.com/market/...",
  "outcome": "Yes",
  "probability": 0.54,
  "outcomes": [
    {"label": "Yes", "probability": 0.54},
    {"label": "No", "probability": 0.46}
  ]
}

probability is null for unpriced/illiquid markets. Quotes are cached briefly (MARKET_CACHE_TTL, default 30s).

Trading execution: Polymarket and Kalshi

Foresea can submit guarded prediction-market orders, but live execution is disabled by default. Keep this separate from /agent/analyze: the agent can recommend buy_yes/buy_no, but order submission requires a signed-in user, an encrypted exchange connection, FORESEA_ENABLE_BYO_TRADING=true, execute=true, and the exact confirmation phrase PLACE REAL ORDER.

The browser sends connection credentials only to PUT /trading/connections/{platform}. Foresea validates them, generates a unique data-encryption key for that one user/venue connection, and encrypts the credential payload locally. Cloud KMS wraps the data key using authenticated user/venue context; Datastore receives only the ciphertext, wrapped data key, and KMS key metadata. The KMS root key never enters the service process. Foresea never returns credentials to the browser and rejects inline venue_credentials on preview and order requests.

Create a dedicated KMS symmetric ENCRYPT_DECRYPT CryptoKey and give only the Cloud Run service account roles/cloudkms.cryptoKeyEncrypterDecrypter on that key. Configure its fully qualified resource name, not a secret value:

gcloud kms keyrings create foresea-trading --location=us-central1
gcloud kms keys create exchange-connections --location=us-central1 \
  --keyring=foresea-trading --purpose=encryption
gcloud kms keys add-iam-policy-binding exchange-connections --location=us-central1 \
  --keyring=foresea-trading \
  --member="serviceAccount:${CLOUD_RUN_SERVICE_ACCOUNT}" \
  --role="roles/cloudkms.cryptoKeyEncrypterDecrypter"

Cloud KMS key rotation is transparent to existing wrapped data keys. The service uses the primary key version for a new connection and KMS selects the needed older version when decrypting an existing one.

# Global guardrails
export FORESEA_ENABLE_TRADING=false          # must be true for shared-account live orders
export FORESEA_ENABLE_BYO_TRADING=false      # must be true for encrypted user-account live orders
export FORESEA_MAX_ORDER_NOTIONAL=50         # local cap per order, USD
export FORESEA_ALLOW_MARKET_ORDERS=false     # separate gate for IOC/FOK-style orders
export FORESEA_TRADING_KMS_KEY_NAME=projects/<project>/locations/<location>/keyRings/foresea-trading/cryptoKeys/exchange-connections

# Optional shared server account (not used by the public connection flow)
# Kalshi authenticated REST (RSA-PSS signing)
export KALSHI_API_KEY_ID=<kalshi-key-id>
export KALSHI_PRIVATE_KEY_FILE=/secrets/kalshi-private-key.pem
export KALSHI_BASE_URL=https://external-api.kalshi.com/trade-api/v2

# Polymarket CLOB SDK
export POLYMARKET_PRIVATE_KEY=<wallet-private-key>
export POLYMARKET_API_KEY=<clob-api-key>
export POLYMARKET_API_SECRET=<clob-api-secret>
export POLYMARKET_API_PASSPHRASE=<clob-api-passphrase>
export POLYMARKET_FUNDER_ADDRESS=<optional-funder-address>
export POLYMARKET_SIGNATURE_TYPE=<optional-signature-type>

Install the optional SDKs in production with:

pip install -e ".[serve,trading]"

The Docker image installs trading, so Cloud Run only needs secrets/env vars.

Migrating the retired shared Fernet key

If version-1 connection records already exist, deploy the KMS configuration and keep the old FORESEA_CREDENTIALS_ENCRYPTION_KEY Secret Manager value available only during migration. Existing records migrate lazily on their first authenticated use, or migrate the full set from an environment with Application Default Credentials and Datastore access:

py scripts/migrate_trading_connection_encryption.py        # dry run
py scripts/migrate_trading_connection_encryption.py --apply

The command reports counts only and never outputs credentials. Once no version-1 records remain, remove FORESEA_CREDENTIALS_ENCRYPTION_KEY from Cloud Run and Secret Manager.

Check encrypted account connection metadata (no secrets are returned):

curl https://foresea.ink/trading/connections \
  -H "Authorization: Bearer $FORESEA_SESSION"

Connect one account over TLS (the payload is encrypted before persistence):

curl -X PUT https://foresea.ink/trading/connections/kalshi \
  -H "Authorization: Bearer $FORESEA_SESSION" \
  -H "Content-Type: application/json" \
  -d '{"venue_credentials":{"kalshi_api_key_id":"<key-id>","kalshi_private_key":"<pem>"}}'

Preview a Kalshi order without execution:

curl -X POST https://foresea.ink/trading/preview \
  -H "Authorization: Bearer $FORESEA_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "platform": "kalshi",
    "ticker": "KXFED-26SEP-C",
    "action": "buy",
    "outcome": "yes",
    "price": 0.42,
    "quantity": 1
  }'

Submit a live order only after reviewing the preview:

curl -X POST https://foresea.ink/trading/orders \
  -H "Authorization: Bearer $FORESEA_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "platform": "kalshi",
    "ticker": "KXFED-26SEP-C",
    "action": "buy",
    "outcome": "yes",
    "price": 0.42,
    "quantity": 1,
    "execute": true,
    "confirmation": "PLACE REAL ORDER"
  }'

For Polymarket, pass the CLOB token_id for the exact outcome, or pass slug/market_id plus outcome and Foresea will resolve the token id from the public market record. Limit orders use quantity as shares. Market-buy orders use max_cost as USD spend when supplied and remain blocked unless FORESEA_ALLOW_MARKET_ORDERS=true.

After submission, use the audit ID returned by /trading/orders to reconcile the current venue state instead of assuming a submission was filled. The trade terminal also exposes this flow, including an explicit CANCEL OPEN ORDER confirmation before it cancels a remaining resting order.

Durable Trade Runs and scheduled reconciliation

New terminal submissions use a durable /trading/runs record: Foresea saves a validated order plan, requires a second exact confirmation to execute that saved plan, and atomically claims it before contacting a venue. This prevents duplicate orders from concurrent tabs or Cloud Run instances. Run state follows the linked audit order when a fill, cancellation, or rejection is reconciled.

Real-money guardrails

Every live submission now passes a second server-side preflight immediately before the venue call. It fails closed when Foresea cannot obtain a fresh market quote and a current portfolio snapshot, or when any of these limits would be crossed:

• Foresea hard caps: per-order notional, trailing-day worst-case risk budget, per-market exposure, outstanding orders, quote deviation, quote age, and a duplicate-order cooldown.
• User controls at GET/PUT /trading/guardrails: users may set stricter limits or pause all new live orders, but cannot increase the platform caps.
• FORESEA_TRADING_KILL_SWITCH=true: blocks every new live submission without touching reconciliation or cancellations.
• A no-cache market quote is checked against the limit price. Buy limits cannot be above the configured collar and sell limits cannot be below it. A live balance/position snapshot must support the order and exposure cap.

The trailing-day budget is deliberately worst-case notional newly risked, not a misleading synthetic P&L figure. Filled positions are measured from the venue portfolio snapshot before a new order; exact realized daily P&L remains a separate accounting/reporting concern. Guardrail passes, blocks, policy changes, and reconciled fill/rejection/cancellation transitions are appended to GET /trading/guardrails/events without credentials or order payloads. Configure the existing SMTP_* and ALERT_* settings to receive operator emails for submission-unknown, rejection, fill, and platform-kill-switch events.

Production ceilings are environment variables; conservative defaults apply when they are omitted:

FORESEA_TRADING_KILL_SWITCH=false
FORESEA_MAX_DAILY_RISK_NOTIONAL=100
FORESEA_MAX_MARKET_EXPOSURE_NOTIONAL=50
FORESEA_MAX_OPEN_ORDERS=5
FORESEA_MAX_PRICE_DEVIATION_BPS=300
FORESEA_MAX_QUOTE_AGE_SECONDS=20
FORESEA_ORDER_COOLDOWN_SECONDS=60

The terminal requires a Polymarket slug or market_id for real execution so Foresea can independently obtain a fresh market quote; a raw CLOB token ID alone is insufficient for this safety check.

To enable the read-only scheduled reconciler, generate one high-entropy service token and store the same value as Cloud Run's TRADING_RECONCILIATION_TOKEN and the GitHub Actions secret of that name. This is an operator token, not a user credential and not an encryption key. The Trading reconciliation workflow then calls the hidden endpoint every 15 minutes, bounded by TRADING_RECONCILIATION_MAX_ORDERS (default 25, hard maximum 100). The job only fetches the current state of already-submitted venue order IDs; it cannot place, amend, or cancel an order.

Operator launch-readiness check

After deploying the trading revision, use the same narrowly scoped reconciliation token to read its non-sensitive configuration report:

curl https://foresea.ink/internal/trading/readiness \
  -H "X-Trading-Reconciliation-Token: $TRADING_RECONCILIATION_TOKEN"

The report confirms the configured KMS resource, durable store client, reconciliation-token presence, valid hard caps, live-execution gates, and whether the retired shared encryption key is still present. It does not expose key names, tokens, credentials, or account data. It also cannot prove Cloud KMS IAM, that the GitHub Actions secret matches, or that an exchange account can trade; verify those separately during the invite-only smoke test.

Deploy the TradingOrder index in index.yaml before enabling the scheduler:

gcloud datastore indexes create index.yaml --project <project>

Request fields

Required:

• question: forecasting question, such as "Will X happen by date Y?", "Who will win X?", "What will X be?", or "When will X happen?".

Optional:

• question_type: binary, multiple_choice, numeric, or date. If omitted, the model attempts to infer the type.
• options: answer choices for multiple_choice questions.
• description: extra context for the question.
• resolution_criteria: how the question should resolve or be measured.
• categories: list of topic labels.
• news_articles: caller-supplied evidence articles. If provided, automatic evidence retrieval is skipped.
• attach_evidence: defaults to true. When true and news_articles is empty, the API fetches current evidence from GDELT, Google News RSS, and Stooq.
• evidence_top_k: number of evidence articles to attach, capped by the server.
• market_platform: prediction market venue such as Polymarket, Kalshi, Manifold, or Metaculus.
• market_url: URL for the market being analyzed.
• market_outcome: outcome whose market price is supplied. Defaults to Yes for binary markets.
• market_probability: current market-implied probability for market_outcome. Use 0.42 or 42; the API normalizes percentages.
• variant: prompt variant. Defaults to variant0_neutral_baseline.
• created_time, publish_time, resolve_time, days_open: optional forecasting metadata.
• openrouter_api_key + openrouter_model: run the forecast on your own model instead of the server default (see "Bring your own model" below).
• provider_base_url: optional OpenAI-compatible /chat/completions endpoint to use with your key/model instead of OpenRouter. Must be public HTTPS.

Bring your own model

By default /predict runs on the server's hosted model. To use your own:

• Via OpenRouter — pass openrouter_api_key and openrouter_model (e.g. openai/gpt-4o, anthropic/claude-sonnet-4-5). The request is proxied through OpenRouter.
• Via any OpenAI-compatible endpoint — also pass provider_base_url (e.g. https://api.openai.com/v1 or https://api.openai.com/v1/chat/completions) with the matching openrouter_model (here just the provider's model ID, e.g. gpt-4o) and your key. Foresea normalizes /v1 base URLs to /v1/chat/completions internally.

For safety, provider_base_url must be public HTTPS; loopback, private, link-local, and cloud-metadata hosts are rejected. In the web app, the sidebar's "Use your own model" panel exposes the provider, endpoint, key, and model.

curl -X POST https://foresea.ink/predict \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Will X happen by 2027?",
    "question_type": "binary",
    "openrouter_api_key": "YOUR_KEY",
    "openrouter_model": "gpt-4o",
    "provider_base_url": "https://api.openai.com/v1/chat/completions"
  }'

Self-hosted vLLM

SCADS AI already exposes Foresea's default models through an OpenAI-compatible hosted endpoint. Use vLLM only when you need direct control over checkpoint, quantization, throughput, or serving hardware.

Start a local vLLM OpenAI-compatible server:

VLLM_API_KEY=token-abc123
vllm serve Qwen/Qwen3-32B \
  --host 0.0.0.0 \
  --port 8001 \
  --api-key "$VLLM_API_KEY" \
  --generation-config vllm

Then point Foresea at the configured qwen3-32b-vllm model:

VLLM_API_KEY=token-abc123 PYTHONPATH=src analyze-llm-rationale smoke-test \
  --model qwen3-32b-vllm

VLLM_API_KEY=token-abc123 PYTHONPATH=src analyze-llm-rationale serve \
  --model qwen3-32b-vllm \
  --variant variant0_neutral_baseline \
  --port 8080

For production, run Foresea and vLLM as separate services. Foresea's public bring-your-own endpoint still requires public HTTPS for provider_base_url; private or loopback vLLM URLs are intended for trusted server-side config.

Binary request

curl -X POST https://foresea.ink/predict \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Will the Federal Reserve cut interest rates at least once before September 30, 2026?",
    "question_type": "binary",
    "market_platform": "Polymarket",
    "market_probability": 42
  }'

Multiple-choice request

curl -X POST https://foresea.ink/predict \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Who will win the 2026 Formula 1 drivers championship?",
    "question_type": "multiple_choice",
    "options": ["Max Verstappen", "Lando Norris", "Charles Leclerc", "Lewis Hamilton", "Other"],
    "attach_evidence": false
  }'

Numeric request

curl -X POST https://foresea.ink/predict \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What will US CPI inflation be in December 2026?",
    "question_type": "numeric",
    "resolution_criteria": "Use the year-over-year CPI-U inflation rate for December 2026."
  }'

Request with caller-provided evidence

curl -X POST https://foresea.ink/predict \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Will Company X report positive net income in Q4 2026?",
    "description": "Resolve using the company earnings release.",
    "resolution_criteria": "Yes if reported GAAP net income is positive.",
    "attach_evidence": false,
    "news_articles": [
      {
        "title": "Company X raises full-year guidance",
        "source": "Example Business News",
        "url": "https://example.com/company-x-guidance",
        "publish_date": "2026-05-29",
        "summary": "Company X raised revenue guidance and reported margin expansion."
      }
    ]
  }'

Python client example

import requests

payload = {
    "question": "Will the Federal Reserve cut interest rates at least once before September 30, 2026?",
    "question_type": "binary",
    "attach_evidence": True,
    "evidence_top_k": 3,
    "market_platform": "Polymarket",
    "market_probability": 42,
}

response = requests.post(
    "https://foresea.ink/predict",
    json=payload,
    timeout=180,
)
response.raise_for_status()
prediction = response.json()

print(prediction["predicted_answer"], prediction["confidence"])
print(prediction["model_rationale"])
if prediction.get("market_analysis"):
    print(prediction["market_analysis"]["summary"])
for source in prediction["evidence_sources"]:
    print(source["source"], source["url"])

Response fields

• question_type: detected or requested type: binary, multiple_choice, numeric, or date.
• predicted_answer: "Yes", "No", the top multiple-choice option, or the median numeric/date estimate.
• confidence: model confidence as a number from 0 to 1 for binary and multiple-choice forecasts; null for numeric/date forecasts.
• options: per-option probabilities for multiple-choice forecasts.
• range_forecast: p10, p50, p90, and optional unit for numeric/date forecasts.
• rationale: model-generated explanation.
• model_rationale: alias for the model-generated explanation, intended for API clients.
• evidence_sources: compact source list with article title, URL, publication date, and relevance score.
• evidence_articles: full evidence records attached to the prompt.
• evidence_error: retrieval error message, or null when evidence retrieval succeeds.
• market_analysis: optional comparison against a supplied market price: market_probability, model_probability, edge, stance, and a short summary. edge is model_probability - market_probability.

Repository Contents

• src/analyzing_llm_rationale/: packaged inference, provider, validation, and CLI logic.
• configs/: model and rationale-variant definitions.
• prompts/: system prompt plus the configured rationale, control, ablation, and no-evidence prompt variants.
• scripts/: evaluation, recovery, SHAP, perturbation, plotting, market-data, and utility scripts.
• slurm/: HPC launchers for the variant/temperature sweeps.
• results/: model outputs and run metadata.
• analysis/: aggregate metric tables and rationale-analysis outputs.
• paper/: paper figures, Draw.io sources, PDFs, and qualitative case studies.
• tests/: unit tests for the package and metric parsing.

See ARTIFACT_MANIFEST.md for the submission checklist and file-level notes.

Install

python -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev,serve,pipeline]"

Use .[dev] for linting and unit tests. Add .[analysis] when regenerating plots, metrics tables, or SHAP analyses. Add .[trading] for local exchange order preview/execution development.

Prompt Variants

Configured variants live in configs/variants.yaml and map directly to prompt files under prompts/.

• variant0 is the neutral baseline.
• variant1 through variant8 cover the original rationale attribute prompts.
• variant9 through variant14 add scratchpad, length-matched, structural, and combined temporal/credibility controls.
• variant15_neutral_no_rationale and variant16_no_evidence_neutral support ablations for rationale and evidence effects.

When adding a variant, update configs/variants.yaml, add the prompt file, and run a bounded smoke test:

PYTHONPATH=src analyze-llm-rationale run-batch \
  --variant <variant_name> \
  --max-records 3

Quick Validation

PYTHONPATH=src python -m analyzing_llm_rationale validate-dataset
python -m unittest discover -s tests
ruff check src tests

PYTHONPATH=src is useful when the repository has not been installed yet or an older user-local install shadows the working tree.

Run the full suite with Python 3.10+ and the relevant extras installed. The server, RAG, tracking, and trading tests import optional dependencies from serve, pipeline, analysis, and trading.

Primary Entry Point

Run the variant 3 pipeline with the packaged CLI:

analyze-llm-rationale run-batch --variant variant3_reasoning_type

For a remote OpenAI-compatible provider:

export PROVIDER_API_KEY=your_token
analyze-llm-rationale run-batch --variant variant3_reasoning_type --model llama-3.3-70b-instruct

If you do not want to install the package into the environment, invoke it directly:

PYTHONPATH=src python -m analyzing_llm_rationale run-batch --variant variant3_reasoning_type

Useful options:

• --variant variant6_step_by_step_reasoning: choose the prompt/output contract.
• --model qwen2.5-7b-instruct: choose a configured model definition.
• --temperature 0.7: control generation temperature and output directory.
• --max-records 10: process only a bounded number of records.
• --reprocess-nulls: rerun existing rows with predicted_answer = null.
• --drop-article-text: remove raw article text from prompts before inference.
• --device auto: select cuda when available, otherwise cpu.
• verify-results --variant ...: verify completeness, duplicates, malformed rows, and missing IDs.
• validate-dataset: validate the dataset schema before a run.

Foresea Autoresearch

Foresea has a Karpathy-style autoresearch harness for prompt experiments: edit one candidate prompt, run a fixed benchmark slice, score one metric, and append an auditable experiment log. The research surface is autoresearch/candidate_prompt.txt; agent instructions live in autoresearch/program.md. The default --model gpt-oss-120b uses the SCADS-hosted OpenAI-compatible endpoint from configs/models.yaml (SCADS_AI_API_KEY or SCADS_AI_API_KEY.txt).

Run one candidate experiment:

PYTHONPATH=src python -m analyzing_llm_rationale autoresearch \
  --model gpt-oss-120b \
  --candidate-prompt-path autoresearch/candidate_prompt.txt \
  --max-records 50 \
  --metric brier_score

Compare against a baseline and promote only if the candidate improves:

PYTHONPATH=src python -m analyzing_llm_rationale autoresearch \
  --model gpt-oss-120b \
  --candidate-prompt-path autoresearch/candidate_prompt.txt \
  --baseline-results-path results/GPT-OSS-120B/temperature_00/results_variant0_neutral_baseline.json \
  --promote-to prompts/variant0_neutral_baseline.txt \
  --max-records 50 \
  --metric brier_score \
  --min-delta 0.001

Each run writes analysis/autoresearch/runs/<run_id>/score.json and appends a machine-readable row to analysis/autoresearch/experiments.jsonl.

Reproducing Core Outputs

Validate an existing result file:

PYTHONPATH=src python -m analyzing_llm_rationale verify-results \
  --model qwen2.5-7b-instruct \
  --variant variant3_reasoning_type \
  --temperature 0.0 \
  --temperature-tag temperature_000

Regenerate aggregate metrics from results/:

python scripts/evaluate_metrics.py

Run the DuckDB SQL analytics suite over the real Metaculus-style dataset and saved model outputs:

python scripts/sql_analytics.py \
  --db analysis/forecasting_analytics.duckdb \
  --ingest --replace \
  --output-dir analysis/sql_analytics

This writes a markdown report plus one CSV per query for 10 medium-level SQL problems: model accuracy, best variants, calibration bins, Brier score, consensus/disagreement cases, prompt lift over baseline, temperature sensitivity, overconfident errors, and category difficulty.

Run the LangChain-powered news retrieval wrapper:

PYTHONPATH=src analyze-llm-rationale fetch-and-rank \
  --question "Will X happen by date Y?" \
  --source gdelt \
  --source google-news \
  --source stooq \
  --top-k 5

The news pipeline uses LangChain for a query-planning step, article summarization, and embedding-based relevance ranking before inference. Evidence sources are configurable with --source for the CLI and --evidence-source when serving the API.

Run or schedule the Prefect DAG for RSS/news fetch, inference, and DuckDB logging:

# One question
python flows/forecasting_flow.py --question-id 124 --top-k 5

# Small batch from the dataset
python flows/forecasting_flow.py --limit 3 --top-k 5

# Daily scheduled deployment at 06:00 UTC
prefect server start
python flows/forecasting_flow.py --deploy --limit 3 --cron "0 6 * * *"

Regenerate paper figures after metrics are present:

python scripts/plot_model_variant_metric_heatmap.py
python scripts/plot_variant_delta_from_v0.py
python scripts/plot_temperature_frontier.py
python scripts/plot_frs_ablation_slopegraph.py
python scripts/plot_uncertainty_language_calibration_disconnect.py
python scripts/plot_shap_importance_attribute_gaps.py

Scripts

Common runner and verification commands:

• python scripts/run_variant.py --variant variant5_key_conditions
• python scripts/run_variant.py --variant variant3_reasoning_type --temperature 0.7 --temperature-tag temperature_07
• python scripts/run_variant.py --variant variant4_credibility --model llama-3.3-70b-instruct
• python scripts/verify_results.py --variant variant3_reasoning_type
• python download_qwen_model.py
• python check_local_inference.py

Repo layout:

• scripts/: modular runner entrypoint
• slurm/: batch launchers

Auditability:

• Each run writes run_metadata_<variant>.json next to the results file.
• Metadata includes provider, normalized provider endpoint, model key, resolved model identifier, temperature, output fields, and prompt SHA-256 hashes.
• Existing malformed results JSON now fails fast instead of being silently ignored.

Quality checks

Always run unit tests, linter, and knowledge graph sync before committing:

# Run full unit test suite (1,140+ unit tests across prediction, MCP, trading, and frontend copy)
python -m unittest discover -s tests

# Linting & code style (E501 line-length is ignored)
ruff check src tests scripts/*.py

# Update codebase knowledge graph (AST extraction, no API cost)
py -m graphify update .

Data, Models, and Secrets

The included dataset is forecasting_qa_news_metaculus_2025-02-01_to_today.metaculus_frs_format.json. Model access is configured in configs/models.yaml. Open-weight Qwen models run locally through Hugging Face; hosted models use OpenAI-compatible endpoints and require API keys through environment variables or local key files.

Never commit key files or tokens. Large local caches (.cache/, envs/, .venv/) are intentionally ignored and excluded from source archives.

Citation

If this repository supports a publication, cite the artifact with the metadata in CITATION.cff and cite the upstream datasets/models according to their licenses.