Provena

Evidence-backed persistent memory for AI agents.
Know what an agent remembers, where it came from, and why it was retrieved.

Provena gives Codex, Claude Code, Gemini CLI, custom agents, and other MCP clients a shared long-term memory layer with explicit provenance. It stores source events separately from structured claims, links every claim to immutable evidence, records review and retrieval history, and keeps source authority separate from semantic relevance.

The result is agent context that can be inspected, challenged, scoped, and explained instead of an opaque collection of vector matches.

Product Demo

See evidence-backed agent memory in action.

A one-minute walkthrough of setup, cross-session recall, and the Provena operator console.

https://github.com/user-attachments/assets/37fe219c-b678-4c6d-811d-bfdd37552628

Try Provena → · See how it works · Contribute

Contributors Wanted

Provena is looking for early contributors interested in Python, TypeScript, MCP, PostgreSQL, agent security, technical writing, and developer tooling.

Start with the open good first issue list. Each beginner task includes its expected skills, estimated effort, likely files, acceptance criteria, and verification commands. Comment on an issue before starting so contributors do not duplicate work.

Read CONTRIBUTING.md for local setup and pull-request expectations.
Read AGENTS.md before changing provenance, trust, scope, or database behavior.
Use GitHub Issues for confirmed bugs and scoped changes.
Report vulnerabilities privately through the repository Security tab as described in SECURITY.md.

Documentation, tests, accessibility improvements, reproducible bug reports, and focused code changes are all useful contributions.

What is Provena?

Provena is a memory service and integration harness for AI-assisted development workflows. It sits between an agent host and durable storage through REST, MCP, or lifecycle hooks.

The harness is responsible for:

capturing selected user and assistant turns as immutable source events;
accepting explicit, structured memories from an agent or application;
extracting candidate facts with a configured local or hosted model;
retrieving relevant claims from one exact organization and scope;
returning source attribution, status, and authority with retrieved context; and
recording which claims were delivered during each retrieval.

Provena does not execute an agent's code or tasks. Its role in the execution context is to make memory capture and context assembly traceable. Retrieval records improve reproducibility by showing which stored claims were supplied to an agent, but Provena does not currently replay a model run or prove that a retrieved claim influenced a later action.

Core model

Record	Meaning
Event	Immutable source material, such as a user statement, assistant inference, hypothesis, or tool observation.
Claim	A structured proposition: subject, predicate, JSON value, validity interval, and review state.
Evidence	An immutable link from a claim to the event that supports it.
Memory action	An append-only status transition or review decision with actor, reason, and version.
Claim relationship	A typed link such as `supports`, `contradicts`, `supersedes`, `derived_from`, or `related_to`.
Retrieval event	An audit record of a query and the exact claims returned to an agent.
Scope	An exact organization, project, or branch boundary for stored and retrieved memory.

Claims can be candidates, active, verified, conflicted, superseded, quarantined, expired, ephemeral, or deleted. Changing state never erases the claim's source evidence.

Why Provenance Matters

Agent memory can be relevant and still be wrong, stale, speculative, or malicious. A vector result alone cannot answer who asserted a fact, what the original source said, whether a human reviewed it, or which execution context received it.

Provena preserves those distinctions:

Traceability: memory_explain follows a claim back to its source event, credential, extraction run, relationships, status history, and recorded retrievals.
Verification: human credentials review state transitions; agent-generated summaries cannot promote themselves into high-authority facts.
Auditability: events, evidence links, relationships, and memory actions are append-only.
Temporal clarity: recorded time and fact-validity time are stored separately.
Conflict awareness: overlapping claims with different values remain visible until a reviewer records a contradiction, temporal change, or dismissal.
Isolation: tenant-owned records include organization IDs, and retrieval requires an exact scope.
Security: remembered content is treated as untrusted data and never grants permission to perform an action.

PostgreSQL is the authoritative system of record. pgvector embeddings are derived indexes; they do not replace evidence or determine authority.

How Provena Works

mermaid

flowchart LR
    A[Agent, CLI, or host application] --> B[REST, MCP, or lifecycle hook]
    B --> C[Provena capture and retrieval harness]
    C --> D[FastAPI policy and transaction boundary]
    D --> E[(PostgreSQL + pgvector)]
    C --> F[Ollama or OpenAI\noptional extraction and embeddings]
    E --> G[Attributed context or explain response]
    G --> A
    E --> H[Next.js operator console]

A typical write and retrieval flow is:

text

source turn or explicit memory
  → immutable event
  → candidate claim linked through evidence
  → duplicate and conflict checks
  → optional human review
  → exact-scope semantic retrieval
  → attributed context plus retrieval audit record

Model output never raises source authority or activates a claim. Candidate claims may be returned as clearly marked provisional context until a human promotes, quarantines, or deletes them.

Use Cases

Cross-session agent memory: share reviewed project facts or user constraints across Codex, Claude Code, Gemini CLI, and custom clients using the same organization and scope.
Explainable preferences: preserve a statement such as a dietary restriction and show the exact event behind the structured preference.
Architecture memory: record decisions such as a production database, runtime, or deployment policy with validity time and source evidence.
Conflict review: distinguish a contradiction from a temporal migration or a fact that belongs to another environment.
Branch experiments: isolate feature-branch facts in a branch scope so they do not silently enter project-scope retrieval.
Context auditing: inspect the exact claims delivered in an agent retrieval without treating operator browsing as another agent retrieval.

Getting Started

Fastest self-hosted setup

Install the published connector with pipx, which manages Provena in its own environment and exposes the command globally. You do not need to create or activate a virtual environment. Choose the agent host you use:

bash

pipx install provena-agent-memory
provena quickstart codex
# Or: provena quickstart claude
# Or: provena quickstart gemini

If pipx is not installed, follow the official pipx installation instructions. A regular pip install remains supported when you already have a persistent Python environment.

This prepares the version-matched Compose deployment, preserves an existing database and .env, starts PostgreSQL and local Ollama models, bootstraps separate agent and reviewer credentials, installs MCP and lifecycle hooks for the selected host, and starts the operator console. The command prints the scope-specific console URL.

Quickstart uses a digest-pinned, third-party CPU-only Ollama image. Its Linux/amd64 image download is about 32 MB; the default extraction and embedding models still download about 1.26 GB. This setup does not use GPU acceleration. See ADR 0022 for the image choice and its trust tradeoff.

After this explicit setup, ordinary prompts and final responses are captured automatically and relevant candidate or reviewed claims are supplied to later turns. Restart the selected host and review Provena under /hooks and /mcp. pip install alone never edits an agent's configuration or begins capture. See ADR 0020 and ADR 0021.

Manual self-hosted release setup

Published releases provide prebuilt API and console images. Download the three deployment files from the matching GitHub release, then create local configuration:

bash

mkdir provena && cd provena
curl -LO https://github.com/admiralpunk/Provena/releases/download/v0.1.13/compose.yaml
curl -LO https://github.com/admiralpunk/Provena/releases/download/v0.1.13/compose.ollama.yaml
curl -Lo .env.example https://github.com/admiralpunk/Provena/releases/download/v0.1.13/default.env.example
cp .env.example .env

Generate separate values for POSTGRES_PASSWORD and BOOTSTRAP_TOKEN, place them in .env, and start core mode:

server.ts

python -c 'import secrets; print(secrets.token_urlsafe(32))'
docker compose up -d postgres api
docker compose exec api provena status --api-url http://127.0.0.1:8000
docker compose exec api provena init --format shell

Core mode supports explicit memories and review without downloading a model. To enable local automatic extraction and semantic retrieval, add the Ollama override:

Terminal

docker compose -f compose.yaml -f compose.ollama.yaml up -d

Save the one-time credentials printed by provena init. Add the human key and scope ID to .env before starting the optional console profile. See deploy/README.md for upgrades and backups.

Prerequisites

For local development:

Python 3.12 or newer
Node.js 20 or newer and npm
Docker Engine with Docker Compose, used for PostgreSQL and local Ollama
curl

For the containerized setup, only Docker Engine, Docker Compose, and curl are required.

bash

git clone https://github.com/admiralpunk/Provena.git
cd Provena

Option 1: Run Locally

Install the Python service and its MCP and test dependencies:

bash

python3 -m venv .venv
.venv/bin/pip install -e '.[server,test]'
cp .env.example .env

Set a private BOOTSTRAP_TOKEN in .env, then start PostgreSQL and Ollama and install the default local models:

Terminal

docker compose up -d postgres ollama
docker compose exec ollama ollama pull qwen2.5:1.5b
docker compose exec ollama ollama pull nomic-embed-text

Migrate the database and start the API:

bash

set -a
source .env
set +a
.venv/bin/alembic upgrade head
.venv/bin/uvicorn provena.api:app --reload --host 127.0.0.1 --port 8000

The API is now available at http://127.0.0.1:8000; interactive OpenAPI documentation is at http://127.0.0.1:8000/docs. Verify API and database readiness with:

bash

.venv/bin/provena status

In a second Bash terminal, load the same configuration and create a local organization, project scope, agent credential, and human review credential:

bash

set -a
source .env
set +a
eval "$(.venv/bin/provena init --format shell)"

The bootstrap credentials are returned once and exported only in the current shell. Start the operator console with the human credential:

bash

cat > frontend/.env.local <<EOF
PROVENA_API_URL=http://127.0.0.1:8000
PROVENA_API_KEY=$PROVENA_HUMAN_KEY
PROVENA_SCOPE_ID=$PROVENA_SCOPE_ID
EOF

cd frontend
npm ci
npm run dev

Open http://127.0.0.1:3000/overview?scope=$PROVENA_SCOPE_ID.

Option 2: Run with Docker

Upgrading from the earlier Compose file without a named PostgreSQL volume?

Back up the existing database before the first restart with this Compose file, then restore it into the new named volume:

Terminal

docker compose exec -T postgres pg_dump -U provena -Fc provena > provena-before-volume.dump
docker compose down
docker compose up -d postgres
until docker compose exec -T postgres pg_isready -U provena -d provena; do sleep 1; done
docker compose exec -T postgres pg_restore -U provena --clean --if-exists --no-owner -d provena < provena-before-volume.dump

Copy the environment template and replace BOOTSTRAP_TOKEN with a private value:

bash

cp .env.example .env
docker compose up --build -d

The first start downloads the configured Ollama extraction and embedding models. Follow progress and verify the API:

Terminal

docker compose logs -f ollama-models api
curl -fsS http://127.0.0.1:8000/openapi.json > /dev/null && echo "Provena API is ready"

Press Ctrl+C after the services are ready; the containers continue running in the background.

Create the initial workspace from inside the API container:

bash

eval "$(docker compose exec -T api python scripts/bootstrap_workspace.py --format shell)"

Then start the console profile with the issued human credential and project scope:

bash

PROVENA_API_KEY="$PROVENA_HUMAN_KEY" \
PROVENA_SCOPE_ID="$PROVENA_SCOPE_ID" \
docker compose --profile console up --build -d console

Open:

API documentation: http://127.0.0.1:8000/docs
Operator console: http://127.0.0.1:3000/overview?scope=$PROVENA_SCOPE_ID

Stop the stack without deleting memory:

Terminal

docker compose --profile console stop

PostgreSQL and Ollama use named volumes. Add docker compose --profile console down --volumes only when you intentionally want to destroy the local database and downloaded models.

Connect an AI Agent

For an existing Provena service, install the connector as a managed command and configure MCP plus automatic capture and retrieval using the agent credential and one exact scope. Replace codex with claude or gemini for that host:

server.ts

pipx install provena-agent-memory
export PROVENA_API_URL=http://127.0.0.1:8000
export PROVENA_API_KEY=paste-agent-key
export PROVENA_SCOPE_ID=paste-project-or-branch-scope-id
provena connect codex --install

Restart the selected host and review the installed integration under /hooks and /mcp. Use provena connect <host> without --install to print configuration without changing the host. Use generic to print standard MCP JSON for another client. The printed MCP command uses:

config.json

{
  "command": "/home/user/.venvs/provena/bin/provena-mcp",
  "env": {
    "PROVENA_API_URL": "http://127.0.0.1:8000",
    "PROVENA_API_KEY": "paste-agent-key",
    "PROVENA_SCOPE_ID": "paste-project-or-branch-scope-id"
  }
}

After adding the configuration, verify the same credential and scope independently:

bash

~/.venvs/provena/bin/provena doctor

The MCP adapter exposes:

memory_context and memory_search for attributed retrieval;
memory_record_event and memory_capture_turn for source capture;
memory_remember and memory_propose_claim for evidence-backed candidate claims; and
memory_explain for provenance, review, conflict, and retrieval history.

Agent keys can create events and candidate claims. Only human credentials can review claim status or resolve conflicts. Host sessions provide provenance labels; they do not create separate memory stores. See ADR 0014 for the model-agnostic integration boundary.

Configuration

Service and model settings

Variable	Default	Purpose
`DATABASE_URL`	`postgresql+psycopg://provena:provena_dev@localhost:5437/provena`	SQLAlchemy connection for the authoritative PostgreSQL store.
`BOOTSTRAP_TOKEN`	empty	Local trust anchor for creating organizations and issuing, rotating, or revoking credentials. Required for bootstrap operations.
`MEMORY_PROVIDER`	`ollama`	Memory intelligence provider: `ollama`, `openai`, or `none`.
`OLLAMA_BASE_URL`	`http://127.0.0.1:11434`	Ollama HTTP endpoint. Compose overrides this with the internal service address.
`EXTRACTION_MODEL`	`qwen2.5:1.5b`	Fact extraction model. Provider-prefixed model names are stored as provenance.
`EMBEDDING_MODEL`	`nomic-embed-text`	Embedding model used for semantic retrieval.
`OPENAI_API_KEY`	empty	Required only when `MEMORY_PROVIDER=openai`.

Console, MCP, and hook settings

Variable	Purpose
`PROVENA_API_URL`	Base URL of the Provena REST API.
`PROVENA_API_KEY`	Server-side console or agent credential. Never expose it as a `NEXT_PUBLIC_` variable.
`PROVENA_SCOPE_ID`	Exact project or branch scope used for capture and retrieval.
`PROVENA_AGENT_HOST`	Optional host label used by portable lifecycle hooks for session provenance.

Use a human credential for the local console if you need review and conflict actions. Use an agent credential for MCP and automatic capture. Do not give a conversational agent the human review key.

Development

Start only the development dependencies:

Terminal

docker compose up -d postgres ollama

Create and migrate the disposable integration-test database, then run the deterministic suite:

Terminal

docker compose exec -T postgres sh -c 'createdb -U provena provena_test 2>/dev/null || true'
DATABASE_URL=postgresql+psycopg://provena:provena_dev@127.0.0.1:5437/provena_test \
  .venv/bin/alembic upgrade head
TEST_DATABASE_URL=postgresql+psycopg://provena:provena_dev@127.0.0.1:5437/provena_test \
  .venv/bin/pytest -q

Validate migrations and the frontend production build:

bash

DATABASE_URL=postgresql+psycopg://provena:provena_dev@127.0.0.1:5437/provena_test \
  .venv/bin/alembic check
cd frontend
npm run typecheck
npm run build

Model-dependent behavior is isolated behind the memory intelligence interface. Deterministic tests use fake transports and do not require model calls.

Project Structure

text

src/
  core/           domain enums and state-transition rules
  persistence/    SQLAlchemy mappings and database invariants
  memory/         extraction and embedding providers
  integrations/   MCP, conversation capture, and host lifecycle hooks
  web/            FastAPI routes, policies, schemas, and operator projections
alembic/           versioned PostgreSQL migrations
frontend/          Next.js operator console
scripts/           local setup helpers
deploy/            versioned self-hosted release Compose files
docs/adr/          durable architecture decisions
examples/          deterministic MCP client flow
tests/             domain and real-PostgreSQL integration tests
compose.yaml       local PostgreSQL, Ollama, API, and optional console stack
server.json        official MCP Registry package metadata

Read the architecture guide for current guarantees and limits. Accepted decisions live in docs/adr/.

Current Limits

Provena currently uses exact-scope retrieval; branch inheritance and cross-scope promotion are not implemented. It records claim delivery but not whether an agent action was caused by that claim. Binary artifact storage, production identity federation, semantic duplicate resolution, automatic temporal resolution, and a general task execution sandbox are outside the current implementation.

Raw event payloads, claims, evidence, actions, extraction metadata, embeddings, and retrieval membership are stored in PostgreSQL. This keeps the provenance transaction atomic while broader artifact storage remains deferred.

Contributing

Choose an open beginner task, comment that you are working on it, and keep the pull request focused on that issue. Preserve the evidence and tenant-boundary invariants, use Alembic for schema changes, add real PostgreSQL coverage for database guarantees, and record durable architecture decisions in docs/adr/.

See CONTRIBUTING.md for setup and verification commands, CODE_OF_CONDUCT.md for community expectations, SECURITY.md for private vulnerability reporting, and CHANGELOG.md for release history.