# kubently [Health: Active]

**Category:** ☁️ Cloud Platforms  
**Repository:** https://github.com/kubently/kubently  
**GitHub Stars:** 45  
**Views:** 0  
**Installs:** 0  
**Upvotes:** 0  
**Directory Page:** https://allmcps.com/mcp/kubently

## Description
Troubleshoot Kubernetes agentically: natural-language cluster diagnosis via ask_kubently

## Claude Desktop Quick Installation
Remote MCP endpoint (confidence: high). Install path detected from listing signals. Add as a URL/SSE server in your client:

```json
"mcpServers": {
  "kubently": {
    "url": "https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE"
  }
}
```

## Documentation & README

# Kubently

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/Python-3.13%2B-blue.svg)](https://www.python.org/)
[![Kubernetes](https://img.shields.io/badge/Kubernetes-1.28%2B-blue.svg)](https://kubernetes.io/)
[![A2A Protocol](https://img.shields.io/badge/A2A-Protocol-green.svg)](https://a2a-protocol.org/)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Helm](https://img.shields.io/badge/Helm-Ready-blue.svg)](https://helm.sh/)
[![Security Policy](https://img.shields.io/badge/Security-Policy-yellow.svg)](SECURITY.md)
[![Contributing](https://img.shields.io/badge/Contributing-Welcome-brightgreen.svg)](CONTRIBUTING.md)

Kubently - Troubleshooting Kubernetes Agentically

## Overview

Kubently (*Kubernetes + Agentically*) is a **free, self-hosted, vendor-neutral multi-cluster Kubernetes troubleshooter**. Ask one question, get AI-diagnosed answers from every cluster in your fleet in parallel — including clusters you can't reach directly: executors dial **outbound** to the central API, so there's no inbound ingress, no shared kubeconfig, and no per-cluster credentials to distribute.

Agents collaborate over the [A2A (Agent-to-Agent) protocol](https://a2a-protocol.org/latest/), and any MCP client (Claude Code, Cursor, Claude Desktop) can use Kubently as a tool out of the box.

## Key Features

- **Multi-Cluster Fleet Troubleshooting**: One question fans out across all registered clusters in parallel
- **Outbound-Dial Executors**: Reach clusters behind firewalls/NAT — no inbound ingress, no shared kubeconfig
- **Natural Language Interface**: Conversational Kubernetes troubleshooting and debugging
- **Comprehensive Analysis**: Automated issue detection, root cause analysis, and solution recommendations
- **Multi-LLM Support**: Compatible with Google Gemini, OpenAI, Anthropic, and other providers
- **A2A Protocol**: Industry-standard agent-to-agent communication for complex workflows
- **MCP Server**: Optional [Model Context Protocol](https://github.com/kubently/kubently/blob/HEAD/docs/MCP.md) endpoint so MCP clients (Claude Desktop, Cursor, custom agents) get direct tool access
- **Security-First**: API key authentication, OAuth/OIDC support, and TLS with cert-manager
- **Persistent Sessions**: Redis-backed conversation history and context management
- **Extensive Tool Suite**: kubectl integration, log analysis, resource inspection, and more

## Quick Start

### For Users: Get Started in 5 Minutes

Point `kubectl` at any cluster (kind, minikube, or real) and run:

```bash
npm install -g @kubently/cli
kubently install
```

That's it. The CLI installs Kubently via Helm, wires up secrets and the
executor, port-forwards the API, and drops you into a debug chat:

```
kubently> why is my nginx pod crashlooping?
```

You'll need an LLM API key (Anthropic, OpenAI, or Google) — the installer
prompts for it, or reads `ANTHROPIC_API_KEY` / `OPENAI_API_KEY` /
`GOOGLE_API_KEY` from your environment. Use `--provider` to pick the LLM,
`--chart ./deployment/helm/kubently` to install from a local checkout, and
`kubently install --help` for everything else.

### Use from Claude Code / Cursor (MCP)

Already ran `kubently install`? Add Kubently to Claude Code:

```bash
claude mcp add kubently -- kubently mcp
```

Or connect directly over HTTP (no bridge process):

```bash
claude mcp add --transport http kubently http://localhost:8080/mcp/ \
  --header "X-API-Key: <your-api-key>"
```

Then ask Claude things like *"use kubently to figure out why payments pods are
crashlooping"*. Any MCP client works — see [docs/MCP.md](https://github.com/kubently/kubently/blob/HEAD/docs/MCP.md) for
Cursor and generic configuration.

### Proactive diagnosis (Alertmanager → Slack)

Set `api.env.SLACK_WEBHOOK_URL` to a Slack incoming-webhook URL and point
Alertmanager at Kubently:

```yaml
receivers:
  - name: kubently
    webhook_configs:
      - url: https://<your-kubently-host>/webhooks/alertmanager
        http_config:
          http_headers:            # Alertmanager >= 0.28
            X-API-Key:
              secrets: ["<your-api-key>"]
```

Each firing alert is diagnosed by the agent and the result is posted to Slack —
the bot often explains the root cause before you've opened your laptop.

### Scheduled fleet health digest

Alerts are reactive. A digest sweeps *every* registered cluster on a schedule and
posts one summary to the same Slack webhook — healthy clusters collapse to a
single line, so what's left is what needs you.

```yaml
fleetReport:
  enabled: true
  schedule: "0 13 * * 1-5"   # weekday mornings
```

Preview it before you schedule it — `dry_run` returns the digest and posts
nothing:

```bash
curl -X POST https://<your-kubently-host>/webhooks/fleet-report \
  -H "X-API-Key: <your-api-key>" -H 'Content-Type: application/json' \
  -d '{"dry_run": true}'
```

The digest question is yours to change. Pass `query` in that request to try one
immediately, then keep the wording you like via `fleetReport.query` in values:

```yaml
fleetReport:
  query: |-
    Check every cluster for pods restarting more than 5 times and for PVCs above
    85% usage. One line per healthy cluster. No preamble.
```

To run the real scheduled path once — image, secrets, in-cluster URL and all:

```bash
kubectl create job --from=cronjob/kubently-fleet-report fleet-report-test -n kubently
```

### Deployment verification (did that deploy actually work?)

Tell Kubently what you just deployed and it watches the rollout settle, then
runs a real investigation — pods ready? events clean? errors in the new logs?
metrics regressed vs the pre-deploy window (when Prometheus is configured)? —
and posts a PASS/FAIL verdict with the evidence to Slack. Wire it into the last
step of your CI pipeline:

```bash
curl -X POST https://<your-kubently-host>/webhooks/verify-deployment \
  -H "X-API-Key: <your-api-key>" -H 'Content-Type: application/json' \
  -d '{"cluster": "prod-east", "namespace": "shop",
       "workload": "deploy/checkout-api", "context": "v1.42.0"}'
```

Add `"dry_run": true` to get the verdict back synchronously without posting.
No CI access? Label the workload instead — `kubently.io/verify=enabled` — and
enable `verifyDeployment.watch` in values: Kubently notices every generation
change and verifies the rollout unprompted.

### Scheduled checks (your recurring questions, on cron)

The digest asks one broad question. Scheduled checks let you ask *your*
questions on *their* schedules — each check is a named prompt with a cron
schedule and optional target clusters, run by the agent and posted to Slack:

```yaml
scheduledChecks:
  enabled: true
  checks:
    - name: cert-expiry
      schedule: "0 8 * * 1"        # Monday mornings
      prompt: |-
        Check TLS secrets for certificates expiring within 21 days.
        List each as namespace/name with days remaining.
    - name: pvc-pressure
      schedule: "0 */6 * * *"
      clusters: [prod-east]
      prompt: Find PersistentVolumeClaims above 85% usage.
```

A passing check posts **nothing** — silence means green (set `notifyOnPass:
true` to hear about passes too). Failures always post, evidence included.
Iterate on a check without waiting for cron:

```bash
curl -X POST https://<your-kubently-host>/webhooks/scheduled-check \
  -H "X-API-Key: <your-api-key>" -H 'Content-Type: application/json' \
  -d '{"check": "cert-expiry", "dry_run": true}'
```

**📖 See [QUICK_START.md](https://github.com/kubently/kubently/blob/HEAD/docs/QUICK_START.md) for full quick-start guide**

**📚 See [GETTING_STARTED.md](https://github.com/kubently/kubently/blob/HEAD/docs/GETTING_STARTED.md) for production deployment**

### For Developers: Local Testing

```bash
# Deploy to a local kind cluster (builds images from HEAD)
ANTHROPIC_API_KEY=sk-... ./deployment/scripts/kind-e2e.sh

# Run comprehensive test suite
./test-automation/run_tests.sh test-and-analyze --api-key test-api-key
```

**📖 See [CLAUDE.md](https://github.com/kubently/kubently/blob/HEAD/CLAUDE.md) for development guidelines**

## Configuration

### LLM Providers

Pick a provider with `LLM_PROVIDER` and supply that provider's key. There is
no default provider — the agent refuses to start without `LLM_PROVIDER`. For
local development with `deployment/docker-compose.yaml`, put both in `.env`
(see `deployment/.env.example`):

```bash
# Anthropic
LLM_PROVIDER=anthropic-claude
ANTHROPIC_API_KEY=your-anthropic-api-key

# OpenAI (also matches Azure and OpenAI-compatible endpoints)
LLM_PROVIDER=openai
OPENAI_API_KEY=your-openai-api-key

# Google Gemini
LLM_PROVIDER=google-gemini
GOOGLE_API_KEY=your-gemini-api-key
```

In Kubernetes the keys come from the `kubently-llm-secrets` secret and
`LLM_PROVIDER` goes under `api.env`.

### Helm Deployment

Customize deployment using Helm values:

Kubently ships as a single chart. Its components are switched on and off with
`api.enabled`, `redis.enabled` and `executor.enabled` — an executor-only
install on a remote cluster is the same chart with the first two disabled.

```bash
# From a checkout
helm install kubently ./deployment/helm/kubently -n kubently \
  -f deployment/helm/test-values.yaml

# Or from the published chart repository
helm repo add kubently https://kubently.github.io/kubently
helm install kubently kubently/kubently -n kubently -f my-values.yaml
```

`LLM_PROVIDER` is required and has no chart default — set it under `api.env`
(`anthropic-claude`, `openai`, or `google-gemini`). See
[ENVIRONMENT_VARIABLES.md](https://github.com/kubently/kubently/blob/HEAD/docs/ENVIRONMENT_VARIABLES.md) for the full
configuration surface, and [GETTING_STARTED.md](https://github.com/kubently/kubently/blob/HEAD/docs/GETTING_STARTED.md) for
the production walkthrough.

### Operator Runbooks

Feed your organization's tribal knowledge into investigations. Runbooks are
hand-written markdown files with lightweight frontmatter; when an
investigation (a chat question, an Alertmanager alert, or an A2A call)
matches a runbook's criteria, the agent receives it as "the operator's
runbook for this situation", follows it where applicable, notes deviations,
and cites it by name in the diagnosis.

A worked example:

```markdown
---
name: Payments CrashLoopBackOff
match:
  alerts: ["KubePodCrashLooping", "PaymentsPod*"]   # alert-name globs
  namespaces: ["payments", "payments-*"]            # namespace selectors
  workloads: ["payment-api*"]                       # matches derived pod names too
  topics: ["crashloop", "OOMKilled", "payment service"]  # free-text tags
---
1. Check recent deploys first: payment-api ships through ArgoCD, and 90% of
   crashloops here follow a bad config sync.
2. OOMKilled almost always means the JVM heap flag drifted from the container
   memory limit — compare `-Xmx` against `resources.limits.memory` before
   blaming traffic.
3. If the DB connection pool is exhausted, do NOT restart the pods; escalate
   to #payments-oncall (restarts thundering-herd the database).
```

Deploy runbooks as Helm values (they become a ConfigMap mounted into the API
pod; edits go live without a pod restart):

```yaml
# production-values.yaml
runbooks:
  payments-crashloop.md: |
    ---
    name: Payments CrashLoopBackOff
    match:
      alerts: ["KubePodCrashLooping"]
      namespaces: ["payments"]
    ---
    1. Check recent deploys first ...
```

Matching is scored: an alert-name hit outranks namespace/workload selector
hits, which outrank topic hits. The best match is injected first, and the
total injected size is capped (`KUBENTLY_RUNBOOKS_MAX_CHARS`, default 8000
characters) — one complete, best-matching runbook beats fragments of many.
Outside Helm, point `KUBENTLY_RUNBOOKS_DIR` at any directory of `.md` files.

### Incident History

Past diagnoses become searchable institutional memory. Whenever an
investigation concludes with a root cause, Kubently stores a compact record —
date, cluster, resources involved, symptom keywords, the root-cause
one-liner, and the resolution when one was stated — in Redis, isolated per
authenticated caller (the same namespace boundary as conversation memory, so
in multi-tenant deployments one tenant's incidents are never visible to
another).

The history is used two ways:

- The agent's **`search_past_incidents`** tool answers "have we seen this
  before?" — keyword search over resources, clusters, symptoms and
  root-cause text, newest first.
- **Auto-surface**: when a new investigation strongly matches a past incident
  (same resources/symptoms/cluster), a one-line
  `SIMILAR PAST INCIDENT (date): <root cause>` note is injected into
  context — framed as something to *verify against fresh evidence*, never to
  assume. When a past incident materially informs the diagnosis, the RCA
  cites it ("same root cause as the 2026-07-03 incident").

This is retrieval over stored summaries, not a learning system: records are
plain data with a TTL (default 90 days, `KUBENTLY_INCIDENT_TTL_SECONDS`) and
a per-tenant cap (default 200, `KUBENTLY_INCIDENT_MAX_PER_NAMESPACE`,
oldest evicted). The feature is on by default; set
`KUBENTLY_INCIDENT_HISTORY=false` (Helm: under `api.env`) to disable both
recording and retrieval.

## Architecture

- **API Server**: FastAPI-based REST API for cluster management and authentication
- **A2A Server**: Implements A2A protocol with LangGraph for workflow orchestration
- **Test Automation**: Comprehensive testing framework with 20+ Kubernetes scenarios
- **CLI Tools**: Modern Node.js CLI for interactive debugging

### Agent Toolset

The diagnostic agent investigates with a small set of read-only tools:

- **`list_clusters`** — enumerate registered clusters
- **`execute_kubectl`** — read-only kubectl against one cluster (whitelist-enforced on the executor)
- **`execute_kubectl_multi`** — one read-only kubectl command fanned out across many clusters
- **`get_recent_changes`** — "what changed?" timeline for a workload or namespace: rollouts (ReplicaSet revisions + change-causes), Helm release history *(opt-in: `changeCorrelation.helmHistory.enabled`)*, ArgoCD sync history *(optional: `changeCorrelation.argocd.url`)*, and Normal+Warning events — correlated against first-error timestamps in the RCA
- **`get_events_for_resource`** — chronological events for a resource and its children (deployment → replicasets → pods)
- **`search_pod_logs`** — structured log search across every pod/container matching a label selector (substring or regex, time bounds, previous-container support). Logs are filtered on the cluster's executor so only matching lines — capped, with explicit truncation notes — come back
- **`query_loki`** *(optional)* — LogQL range queries against a cluster's Loki for aggregated/historical log search, including logs from pods that have restarted or been deleted. Enabled by setting `loki.url` in Helm values (unset by default); queries execute on each cluster's executor through the same outbound channel as kubectl commands
- **`query_prometheus`** *(optional)* — instant and range PromQL queries for latency, saturation, OOM-trend and restarts-over-time evidence. Enabled by setting `prometheus.url` in Helm values (unset by default); queries execute on each cluster's executor through the same outbound channel as kubectl commands
- **`search_past_incidents`** — keyword search over this deployment's incident history (see below). On by default when Redis is available; disable with `KUBENTLY_INCIDENT_HISTORY=false`
- **`get_manifest_file`** *(optional)* — read-only fetch of a file from the configured GitOps manifests repo, so proposed fixes are diffed against the real manifest instead of a hallucinated one. Enabled with `gitRemediation` in Helm values (off by default)
- **`propose_fix_pr`** *(optional)* — proposes a high-confidence manifest fix as a **pull request** against the configured GitOps manifests repo (GitHub or GitLab): branch → commit → PR with the investigation evidence in a body clearly marked machine-proposed. The agent **never merges** — a human reviews and merges, and your GitOps controller applies. Size-capped (files/changed lines), token never enters model context, cluster access stays read-only. See [GitOps PR Remediation](https://github.com/kubently/kubently/blob/HEAD/docs/GITOPS_REMEDIATION.md)
- **`query_cloud_logs`**, **`query_cloud_metrics`**, **`get_recent_cloud_changes`** *(optional)* — read-only cloud telemetry for a cluster: CloudWatch Logs Insights / CloudWatch metrics / CloudTrail on AWS, Cloud Logging / Cloud Monitoring / GKE audit logs on GCP. The executor answers them using the workload identity **you** grant its ServiceAccount (EKS Pod Identity, IRSA, or GKE Workload Identity) — no cloud key is ever stored by Kubently, and revoking the IAM role kills the capability instantly. Enabled per cluster with `executor.cloud.enabled` in Helm values (off by default); each call re-checks that the target cluster's executor actually reports an identity. Operations are additionally limited by a code-level allowlist. See [Cloud Telemetry](https://github.com/kubently/kubently/blob/HEAD/docs/CLOUD_TELEMETRY.md)
- **`mcp_<server>_*`** *(optional)* — tools from external MCP servers (streamable HTTP, e.g. Grafana Cloud's or Datadog's remote MCP) configured via `mcpServers` in Helm values (unset by default). Tool names are prefixed with the server name to avoid collisions; results are treated as untrusted input (framed and size-capped) and credentials stay in Kubernetes secrets. **Connect read-scoped servers/credentials only** — Kubently cannot enforce read-only semantics on a remote server's tools. See `docs/MCP_CLIENT_TOOLS.md`

## Documentation

### Getting Started
- **[Quick Start Guide](https://github.com/kubently/kubently/blob/HEAD/docs/QUICK_START.md)** - Get running in 5 minutes
- **[Getting Started](https://github.com/kubently/kubently/blob/HEAD/docs/GETTING_STARTED.md)** - Complete setup for production use
- **[Deployment Guide](https://github.com/kubently/kubently/blob/HEAD/docs/DEPLOYMENT.md)** - Detailed deployment options and configuration

### Usage & Operations
- **[CLI Admin Guide](https://github.com/kubently/kubently/blob/HEAD/docs/GETTING_STARTED.md#step-5-register-and-deploy-executors)** - Managing clusters and executors
- **[Test Queries](https://github.com/kubently/kubently/blob/HEAD/docs/TEST_QUERIES.md)** - Example API requests and A2A protocol usage
- **[MCP Connect Guide](https://github.com/kubently/kubently/blob/HEAD/docs/MCP.md)** - Connect MCP clients (Claude Desktop, Cursor, custom agents)
- **[Environment Variables](https://github.com/kubently/kubently/blob/HEAD/docs/ENVIRONMENT_VARIABLES.md)** - Configuration reference
- **[GitOps PR Remediation](https://github.com/kubently/kubently/blob/HEAD/docs/GITOPS_REMEDIATION.md)** - Agent-proposed fix PRs (human-merged, default off)
- **[Cloud Telemetry](https://github.com/kubently/kubently/blob/HEAD/docs/CLOUD_TELEMETRY.md)** - Read-only CloudWatch / Cloud Logging access via workload identity (default off)
- **[External MCP Tools](https://github.com/kubently/kubently/blob/HEAD/docs/MCP_CLIENT_TOOLS.md)** - Mounting third-party MCP servers into the agent, including per-request injection

### Architecture & Development
- **[Architecture](https://github.com/kubently/kubently/blob/HEAD/docs/ARCHITECTURE.md)** - System design and components
- **[A2A Protocol Spec](https://a2a-protocol.org/latest/)** - Official protocol documentation
- **[Development Guide](https://github.com/kubently/kubently/blob/HEAD/CLAUDE.md)** - Guidelines for contributors

### Troubleshooting
- **[Getting Started - Common Issues](https://github.com/kubently/kubently/blob/HEAD/docs/GETTING_STARTED.md#common-issues)** - Troubleshooting guide

## Contributing

See [CLAUDE.md](https://github.com/kubently/kubently/blob/HEAD/CLAUDE.md) for development guidelines and contribution instructions.

## Maintainer

**Kubently Team** - [hello@kubently.io](mailto:hello@kubently.io)

## License

Apache 2.0 License - See LICENSE file for details