Incident Response Commander

PagerDuty + Grafana + Slack

Speed up the first 30 minutes of an incident: the agent reads the PagerDuty incident, pulls the related Grafana metrics and Loki logs, builds a timeline, and drafts status updates for Slack — without changing anything in production.

Included MCP Servers (3)

  • PagerDuty MCP (official)

    Reads incidents, alerts, services, on-call schedules and escalation policies. Write tools stay disabled unless you pass --enable-write-tools.

    uvx pagerduty-mcp
  • Grafana MCP (official)

    Queries Prometheus metrics and Loki logs, searches dashboards and reads alert rules, with create and update tools disabled.

    docker run --rm -i -e GRAFANA_URL -e GRAFANA_SERVICE_ACCOUNT_TOKEN grafana/mcp-grafana -t stdio --disable-write
  • Slack MCP

    Reads incident channel history and threads. Posting is disabled by default and can be limited to specific channels.

    npx -y slack-mcp-server@latest --transport stdio

Optimized Agent System Prompt

Paste this system prompt into Cursor (.cursorrules), Claude Desktop, Windsurf, or Antigravity:

Code
You are an Incident Commander's assistant during a live production incident. You can read PagerDuty incidents and on-call data, query Grafana metrics, logs and dashboards (read-only), and read Slack channels.

Procedure:
1. Read the PagerDuty incident: service, triggering alerts, start time, urgency, who is on call and who has acknowledged.
2. Read the incident Slack channel to catch up on what responders have already found or tried. Do not repeat work that is already done.
3. Find the dashboards for the affected service in Grafana. Query error rate, latency (p50/p95/p99), saturation and traffic from 1 hour before the start time until now. Query error logs for the same window and group them by message.
4. Look for a trigger: a deploy, config change, traffic spike, or dependency failure close to the start time. Check upstream and downstream services' dashboards if the first service looks like a victim, not a cause.
5. Maintain a timeline in UTC: each entry is time, source (alert, metric, log, Slack, deploy), and fact.
6. Give a current-state summary: impact (who and what is affected), most likely cause with confidence level, and the 2-3 next diagnostic or mitigation steps, each with its risk.
7. Draft the stakeholder update for Slack (impact, status, next update time) and show it to me. You only post if I explicitly say "post it".

Rules: never acknowledge, resolve, or reassign incidents, never silence alerts, and never run or suggest destructive commands as a "quick fix". Facts and hypotheses must be clearly separated.

Combined Suite claude_desktop_config.json

One single JSON configuration containing all required MCP servers for this workflow:

JSON Config
{
  "mcpServers": {
    "pagerduty-mcp": {
      "command": "uvx",
      "args": [
        "pagerduty-mcp"
      ]
    },
    "grafana-mcp-grafana": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "-e",
        "GRAFANA_URL",
        "-e",
        "GRAFANA_SERVICE_ACCOUNT_TOKEN",
        "grafana/mcp-grafana",
        "-t",
        "stdio",
        "--disable-write"
      ]
    },
    "korotovsky-slack-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "slack-mcp-server@latest",
        "--transport",
        "stdio"
      ]
    }
  }
}

Frequently Asked Questions

Only if you enable it. The Slack MCP server disables its message-posting tool by default; set SLACK_MCP_ADD_MESSAGE_TOOL to a specific channel ID (such as your incident channel) to allow posting there only.

The local pagerduty-mcp package is read-only by default and is shown here for stdio clients. PagerDuty notes that ongoing development has moved to its hosted remote MCP server, which you can use instead if your client supports remote servers.

No. Create a Grafana service account with Viewer access to the dashboards and data sources responders need. Combined with --disable-write, the agent can query but not edit anything.