Ticket AI README
The full upstream README, mirrored here for reference. Install config, tool schemas, adoption signals, and an original overview live on the Ticket AI listing page.
ticket-ai
An MCP server and CLI that measures how your team actually writes tickets, and holds new ones to that. Not this ticket is bad — 31 of the 40 tickets that shipped here have an acceptance-criteria section, and this one does not.
No 'Akzeptanzkriterien' section. 31 of the 40 exemplar tickets (78%) have one.
Most ticket advice is free and therefore ignored. "Add acceptance criteria", "include steps to reproduce" — everyone has heard it, nobody changed anything. A count of what already happened in your own project is harder to wave away, and it is not an opinion about tickets.
There is no model in the loop for any of that. Learning the house style, finding related tickets and measuring a draft are counting, and run with no key and no network beyond your tracker. Writing a ticket needs a model, so that part is opt-in and which one is your choice. What it found when it was run against real boards →
Install
Configuration is environment variables only — a token passed as --token ends
up in your shell history and in the process list.
The token is optional on a public project. TICKET_AI_GITLAB_URL=https://gitlab.com
with no token reads any public board, which is the quickest way to see what
this does before pointing it at your own instance.
Jira and GitHub
mcp-name: io.github.syrian963/ticket-ai-mcp
One command
learn takes a minute or two: ranking needs each ticket's comments and linked
merge requests, which is an extra request or two per ticket. It caches to
TICKET_AI_CACHE_DIR if you set one and to .ticket-ai/ otherwise, so it
happens once rather than once per review.
Then the rest:
draft is the one worth building a habit around. Checking a ticket after you
create it puts the review past the point of no return: the board has already
been notified and every fix is now an edit with a history.
--from is taken as given: no filtering, no scoring against your choices. Name
a ticket with a three-word description and that is your answer about how this
team writes tickets, and the profile will say so.
As an MCP server
Eight tools, all read-only: learn_conventions, house_style,
ticket_template, ticket_context, template_gaps, review_draft,
review_ticket, review_open_tickets.
Add TICKET_AI_REPO to the env block if the checkout you want searched is
not the assistant's working directory.
Writing a ticket this way
The path to reach for if you already use Claude Code: no key, no compose, no
second model call. Ask for a ticket and the assistant does five things, three
of them here:
ticket_template— the shape: which sections, how long, what language, which labels.ticket_context— what exists: related tickets, the files their merge requests changed, the files in the checkout that mention it.- It reads those files.
ticket_contextruns a text search, not an analysis; it says where to look, it does not save you looking. - It writes the ticket.
review_draft— measures what it wrote, and fixes what that finds before showing you anything.
Where the "AI" is
Counting cannot produce a paragraph of German, so writing a ticket needs a model. Everything else needs nothing.
| Writes | Needs | |
|---|---|---|
| MCP, in Claude Code | yes | nothing — the assistant is already a model |
--writer ollama | yes | a model on your machine. No key, no account, nothing leaves the laptop |
--writer openai | yes | a base URL and a key. OpenRouter, Azure AI Foundry, vLLM, any provider |
| no writer (default) | no | nothing. Measures and gathers; you write |
compose writes the body, measures it, hands the findings back to the model
once, and prints the review to stderr so the body alone can be redirected.
--fail-under makes it refuse to emit a draft that missed the house style.
Whichever model writes, the draft goes through the same measurement as any other ticket. That loop is why a small local model is usable here: it writes into a shape worked out by counting, and is marked against your team's own tickets afterwards.
A page instead
A local page on 127.0.0.1:8760 with three tabs: the house style, a box to
paste a draft into, and the open backlog worst-first. Loopback only, because
this process holds a tracker token, and there is no flag to change that.
No build step and no CDN — one HTML file with its CSS and JavaScript inline.
Two languages, and they are separate
The distinction is easy to collapse and worth keeping. A German team may want the tool's own buttons in English; someone joining a German board still has to write the ticket in German.
TICKET_AI_TICKET_LANGUAGE overrides what the corpus measured and takes effect
without re-learning. Leave it unset unless the board is mid-switch — a
measurement beats a setting, and forcing a language the board does not use
makes every existing ticket fail the language check.
Findings, the caveats under them and the list of what a ticket already got right are all rendered in the reader's language at the moment they are shown. The section names inside them are not: those are the team's own headings, and translating one turns it into a section the team does not have.
In CI
What it measures
The house style. Sections, length, labels, title markers, language — over a corpus it either took from you or found itself. Every finding cites a count over that corpus.
The tickets that failed, not only the ones that worked. A rate can be a comparison: not "78% of tickets have acceptance criteria" but "78% of the ones that shipped, and 30% of the ones that stalled". The second is evidence; the first invites a shrug.
The split is on outcome alone — a merged change, a reopen, a run of clarifying questions — and never on what the ticket contains, because splitting on content and then comparing content would be circular. Tickets a staleness bot closed are left out of both groups: those say something about attention rather than about writing. On a board where nothing separates the two, it says so, which is worth knowing before anyone is asked to write differently. docs/shipped-against-stalled.md has the guards and the real board that forced the bot exclusion.
Sections that only travel together. A section carried by 41% of tickets is under any threshold worth having — but if it is on 76% of the tickets that also have a Ziel section and 20% of the ones that do not, the team has a template and applies it to one kind of work. A board-wide rate hides that entirely.
The form you declared, against the tickets you got. With a
.github/ISSUE_TEMPLATE or .gitlab/issue_templates in the checkout, gaps
lines each field up against how often tickets actually carry it — and the gap
runs both ways. A required field that turns up in 10% of tickets is a form
asking for something people cannot easily give. A section most tickets carry
that no form mentions is a convention the project grew and never wrote down.
What you need to write one. Give it a subject and it returns the related past tickets, the files the merge requests for those tickets actually changed, and the files in your checkout that mention it. That middle one lives only in the tracker's history — no amount of reading the code produces it, and it is usually the fastest way to find where the work will land.
How the mining works
Only closed tickets are sampled: an open ticket may be beautifully written, but nothing about it yet shows anyone could act on it. Each is scored on
| Signal | Weight | Why |
|---|---|---|
| A merged MR is attached | 0.30 | Strongest evidence someone could build it as written |
| Substantial description | 0.20 | A stub teaches nothing about a template |
| Has sections | 0.15 | The template is the thing being learned |
| No clarifying questions | 0.15 | Eleven "what do you mean?" comments means it was not clear |
| Never reopened | 0.10 | Reopened means closed before it was understood |
| Labelled | 0.10 |
Tickets opened and closed inside an hour are halved — usually duplicates or typo fixes, whose shape is not the shape of real work. Bot authors are dropped outright; learning a house style from Renovate is a real failure mode. No one author may supply more than 40% of the corpus, or the profile describes your most prolific ticket-writer instead of your team.
Every score carries the reasons that produced it. A corpus you cannot argue with is one you will not trust.
What it will not tell you
Whether your tickets are any good. Nothing here reads for meaning; it counts. It cannot tell you whether your acceptance criteria make sense. It can tell you that the 40 tickets that shipped in this project all had some and this one does not.
Alignment is not quality. The score is distance from the tickets that historically got built here. A one-line ticket from someone who knows exactly what they mean can score badly and be completely fine. The tool says this about itself, and so should you when you quote it at a colleague.
Nothing about your tracker's contents changes. Every operation is a read.
The ranking signals are also circumstantial. A well-written ticket closed as out-of-scope with no MR scores badly, and that is an acceptable error: the goal is thirty representative tickets, not the thirty best ones. A thin sample says so, loudly, in the report and in every review built on it.
Jira has no public API for linked branches and merge requests, so the "shipped" signal there falls back to remote links, and a board that does not post them cannot be split into shipped and stalled at all. It says that rather than reporting every ticket as stalled. That is a limit of the API, not of the corpus.
Measuring the part a model writes
Every test in this repository mocks the writer, so none of them says anything
about the prose that comes back. evals/ is the answer to that: a frozen set of
titles from public boards, each stored next to the board profile as it stood
when the case was collected.
The cases were held out of the profile before it was built. A case the profile was fitted to flatters the model for a reason that has nothing to do with the model.
The runner records and does not score. A case costs thirty to seventy
seconds on a local model, so scoring separately is what makes a changed metric
free to try. --runs is that path.
Two of the figures are not in review_draft, deliberately. It asks whether the
sections a board uses are present; it never asks what else the draft
invented, because people rarely add a heading their board has never seen and
models do it constantly. The other is spread: one number per case reads as
precision a non-deterministic writer cannot support, so --repeats exists and
the report prints the deviation next to the mean.
The judge, and why its number never travels alone
Counting cannot say whether the prose is about the title, and that is the
failure worth catching: a draft can carry every section, hit the length, take
the right labels and describe something else entirely. review_draft gives it
full marks.
--judge asks a model that one question and nothing else. Not "is this a good
ticket" - that is the question this tool exists to replace with counts.
Raw agreement is close to useless on its own. If nine drafts in ten are on topic, a judge that answers "on topic" every time agrees ninety percent of the time and has learned nothing. Cohen's kappa subtracts the agreement two people guessing at those rates would reach, so that judge scores zero.
The labels come from a person and there is no way around it. A calibration
whose human half was generated compares one model to another. Without
--labels, the judge line prints uncalibrated every time rather than
documenting the caveat once and letting the number travel without it.
The dataset is 134 cases over ten public boards. Thirty are German, from
kern-ux/pattern-library and fitko/fim/portal on gitlab.opencode.de, a public
German GitLab that reads without a token. Three boards are Jira (KAFKA and CASSANDRA on issues.apache.org, HHH
on hibernate.atlassian.net), and four write pure prose with no recurring
sections at all.
That is not variety for its own sake. compose branches on each of them: a
German prompt, a tracker abstraction, and a skeleton block that either lists
sections or tells the model to write no headings at all. CI asserts all three
survive, because a dataset that lost one would stop measuring it without the
average moving, and the loss would be invisible in the one number people read.
kern-ux is why jsonl_lines exists: one of its
tickets carries a literal U+2028, str.splitlines treats that as a line break
and json.dumps does not escape it, so a valid file read as a truncated record
and the error blamed the file.
evals/README.md has the rules about what may become a case. Public boards only.
Where it runs
The cheap half runs on every pull request that touches the harness: every board loads, no case is also an exemplar, and a committed results file is re-scored. No model, so it costs seconds.
The suite itself is workflow_dispatch. Composing 134 cases is minutes and
money, and paying for it on a branch about a README typo buys nothing.
terraform/ is the report's address: an S3 bucket behind CloudFront, and an
IAM role GitHub Actions assumes with a short-lived OIDC token. There is no
AWS key in the repository secrets to leak or rotate. The trust policy is
scoped to one repository; repo:owner/* would hand the role to every
repository that owner ever creates. Everything works without any of it, and the
publish step skips itself when the role variable is unset.
Documentation
| thirty-boards.md | What running it against forty-three real boards, an MCP client, a browser and a clean install found. Almost all of it had full line coverage at the time |
| what-it-produces.md | Five tickets it wrote, unedited, with the score each one got |
| shipped-against-stalled.md | How the two groups are split, and the guards that keep the comparison honest |
| evaluating-compose.md | The harness over the frozen dataset: what was held out and why, the two metrics review_draft does not have, why the gate tolerates noise, and what is still missing |
| local-models.md | Running the writer on a 2 GB local model: measured timings, and what it trades away |
Verified against
Run end to end, read-only, against home-assistant/core, pydantic/pydantic,
astral-sh/uv, fastapi/fastapi (GitHub), inkscape/inkscape and
gitlab-org/gitlab-runner (GitLab), hibernate.atlassian.net HHH (Jira Cloud),
issues.apache.org/jira KAFKA (Jira Data Center), and one private GitLab board.
Home Assistant produces no conditional rules at all, which is the right answer: their issue form is mandatory, so every section already clears the board-wide threshold and a conditional has nothing to add.
Development
License
MIT.