How Cawdit reviews
This page explains what happens between a push and a review, and why Cawdit works the way it does.
Two ways to run
Cawdit runs in one of two modes, which share the same review engine.
cawdit review runs once inside a CI step, such as the Crow plugin.
It needs no server and no database, and each repository configures its own step and secrets.
It suits a few repositories, or a team that wants the review next to its other checks.
cawdit serve is a long-running server that takes webhooks from the forge.
It reviews every allowed repository without a pipeline in each, keeps a queue in Postgres, and exports metrics.
It suits an organisation that wants one Cawdit for many repositories and one place for the model key.
The steps of a review
| Step | What happens |
|---|---|
| Settings | Cawdit reads .cawdit.toml and the rule files from the base branch, and skips drafts and skipped authors. |
| History | It finds the commit it reviewed last, from its own earlier reviews, and skips a commit it reviewed already. |
| Context | It collects the changed hunks, the code around them, and the repository’s rules, within context.budget. On a re-review it looks only at the files that changed since the last reviewed commit. |
| Review pass | The review model reads each part and answers with findings, each on a line range with a severity and a category. Cawdit drops findings it cannot use, such as ones outside the diff or below min_severity. |
| Verify pass | The verify model reads each finding with its code and keeps or drops it with a reason. This pass removes much of the noise that a single model call produces. |
| Dedupe | Cawdit compares each finding with the open findings of earlier reviews, by a fingerprint of the code it points at, and drops repeats. |
| Resolve | It checks each earlier finding and marks it fixed when the code it points at changed and the model did not report it again. |
| Publish | It posts one review, with a summary and one inline comment per new finding, and sets the commit status cawdit. |
The summary counts the findings by severity and names the files it left out, with the reason. Its footer names the commit, the model, the tokens and the cost.
State without a database
Every comment Cawdit posts ends with a hidden HTML comment, a marker, that the forge does not render:
The marker records the finding’s fingerprint, severity and state, and the commit it was made on.
On the next run, Cawdit reads the markers back from the pull request and knows what it reported before.
That is why cawdit review needs no database, and why the CI step and the server can review the same pull request.
Anyone can write a marker into a comment.
Cawdit trusts a marker only in a comment that its own account wrote, which is why its token needs read:user.
Fixed findings
A finding counts as fixed when the lines it points at changed and the model did not report the same problem again. A finding on code that the push left alone stays open, even when the model did not repeat it, because a re-review looks only at what changed.
For a fixed finding, Cawdit edits its own comment to say so and sets the marker’s state to fixed. On Arbinel, it also resolves the comment’s thread. It resolves a thread only when Cawdit started it and nobody but Cawdit and the author wrote in it, so it never hides a conversation between people. Plain Forgejo cannot resolve threads through its API, so there the edited comment is the only sign.
Commands
With cawdit serve, people steer Cawdit with comments that mention its account, written here as @cawdit:
| Comment | Where | What Cawdit does |
|---|---|---|
@cawdit review |
Anywhere on the pull request | Reviews the whole pull request at its head, also a draft, a skipped author, or a head it reviewed already. |
@cawdit ignore or @cawdit dismiss |
In a finding’s thread | Marks the finding dismissed and never posts it on the pull request again. |
| Any other mention | In a finding’s thread | Answers the question with the model, from the finding, the code at the head and the thread. |
The first word after the mention is the command.
A mention in a quote or in code does not count, so quoting a command does not run it again.
Cawdit responds only to accounts with write access to the repository, and never to its own comments.
Dismissing needs an exact answer from the forge, which in an organisation’s repository may need admin access for the bot account, see the security model.
cawdit review in a CI step does not see comments, so commands need the server.
The commit status
Cawdit sets the status cawdit to pending when a review starts.
It ends as failure while open findings at or above review.fail_at remain, as success otherwise, and as error when the review failed.
Branch protection can require the status to block merging until the findings are fixed.
Severity
| Severity | Meaning |
|---|---|
error |
Incorrect behaviour, data loss, a crash or a security hole under normal use. |
warning |
A problem that is likely under realistic conditions, or a clear violation of a repository rule. |
info |
A real but minor problem that is still worth fixing. Hidden by default. |
Cawdit reports bugs, security problems, performance problems with real impact, reliability problems, and violations of the repository’s rules. It does not comment on style, naming, missing tests or anything a compiler or linter catches, unless a repository rule asks for it.