Command line
Cawdit is one binary, cawdit, with one subcommand per job.
Every flag that names a variable can also come from that variable, and the flag wins when both are set.
Global flags
| Flag | Variable | Default | Description |
|---|---|---|---|
--log-level |
CAWDIT_LOG_LEVEL |
info |
The lowest level to log: debug, info, warn or error. |
--log-format |
CAWDIT_LOG_FORMAT |
text |
text or json. |
Cawdit writes its logs to standard error.
Put the global flags before the subcommand, as in cawdit --log-level debug review.
Exit codes
| Code | Meaning |
|---|---|
| 0 | The command succeeded. For cawdit review, the pull request was reviewed or skipped. |
| 1 | The command failed, such as on an invalid configuration or an unreachable forge. |
| 2 | The command line is invalid, such as an unknown flag or a missing pull request number. |
| 3 | cawdit review --fail-on found open findings at or above the severity. |
cawdit review
Reviews one pull request once, as a CI step, and needs no database. It reads the state of earlier reviews from the hidden markers in Cawdit’s own comments.
| Flag | Variable | Description |
|---|---|---|
--config |
CAWDIT_CONFIG |
A deployment configuration file. The environment overrides it, and without it Cawdit reads only the environment. |
--forge-url |
CI_FORGE_URL |
The base URL of the forge, such as https://codefloe.com. |
--repo |
CI_REPO |
The repository, as owner/name. |
--pull-request, --pr |
CI_COMMIT_PULL_REQUEST |
The number of the pull request. |
--sha |
CI_COMMIT_SHA |
The head commit to review. Empty reviews the pull request’s current head. |
--target-url |
CI_STEP_URL, then CI_PIPELINE_URL |
The link of the commit status, usually the CI step. |
--dry-run |
none | Post nothing, and print the review as Markdown on standard output instead. |
--fail-on |
CAWDIT_FAIL_ON |
Exit with code 3 while open findings of this severity or higher exist: info, warning or error. |
The CI_* variables are the ones Crow sets in a pull_request pipeline, so the command needs no flags inside Crow.
Outside Crow, pass the flags instead.
The forge token comes from CAWDIT_FORGE_TOKEN, which needs the forge URL from --forge-url or CI_FORGE_URL.
A configuration file or CAWDIT_FORGES_<NAME>_* variables can describe the forge instead.
The model comes from the CAWDIT_PROVIDERS_<NAME>_* variables or the configuration file.
A dry run still reads the pull request and calls the model, so it costs as much as a real review. It writes nothing to the forge, neither the review nor comment edits nor the commit status.
When it posts a review, the command prints a line such as posted 2 new findings on owner/name#7.
cawdit serve
Runs the server that reviews pull requests on webhooks, with the review workers in the same process. The HTTP server reference lists its routes and metrics.
| Flag | Variable | Default | Description |
|---|---|---|---|
--config |
CAWDIT_CONFIG |
none | The deployment configuration file. |
--workers |
CAWDIT_WORKERS |
4 |
How many reviews run at once. |
--migrate |
CAWDIT_MIGRATE |
false |
Apply the database migrations before serving, as cawdit migrate does. |
--drain-timeout |
CAWDIT_DRAIN_TIMEOUT |
5m |
How long to wait for running reviews on shutdown before cancelling them. |
--print-config |
none | false |
Print the configuration, with secrets as their references, and exit. |
On SIGINT or SIGTERM the server stops taking jobs, waits up to --drain-timeout for the running reviews, and exits.
A cancelled review runs again after the next start, and webhooks that arrive while it drains are queued for then.
cawdit migrate
Applies the database migrations, for Cawdit’s tables and the job queue’s, and prints what it applied.
It reads the same configuration as cawdit serve and requires the same keys.
| Flag | Variable | Description |
|---|---|---|
--config |
CAWDIT_CONFIG |
The deployment configuration file. |
Run it before you start a new version of cawdit serve, or start the server with --migrate.
A migration that already ran is skipped, so running the command twice is safe.
cawdit eval
Replays a dataset of pull requests through the review engine and scores the findings.
It sends nothing to a forge.
eval/README.md explains the dataset and the scores.
| Flag | Default | Description |
|---|---|---|
--dataset |
eval/cases |
The directory of the cases. |
--case |
every case | Run only the case with this ID. Repeat it for more cases. |
--config |
none | The deployment configuration file that configures the providers, also from CAWDIT_CONFIG. |
--provider |
the default provider | The configured provider to review with, or oracle for the scripted provider that CI uses. |
--model |
the provider’s | The model of the review pass, and of the verify pass unless --verify-model is set. |
--verify-model |
the provider’s | The model of the verify pass. |
--no-verify |
false |
Skip the verify pass. |
--out |
none | Write the JSON result to this path. |
--compare |
none | Compare with this earlier result file. |
--tolerance |
3 |
How many lines a finding may lie outside an expected range and still match it. |
--concurrency |
2 |
How many cases run at once. |
--timeout |
20m |
The time limit of each case. |
--min-recall |
none | Fail when the share of expected findings caught is below this, from 0 to 1. |
cawdit eval capture
Writes a pull request from a forge as a new case of the dataset, without expected findings.
| Flag | Default | Description |
|---|---|---|
--dataset |
eval/cases |
The directory of the cases. |
--forge-url |
https://codefloe.com |
The base URL of the forge. |
--token |
none | An access token, needed only for a private repository, also from CAWDIT_EVAL_TOKEN. |
--id |
<name>-<number> |
The ID of the case. |
--include |
every file | Keep only changed files that match this pattern or lie below this directory. Repeat it for more. |
--license |
none | The SPDX identifier of the repository’s licence. Required. |
--overwrite |
false |
Replace a case that exists already. |
cawdit version
Prints the version the binary was built from.