Label review¶
Label review checks whether each issue's labels and GitHub issue type are right, and suggests changes.
How it works¶
./triage.py labels sync fetches the repository's label catalogue (names and descriptions) and its issue types. Good label descriptions matter: they are what the model reads to decide whether a label applies.
For each issue, a decision model gets the issue's title, body, current labels and type, plus the catalogue, and answers one question per label ("does this label apply?") and one choice question for the issue type. Labels above add_threshold that the issue lacks are suggested for adding; current labels below remove_threshold are suggested for removal. If the decision model is replaced by an LLM, it returns a structured verdict with a reason per label instead of probabilities.
[labels]
add_threshold = 0.75
remove_threshold = 0.25
show_threshold = 0.2 # scores shown in the detail pane
exclude = ["dependencies", "python", "javascript"] # e.g. labels only bots apply
Reviewing suggestions¶
On the Labels page choose which issues to review (not reviewed yet, without labels, triaged, reviewed against an older label list, or all open issues), how many and how many in parallel, then Review labels. Results appear as they come in.
For each issue the detail pane has an Issue type dropdown and a searchable Labels field over the whole catalogue, both starting from the model's suggestion; you can pick any type and add any label. Below them, a bar shows how likely each label applies, for every label from show_threshold (20% by default) up plus the current ones, tagged current, will be added or will be removed.
Stage changes records the difference between the issue and your selection and moves to the next issue; Dismiss (or staging with nothing changed) leaves the labels as they are. Stage all high-confidence stages every confident suggestion at once. Staged changes are applied from the Outbox, and can be undone from its History.
Each review records a fingerprint of the label catalogue it was made against. When labels are added, renamed or re-described and the catalogue is synced, older reviews show as reviewed against an older label list, and that scope re-runs them.
From the command line: