Source: Mayank A.'s LinkedIn post (2026-11-12) on alibaba/open-code-review (Go, Apache-2.0, about 50k stars, v1.13.8).
TL;DR
Open Code Review (OCR) is a CLI that reviews Git changes with an LLM agent, and that Alibaba says ran as its internal reviewer for about two years. Its thesis is that a general coding agent with a review skill fails in three predictable ways: it skips files on large changes, its comments drift off the right lines, and its quality swings with small prompt edits. OCR's fix is to take the steps that must not go wrong away from the model. Plain code picks the files, bundles related files, matches rules to files, and places and re-checks comments; the agent only reads code, pulls context and reasons about defects. On Alibaba's own 200-PR benchmark this gives roughly double the F1 and four to five times the precision of Claude Code on the same model, at about a ninth of the tokens. Recall is lower, by design.
Architecture
Deterministic layer (hard constraints)
File selection. Code decides which changed files must be reviewed and which are filtered out, so coverage does not depend on the model choosing to look.
Bundling. Related files (for example
message_en.propertiesandmessage_zh.properties) form one review unit. Each bundle runs as a sub-agent with its own isolated context, which keeps large change sets bounded and makes bundles parallel.Rule matching. A template engine attaches review rules to each file by its traits (built-in rules for null dereference, thread safety, XSS, SQL injection and so on, plus custom rules with path targeting). The model sees only the rules that apply.
Positioning and reflection. Separate modules map each comment to its file and line and re-check its content before it is emitted. This is the fix for position drift.
Agent layer (dynamic decisions)
Review-specific prompt templates.
A narrow toolset (read file, search the codebase, inspect other changed files) distilled from production tool-call traces: call frequency, per-tool repetition rates, and how adding a tool changed the rest of the call chain.
Execution modes
ocr review(working tree, merge-base branch range, or one commit; resumable sessions) andocr scan(whole files or folders, no diff).Delegation mode (
ocr delegate): OCR does file selection and rule resolution, and the host agent (Claude Code, Codex, Cursor, Kimi Code, OpenCode) reviews with its own model. This splits the deterministic half from the model half as a product feature.JSON output, CI integrations (GitHub Actions, GitLab, GitFlic, Gerrit), an MCP server for extra tools, a session viewer to mark findings fixed or ignored, and OpenTelemetry tracing.
Benchmark (AACR-Bench)
200 real PRs from 50 open-source repos in 10 languages, with 1,505 ground-truth issues labelled by 80+ senior engineers; dataset on Hugging Face (Alibaba-Aone/aacr-bench). Selected rows from the README table:
| Harness + model | F1 | Precision | Recall | Avg time | Avg tokens |
|---|---|---|---|---|---|
| OCR v1.3.1 + Claude 4.6 Opus | 25.1% | 33.9% (301/889) | 20.0% | 1m23s | 385K |
| Claude Code v2.1.169 + Claude 4.6 Opus | 11.6% | 7.2% (435/5,980) | 28.9% | 13m6s | 5,664K |
| OCR v1.3.1 + Claude 4.8 Opus | 17.9% | 37.8% (176/465) | 11.7% | 1m6s | 352K |
| Claude Code v2.1.169 + Claude 4.8 Opus | 14.1% | 15.9% (191/1,200) | 12.7% | 5m38s | 2,062K |
| OCR v1.8.7 + Qwen3.8-Max | 23.0% | 33.9% | 17.4% | 5m14s | 334K |
| OCR v1.3.1 + GPT-5.5 | 21.0% | 32.1% | 15.5% | 2m51s | 422K |
| Codex v0.140.0 + GPT-5.5 | 8.4% | 27.8% | 4.9% | 2m58s | 525K |
What the table actually says:
The win is mostly precision. Claude Code with Opus 4.6 finds more real issues (435 vs 301) but buries them in about 6,000 comments; OCR posts 889. For a human triaging comments, that is the number that matters.
Recall goes down, and with Opus 4.8 it falls to 11.7%, so the newer model scores lower F1 inside OCR than the older one. Rule focus and tight tools seem to make a more conservative model even more conservative.
The token gap (about 9 to 15 times) comes from bounded bundles and a narrow toolset, not from a cheaper model.
Absolute numbers are low for everyone. The best system finds one in five labelled issues.
Why it matters
This is a production-scale instance of "the model proposes, the harness enforces" (R020) applied to review rather than testing: coverage, locality and rule selection are invariants, so they belong in code. It also shows that the review task decomposes cleanly into a deterministic plan (which files, which rules, where comments go) and a bounded model step, which is why delegation mode can hand the second half to any host agent.
Tradeoffs and failure modes
Precision over recall is a policy, not a free win. Fewer comments means fewer real bugs caught. Fine for a pre-merge assistant next to human review; risky as the only gate.
Rule matching narrows attention. A defect outside the matched rule set (cross-module logic, design errors, a bug in an unchanged caller) is less likely to be raised. File selection that filters too aggressively hides changes the same way.
Bundles isolate context. A bug that spans two bundles may be missed by both, since neither sub-agent sees the other's files except through search.
Toolset tuned on Alibaba's traces. It was optimised for Alibaba's codebases and languages; another org's repos may want different tools.
Not an independent evaluation. Alibaba built both the tool and the benchmark; the baselines are untuned general agents with skills, and the OCR rows mostly use v1.3.1 while the current release is v1.13.8.
Review checklist (adopting an AI reviewer)
Is coverage enforced by code (every changed file in some bundle), or left to the model?
Are comment locations computed and validated outside the model?
What is the comment volume per PR, and who triages false positives?
Measure recall on your own seeded bugs, not just vendor precision.
Which rules apply to which paths, and is there a catch-all pass for issues no rule names?
Does a cross-bundle bug get seen by anyone?
In CI, does the reviewer block merges or only comment? Low recall argues for comment-only.
Is the model call delegable to your existing agent and keys, or does it need its own provider credentials?
Related entries
R020 The model proposes, the harness enforces (same principle, applied to tests).
R015 Psychology of programming and deterministic workflows (workflows over free-form agents).
R044 Formal Skill (skills as state machines and guards, the same critique of natural-language skills).
R017 Elephant-Goldfish Model (isolated-context reviewers for specs; OCR isolates per bundle for code).
R016 Human verification of AI output (who checks the checker; planted canaries to measure reviewer recall).
R029 Synthetic enterprise tenants (evaluation data; AACR-Bench is a labelled review benchmark).
R006 Harness building blocks.