Code Review Rules as Code: Policies and Tests
Turn team code review rules into versioned policy files, test them against five cases, and track the rule behind every AI finding.
On this page
Generic code review finds common bugs. It does not know every rule your team cares about. One team may ban direct database calls in handlers. Another may require a tenant check before each query. These rules often live in a wiki or in one senior engineer’s memory.
Rules as code means storing review guidance in versioned policy files with an ID, scope, evidence, severity, exception, and tests. Then you test the rules like code. This makes AI review more useful because every finding can point to a rule ID, a version, and the evidence that triggered it.
Turn review guidance into versioned rules as code
A rule should be narrow enough to test. It should say where it applies, what evidence to look for, how serious the issue is, and when an exception is allowed.
rule_id: CRR-014
version: 3
title: Require tenant scope
applies_to: services/**/*.ts
severity: blocker
check: every database query uses tenantId
evidence: show the query and missing scope
exception: migration files with owner approval Keep the rule ID stable when the meaning stays the same. Raise the version when the logic changes. This lets teams compare old and new findings without guessing which policy ran.
Test five cases before rollout
Run every rule against the same five cases before it can block a merge.
| Case | Expected result | What it proves |
|---|---|---|
| Passing change | No finding | The rule stays quiet when code is safe. |
| Failing change | One clear finding | The rule catches the target issue. |
| False positive trap | No finding | Similar code does not create noise. |
| Missing context | Ask or lower confidence | The reviewer does not invent facts. |
| Rule update | Result matches version 4 | The change from version 3 to 4 is covered by tests. |
Use small diffs as test fixtures
A test pack can be a folder of tiny diffs plus expected findings. Keep each fixture focused on one rule. The failing case below removes tenant scope from a query.
--- a/services/orders.ts
+++ b/services/orders.ts
@@ -12 +12 @@
- return db.orders.find({ tenantId, id });
+ return db.orders.find({ id }); expected:
rule_id: CRR-014
result: fail
file: services/orders.ts
line: 12 # line number in the changed file, not the hunk
evidence: tenantId is missing The passing fixture keeps tenantId. The false positive fixture uses an in-memory list, not the database. The missing-context fixture hides the data layer and expects a question or a lower-confidence result. The update fixture raises CRR-014 from version 3 to version 4 and confirms that old behavior does not slip through.
Make every finding traceable

The rule ID links policy, tests, review output, and later updates.
A useful finding should include the rule ID and version, the file and line, the evidence, and the next check a developer can run. This is more useful than a vague note such as “check access control.” It also gives engineering leaders a way to see which rules create noise and which rules stop real defects.
Use repository controls to enforce the final gate. GitHub rulesets can require checks before merge. The AI reviewer supplies evidence; the merge rule decides what must pass. Our guide to adding an AI code review merge gate on GitHub and GitLab walks through that setup.
Roll out one rule at a time
Start with a rule that reviewers already enforce by hand. Run it in report-only mode. Review misses and false positives with the team, using the precision and noise measures from measuring AI code review agents. Fix the policy and test pack. Only then make it block a merge.
Teams can apply this model with Vera AI Code Review and connect it to a wider workflow through VibeFlow, where review sits beside other software delivery agents. Keep the rule file in your repository so the team—not a hidden prompt—owns the policy.
Make policy changes safe
Rules as code do not remove human judgment. They make the repeated part clear and testable. Give each rule an owner, ID, version, fixtures, and review date. When the rule changes, update the tests in the same pull request. That keeps the reviewer aligned with the team as the codebase changes.
Frequently Asked Questions
What does rules as code mean for code review?
Rules as code means storing important review guidance in versioned policy files with a rule ID, scope, evidence, severity, exception, and tests.
Which cases should an AI code review rule test?
Test a passing change, a failing change, a false-positive trap, missing context, and a rule update before making the rule blocking.
How can an AI finding be traceable?
Include the rule ID and version, file and line, evidence, and the next check a developer can run in every finding.
When should a code review rule block a merge?
Run a new rule in report-only mode first, review misses and false positives, fix the policy and test pack, and only then make it block a merge.
Written by
Axiom Studio