Skip to main content
Back to Blog

Code Review Rules as Code: Policies and Tests

Turn team code review rules into versioned policy files, test them against five cases, and track the rule behind every AI finding.

Axiom Studio Axiom Studio November 11, 2026 4 min read

Generic code review finds common bugs. It does not know every rule your team cares about. One team may ban direct database calls in handlers. Another may require a tenant check before each query. These rules often live in a wiki or in one senior engineer’s memory.

Rules as code means storing review guidance in versioned policy files with an ID, scope, evidence, severity, exception, and tests. Then you test the rules like code. This makes AI review more useful because every finding can point to a rule ID, a version, and the evidence that triggered it.

Turn review guidance into versioned rules as code

A rule should be narrow enough to test. It should say where it applies, what evidence to look for, how serious the issue is, and when an exception is allowed.

yaml
rule_id: CRR-014
version: 3
title: Require tenant scope
applies_to: services/**/*.ts
severity: blocker
check: every database query uses tenantId
evidence: show the query and missing scope
exception: migration files with owner approval

Keep the rule ID stable when the meaning stays the same. Raise the version when the logic changes. This lets teams compare old and new findings without guessing which policy ran.

Test five cases before rollout

Run every rule against the same five cases before it can block a merge.

Case Expected result What it proves
Passing change No finding The rule stays quiet when code is safe.
Failing change One clear finding The rule catches the target issue.
False positive trap No finding Similar code does not create noise.
Missing context Ask or lower confidence The reviewer does not invent facts.
Rule update Result matches version 4 The change from version 3 to 4 is covered by tests.

Use small diffs as test fixtures

A test pack can be a folder of tiny diffs plus expected findings. Keep each fixture focused on one rule. The failing case below removes tenant scope from a query.

diff
--- a/services/orders.ts
+++ b/services/orders.ts
@@ -12 +12 @@
-  return db.orders.find({ tenantId, id });
+  return db.orders.find({ id });
yaml
expected:
  rule_id: CRR-014
  result: fail
  file: services/orders.ts
  line: 12 # line number in the changed file, not the hunk
  evidence: tenantId is missing

The passing fixture keeps tenantId. The false positive fixture uses an in-memory list, not the database. The missing-context fixture hides the data layer and expects a question or a lower-confidence result. The update fixture raises CRR-014 from version 3 to version 4 and confirms that old behavior does not slip through.

Make every finding traceable

Five-step code review rule lifecycle from writing through revision.

The rule ID links policy, tests, review output, and later updates.

A useful finding should include the rule ID and version, the file and line, the evidence, and the next check a developer can run. This is more useful than a vague note such as “check access control.” It also gives engineering leaders a way to see which rules create noise and which rules stop real defects.

Use repository controls to enforce the final gate. GitHub rulesets can require checks before merge. The AI reviewer supplies evidence; the merge rule decides what must pass. Our guide to adding an AI code review merge gate on GitHub and GitLab walks through that setup.

Roll out one rule at a time

Start with a rule that reviewers already enforce by hand. Run it in report-only mode. Review misses and false positives with the team, using the precision and noise measures from measuring AI code review agents. Fix the policy and test pack. Only then make it block a merge.

Teams can apply this model with Vera AI Code Review and connect it to a wider workflow through VibeFlow, where review sits beside other software delivery agents. Keep the rule file in your repository so the team—not a hidden prompt—owns the policy.

Make policy changes safe

Rules as code do not remove human judgment. They make the repeated part clear and testable. Give each rule an owner, ID, version, fixtures, and review date. When the rule changes, update the tests in the same pull request. That keeps the reviewer aligned with the team as the codebase changes.

Frequently Asked Questions

What does rules as code mean for code review?

Rules as code means storing important review guidance in versioned policy files with a rule ID, scope, evidence, severity, exception, and tests.

Which cases should an AI code review rule test?

Test a passing change, a failing change, a false-positive trap, missing context, and a rule update before making the rule blocking.

How can an AI finding be traceable?

Include the rule ID and version, file and line, evidence, and the next check a developer can run in every finding.

When should a code review rule block a merge?

Run a new rule in report-only mode first, review misses and false positives, fix the policy and test pack, and only then make it block a merge.

Axiom Studio

Written by

Axiom Studio

Turn AI governance insight into evidence

Get weekly governance insights for engineering leaders, then put them to work with VibeFlow.