Guide

How to review an AI pull request

AI-generated diffs are fluent. Fluency is not correctness. Treat the patch as untrusted work from a fast intern who has never been on-call for your system.

1. Confirm the ticket still matches the diff

Agents “helpfully” refactor neighbors. If the ticket was “add a query param,” a 400-line rewrite is a process failure even if tests pass.

2. Search for invented truth

Grep for new endpoints, feature flags, and package names. If they are not in the repo today, the model may have fabricated them. Do not merge on the comment that “this is the usual pattern.”

3. Read the tests like an attacker

Passing tests that only assert status 200 are marketing. Ask: what input should be rejected? What happens when the dependency times out? Did the agent stub the failure away?

4. Secrets, licenses, and comments

Reject API keys, customer samples, and copied snippets with incompatible licenses. AI tools can reproduce training-set code. You are responsible for what enters the repository.

5. Rollback

If the change includes a migration, require a documented down path. “We’ll fix forward” is not a review outcome.