Here is the short version, from the reviewer's side of the pull-request queue: modern AI code review tools are genuinely good at the mechanical half of reviewing and reliably irritating at the human half. The bots that survive contact with a real backlog are quiet, specific and overridable; the ones that get muted within a week are chatty, vague, or allowed to block merges on vibes. Most of that difference comes down to plumbing you control — which model reviews what, and whether the whole loop runs through something like OrcaRouter, where those choices are visible and logged rather than baked into somebody else's hosted product.
This matters now because review capacity, not typing capacity, is the bottleneck engineering teams actually feel. AI-assisted coding has made diffs grow faster than it has made reviewers faster, and review bots arrived into exactly that gap. Used well, one clears the underbrush so the first human to open the PR starts at "is this the right shape" instead of "does this even run". Used badly, one trains your most experienced people to stop reading comments at all — a far more expensive outcome than any single missed bug. The behaviours that decide which way it goes are small, concrete, and worth choosing deliberately rather than discovering by attrition.
What AI review is genuinely good at
Strip away the marketing and today's review bots do four things well.
They catch the mechanical class of mistake with patience no human can match: unhandled error paths, missing input validation, tests that assert nothing, dead code, a dependency bumped in one place and not another, a credential that slipped into a diff. None of this is glamorous. All of it is the kind of thing a tired reviewer skims past late in the afternoon on their ninth PR of the day.
They are consistent. A human who has explained "this repo needs a test for every new public function" forty times stops explaining it on the forty-first; a bot never does. That consistency is worth more than it sounds, because most review friction is not disagreement about architecture — it is the slow erosion of standards under deadline pressure.
They give every file the same attention. Human attention decays predictably across a large diff; the third file of forty gets a skim. A bot reads file thirty-nine with the same eyes as file one, which is exactly where the embarrassing bugs hide.
And they anchor their findings. A comment pinned to a specific line, with a reason attached, is checkable in ten seconds. That habit — cite the line, state the reason, stop talking — is the single behaviour that separates a useful bot from wallpaper, and it is the one to demand from whatever you deploy.
The noise problem, and how it kills adoption
Every team that has adopted a review bot badly knows the archetypal first week: a modest PR comes back with a dozen comments, most of them style nits the repo's formatter already handles, a couple confidently wrong, and one genuinely useful catch buried in the middle. The author dismisses the comments in bulk to get the queue moving. The reviewer, seeing comments resolved without discussion, learns they are not worth reading. The next genuinely useful finding dies in the same scroll as the last ten nits. That is how a tool gets muted, and it happens faster than any evaluation cycle can catch.
The specific behaviours that get a bot silenced are worth naming, because every one of them is avoidable:
- Commenting on lockfiles, generated code and vendored dependencies.
- Restating the diff in prose — "this function adds two numbers" helps nobody.
- Praising the code. Compliments from a bot are noise with extra steps.
- Re-raising a comment after it has been addressed, or arguing style against the repo's own configured formatter.
- Essay-length opinions on a one-line fix, and equal-length comments on every PR in a stacked branch, including the ones already merged.
What survives contact with a real backlog is the opposite profile: few comments, each anchored to a line, each carrying a reason, and silence otherwise. Severity tiers help — a finding should know whether it is "will break production" or "consider a clearer name". Per-path ignore lists help more. A bot that finds one real thing per pull request will outlast one that finds ten, because trust compounds and noise compounds faster.
Merge gates: when should a bot be allowed to block?
The default answer is: not yet. Run the bot in advisory mode across every PR for its first weeks, and watch one number — how often do humans, reading the same diff, agree with what it flagged? That agreement rate is the only licence a merge gate should recognise, and it is earned per class of finding, not for the bot as a whole.
A bot may block when its finding is either objectively verifiable or empirically almost never wrong on your code. Verifiable is easy: the test suite fails, a file marked generated was hand-edited, a migration shipped without a rollback, a credential pattern appeared in the diff. Those are facts a script could double-check, and a gate that blocks on facts never becomes the story of why the release slipped. "This design would be cleaner" is not a fact, and a bot that holds a merge open on taste will be overridden on taste — correctly.
Two design rules keep a gate honest. First, any human must be able to override it in one click, with a written reason; a gate nobody can bypass gets routed around, and the workaround is worse than the gate. Second, the override reason is logged wherever the findings are logged. When someone asks in three months why the hotfix went out with a failing check, "the bot blocked it, here is the override and its reason" is an answer. "I think the bot was confused" is not.
Self-hosted or hosted — and why the audit trail matters
A pull-request diff is the most concentrated form of your IP your repository produces: unreleased features, internal endpoints, the occasional customer record in a fixture nobody sanitised. A hosted review service sends exactly that to someone else's model endpoint, under someone else's retention and training policy — and the reviewer clicking approve is rarely the person who read those terms. For some organisations that trade is fine. For many it is a policy violation waiting for an auditor to notice.
Self-hosting flips the trade: the model runs inside your perimeter, the diff never leaves, and you choose which model reviews what — and pin it, so an upstream "upgrade" cannot silently change what your reviewers' bot sounds like. The cost is operating the thing. Both routes are legitimate; what is not legitimate is not knowing which data went where.
Whichever deployment you pick, insist on two things. Per-comment provenance: which model, at which version, under which prompt, produced each finding. And an append-only log of what was raised, what was dismissed, and who overrode what. This is not compliance theatre. It is how you debug the bot — was that confidently wrong comment the new model, or last week's prompt change? — and it is the only acceptable answer when a blocked merge becomes a release post-mortem. It is also why running reviews through a gateway rather than a pile of scattered API keys pays off: one place holds the model choices, the keys and the log. The homepage below shows the shape of that — routing, self-hosting and logging presented as first-class controls rather than enterprise add-ons.
What a multi-model review harness actually looks like
No single model is the best reviewer. The cheap one is fast enough to run on every commit but misses the subtle failures; the strong one catches real design flaws but costs too much to invoke on a typo fix; the model that reasons best about security is not the one that best notices an untested branch. A review harness accepts this instead of pretending otherwise, and does three things.
It routes passes. A broad, cheap pass runs on every diff. Specialist passes — security reasoning, test-gap analysis — run on the paths where they earn their keep, or when the cheap pass flags something worth a second opinion.
It verifies before it speaks. A second model checks the first model's findings before any of them reach a human, and is required to cite a line for anything it lets through. That one habit removes the most expensive noise in the pipeline: the confidently wrong comment. Two models agreeing is a signal. One model being eloquent is not.
It keeps the wiring in one place. Passes, prompts, model assignments and fallbacks live in a single config, so you can swap a model on Tuesday without touching CI on Wednesday — and the log from the previous section tells you exactly which model wrote which comment.
When you evaluate any of this, the vendor's public product page is a surprisingly honest artefact: it shows whether multi-model review is part of the design or a roadmap bullet, how findings are graded, and whether a verification step exists at all. The screenshot below is what that looks like for one gateway-based review product.
The takeaway
Adopt AI code review — on the reviewer's terms. Give the bot the mechanical half of the job and keep the human half human. Start advisory, let blocking be earned class by class on findings that are verifiable or near-never wrong, and keep the override one click away with the reason logged. Prefer a setup where you can see which model is reviewing, swap it without a migration, and read afterwards exactly what it said and why. A bot with those properties survives a real backlog, because it makes reviewers faster without making them stop reading. A bot without them gets muted in a week — and your reviewers will be right to mute it.
Sourcing note: written 2026-09-04. Everything above reflects the author's experience running review bots against live backlogs; no vendor benchmarks are cited, and the two links in the opening paragraph are the only commercial references.
