Starter
Try AI PR review.
- 2 reviewers
- Code Review
- Security Review
- 3 seats included
- 15,000 credits to start, then 5,000/mo
- About 70 reviews to start, then about 25 a month
- Unlimited repos
The best LLM for code review we have measured is Muse Spark 1.3. On the 50 pull requests of Martian's open code review benchmark,[1] which hold 158 known issues, it found 79, with 42 findings that matched none: an F1 of 56.6%, for 42 cents a review.[2] We ran five models and four review apps, and Claude Opus 4.5 graded every finding (the 6 October 2026 release of our AI code review benchmarks). The Qodo app found the most, 98, and scored a little higher: 58.5%.
What lands on your pull request · Code Review
billing/checkout.ts:42 · A timed-out response can charge the order twice- await payments.charge(order);+ await payments.charge(order, {+ idempotencyKey: order.id,+ });
DeepSeek V4.1 Flash, the model IonWarp's Code Review runs by default, scored 49.2% for 10 cents a review. The most precise reviewers found the fewest bugs: 73% of GPT 5.6 Luna Pro's findings matched a known issue, but it found only 46.[2] No model wins every column, or every kind of review: five different models lead our nine review types, and the last table names the winner of each.
Runs DeepSeek V4.1 Flash. We chose it on reviews of our own repositories from 1 to 21 September 2026: per 100 reviews it filed as many P0 and P1 findings as DeepSeek V4 Pro (14.8 on 549 reviews, against 13.5 on 104) for about a quarter of the cost.
Code ReviewOn Starter, Pro and Max
A Code Review run that fails gets one retry, which can move to Muse Spark 1.3, the best model on the code benchmark.
Code ReviewOn Starter, Pro and Max
Security, docs, logs, performance, CI/CD, cost and SOC 2 are measured on 11 to 20 pull requests each, with provisional answer keys. Their leaders in the table are a guide; the models those reviewers run were picked on production reviews.
A Discourse change set X-Frame-Options to ALLOWALL and trusted the Referer header, which any client can fake.[3] DeepSeek V4.1 Flash flagged it.
Security ReviewOn Starter, Pro and Max
In Cal.com, the endpoint that turns off two-factor login answered with an error about backup code login.[4] Gemini 3.8 Flash flagged it.
Docs ReviewOn Pro and Max
Each model ran as IonWarp's Code Review on the same 50 pull requests, and the IonWarp column says where our reviewers use it.[2] Costs here are the benchmark's: the quarter-of-the-cost comparison above comes from reviews of our own repositories. Factory ran 13 other models in April on 50 pull requests from the same five repositories, and GPT-5.2 led with an F1 of 60.5%.[5] Its prompts and grader are its own, so the two lists don't compare score for score.
| Criterion | IonWarp | F1 | Found, of 158 | Matched none | Cost per review | Median time |
|---|---|---|---|---|---|---|
| Muse Spark 1.3 | Retry for Code Review | 56.6%[2] | 79[2] | 42[2] | $0.42[2] | 3m 18s[2] |
| DeepSeek V4 Pro | Retry for CI/CD Performance | 51.4%[2] | 64[2] | 27[2] | $0.16[2] | 5m 59s[2] |
| DeepSeek V4.1 Flash | Default for Code Review | 49.2%[2] | 77[2] | 78[2] | $0.10[2] | 6m 56s[2] |
| GLM 5.3 Flash | Default for Cost Review | 42.8%[2] | 59[2] | 59[2] | $0.02[2] | 2m 15s[2] |
| GPT 5.6 Luna Pro | Default for Analytics Review | 41.6%[2] | 46[2] | 17[2] | $0.12[2] | 1m 47s[2] |
| Review | Best model | F1 | Precision | Recall | $ per review | Median time |
|---|---|---|---|---|---|---|
| Code Review50 of 50 PRs | Muse Spark 1.3 | 56.6% | 65.3% | 50.0% | $0.42 | 3m 18s |
| Security Review13 of 13 PRs | GLM 5.3 Flash | 66.7% | 83.3% | 55.6% | $0.01 | 1m 38s |
| AI Architecture Review11 of 11 PRs | DeepSeek V4.1 Flash | 47.1% | 36.4% | 66.7% | $0.05 | 6m 14s |
| Log & Errors Review17 of 17 PRs | Gemini 3.8 Flash | 67.7% | 67.7% | 67.7% | $0.10 | 1m 28s |
| Docs Review19 of 19 PRs | Gemini 3.8 Flash | 68.6% | 66.7% | 70.6% | $0.08 | 59 sec |
| API Performance Review18 of 18 PRs | Muse Spark 1.3 | 56.0% | 58.3% | 53.8% | $0.06 | 42 sec |
| SOC 2 Review20 of 20 PRs | GLM 5.3 Flash | 66.7% | 61.1% | 73.3% | $0.01 | 34 sec |
| CI/CD Performance18 of 18 PRs | DeepSeek V4 Pro | 72.7% | 60.0% | 92.3% | $0.03 | 1m 22s |
| Cost Review20 of 20 PRs | DeepSeek V4.1 Flash | 57.1% | 47.6% | 71.4% | $0.07 | 10m 39s |
The top F1 for each review on its own benchmark pull requests, read from the published results. Only the code review table is ranked; the others use provisional answer keys.
Measured on open-source pull requests with known issues. See the benchmark
One workspace price with seats included. Credits pool across your team. Unlimited repos on every plan.
Try AI PR review.
Get AI PR review right from day 1.
Scale AI PR review.
Custom reviewers + credits · GitHub Enterprise · SSO/SAML + audit logs · Custom security review + SLA
On IonWarp's benchmark of 50 pull requests and 158 known issues, Muse Spark 1.3 had the best F1 of the models we ran, 56.6%: it found 79 issues, with 42 findings that matched none. DeepSeek V4 Pro scored 51.4% and DeepSeek V4.1 Flash, the model Code Review runs, 49.2%. The Qodo app scored higher than every model, 58.5%, and found 98.
We did not test that, and a coding ranking cannot say. Rankings such as LiveCodeBench grade code a model writes; this benchmark grades whether a model finds known issues in a pull request someone else wrote. Even within review the scores pull apart: GPT 5.6 Luna Pro was the most precise model here and found the fewest issues.
We have not benchmarked Claude models as reviewers: Claude Opus 4.5 is our benchmark's grader, the judge of whether each finding matches a known issue. Factory has, with its own prompts and grader, on 50 pull requests from the same five repositories: Claude Opus 4.6 scored an F1 of 59.8% and Sonnet 4.6 57.4% (Factory, Apr 2026). Those scores don't compare directly with ours.
GPT 5.6 Luna Pro, at a median of 107 seconds a review, against 198 for Muse Spark 1.3 and 359 for DeepSeek V4 Pro. It was also the most precise, and it found the fewest issues.
Yes. Every reviewer has a pinned model on each plan, and you can switch any reviewer's model in settings. On Enterprise you can route reviews through your own AI gateway and model keys.
It depends on the review, and five different models lead our nine. Muse Spark 1.3 leads code review (F1 56.6%) and API performance (56.0%). GLM 5.3 Flash leads security (66.7%) and SOC 2 (66.7%, a tie with DeepSeek V4.1 Flash that goes to the cheaper model). Gemini 3.8 Flash leads logs and errors (67.7%) and docs (68.6%), DeepSeek V4.1 Flash AI architecture (47.1%) and cost (57.1%), and DeepSeek V4 Pro CI/CD (72.7%). Only the code review set has an independent answer key: the other eight cover 11 to 20 pull requests each, with provisional keys we scoped ourselves.