Skip to content
IonWarpTry for free
GPT 5.6 code review

GPT 5.6 code review: GPT 5.6 Luna Pro benchmarks on real pull requests

For GPT 5.6 code review we ran one model. GPT 5.6 Luna Pro is the most precise model we have run on code review, and the one that finds the fewest bugs. On 50 open-source pull requests with 158 known issues, 73% of its findings matched a known issue, but it found only 46: an F1 of 41.6%, for about 12 cents a review in a median 107 seconds.[1]

Example pull request#310

What lands on your pull request · Code Review

P1Retry charges the same order twice
billing/checkout.ts:42 · A timed-out response can charge the order twice
- await payments.charge(order);+ await payments.charge(order, {+   idempotencyKey: order.id,+ });
When payment succeeds but the response times out, the caller retries the same order. Both calls charge it. Pass the stable order ID as the payment idempotency key.

On the code review benchmark it was the only one of our five models to flag a Discourse controller that built SQL by pasting in host names, open to SQL injection.[2]

Where GPT 5.6 Luna Pro leads, and where it does not

  • Code review: precise, and it misses most bugs

    46 of 158 known issues found, with 17 findings that matched none. Muse Spark 1.3 found 79 and DeepSeek V4.1 Flash 77.[1] Every model's score.

    Code ReviewOn Starter, Pro and Max

  • Security: every finding matched, 3 of 18 found

    Every finding it posted on 13 security pull requests matched a known issue, but it found 3 of 18: an F1 of 28.6%, last of the seven models. GLM 5.3 Flash found 10.[3]

    Security ReviewOn Starter, Pro and Max

  • SOC 2: near the top, in 37 seconds

    10 of 15 known control gaps found, an F1 of 64.5% against 66.7% for the leaders, DeepSeek V4.1 Flash and GLM 5.3 Flash.[4]

    SOC 2 ReviewOn Max

  • CI/CD, API performance, logs and AI architecture: behind

    It scored 54.5% on CI/CD, fourth of five models;[5] 50.0% on API performance, fourth of six;[6] 36.0% on logs, last of five;[7] and 35.3% on AI architecture, fourth of eight.[8] We did not run it on docs or cost reviews.

What does Luna Pro miss on real pull requests?

  1. A timestamp frozen when the class loads, in Sentry

    The AssignmentSource dataclass set its queued field's default to timezone.now(), which runs once when the class is defined, so every object shares one time. Luna Pro posted no finding on that pull request. The other four models and all four review apps on that case flagged it.[9]

  2. A null that Keycloak's interface promises never to return

    getSubGroupsCount() returned null although the interface's Javadoc says it never does, which can throw a null pointer error in a caller expecting a count. The benchmark rates it Critical. The other four models caught it, and three of the four review apps missed it as well.[10]

  3. A negative offset passed straight into a Sentry query

    One pagination branch handed a negative cursor offset directly to queryset slicing, and the benchmark rates that Critical too. All four other models caught it, and so did three of the four review apps.[11]

  4. What a quiet review from it tells you

    We picked these three because the other models caught them, so they are examples and not a sample. The totals above are the sample, and on code review they say most of what it reports is real and most of what is there goes unreported. That precision does not travel, either, because on three of the seven review types in the table fewer than half its findings matched a known issue. We would run it as an extra pass beside a model that finds more, and would not read silence from it as a clean pull request.

How GPT 5.6 scored on each review type

ReviewModelF1PrecisionRecall$ per reviewMedian timeRank
Code Review50 of 50 PRsGPT 5.6 Luna Pro41.6%73.0%29.1%$0.121m 47s5 of 5
Security Review13 of 13 PRsGPT 5.6 Luna Pro28.6%100.0%16.7%$0.111m 16s7 of 7
AI Architecture Review11 of 11 PRsGPT 5.6 Luna Pro35.3%27.3%50.0%$0.0551 sec4 of 8
Log & Errors Review17 of 17 PRsGPT 5.6 Luna Pro36.0%47.4%29.0%$0.0544 sec5 of 5
API Performance Review18 of 18 PRsGPT 5.6 Luna Pro50.0%54.5%46.2%$0.0328 sec4 of 6
SOC 2 Review20 of 20 PRsGPT 5.6 Luna Pro64.5%62.5%66.7%$0.0737 sec3 of 5
CI/CD Performance18 of 18 PRsGPT 5.6 Luna Pro54.5%45.0%69.2%$0.0133 sec4 of 5

Rank is among the models we ran on that review, by F1; a tie goes to the cheaper one. See the benchmark

Pricing

Simple pricing for your whole team.

One workspace price with seats included. Credits pool across your team. Unlimited repos on every plan.

Starter

$0 /mo

Try AI PR review.

  • 2 reviewers
    • Code Review
    • Security Review
  • 3 seats included
  • 15,000 credits to start, then 5,000/mo
  • About 70 reviews to start, then about 25 a month
  • Unlimited repos
Get Started

Max

$149 /mo

Scale AI PR review.

  • +9 reviewers
    • UX Review
    • API Performance
    • Analytics Review
    • SEO Review
    • Changelog
    • CI/CD Performance
    • Plan Review
    • Repeat-Failure Learnings
    • SOC 2 Review
  • 10 seats included
  • +75,000 credits/mo
  • About 360 reviews a month
  • +$29/mo per added seat
  • Unlimited repos
  • Priority support
Choose Max
Enterprise

Custom reviewers + credits · GitHub Enterprise · SSO/SAML + audit logs · Custom security review + SLA

Contact us
FAQ

Frequently asked questions

Is GPT 5.6 good for code review?

GPT 5.6 Luna Pro is precise and fast, and it finds few bugs: 46 of 158 known issues on our code benchmark, against 79 for Muse Spark 1.3. We have not run the other GPT 5.6 models.

Is GPT 5.6 Luna Pro good for coding?

We measured it as a reviewer, not as a coding agent: it reads a pull request someone else wrote and points out defects. There it was precise and found few issues, 46 of 158. How well it writes code is outside this benchmark.

Which GPT 5.6 models did you benchmark?

Only GPT 5.6 Luna Pro, on seven of our nine review types (all but docs and cost). We have not run GPT 5.6 Sol, GPT 5.6 Terra or GPT 5.6 Luna, so none of these numbers speak for them. The Cost Review benchmark ran GPT-6 Luna Pro, a different model.

What is the difference between GPT 5.6 Luna and Luna Pro?

They are the same underlying model, and Luna Pro serves it in OpenAI's pro reasoning mode, which does more model work per answer and uses more tokens (OpenRouter and OpenAI, Nov 2026). Our code review run used medium effort. We have not run plain GPT 5.6 Luna, so these numbers say nothing about it.

What does a GPT 5.6 Luna Pro review cost?

In model fees on our benchmarks, about 12 cents a code review and 1 to 11 cents on the smaller specialist reviews. IonWarp plans include the model cost: Pro is $49/mo for the workspace.

Can IonWarp run GPT 5.6 Luna Pro?

Yes. It is one of the models IonWarp's reviewers are pinned to (DeepSeek V4.1 Flash, GPT 5.6 Luna Pro, GLM 5.3 Flash), and you can switch any reviewer's model in settings.

Sources

  1. The code review benchmark: 50 pull requests, 158 known issues (release of 2026-10-06)IonWarp benchmark · read
  2. discourse #10 (embeddable hosts), graded code review findingsIonWarp benchmark · read
  3. The security review benchmark: 13 pull requests, 18 known issues (release of 2026-09-24)IonWarp benchmark · read
  4. The SOC 2 review benchmark: 20 pull requests, 15 known control gaps (release of 2026-09-24)IonWarp benchmark · read
  5. The CI/CD benchmark: 18 pull requests, 13 known pipeline defects (release of 2026-09-20)IonWarp benchmark · read
  6. The API performance review benchmark: 18 pull requests, 13 known issues (release of 2026-09-24)IonWarp benchmark · read
  7. The log and error review benchmark: 17 pull requests, 31 known issues (release of 2026-09-24)IonWarp benchmark · read
  8. The AI architecture review benchmark: 11 pull requests, 6 known issues (release of 2026-09-24)IonWarp benchmark · read
  9. sentry #77754 (assignment timestamp), graded code review findingsIonWarp benchmark · read
  10. keycloak #40940 (sub-group count), graded code review findingsIonWarp benchmark · read
  11. sentry-greptile #2 (cursor pagination), graded code review findingsIonWarp benchmark · read