Skip to content
IonWarpGet started
GLM code review

GLM code review: GLM 5.3 Flash benchmarks on real pull requests

GLM 5.3 Flash was the cheapest model in every benchmark we ran it on. IonWarp runs it for Cost Review, and a failed Security Review or SOC 2 Review run retries on it. It reviewed the same public pull requests as DeepSeek, Muse Spark, GPT and Gemini models, and a model grader matched every finding to a known issue (how the benchmarks work).

Example pull request#310

What lands on your pull request · Code Review

P1Retry charges the same order twice
billing/checkout.ts:42 · A timed-out response can charge the order twice
- await payments.charge(order);+ await payments.charge(order, {+   idempotencyKey: order.id,+ });
When payment succeeds but the response times out, the caller retries the same order. Both calls charge it. Pass the stable order ID as the payment idempotency key.

Its best result is on security. On 13 pull requests with 18 known security issues, GLM 5.3 Flash scored an F1 of 66.7% for about a cent a review, ahead of Grok 4.6 (53.3%) and Gemini 3.8 Flash (51.6%), which cost 44 and 49 times as much.[1] It trails on general code review, at 42.8% on 50 pull requests against 49.2% for DeepSeek V4.1 Flash.[2]

Where GLM 5.3 Flash leads, and where it does not

  • Security: the best model we ran

    10 of 18 known issues found, with 2 that matched none, for about a cent a review. Only the Qodo app scored higher, 73.3%.[1]

    Security ReviewOn Starter, Pro and Max

  • SOC 2: a tie for first, in 34 seconds

    GLM 5.3 Flash and DeepSeek V4.1 Flash both scored 66.7% on 20 pull requests with 15 known control gaps. GLM found 11 in a median 34 seconds for about a cent; DeepSeek found 13 in six minutes.[3]

    SOC 2 ReviewOn Max

  • API performance: half a cent a review

    48.3% on 18 pull requests. In Grafana #97529, moving a lock let several goroutines build the same expensive index at once; GLM flagged it, and Gemini 3.8 Flash, GPT 5.6 Luna Pro and Muse Spark 1.3 did not.[4] The apps lead this set: Greptile scored 76.9%.[5]

    API PerformanceOn Max

  • Cost review: one bug only GLM caught

    55.2% on 20 pull requests for about 1.4 cents, against 57.1% for DeepSeek V4.1 Flash at 7 cents.[6] In PostHog's Python SDK (#346), a cache-token subtraction meant for Anthropic ran for every provider and undercounted OpenAI input tokens. Of the three models we ran, only GLM flagged it.[7]

    Cost ReviewOn Pro and Max

  • Logs and AI architecture: behind

    36.1% on 17 log and error pull requests, fourth of five models,[8] and 15.4% on the 11-pull-request AI architecture set.[9]

How GLM scored on each review type

ReviewModelF1PrecisionRecall$ per reviewMedian timeRank
Code Review50 of 50 PRsGLM 5.3 Flash42.8%50.0%37.3%$0.022m 15s4 of 5
Security Review13 of 13 PRsGLM 5.3 Flash66.7%83.3%55.6%$0.011m 38s1 of 7
AI Architecture Review11 of 11 PRsGLM 5.347.1%36.4%66.7%$0.071m 29s2 of 8
AI Architecture Review11 of 11 PRsGLM 5.3 Flash15.4%14.3%16.7%$0.011m 2s8 of 8
Log & Errors Review17 of 17 PRsGLM 5.3 Flash36.1%36.7%35.5%$0.011m 14s4 of 5
API Performance Review18 of 18 PRsGLM 5.3 Flash48.3%43.8%53.8%<$0.0149 sec5 of 6
SOC 2 Review20 of 20 PRsGLM 5.3 Flash66.7%61.1%73.3%$0.0134 sec1 of 5
CI/CD Performance18 of 18 PRsGLM 5.3 Flash55.0%40.7%84.6%<$0.0150 sec3 of 5
Cost Review20 of 20 PRsGLM 5.3 Flash55.2%53.3%57.1%$0.012m 23s2 of 3

Rank is among the models we ran on that review, by F1; a tie goes to the cheaper one. See the benchmark

Pricing

Simple pricing for your whole team.

One workspace price with seats included. Credits pool across your team. Unlimited repos on every plan.

Starter

$0 /mo

Try AI PR review.

  • 2 reviewers
    • Code Review
    • Security Review
  • 3 seats included
  • 15,000 credits to start, then 5,000/mo
  • About 70 reviews to start, then about 25 a month
  • Unlimited repos
Get Started

Max

$149 /mo

Scale AI PR review.

  • +9 reviewers
    • UX Review
    • API Performance
    • Analytics Review
    • SEO Review
    • Changelog
    • CI/CD Performance
    • Plan Review
    • Repeat-Failure Learnings
    • SOC 2 Review
  • 10 seats included
  • +75,000 credits/mo
  • About 360 reviews a month
  • +$29/mo per added seat
  • Unlimited repos
  • Priority support
Choose Max
Enterprise

Custom reviewers + credits · GitHub Enterprise · SSO/SAML + audit logs · Custom security review + SLA

Contact us
FAQ

Frequently asked questions

Is GLM good for code review?

For some reviews. GLM 5.3 Flash was the best model we ran on security and tied for first on SOC 2, at about a cent a review. On general code review, logs and AI architecture, other models scored higher.

GLM 5.3 Flash or DeepSeek V4.1 Flash?

On our benchmarks GLM is cheaper and faster, and scores higher on security (66.7% to 62.5%). DeepSeek V4.1 Flash scores higher on code review (49.2% to 42.8%) and on logs (59.0% to 36.1%). IonWarp runs DeepSeek for most reviewers and GLM for Cost Review.

What does a GLM code review cost?

In model fees on our benchmarks, GLM 5.3 Flash cost from about a quarter of a cent a review on CI/CD to about 2 cents on general code review. IonWarp plans include the model cost: Pro is $49/mo for the workspace.

Which GLM models has IonWarp benchmarked?

GLM 5.3 Flash on eight of our nine review types (all but docs). We also ran the larger GLM 5.3 on AI architecture, where it tied DeepSeek V4.1 Flash at 47.1%; GLM 5.3 Flash scored 15.4% there.

Sources

  1. The security review benchmark: 13 pull requests, 18 known issues (release of 2026-09-24)IonWarp benchmark · read
  2. The code review benchmark: 50 pull requests, 158 known issues (release of 2026-10-06)IonWarp benchmark · read
  3. The SOC 2 review benchmark: 20 pull requests, 15 known control gaps (release of 2026-09-24)IonWarp benchmark · read
  4. grafana/grafana #97529, graded API performance findingsIonWarp benchmark · read
  5. The API performance review benchmark: 18 pull requests, 13 known issues (release of 2026-09-24)IonWarp benchmark · read
  6. The cost review benchmark: 20 pull requests, 14 known issues (release of 2026-09-24)IonWarp benchmark · read
  7. PostHog/posthog-python #346, graded cost findingsIonWarp benchmark · read
  8. The log and error review benchmark: 17 pull requests, 31 known issues (release of 2026-09-24)IonWarp benchmark · read
  9. The AI architecture review benchmark: 11 pull requests, 6 known issues (release of 2026-09-24)IonWarp benchmark · read