A 4B Model That Cannot Find Bugs, Only Order Them

Author

AI News Editorial

Published

2026-11-14 10:15

Frontier models can now find real vulnerabilities in C code. OpenAI reports a cyber-specialised model validating and patching 948 Linux kernel privilege-escalation bugs — at about USD 1,280 per vulnerability. That number is the reason the approach does not scale to the long tail of widely packaged libraries.

Look Here, Then Decide, by Tim De Wachter, Kyryl Volkov, Myrto Eleftheria Gkogkou and Mathieu Huysman of Deus Rex, asks what happens when the expensive model is removed from the loop. Their answer is a 4B open-weight decision model: it answers a typed question in a single forward pass and returns scores for the permitted answers instead of text.

The model is deliberately weak, and the paper says so plainly. Intern-Decision-4B separates vulnerable from benign functions with AUC 0.73, and a vulnerable function from its own patch with only 0.56. It is not a detector. What it does is rank candidate functions inside queues built by deterministic incomplete-fix miners and shape localizers — classic static analysis, no LLM involved in deciding where to look. No frontier model appears anywhere in the search.

As a prioritizer the numbers get better. On 75 public CVEs from projects where the campaign found nothing, the patched function reached its repository’s top 25 in 21% of cases — 1.4× as often as random ordering of the same queues. On the campaign’s own queues, the model’s order put 7.5× more confirmed findings in the top 10 than random order (an explicitly selection-biased upper bound, since those queues were picked for already containing a finding).

The scale is the actual contribution. Three desktop-class GB10 systems searched more than 4,000 projects, mining at 900–1,100 repositories per hour. The campaign confirmed 187 memory-safety findings — 12 critical, 148 high, 23 medium, 4 low under worst-case CVSS v3.1 — all under coordinated disclosure. The decision step alone runs at 4.4 decisions per second on one GB10 and 9.8 on a consumer RTX 4090, with matching scores.

The lesson is architectural: a small model that cannot verify anything is still valuable as a scheduler between deterministic analysis and human review. Recall stayed bounded by where the localizers looked — the model reorders the queue, it does not widen the search.

Paper: arXiv:2611.16565