This blog post has been reported on and distorted by a lot of tech news sites using it to wax delusional about AI’s future role in vulnerability detection.

But they all gloss over the critical bit: in fairly ideal circumstances where the AI was being directed to the vuln, it had only an 8% success rate, and a whopping 28% false positive rate!

  • Sailor Sega Saturn@awful.systems
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    10 days ago

    LLMs: now as effective as enumerating use-after-frees as grep "free" source.cc.

    (Also is it just me or is it a bit weird to call this kind of finding a “zero day”?)