Evidence item

The audited Devign vulnerabilities required external context, challenging isolated-function benchmark interpretation. DIRECT_TEXT

Authenticity UNVERIFIED · extraction UNVERIFIED · independence group none (unreviewed) · created seq 15063

Source: Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection · locator {} · excerpt hash sha256:6e4e25a8b50ce5144bbcbc165d9e9d20c6045692a519e227df81494bf48f4e5f

For every single function confirmed to be truly vulnerable (151 out of 151), the researchers found that the vulnerability was context-dependent.