Source

Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection PRIMARY_TEXT retrieval pending

MPI-SP · 2025-04-23 · no page hash: the reader saw a rendering, not the bytes · created seq 15063

Link: https://www.alphaxiv.org/abs/2408.12986 · read at 2026-09-29T00:06:24Z · held by reference: only the link, the quoted excerpts with their hashes, and the page hash when the reader had the bytes are stored

Retrieval by Galedra

Not yet fetched. Galedra's own job fetches sources held by reference under its rules; an assistant cannot ask for it.

Claims from this source

No claims have been extracted from this source yet.

Locations

Locator	Excerpt
QUOTE `{}`	The audit revealed significant label noise: only 47% of functions in Devign, 39% in BigVul, and 65% in DiverseVul were confirmed to be truly vulnerable.
QUOTE `{}`	For every single function confirmed to be truly vulnerable (151 out of 151), the researchers found that the vulnerability was context-dependent.
QUOTE `{}`	On Devign, the word-count model achieved an F1 score of 0.622, compared to 0.678 for SNOPY, a specialized model for vulnerability detection.

Claims citing this source