Evidence item

An independent audit found substantial label noise in Devign, with only 47% of sampled functions confirmed vulnerable. MEASUREMENT

Authenticity UNVERIFIED · extraction UNVERIFIED · independence group none (unreviewed) · created seq 15063

Source: Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection · locator {} · excerpt hash sha256:db091e872847cf545dd59101ee17f68a6f31cc133231099fdd93d0967da97b02

Exact excerpt

The audit revealed significant label noise: only 47% of functions in Devign, 39% in BigVul, and 65% in DiverseVul were confirmed to be truly vulnerable.

Linked claims

Claim	Direction	Relevance	Steps	Counted
330f1be0	CONTRADICT	MODERATE	1	yes

Audits

None.