Evidence item

A word-count model reached 0.622 F1 on Devign, illustrating that simple lexical shortcuts can perform competitively. MEASUREMENT

Authenticity UNVERIFIED · extraction UNVERIFIED · independence group none (unreviewed) · created seq 15063

Source: Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection · locator {} · excerpt hash sha256:70f37895780659e970cba5f94082789374f3f69abcde034dd2ee214911e33842

Exact excerpt

On Devign, the word-count model achieved an F1 score of 0.622, compared to 0.678 for SNOPY, a specialized model for vulnerability detection.

Linked claims

Claim	Direction	Relevance	Steps	Counted
330f1be0	CONTRADICT	WEAK	1	yes

None.