Claim

Model ledger-default@0.1.0 ledger-strict@0.1.0 ledger-default@0.2.0 ledger-strict@0.2.0 ledger-default@0.3.0 (default) ledger-strict@0.3.0 Snapshot seq

Topics

AI · 1efd24fe Statistics · 1efd24fe

Assessment

Why?

Strongest counted support: A public release summary repeats Paper2Agent’s 91.2% accuracy on 300 benchmark questions. weight 0.459000 evidence

Suppressed as dependent: The supplied passage directly reports 91.2 ± 1.6% versus 80.3 ± 2.3% on 300 tutorial-derived questions using Sonnet 4.; The primary Nature article reports 91.2 ± 1.6% accuracy for Paper2Agent with Sonnet 4 on 300 tutorial-derived questions versus 80.3 ± 2.3% for direct-repository Claude Code with Sonnet 4.

Review gaps: qualifiers reviewed, opposing search done, independence reviewed

What would most change this: One independent, direct measurement that contradicts this claim would move it from Leans supported to Contradicted.

Show calculation

Your view

Registered views

Checked as part of

Statements this claim was broken out of. The claim is scored on its own; a statement's reading only counts its claims.

• “On 300 tutorial-derived benchmark questions, Paper2Agent with Sonnet 4 achieved 91.2 ± 1.6% accuracy, compared with 80.3 ± 2.3% for direct-repository Claude...” · 1 claim · 29 September 2026
• “The Nature study evaluated Paper2Agent across heterogeneous paper corpora and execution benchmarks: https://www.nature.com/articles/s41586-026-11044-y” · 4 claims · 29 September 2026

Evidence for

• Paper2Agent scored 91.2 ± 1.6% on 300 tutorial questions versus 80.3 ± 2.3% for direct-repository Claude Code using Sonnet 4.

  From Reimagining research papers as interactive and reliable AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted

• A public release summary repeats Paper2Agent’s 91.2% accuracy on 300 benchmark questions.

  From Stanford Researchers Release Paper2Agent: Turning Research Papers Into AI Agents That Reproduce Results and Run on New Data Read 2026-09-29 Relevance MODERATE Steps 1 Counted

  Public release summary corroborates the headline benchmark but omits the reported uncertainty and comparator details.

• The supplied passage directly reports 91.2 ± 1.6% versus 80.3 ± 2.3% on 300 tutorial-derived questions using Sonnet 4.

  From Reimagining research papers as interactive and reliable AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted

  The excerpt exactly states the sample size, model condition, and both accuracy values.

• The primary Nature article reports 91.2 ± 1.6% accuracy for Paper2Agent with Sonnet 4 on 300 tutorial-derived questions versus 80.3 ± 2.3% for direct-repository Claude Code with Sonnet 4.

  From Reimagining research papers as interactive and reliable AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted

  The source directly states the benchmark size, both accuracy values, and model condition.

Evidence against

None.

Qualifications

None.

Suppressed as dependent

Evidence	Group	Kept instead
afd49faa		f2d7acc8
a9dda155		f2d7acc8

Inferences

Concluded from

None.

Used as a premise in

None.

An inference is interpretation, not evidence: it never changes the assessment of its conclusion. Each premise is a claim with its own evidence.

Threads

None. Threads are for disagreements about how a determination was made — a statement whose figures are not in the passage it rests on, two sources that may share an origin. Something about the world belongs in the evidence above.

Claim edges

None. Edges are stored and displayed; v0.1 does not propagate probability across them.

Review checklist

Review checks: 1 of 4

3 checks by this claim's own author, alongside the independent review above. 1 OPPOSING EVIDENCE SEARCH · 1 QUALIFIER CHECK · 1 EVIDENCE VERIFICATION. A check by the person who recorded a claim does not raise its review coverage.

Check	Done	By
qualifiers reviewed	no
opposing search done	no
independence reviewed	no
primary source reviewed	yes	fce18e0c, 96fae335, f2d7acc8

Summary

Contribution history

seq	event	detail
22869	APPENDED
22870	ACCEPT	AUTOMATIC_AFTER_VALIDATION
22891	LINK_EVIDENCE	by 1efd24fe (accepted)
22954	LINK_EVIDENCE	by 1efd24fe (accepted)
22956	TASK_RESULT	by 1efd24fe (accepted)
24382	TASK_RESULT	by 1efd24fe (accepted)