Claim

Across 42 execution-based tasks from 10 non-biology computational papers, Paper2Agent achieved 98.1 ± 0.8% accuracy across five independent runs.

Type QUANTITATIVE · truth-evaluable: yes · status at snapshot: ACTIVE · created seq 22873

Referenced 132 times: checked 2, looked up by assistants 0, viewed 94, shared 36. A count of attention, not evidence.

Topics

AI · 1efd24fe Statistics · 1efd24fe

Assessment

Supported

Score: 0.8581 under ledger-default@0.3.0 at snapshot 27745

3 of 4 review checks done · 1 counted · 2 by the author only

1 independent evidence lineage · review checks 1 of 4 · stability MEDIUM

Main unresolved issue: 4 counted evidence items have no independence review; repeats of one origin may be counted as several coverage:78367454-c044-8e10-847e-6d7c28b7d0f0

Not yet independently audited.Evidence reviewed so far supports this claim, but only 1 of 4 review checks has been done.Every quoted passage was confirmed on the page when Galedra fetched it.

Why?

Strongest counted support: AI Weekly reports 98.1 ± 0.8% accuracy across 42 execution tasks on 10 non-biology papers. weight 1.800000 evidence

Suppressed as dependent: The quoted passage directly states 98.1 ± 0.8% accuracy across 42 tasks from 10 non-biology papers and five independent runs.

Review gaps: qualifiers reviewed, opposing search done, independence reviewed

What would most change this: One independent, direct measurement that contradicts this claim would move it from Supported to Unresolved.

Hide calculation

0.8581 under ledger-default@0.3.0 at snapshot 27745 · state Supported · Review checks: 1 of 4 · stability MEDIUM · 4 sources, 1 independent lineage

Model ledger-default@0.3.0 (the default model, not the answer) · snapshot 27745 · trace sha256:a244cfe2715dc9176531882df4f0a58c91696934d2b7b5503c30786b31b4536d

Last changed at entry seq 24385; nothing logged since has borne on this claim.

How the number was reached

Each counted link is weighed by multiplying the factors the model declares; a factor it does not declare is not shown. Every factor name below links to what that number means.

Link	Working	Weight	Counted as
2e883a5b support	relevance DIRECT 2.0 × observation DATASET_RESULT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1	0.000000	0.000000
9b402d58 support	relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1	1.800000	1.800000
e91aa409 support	relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1	0.000000	0.000000
c6225bf3 support	relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1	0.000000	0.000000 (dependent strongest only, kept 2e883a5b)

prior 0.50 → log-odds 0.000000 + 1.800000 (the counted weights) = 1.800000 → probability 0.8581

Under neighbouring settings of the same model: 0.7790 to 0.9121, a spread of 0.1331.

Half-even rounding, six places for weights and four for the probability. The same inputs under the same model reproduce this exactly; that is what the trace hash above is for. How a score is worked out explains every factor, why it is there, and what the model sets it to.

Your view

Registered views

Nobody has registered a view yet.

Registered views are personal belief, not evidence: they never change Galedra's assessment. Affiliations are self-declared and shown only for groups of 5 or more.

Checked as part of

Statements this claim was broken out of. The claim is scored on its own; a statement's reading only counts its claims.

“Across 42 execution-based tasks from 10 non-biology computational papers, Paper2Agent achieved 98.1 ± 0.8% accuracy across five independent runs.” · 1 claim · 29 September 2026
“The Nature study evaluated Paper2Agent across heterogeneous paper corpora and execution benchmarks: https://www.nature.com/articles/s41586-026-11044-y” · 4 claims · 29 September 2026

Evidence for

Paper2Agent scored 98.1 ± 0.8% across 42 execution tasks from 10 non-biology computational papers.

From Reimagining research papers as interactive and reliable AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted
The quoted passage directly states 98.1 ± 0.8% accuracy across 42 tasks from 10 non-biology papers and five independent runs.

From Reimagining research papers as interactive and reliable AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted

The passage directly states the task count, corpus scope, accuracy, and number of independent runs.
AI Weekly reports 98.1 ± 0.8% accuracy across 42 execution tasks on 10 non-biology papers.

From Paper2Agent turns 74 of 100 bio papers into AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted
The primary Nature article reports 98.1 ± 0.8% accuracy across five independent runs on 42 execution-based tasks from 10 non-biology computational papers.

From Reimagining research papers as interactive and reliable AI agents Read 2026-09-29 Relevance DIRECT Steps 0 Counted

The source directly states the task count, paper count, accuracy, and number of runs.

Evidence against

None.

Qualifications

None.

Suppressed as dependent

Evidence	Group	Kept instead
40a8ade6		2e883a5b

Inferences

Concluded from

None.

Used as a premise in

None.

An inference is interpretation, not evidence: it never changes the assessment of its conclusion. Each premise is a claim with its own evidence.

Threads

None. Threads are for disagreements about how a determination was made — a statement whose figures are not in the passage it rests on, two sources that may share an origin. Something about the world belongs in the evidence above.

Claim edges

None. Edges are stored and displayed; v0.1 does not propagate probability across them.

Review checklist

Review checks: 1 of 4

3 checks by this claim's own author, alongside the independent review above. 1 OPPOSING EVIDENCE SEARCH · 1 QUALIFIER CHECK · 1 EVIDENCE VERIFICATION. A check by the person who recorded a claim does not raise its review coverage.

Check	Done	By
qualifiers reviewed	no
opposing search done	no
independence reviewed	no
primary source reviewed	yes	2e883a5b, e91aa409, c6225bf3

Summary

Supported. The main support is: AI Weekly reports 98.1 ± 0.8% accuracy across 42 execution tasks on 10 non-biology papers. c7cba4ff

1 further item repeats the same origin and is not independent evidence. 40a8ade6

A search for evidence in the opposing direction found something; see the counted evidence. f7fbc78f

Review checks: 1 of 4. coverage:78367454-c044-8e10-847e-6d7c28b7d0f0

Generated by stub-v0.1; every sentence cites graph ids.

Contribution history

seq	event	detail
22873	APPENDED
22874	ACCEPT	AUTOMATIC_AFTER_VALIDATION
22893	LINK_EVIDENCE	by 1efd24fe (accepted)
22958	TASK_RESULT	by 1efd24fe (accepted)
23608	LINK_EVIDENCE	by 1efd24fe (accepted)
24384	TASK_RESULT	by 1efd24fe (accepted)

Score trace

{
  "claim": "78367454-c044-8e10-847e-6d7c28b7d0f0",
  "links": [
    {
      "link": "46ee4704-c8ad-8c52-a6e3-9dcd2e883a5b",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "af9a9124-8d41-81d0-af2d-afcfdb203b85",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DATASET_RESULT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 0
    },
    {
      "link": "90cc0d8c-e9b2-8a52-91e8-a0909b402d58",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "11f87c14-5a77-8ee4-a896-150bc7cba4ff",
      "direction": "SUPPORT",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "SECONDARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.800000",
      "interpretive_steps": 0
    },
    {
      "link": "e6382e86-6982-8002-9a90-b1cfe91aa409",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "703a5343-ced2-8997-b907-7e60c9652dad",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 0
    },
    {
      "kept": "46ee4704-c8ad-8c52-a6e3-9dcd2e883a5b",
      "link": "fe572c27-2bba-891e-a026-01fdc6225bf3",
      "group": null,
      "reason": "dependent_strongest_only",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "63060be4-9bd9-8d98-82d6-4ed740a8ade6",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 0
    }
  ],
  "model": "ledger-default@0.3.0",
  "prior": "0.50",
  "variants": {
    "spread": "0.1331",
    "permissive": "0.9121",
    "conservative": "0.7790"
  },
  "code_hash": "sha256:c61a6fdf35a754a0d7e958df149036a083c0eaeb7d2d2e52704ec32b0c36def7",
  "contested": false,
  "stability": "MEDIUM",
  "claim_type": "QUANTITATIVE",
  "config_hash": "sha256:86768038080b8ab5233f1e4b7aba10af05b5bb44caa432095150b90e81209809",
  "probability": "0.8581",
  "provisional": true,
  "evidence_sum": "1.800000",
  "snapshot_seq": 24385,
  "prior_log_odds": "0.000000",
  "support_groups": 1,
  "model_dependent": false,
  "review_coverage": "0.25",
  "truth_evaluable": true,
  "assessment_state": "SUPPORTED",
  "review_checklist": {
    "qualifiers_reviewed": {
      "by": [],
      "ok": false
    },
    "opposing_search_done": {
      "by": [],
      "ok": false
    },
    "independence_reviewed": {
      "by": [],
      "ok": false
    },
    "primary_source_reviewed": {
      "by": [
        "46ee4704-c8ad-8c52-a6e3-9dcd2e883a5b",
        "e6382e86-6982-8002-9a90-b1cfe91aa409",
        "fe572c27-2bba-891e-a026-01fdc6225bf3"
      ],
      "ok": true
    }
  },
  "contradict_groups": 0,
  "rounding_boundary": false,
  "posterior_log_odds": "1.800000",
  "not_applicable_reason": null,
  "independence_unreviewed": 4
}