Claim

In that study, GPT-4-derived predictions were strongly correlated with observed treatment effects at a level similar to pooled human forecasts, but systematically overestimated effect sizes.

Type COMPARATIVE · truth-evaluable: yes · status at snapshot: ACTIVE · created seq 18342

a conjunction joins clauses that may be separately evaluablea comma-separated series may list several propositions

Referenced 95 times: checked 1, looked up by assistants 0, viewed 70, shared 24. A count of attention, not evidence.

Topics

Media · 1efd24fe AI · 1efd24fe

Assessment

Supported

Score: 0.9734 under ledger-default@0.3.0 at snapshot 27745

3 of 4 review checks done · 1 counted · 2 by the author only

2 independent evidence lineages · review checks 1 of 4 · stability HIGH

Main unresolved issue: 3 counted evidence items have no independence review; repeats of one origin may be counted as several coverage:e7267b1d-9faa-8f3a-988c-9e5ebbe65c84

Not yet independently audited.Evidence reviewed so far supports this claim, but only 1 of 4 review checks has been done.A quoted passage from nature.com was not found on the page when Galedra fetched it.

Why?

Strongest counted support: The passage directly reports strong GPT-4 correlations, human-forecast similarity, and systematic effect-size overestimation. weight 1.800000 evidence

Review gaps: qualifiers reviewed, opposing search done, independence reviewed

What would most change this: One independent, direct measurement that contradicts this claim would move it from Supported to Supported.

Hide calculation

0.9734 under ledger-default@0.3.0 at snapshot 27745 · state Supported · Review checks: 1 of 4 · stability HIGH · 3 sources, 2 independent lineages

Model ledger-default@0.3.0 (the default model, not the answer) · snapshot 27745 · trace sha256:343424a32ae25e21802ec641bb3d999b40f32ff8326099d6c89977c40a84b306

Last changed at entry seq 24301; nothing logged since has borne on this claim.

How the number was reached

Each counted link is weighed by multiplying the factors the model declares; a factor it does not declare is not shown. Every factor name below links to what that number means.

Link	Working	Weight	Counted as
0fad4404 support	relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1	1.800000	1.800000
9bd30046 qualify	relevance DIRECT 2.0 × observation DATASET_RESULT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1	1.800000	0.000000 (non directional)
d37b3c19 support	relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1	1.800000	1.800000

prior 0.50 → log-odds 0.000000 + 3.600000 (the counted weights) = 3.600000 → probability 0.9734

Under neighbouring settings of the same model: 0.9255 to 0.9908, a spread of 0.0653.

Half-even rounding, six places for weights and four for the probability. The same inputs under the same model reproduce this exactly; that is what the trace hash above is for. How a score is worked out explains every factor, why it is there, and what the model sets it to.

Your view

Registered views

Nobody has registered a view yet.

Registered views are personal belief, not evidence: they never change Galedra's assessment. Affiliations are self-declared and shown only for groups of 5 or more.

Checked as part of

Statements this claim was broken out of. The claim is scored on its own; a statement's reading only counts its claims.

“A Nature study evaluated LLM predictions of social-science experiment results and found strong correlations with observed effects but systematic...” · 2 claims · 29 September 2026

Evidence for

The passage directly reports strong GPT-4 correlations, human-forecast similarity, and systematic effect-size overestimation.

From Large language models can predict the results of social science experiments Read 2026-09-29 Relevance DIRECT Steps 0 Counted

The excerpt explicitly states all comparative components.
The primary Nature article directly reports strong correlation with actual treatment effects, accuracy similar to pooled human forecasts, and systematic overestimation of effect sizes.

From Large language models can predict the results of social science experiments Read 2026-09-29 Relevance DIRECT Steps 0 Counted

The source exactly supports all three comparative claims.

Evidence against

None.

Qualifications

The Nature study reports strong GPT-4 correlations with observed effects, similar to pooled human forecasts, alongside systematic effect-size overestimation.

From Large language models can predict the results of social science experiments Read 2026-09-29 Relevance DIRECT Steps 0 Counted

Primary results include both correlation and systematic overestimation.

Suppressed as dependent

None.

Inferences

Concluded from

None.

Used as a premise in

None.

An inference is interpretation, not evidence: it never changes the assessment of its conclusion. Each premise is a claim with its own evidence.

Threads

None. Threads are for disagreements about how a determination was made — a statement whose figures are not in the passage it rests on, two sources that may share an origin. Something about the world belongs in the evidence above.

Claim edges

None. Edges are stored and displayed; v0.1 does not propagate probability across them.

Review checklist

Review checks: 1 of 4

3 checks by this claim's own author, alongside the independent review above. 1 OPPOSING EVIDENCE SEARCH · 1 QUALIFIER CHECK · 1 EVIDENCE VERIFICATION. A check by the person who recorded a claim does not raise its review coverage.

Check	Done	By
qualifiers reviewed	no
opposing search done	no
independence reviewed	no
primary source reviewed	yes	0fad4404, 9bd30046, d37b3c19

Summary

Supported. The main support is: The passage directly reports strong GPT-4 correlations, human-forecast similarity, and systematic effect-size overestimation. 34fcfa87

Qualifier: The Nature study reports strong GPT-4 correlations with observed effects, similar to pooled human forecasts, alongside systematic effect-size overestimation. 3dc05238

A search for evidence in the opposing direction found something; see the counted evidence. b302f5a5

Review checks: 1 of 4. coverage:e7267b1d-9faa-8f3a-988c-9e5ebbe65c84

Generated by stub-v0.1; every sentence cites graph ids.

Contribution history

seq	event	detail
18342	APPENDED
18343	ACCEPT	AUTOMATIC_AFTER_VALIDATION
18352	LINK_EVIDENCE	by 1efd24fe (accepted)
20731	TASK_RESULT	by 1efd24fe (accepted)
24300	TASK_RESULT	by 1efd24fe (accepted)

Score trace

{
  "claim": "e7267b1d-9faa-8f3a-988c-9e5ebbe65c84",
  "links": [
    {
      "link": "6ee6e27d-6b26-821f-a79c-f59f0fad4404",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "43062cee-278a-8c70-977b-9aea34fcfa87",
      "direction": "SUPPORT",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.800000",
      "interpretive_steps": 0
    },
    {
      "link": "cc0fef75-0aae-8429-b2cc-417f9bd30046",
      "group": null,
      "reason": "non_directional",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "de16d714-4404-87d9-b95b-91253dc05238",
      "direction": "QUALIFY",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DATASET_RESULT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 0
    },
    {
      "link": "e551b347-bf1f-8d00-a9a0-07ccd37b3c19",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "73e2a2be-3ea1-86ad-95d2-0244c2342df7",
      "direction": "SUPPORT",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.800000",
      "interpretive_steps": 0
    }
  ],
  "model": "ledger-default@0.3.0",
  "prior": "0.50",
  "variants": {
    "spread": "0.0653",
    "permissive": "0.9908",
    "conservative": "0.9255"
  },
  "code_hash": "sha256:c61a6fdf35a754a0d7e958df149036a083c0eaeb7d2d2e52704ec32b0c36def7",
  "contested": false,
  "stability": "HIGH",
  "claim_type": "COMPARATIVE",
  "config_hash": "sha256:86768038080b8ab5233f1e4b7aba10af05b5bb44caa432095150b90e81209809",
  "probability": "0.9734",
  "provisional": true,
  "evidence_sum": "3.600000",
  "snapshot_seq": 24301,
  "prior_log_odds": "0.000000",
  "support_groups": 2,
  "model_dependent": false,
  "review_coverage": "0.25",
  "truth_evaluable": true,
  "assessment_state": "SUPPORTED",
  "review_checklist": {
    "qualifiers_reviewed": {
      "by": [],
      "ok": false
    },
    "opposing_search_done": {
      "by": [],
      "ok": false
    },
    "independence_reviewed": {
      "by": [],
      "ok": false
    },
    "primary_source_reviewed": {
      "by": [
        "6ee6e27d-6b26-821f-a79c-f59f0fad4404",
        "cc0fef75-0aae-8429-b2cc-417f9bd30046",
        "e551b347-bf1f-8d00-a9a0-07ccd37b3c19"
      ],
      "ok": true
    }
  },
  "contradict_groups": 0,
  "rounding_boundary": false,
  "posterior_log_odds": "3.600000",
  "not_applicable_reason": null,
  "independence_unreviewed": 3
}