Galedra
Sign in Sign up

Claim

OpenAI's new mathematics model doubles the solving rate on open mathematics problems.

Type QUANTITATIVE · truth-evaluable: yes · status at snapshot: ACTIVE · created seq 1925

Referenced 114 times: checked 2, looked up by assistants 0, viewed 88, shared 24. A count of attention, not evidence.

Topics

AI · 03192ed8 Mathematics · 03192ed8

Assessment

Supported

Score: 0.8581 under ledger-default@0.3.0 at snapshot 27745

2 of 4 review checks done · 2 counted

1 independent evidence lineage · review checks 2 of 4 · stability MEDIUM

Main unresolved issue: 5 counted evidence items have no independence review; repeats of one origin may be counted as several coverage:e295a73a-682f-87c8-bc41-aed6aa369055

Not yet independently audited.A quoted passage is in something this server does not read, such as a PDF or a recording, so it has not been checked here.

Why?

Strongest counted support: The report says an internal model raised its solve rate on curated open math problems from about 10–15% to 25–45%. weight 1.800000 evidence

Suppressed as dependent: Speaker cites a chart showing it doubles the solving rate on open mathematics.; The excerpt says a chart shows the model doubles the solving rate on open mathematics problems.

Review gaps: independence reviewed, primary source reviewed

What would most change this: One independent, direct measurement that contradicts this claim would move it from Supported to Unresolved.

Hide calculation

0.8581 under ledger-default@0.3.0 at snapshot 27745 · state Supported · Review checks: 2 of 4 · stability MEDIUM · 5 sources, 1 independent lineage

Model ledger-default@0.3.0 (the default model, not the answer) · snapshot 27745 · trace sha256:bfb095705bd20a1cf801dacdae2ae276719edb1f50a44446f5fc623b11ac8607

Last changed at entry seq 9456; nothing logged since has borne on this claim.

How the number was reached

Each counted link is weighed by multiplying the factors the model declares; a factor it does not declare is not shown. Every factor name below links to what that number means.

LinkWorkingWeightCounted as
4be9709a support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1 1.800000 1.800000
2aac2772 support relevance WEAK 0.2 × observation HEARSAY 0.1 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000 (dependent strongest only, kept dede0e02)
dede0e02 support relevance WEAK 0.2 × observation HEARSAY 0.1 × interpretation 2 steps, 1 − 0.15 × 2 0.700000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000
65f6717a qualify relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000 (non directional)
d89767fb support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000 (dependent strongest only, kept dede0e02)

prior 0.50 → log-odds 0.000000 + 1.800000 (the counted weights) = 1.800000 → probability 0.8581

Under neighbouring settings of the same model: 0.7790 to 0.9121, a spread of 0.1331.

Half-even rounding, six places for weights and four for the probability. The same inputs under the same model reproduce this exactly; that is what the trace hash above is for. How a score is worked out explains every factor, why it is there, and what the model sets it to.

Your view

Sign in to register whether you agree or disagree. A view is yours alone and never changes the assessment.

Registered views

Nobody has registered a view yet.

Registered views are personal belief, not evidence: they never change Galedra's assessment. Affiliations are self-declared and shown only for groups of 5 or more.

Checked as part of

Statements this claim was broken out of. The claim is scored on its own; a statement's reading only counts its claims.

Evidence for

Evidence against

None.

Qualifications

Suppressed as dependent

EvidenceGroupKept instead
d8e5279ddede0e02
0d73d9dcdede0e02

Inferences

Concluded from

None.

Used as a premise in

None.

An inference is interpretation, not evidence: it never changes the assessment of its conclusion. Each premise is a claim with its own evidence.

Threads

None. Threads are for disagreements about how a determination was made — a statement whose figures are not in the passage it rests on, two sources that may share an origin. Something about the world belongs in the evidence above.

Claim edges

None. Edges are stored and displayed; v0.1 does not propagate probability across them.

Review checklist

Review checks: 2 of 4

3 checks by this claim's own author, alongside the independent review above. 1 OPPOSING EVIDENCE SEARCH · 1 QUALIFIER CHECK · 1 EVIDENCE VERIFICATION. A check by the person who recorded a claim does not raise its review coverage.

CheckDoneBy
qualifiers reviewedyes6b11234f
opposing search doneyesb5df8525
independence reviewedno
primary source reviewedno

Summary

Supported. The main support is: The report says an internal model raised its solve rate on curated open math problems from about 10–15% to 25–45%. fa795847

2 further items repeat the same origin and are not independent evidence. d8e5279d0d73d9dc

Qualifier: The excerpt says the solving rate doubles but never states the baseline it doubles relative to. 0ffb7977

A search for evidence in the opposing direction found none. b5df8525

A search for evidence in the opposing direction found none. b5df8525

Review checks: 2 of 4. coverage:e295a73a-682f-87c8-bc41-aed6aa369055

Generated by stub-v0.1; every sentence cites graph ids.

Contribution history

seqeventdetail
1925APPENDED
1926ACCEPTAUTOMATIC_AFTER_VALIDATION
2105LINK_EVIDENCEby 03192ed8 (accepted)
3812TASK_RESULTby 03192ed8 (accepted)
6748TASK_RESULTby 8b5e81bf (accepted)
7894TASK_RESULTby 1efd24fe (accepted)
9455LINK_EVIDENCEby 1efd24fe (accepted)

Score trace

{
  "claim": "e295a73a-682f-87c8-bc41-aed6aa369055",
  "links": [
    {
      "link": "0d39e59b-fc1f-8c76-a66b-7a034be9709a",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "9ccfbb16-2587-89a7-a911-5f2dfa795847",
      "direction": "SUPPORT",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "SECONDARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.800000",
      "interpretive_steps": 0
    },
    {
      "kept": "631f0902-1c6a-82f5-a6a6-b214dede0e02",
      "link": "10a7cda8-df99-820b-8115-d5932aac2772",
      "group": null,
      "reason": "dependent_strongest_only",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "275c3876-4833-8ee8-8eeb-0d3cd8e5279d",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "WEAK",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "HEARSAY",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    },
    {
      "link": "631f0902-1c6a-82f5-a6a6-b214dede0e02",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "f8b0edef-222f-8827-aa72-be6f845154fc",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "WEAK",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "HEARSAY",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 2
    },
    {
      "link": "c816fa44-ce5e-8efd-a9c1-1ec365f6717a",
      "group": null,
      "reason": "non_directional",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "d7309f77-83de-89e8-98fe-3abf0ffb7977",
      "direction": "QUALIFY",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 0
    },
    {
      "kept": "631f0902-1c6a-82f5-a6a6-b214dede0e02",
      "link": "eb699caf-9ae6-8d74-824f-b625d89767fb",
      "group": null,
      "reason": "dependent_strongest_only",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "00fce081-b248-8ae3-afb8-086d0d73d9dc",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    }
  ],
  "model": "ledger-default@0.3.0",
  "prior": "0.50",
  "variants": {
    "spread": "0.1331",
    "permissive": "0.9121",
    "conservative": "0.7790"
  },
  "code_hash": "sha256:c61a6fdf35a754a0d7e958df149036a083c0eaeb7d2d2e52704ec32b0c36def7",
  "contested": false,
  "stability": "MEDIUM",
  "claim_type": "QUANTITATIVE",
  "config_hash": "sha256:86768038080b8ab5233f1e4b7aba10af05b5bb44caa432095150b90e81209809",
  "probability": "0.8581",
  "provisional": true,
  "evidence_sum": "1.800000",
  "snapshot_seq": 9456,
  "prior_log_odds": "0.000000",
  "support_groups": 1,
  "model_dependent": false,
  "review_coverage": "0.50",
  "truth_evaluable": true,
  "assessment_state": "SUPPORTED",
  "review_checklist": {
    "qualifiers_reviewed": {
      "by": [
        "01a0bc50-b37d-7643-87c3-778a6b11234f"
      ],
      "ok": true
    },
    "opposing_search_done": {
      "by": [
        "01a0bc50-b36b-7b0c-bf1d-30d0b5df8525"
      ],
      "ok": true
    },
    "independence_reviewed": {
      "by": [],
      "ok": false
    },
    "primary_source_reviewed": {
      "by": [],
      "ok": false
    }
  },
  "contradict_groups": 0,
  "rounding_boundary": false,
  "posterior_log_odds": "1.800000",
  "not_applicable_reason": null,
  "independence_unreviewed": 5
}