Galedra
Sign in Sign up

Claim

In mid-2023, within an OpenAI project called RL Slow, Pachocki's team saw the first results showing that reasoning models could scale.

Type HISTORICAL · truth-evaluable: yes · status at snapshot: ACTIVE · created seq 2241

a comma-separated series may list several propositions

Referenced 99 times: checked 2, looked up by assistants 0, viewed 73, shared 24. A count of attention, not evidence.

Topics

AI · 03192ed8

Assessment

Supported

Score: 0.9734 under ledger-default@0.3.0 at snapshot 27745

3 of 4 review checks done · 3 counted

2 independent evidence lineages · review checks 3 of 4 · stability HIGH

Main unresolved issue: 4 counted evidence items have no independence review; repeats of one origin may be counted as several coverage:9e229a22-40df-818f-b566-5ed4e7640ae5

Not yet independently audited.A quoted passage from openai.com was not found on the page when Galedra fetched it.A quoted passage is in something this server does not read, such as a PDF or a recording, so it has not been checked here.

Why?

Strongest counted support: Pachocki writes that in mid-2023, within the RLSlow research project, they saw the first results. weight 1.800000 evidence

Review gaps: independence reviewed

What would most change this: One independent, direct measurement that contradicts this claim would move it from Supported to Supported.

Hide calculation

0.9734 under ledger-default@0.3.0 at snapshot 27745 · state Supported · Review checks: 3 of 4 · stability HIGH · 5 sources, 2 independent lineages

Model ledger-default@0.3.0 (the default model, not the answer) · snapshot 27745 · trace sha256:bc38f29926f0a29b49e109dec17e029a9c365caf8b4af1173c0a69c15181b1d8

Last changed at entry seq 10714; nothing logged since has borne on this claim.

How the number was reached

Each counted link is weighed by multiplying the factors the model declares; a factor it does not declare is not shown. Every factor name below links to what that number means.

LinkWorkingWeightCounted as
bb4601ac support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1 1.800000 1.800000
f818cb3a support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000
ba726c94 support relevance MODERATE 0.6 × observation HEARSAY 0.1 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000
c448bf9d qualify relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000 (non directional)
6c22fce1 support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1 1.800000 1.800000

prior 0.50 → log-odds 0.000000 + 3.600000 (the counted weights) = 3.600000 → probability 0.9734

Under neighbouring settings of the same model: 0.9255 to 0.9908, a spread of 0.0653.

Half-even rounding, six places for weights and four for the probability. The same inputs under the same model reproduce this exactly; that is what the trace hash above is for. How a score is worked out explains every factor, why it is there, and what the model sets it to.

Your view

Sign in to register whether you agree or disagree. A view is yours alone and never changes the assessment.

Registered views

Nobody has registered a view yet.

Registered views are personal belief, not evidence: they never change Galedra's assessment. Affiliations are self-declared and shown only for groups of 5 or more.

Checked as part of

Statements this claim was broken out of. The claim is scored on its own; a statement's reading only counts its claims.

Evidence for

Evidence against

None.

Qualifications

Suppressed as dependent

None.

Inferences

Concluded from

None.

Used as a premise in

None.

An inference is interpretation, not evidence: it never changes the assessment of its conclusion. Each premise is a claim with its own evidence.

Threads

None. Threads are for disagreements about how a determination was made — a statement whose figures are not in the passage it rests on, two sources that may share an origin. Something about the world belongs in the evidence above.

Claim edges

None. Edges are stored and displayed; v0.1 does not propagate probability across them.

Review checklist

Review checks: 3 of 4

2 checks by this claim's own author, alongside the independent review above. 1 QUALIFIER CHECK · 1 EVIDENCE VERIFICATION. A check by the person who recorded a claim does not raise its review coverage.

CheckDoneBy
qualifiers reviewedyes5802fa5f
opposing search doneyesa1233687
independence reviewedno
primary source reviewedyesbb4601ac, 6c22fce1

Summary

Supported. The main support is: Pachocki writes that in mid-2023, within the RLSlow research project, they saw the first results. 8d666a02

Qualifier: The passage says 'he' saw first results in mid-2023 in project RL Slow proving reasoning models could scale, but names neither Pachocki nor OpenAI. 3c646075

A search for evidence in the opposing direction found something; see the counted evidence. a1233687

Review checks: 3 of 4. coverage:9e229a22-40df-818f-b566-5ed4e7640ae5

Generated by stub-v0.1; every sentence cites graph ids.

Contribution history

seqeventdetail
2241APPENDED
2242ACCEPTAUTOMATIC_AFTER_VALIDATION
2429LINK_EVIDENCEby 03192ed8 (accepted)
4413LINK_EVIDENCEby 03192ed8 (accepted)
6628TASK_RESULTby 8b5e81bf (accepted)
8262TASK_RESULTby 1efd24fe (accepted)
10713TASK_RESULTby 1efd24fe (accepted)

Score trace

{
  "claim": "9e229a22-40df-818f-b566-5ed4e7640ae5",
  "links": [
    {
      "link": "182d7b63-5566-840d-b355-9dcebb4601ac",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "0b482ddd-16c5-865d-8d50-8fe08d666a02",
      "direction": "SUPPORT",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.800000",
      "interpretive_steps": 0
    },
    {
      "link": "5232ba01-57ef-8586-81f4-8f30f818cb3a",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "f658414c-a450-8070-8c51-3c07f933f315",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    },
    {
      "link": "6cc1e139-47c0-8a3a-94b6-0d26ba726c94",
      "group": "2e2a4412-5473-8129-992a-4b58484fd863",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "9b80388a-21a3-8cb4-9a8e-e01d9e0f0623",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "MODERATE",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "HEARSAY",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    },
    {
      "link": "77025742-61d7-8bd3-9763-d601c448bf9d",
      "group": null,
      "reason": "non_directional",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "d42e71ca-7d58-8c6c-b168-28823c646075",
      "direction": "QUALIFY",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    },
    {
      "link": "af957c57-21a7-8993-83cf-71bf6c22fce1",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "3004a96a-1c6d-83ce-b00e-e2a6a8e10434",
      "direction": "SUPPORT",
      "magnitude": "1.800000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "PRIMARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.800000",
      "interpretive_steps": 0
    }
  ],
  "model": "ledger-default@0.3.0",
  "prior": "0.50",
  "variants": {
    "spread": "0.0653",
    "permissive": "0.9908",
    "conservative": "0.9255"
  },
  "code_hash": "sha256:c61a6fdf35a754a0d7e958df149036a083c0eaeb7d2d2e52704ec32b0c36def7",
  "contested": false,
  "stability": "HIGH",
  "claim_type": "HISTORICAL",
  "config_hash": "sha256:86768038080b8ab5233f1e4b7aba10af05b5bb44caa432095150b90e81209809",
  "probability": "0.9734",
  "provisional": true,
  "evidence_sum": "3.600000",
  "snapshot_seq": 10714,
  "prior_log_odds": "0.000000",
  "support_groups": 2,
  "model_dependent": false,
  "review_coverage": "0.75",
  "truth_evaluable": true,
  "assessment_state": "SUPPORTED",
  "review_checklist": {
    "qualifiers_reviewed": {
      "by": [
        "01a0bc53-0d79-7b9c-8ff7-d9f45802fa5f"
      ],
      "ok": true
    },
    "opposing_search_done": {
      "by": [
        "01a0bc53-0d65-7c36-bbaa-971ea1233687"
      ],
      "ok": true
    },
    "independence_reviewed": {
      "by": [],
      "ok": false
    },
    "primary_source_reviewed": {
      "by": [
        "182d7b63-5566-840d-b355-9dcebb4601ac",
        "af957c57-21a7-8993-83cf-71bf6c22fce1"
      ],
      "ok": true
    }
  },
  "contradict_groups": 0,
  "rounding_boundary": false,
  "posterior_log_odds": "3.600000",
  "not_applicable_reason": null,
  "independence_unreviewed": 4
}