Galedra
Sign in Sign up

Claim

Frontier labs' unreleased internal models are only a few months ahead of the models they have released publicly.

Type INTERPRETIVE · truth-evaluable: yes · status at snapshot: ACTIVE · created seq 1731

Referenced 156 times: checked 1, looked up by assistants 0, viewed 153, shared 2. A count of attention, not evidence.

Topics

AI · 03192ed8

Assessment

Supported

Score: 0.9843 under ledger-default@0.3.0 at snapshot 27745

2 of 4 review checks done · 2 counted

3 independent evidence lineages · review checks 2 of 4 · stability LOW

Main unresolved issue: 6 counted evidence items have no independence review; repeats of one origin may be counted as several coverage:12aba7bf-dfeb-8be5-ba0b-fbceaa373ec5

Not yet independently audited.This assessment depends heavily on modeling choices.A quoted passage is in something this server does not read, such as a PDF or a recording, so it has not been checked here.

Related: NARROWS incoming: Leans supported

Why?

Strongest counted support: Internal frontier models at four major labs were only about two months ahead of public releases. weight 1.530000 evidence

Suppressed as dependent: The excerpt says internal frontier models are only a few months ahead, while also stating recursive self-improvement is here.

Review gaps: independence reviewed, primary source reviewed

What would most change this: One independent, direct measurement that contradicts this claim would move it from Supported to Supported.

Hide calculation

0.9843 under ledger-default@0.3.0 at snapshot 27745 · state Supported · Review checks: 2 of 4 · stability LOW · 6 sources, 3 independent lineages

Model ledger-default@0.3.0 (the default model, not the answer) · snapshot 27745 · trace sha256:d50b28f2952ca0a7c91c8d05fcdca4d425e8336879ee3f4b4a16c8a64e47042f

Last changed at entry seq 10367; nothing logged since has borne on this claim.

How the number was reached

Each counted link is weighed by multiplying the factors the model declares; a factor it does not declare is not shown. Every factor name below links to what that number means.

LinkWorkingWeightCounted as
fead29a0 support relevance STRONG 1.2 × observation DIRECT_TEXT 0.9 × interpretation 0 steps, 1 − 0.15 × 0 1.000000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1 1.080000 1.080000
cfee36f2 support relevance WEAK 0.2 × observation EXPERT_ANALYSIS 0.3 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000
f0428711 qualify relevance MODERATE 0.6 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000 (non directional)
611cf127 support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1 1.530000 1.530000
bbb1814b support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance INDEPENDENT 1.0 × edition NAMED_OR_UNRELATED 1 1.530000 1.530000
257eb5f5 support relevance DIRECT 2.0 × observation DIRECT_TEXT 0.9 × interpretation 1 step, 1 − 0.15 × 1 0.850000 × authenticity UNVERIFIED 1.0 × extraction UNVERIFIED 1.0 × provenance SELF 0.0 × edition NAMED_OR_UNRELATED 1 0.000000 0.000000 (dependent strongest only, kept cfee36f2)

prior 0.50 → log-odds 0.000000 + 4.140000 (the counted weights) = 4.140000 → probability 0.9843

Under neighbouring settings of the same model: 0.9477 to 0.9954, a spread of 0.0477.

Half-even rounding, six places for weights and four for the probability. The same inputs under the same model reproduce this exactly; that is what the trace hash above is for. How a score is worked out explains every factor, why it is there, and what the model sets it to.

Your view

Sign in to register whether you agree or disagree. A view is yours alone and never changes the assessment.

Registered views

Nobody has registered a view yet.

Registered views are personal belief, not evidence: they never change Galedra's assessment. Affiliations are self-declared and shown only for groups of 5 or more.

Checked as part of

Statements this claim was broken out of. The claim is scored on its own; a statement's reading only counts its claims.

Evidence for

Evidence against

None.

Qualifications

Suppressed as dependent

EvidenceGroupKept instead
9f8b2be9cfee36f2

Inferences

Concluded from

None.

Used as a premise in

None.

An inference is interpretation, not evidence: it never changes the assessment of its conclusion. Each premise is a claim with its own evidence.

Threads

None. Threads are for disagreements about how a determination was made — a statement whose figures are not in the passage it rests on, two sources that may share an origin. Something about the world belongs in the evidence above.

Claim edges

FromRelationshipTo
ac095bbcNARROWSaa373ec5

Review checklist

Review checks: 2 of 4

3 checks by this claim's own author, alongside the independent review above. 1 OPPOSING EVIDENCE SEARCH · 1 QUALIFIER CHECK · 1 EVIDENCE VERIFICATION. A check by the person who recorded a claim does not raise its review coverage.

CheckDoneBy
qualifiers reviewedyescb6b9cc9
opposing search doneyes50a696c3
independence reviewedno
primary source reviewedno

Summary

Supported. The main support is: AI Frontiers reports that major labs keep models internal for months before public release during testing and refinement. 8d713367

1 further item repeats the same origin and is not independent evidence. 9f8b2be9

Qualifier: The speaker says 'they' are only a few months ahead, hedged with 'I think', but names no frontier labs and no internal versus public models. f9ce8158

A narrower version of this claim is leans supported. ac095bbc

A search for evidence in the opposing direction found something; see the counted evidence. 50a696c3

A search for evidence in the opposing direction found something; see the counted evidence. 50a696c3

Generated by stub-v0.1; every sentence cites graph ids.

Contribution history

seqeventdetail
1731APPENDED
1732ACCEPTAUTOMATIC_AFTER_VALIDATION
1801LINK_EVIDENCEby 03192ed8 (accepted)
6474TASK_RESULTby 8b5e81bf (accepted)
7355TASK_RESULTby 8b5e81bf (accepted)
8054TASK_RESULTby 1efd24fe (accepted)
10366TASK_RESULTby 1efd24fe (accepted)

Score trace

{
  "claim": "12aba7bf-dfeb-8be5-ba0b-fbceaa373ec5",
  "links": [
    {
      "link": "08fa6a44-3fed-8aea-b76c-806dfead29a0",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "94385054-bbd8-8c98-b1ac-4a1d8d713367",
      "direction": "SUPPORT",
      "magnitude": "1.080000",
      "relevance": "STRONG",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "SECONDARY_TEXT",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.080000",
      "interpretive_steps": 0
    },
    {
      "link": "1f04e806-6e29-8ce7-aa62-5d43cfee36f2",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "588050d4-85d9-8883-860e-ef29687743d2",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "WEAK",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "EXPERT_ANALYSIS",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    },
    {
      "link": "31bfbe62-63db-86ef-8206-7371f0428711",
      "group": null,
      "reason": "non_directional",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "3cb0a62c-5ad1-82c3-bd3c-cc17f9ce8158",
      "direction": "QUALIFY",
      "magnitude": "0.000000",
      "relevance": "MODERATE",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    },
    {
      "link": "62ea7c60-94fd-8ed2-8c98-591d611cf127",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "86e2dd72-19f6-8795-9ae5-f63b68b0507b",
      "direction": "SUPPORT",
      "magnitude": "1.530000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "WEBSITE",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.530000",
      "interpretive_steps": 1
    },
    {
      "link": "780094f4-d0d6-8573-afb2-ec16bbb1814b",
      "group": null,
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "fa910d98-d049-80b1-8545-bb4893d0fa88",
      "direction": "SUPPORT",
      "magnitude": "1.530000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "INDEPENDENT",
      "observation": "DIRECT_TEXT",
      "source_type": "WEBSITE",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "1.530000",
      "interpretive_steps": 1
    },
    {
      "kept": "1f04e806-6e29-8ce7-aa62-5d43cfee36f2",
      "link": "d840454c-5dcf-8901-a519-67ce257eb5f5",
      "group": null,
      "reason": "dependent_strongest_only",
      "edition": "NAMED_OR_UNRELATED",
      "evidence": "5e17d006-08dd-829e-89eb-c7ba9f8b2be9",
      "direction": "SUPPORT",
      "magnitude": "0.000000",
      "relevance": "DIRECT",
      "extraction": "UNVERIFIED",
      "provenance": "SELF",
      "observation": "DIRECT_TEXT",
      "source_type": "VIDEO",
      "authenticity": "UNVERIFIED",
      "audit_confirmed": false,
      "effective_weight": "0.000000",
      "interpretive_steps": 1
    }
  ],
  "model": "ledger-default@0.3.0",
  "prior": "0.50",
  "variants": {
    "spread": "0.0477",
    "permissive": "0.9954",
    "conservative": "0.9477"
  },
  "code_hash": "sha256:c61a6fdf35a754a0d7e958df149036a083c0eaeb7d2d2e52704ec32b0c36def7",
  "contested": false,
  "stability": "LOW",
  "claim_type": "INTERPRETIVE",
  "config_hash": "sha256:86768038080b8ab5233f1e4b7aba10af05b5bb44caa432095150b90e81209809",
  "probability": "0.9843",
  "provisional": true,
  "evidence_sum": "4.140000",
  "snapshot_seq": 10367,
  "prior_log_odds": "0.000000",
  "support_groups": 3,
  "model_dependent": true,
  "review_coverage": "0.50",
  "truth_evaluable": true,
  "assessment_state": "SUPPORTED",
  "review_checklist": {
    "qualifiers_reviewed": {
      "by": [
        "01a0bc4e-1129-7b3e-b49b-804dcb6b9cc9"
      ],
      "ok": true
    },
    "opposing_search_done": {
      "by": [
        "01a0bc4e-1115-7143-9476-28cf50a696c3"
      ],
      "ok": true
    },
    "independence_reviewed": {
      "by": [],
      "ok": false
    },
    "primary_source_reviewed": {
      "by": [],
      "ok": false
    }
  },
  "contradict_groups": 0,
  "rounding_boundary": false,
  "posterior_log_odds": "4.140000",
  "not_applicable_reason": null,
  "independence_unreviewed": 6
}