Evidence item

The Nature study reports strong GPT-4 correlations with observed effects, similar to pooled human forecasts, alongside systematic effect-size overestimation. DATASET_RESULT

Authenticity UNVERIFIED · extraction UNVERIFIED · independence group none (unreviewed) · created seq 18348

Source: Large language models can predict the results of social science experiments · locator {} · excerpt hash sha256:9bad3a24d6d2d7149f5984100c009b1c79351a8a80925d0d55b8a27e5c14f5ba

Predictions derived from GPT-4 ... were strongly correlated with actual treatment effects, achieving accuracy similar to pooled human forecasts. ... Despite high correlations, predictions systematically overestimated effect sizes.