Evidence item

The primary Nature article directly reports strong correlation with actual treatment effects, accuracy similar to pooled human forecasts, and systematic overestimation of effect sizes. DIRECT_TEXT

Authenticity UNVERIFIED · extraction UNVERIFIED · independence group none (unreviewed) · created seq 24300

Source: Large language models can predict the results of social science experiments · locator {} · excerpt hash sha256:0b460bb4329c9bbbf43d2f81e736fd33739edc4199e6b577d2fa7e2930c54b8b

Predictions derived from GPT-4, whose training-data cutoff predated the publication of many studies in our archive, were strongly correlated with actual treatment effects, achieving accuracy similar to pooled human forecasts. ... Despite high correlations, predictions systematically overestimated effect sizes.