Evidence item

OpenAI describes language models as pretrained on internet text and distinguishes that objective from human instruction-following. DIRECT_TEXT

Authenticity UNVERIFIED · extraction UNVERIFIED · independence group none (unreviewed) · created seq 10485

Source: Training language models to follow instructions with human feedback · locator {} · excerpt hash sha256:20c073da5d5d25b8684b56d6d28755616b87567c64c513b95f7611cf5f930ea2

The language modeling objective used for many recent large LMs—predicting the next token on a webpage from the internet—is different from the objective “follow the user’s instructions helpfully and safely”.