We’ve trained language models which can be significantly better at following user intentions than GPT-3 while also making them more truthful and fewer toxic, using techniques developed through our alignment research. These InstructGPT models, that are trained with humans within the loop, are actually deployed because the default language models on our API.