During my first 2.5 years at OpenAI, I worked on the Robotics team on a moonshot idea: we desired to teach a single, human-like robot hand to resolve Rubik’s cube. It was a tremendously exciting, difficult, and emotional experience. We solved the challenge with deep reinforcement learning (RL), crazy amounts of domain randomization, and no real-world training data. More importantly, we conquered the challenge as a team.

From simulation and RL training to vision perception and hardware firmware, we collaborated so closely and cohesively. It was an incredible experiment and through that point, I often considered Steve Jobs’ reality distortion field: if you consider in something so strongly and carry on pushing it so persistently, someway you possibly can make the unattainable possible.

For the reason that starting of 2021, I began leading the Applied AI Research team. Managing a team presents a distinct set of challenges and requires working style changes. I’m most happy with several projects related to language model safety inside Applied AI:

1. We designed and constructed a set of evaluation data and tasks to evaluate the tendency of pre-trained language models to generate hateful, sexual, or violent content.
2. We created an in depth taxonomy and built a powerful classifier to detect unwanted content in addition to the rationale why the content is inappropriate.
3. We’re working on various techniques to make the model less prone to generate unsafe outputs.

Because the Applied AI team is practicing the most effective strategy to deploy cutting-edge AI techniques, equivalent to large pre-trained language models, we see how powerful and useful they’re for real-world tasks. We’re also aware of the importance of safely deploying the techniques, as emphasized in our Charter.