The Key Thing Human Brains Have That AI Is Trying To Learn

The gist

1. Humans learn new skills from a few examples, while current AI needs tens of thousands of data points, because humans mentally simulate outcomes before acting.
2. Reinforcement learning breaks down in complex tasks like self-driving, where there are around 365,000 possible actions versus only 361 in the game of Go.
3. Video diffusion models can be fine-tuned with a small amount of action-labeled data to generate synthetic training rollouts, reducing the need for real-world demonstrations.
4. Training robot policies on imagined rollouts from a learned world model, as in the Dreamer framework, can outperform training on limited real data.