From OpenAI to DeepSeek: New Scaling Laws for LLMs that can Reason

With o1, OpenAI ushered a new era: LLMs with reasoning capabilities. This new breed of models broadened the concept of scaling laws, shifting focus from train-time to inference-time compute. But how do these models work? What do we think their architectures look like, and what data do we use to train them? And finally - and perhaps more importantly: how expensive can they get, and what can we use them for?