It's been 7 years from GPT-2 to GPT-6, and a lot has changed. The diagram below shows each generation and its key highlights, based on the public information we know. GPT-2 (2019): More like a prototype to validate the potential of the decoder-only transformer. This was a 1.5B-parameter model that only went through pre-training with next-token prediction. The trained model was only capable of continuing text, not answering questions. GPT-3 (2020): The same recipe as GPT-2, just much bigger and trained on more data to test how far scale alone could go. GPT-3 had 175B parameters, more than 100 times the size of GPT-2, and still only went through pre-training. The scale allowed it to show in-context learning capabilities. When shown some examples, the model could follow them. GPT-4 (2023): On top of pre-training, OpenAI fine-tuned it with RLHF (reinforcement learning from human feedback), so it was not just a completion model but could actually answer questions. It was also multimodal, meaning that it could process input images as well as text, and it handled up to 32K tokens of context. GPT-5 (2025): The first GPT generation with built-in reasoning. Given a prompt, tools, and an effort level, the trained model could generate reasoning tokens before the answer. Its context window grew to 400K tokens, with a router in ChatGPT to decide when a prompt needed thinking mode. GPT-6 (2026): This new model has a bigger context window (1.05M), supports async tool calling, and is generally safer. An interesting feature, called mid-turn steering, is that it can take a new instruction while it's thinking.

It's been 7 years from GPT-2 to GPT-6, and a lot has changed. The diagram below shows each generation and its key highlights, based on the public information we know. GPT-2 (2019): More like a prototype to validate the potential of the decoder-only transformer. This was a 1.5B-parameter model that only went through pre-training with next-token prediction. The trained model was only capable of continuing text, not answering questions. GPT-3 (2020): The same recipe as GPT-2, just much bigger and trained on more data to test how far scale alone could go. GPT-3 had 175B parameters, more than 100 times the size of GPT-2, and still only went through pre-training. The scale allowed it to show in-context learning capabilities. When shown some examples, the model could follow them. GPT-4 (2023): On top of pre-training, OpenAI fine-tuned it with RLHF (reinforcement learning from human feedback), so it was not just a completion model but could actually answer questions. It was also multimodal, meaning that it could process input images as well as text, and it handled up to 32K tokens of context. GPT-5 (2025): The first GPT generation with built-in reasoning. Given a prompt, tools, and an effort level, the trained model could generate reasoning tokens before the answer. Its context window grew to 400K tokens, with a router in ChatGPT to decide when a prompt needed thinking mode. GPT-6 (2026): This new model has a bigger context window (1.05M), supports async tool calling, and is generally safer. An interesting feature, called mid-turn steering, is that it can take a new instruction while it's thinking.

LLMs are not the only way to model intelligence. Many strong voices believe the future of intelligence lies in other routes, like energy-based modeling. We spoke with @evelovesolive, founder and CEO of Logical Intelligence, to understand how energy-based reasoning models (EBRMs) work and what kinds of problems are well suited for them. In this deep dive, we'll explore: - Why LLMs struggle with problems like Sudoku - What is an energy-based model - How energy-based reasoning works - What EBRMs can and cannot do - How EBRMs handle verification and safety Read full deep dive here: