Epoch AI(@EpochAIResearch):OpenAI GPT-5.6 models and Anthropic Claude 5 models have different pricing structures at long context lengths. GPT model costs increase in price past 272k input tokens, while Claude model costs remain fixed. Does this reflect an underlying difference in the architecture of these models? Our measurements of serving latency suggest so. We studied time to first token (TTFT) on these models and how it scales with increasing context length. We found a significant difference in how they scale, with GPT showing a noticeable quadratic component, while Claude models remain closer to linear.

2026.09.08 21:12

OpenAI GPT-5.6 models and Anthropic Claude 5 models have different pricing structures at long context lengths. GPT model costs increase in price past 272k input tokens, while Claude model costs remain fixed. Does this reflect an underlying difference in the architecture of these models? Our measurements of serving latency suggest so. We studied time to first token (TTFT) on these models and how it scales with increasing context length. We found a significant difference in how they scale, with GPT showing a noticeable quadratic component, while Claude models remain closer to linear.