Why can a longer prompt make every token more expensive? Google's Gemini 3.1 Pro Preview charges $2 per million input tokens for prompts up to 200,000 tokens. Above that threshold, the rate becomes $4. Same model. Twice the price per input token. A longer prompt gives the model more to process. It also gives it more to keep in memory. While generating your answer, the model stores information about the conversation so it can reuse earlier work. That store is called the KV cache. The longer the context, the larger it generally becomes. This uses limited fast memory on the hardware. Each request needs memory for its conversation, and a request that takes more space can leave less room for other requests to run alongside it. Think of a restaurant. A large party takes more seats, so there is room for fewer other customers. Longer conversations create a similar capacity tradeoff on AI hardware. Memory is only part of the cost. Longer context also makes attention more expensive, and providers decide how those costs turn into prices. The 200,000-token threshold is a pricing boundary, not the point where the hardware suddenly runs out of room. An AI request buys more than computation. You are also renting room on the machine, for as long as the model is writing your answer.