Moonshot run above 90% cache hit rate on their own coding traffic. Almost everyone building on that same API sits at zero. Same rate card, a tenfold difference on the invoice. Eight rules, and one of them is about where the date lives.
They published the whole mechanism.
The idea is the opposite of the usual one. A prompt is not text for the model. It is a byte sequence, compared left to right and dropped at the first difference.
Eight rules. Every one about placement, not wording:
> SYSTEM PROMPT - frozen and versioned. Not one dynamic value
> DOCUMENTS - top, once, byte for byte
> HISTORY - append only. Sliding windows are banned
> TOOLS - declared once, new ones appended at the end
> RAG CHUNKS - bottom, next to the question, in a fixed sort order
> DATE - last message only, rounded to the day
> REASONING - returned into history whole, exactly as it came back
> cached_tokens - logged on every call, or you know nothing
Seven rules about where things sit. The eighth is why the other seven are verifiable at all.
The whole design is one idea: the top of your prompt holds only what you refuse to change. A prompt that moves up top gets repriced in full. Every request.
Measured on a live run: one agent, 30 steps, 1.9M input tokens. Missing the cache, $6.08. Hitting it, $1.22. A thousand runs a day and the gap is $1.77M a year.
Same model. Same prompt in meaning. Answer quality never moved.
And the uncomfortable part: Moonshot's own multi-turn guide tells you to keep the last 20 messages so you do not blow the context window. Right for the window, catastrophic for the bill. Drop one message off the front and the prefix shifts, never to match again.
Five years of tuning prompts for the answer. The invoice was decided by line order the whole time, and nobody touched it.
The article below is the full build: eight rules with code, an audit prompt that finds every leak in your pipeline in one pass, and the dollar math on three workloads.
Save it. You will want it open in the other tab when you go into your prompt builder.
もっと見る