Among open-weight models, do the Chinese have the best pre-training? I measured how well LLMs compress fresh, unseen human text (bits-per-byte). Of Western models, only NVIDIA's Nemotron sits squarely in the frontier cluster (Thinking Machines' Inkling is just behind). You can't measure compression directly for closed models like Fable or Sol... commercial APIs block the log probs you'd need. And my attempts to estimate it by sampling were too noisy. One corollary of challenges estimating logprobs is that KL/log-prob distillation against a commercial API is very hard. Sampling can't recover the tails of the probability distribution, and hidden top-p sampling truncates them altogether. That bpb is where it is for frontier Chinese models suggests to me there is quite strong pretraining on raw - not synthetic - text. [Which is not to say there isn't also lots of synthetic data, in both Western and Chinese models.]