Part 4 of my TensorFold 1.0 test: long prompts. Does
@ashxhart's new native engine hold up at 128k tokens?
Same GB10 box (Lenovo PGX), Nemotron 3.5 Lightning, real book text from Project Gutenberg, old Python engine 0.6.6 vs 1.0.1, median of 3 different book passages per length:
decode at 8k: 118.7 → 138.0 tok/s
decode at 32k: 110.5 → 124.0 tok/s
decode at 64k: 97.5 → 111.8 tok/s
decode at 128k: 85.9 → 96.2 tok/s
So 1.0.1 decodes 12–16% faster at every length (short prompts in part 1: 1–9%). Prefill is a tie: 0.6.6 is up to 2% faster (a 128k prompt takes 31.8 s vs 32.3 s to the first token).
And all 12 answers to these long prompts are bit-identical between the two engines, from 8k to 128k.
One limit: with 4 long prompts (32k each) at once, both engines read them one after another. The 4th request waits about 22–24 s for its first token on either engine, and all 4 finish in about 24 s.