Register and share your invite link to earn from video plays and referrals.

DaRazor
@akramsrazor
Hyyypergrowth, SaaS, Digital money enthusiast, CEFI Skeptic, Oil bull, and Futurist. If you ain't long, you're wrong. Occasional Security Analysis.
Joined December 2011
741 Following    24.8K Followers
Christopher is getting so smart... Tokens Are Not Barrels of Oil A token is a logical event generated by an evolving software architecture. A cached input token, an uncached input token, a reasoning token, and an output token can have radically different physical costs. The same nominal token can cost differently depending on: · model size; · active parameter count; · architecture; · precision; · sequence length; · batch size; · hardware; · cache state; · serving software; · utilization. A system can generate more tokens while using less physical compute. A system can generate fewer tokens while costing more if the intervention destroys cache reuse, lengthens the trajectory, or shifts work toward more expensive outputs.
Show more