SambaNova
@SambaNovaAI
SambaNova is the fastest AI inference platform — running AI securely, profitably, and at scale, with ROI received at a remarkable speed that matters⚡
参加 March 2018
165 フォロー中 13K ファン
The slow part is writing the answer, not reading the prompt.
That's why agentic inference needs a different division of labor. GPUs for prefill, SN50 for decode.
Great breakdown of Sven's talk at the @tetrateio Inference Routing Summit!
もっと見る
I thought long context was what made agents slow. At @tetrateio's Inference Routing Summit, Sven Mawson of @SambaNovaAI said agent sessions hit cache 90-95% of the time, so 80-95% of the time goes to decode.
The slow part is writing the answer, not reading the prompt.
もっと見る
