注册并分享邀请链接,可获得视频播放与邀请奖励。

DigitalOcean
@digitalocean
The AI-Native Cloud. ☁️ Status: @DOstatus Support:
加入 January 2012
287 正在关注    226.8K 粉丝
A typical agentic session runs 43 turns with 142K tokens in and 444 tokens out. Serving that well comes down to the cache. @yifandotqiao of @Inferact (the team behind vLLM) on taking one card from 84K to 480K tokens a minute on @Kimi_Moonshot K3.
显示更多