Suso Fontán
@susofontan
Software Architect.
Side quest: AI research & AI DevOps.
加入 December 2011
134 正在關注 85 粉絲
Implemented batching support for footless-orcasaq-2-27B-nvidia-p100. Benchmarks show 40 tok/s at concurrency = 4 running OrcaSAQ-2-27B (@OrcaRouter) on a single NVIDIA Tesla P100.
More updates to follow.
• Repo:
• Demo (pure prose):
顯示更多
