๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Misha Goin
@mgoin_
maintainer @vllm_project, inference perf @RedHat_AI (acq @neuralmagic)
๊ฐ€์ž… April 2022
487 ํŒ”๋กœ์ž‰ ์ค‘    3.1K ํŒฌ
the strongest open model yet runs at 800+ tok/s/user on VR200 vLLM *on day 0* canโ€™t wait to see how much faster we make it over the coming weeks ๐Ÿ”ฅ