๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
๊ฐ€์ž… March 2024
37 ํŒ”๋กœ์ž‰ ์ค‘    54.1K ํŒฌ
๐ŸŽ‰ Spark-X2.6-1.7B from @SparkLLM is out โ€” a compact on-device agent model with 1M native context and controllable thinking (none/low/high), nearly matching its 4B sibling on SWE-Verified and ฯ„ยฒ-bench. Day-0 vLLM support via trust-remote-code, incl. vLLM-Ascend for NPU deployment.
๋” ๋ณด๊ธฐ
Spark-X2.6-1.7B: less than half the parameters of our 4B, close on two benchmarks: SWE Verified: 59.8 vs 62.8 ฯ„ยฒ-bench: 85.3 vs 86.9 Both at high thinking. Try it & Like on HF! @huggingface @Ollama @ModelScope2022
๋” ๋ณด๊ธฐ