가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Mia
@MiaAI_lab
Building with AI & LLMs // Running frontier open models locally
가입 July 2022
413 팔로잉 중    48.3K 팬
DeepSeek V4.1 Flash 1017 for 4x DGX Sparks got a lot of updates, with support for @NVIDIAAI's NVFP4, and a lot PRs/issues merged/fixed. NVIDIA's NVFP4 - Can be enabled by setting NVFP4=1. - 4-6% increase on prefill speeds. - A bit faster decode on multi-concurrency. ✨Faster loading - Each worker can load the model from its own disk instead of over the network (opt-in); reported load time from 315s to 96s. - The weight file is read ahead of the loader; reported node load time from 83s to 27s. ✨More reliable replies - A reply format rule that can't start now fails only its own request. - A failed first GPU launch leaves the cached graph safely in its cache. - JSON answers stay JSON instead of being mistaken for tool calls. - A reply containing only stray tool-call closing tags now gets a warning, not a guessed repair. ✨Safer scripts - Long-running background processes no longer hold on to the launcher's locks. - On switched networks, nodes prefer their shared fast network when connecting at startup. - ./stop.sh can now warn about requests still in progress, and the ./start.sh test needs a real model reply. - IPv6 addresses now work in the URLs ./start.sh checks and prints. - Preparation tools now run on the same image as the server. - ./prepare.sh now stops if indexing fails, instead of deleting the weight file. Updates also coming for 2x DGX Spark version! Thanks to shalk3, @SvangenStudios, @tfolkman, Christopher A. Williamson (cawilliamson), and Toshiaki Maki. Link below.
더 보기