filipe
@filicroval
data eng | 1xAsus Ascent GX10 | benchmarking local models so you don't have to
๊ฐ์
May 2023
214 ํ๋ก์ ์ค 153.4K ํฌ
Benchmarking @NVIDIAAI's Nemotron Puzzle 75B locally on the GX10.
NVFP4 via vLLM's OpenAI API, MTP speculative decoding, forced 1,500-token generations.
๐โโ๏ธ22.75 tok/s in a single session into 88.85 cumulative at 7 sessions.
๐งBaseline without MTP: ~16.3.
Scripts + setup:
๋ ๋ณด๊ธฐ
