Sudo su(@sudoingX):for anyone starting in local ai, the first thing you do is buy hardware and install ollama or lm studio without thinking about what powers it underneath and what actually runs it the first instinct should be learning the stack, use them but transition to the layer beneath with llama.cpp and try to understand it, then keep buying gpus, move to vllm and unlock your tp= do not settle there, even try to tweak the model and use mtp and all to speed it up, and try kernels along the way once you understand inference from the foundational level you will never have to worry about getting a job ever again learn inference

2026.10.29 00:31

for anyone starting in local ai, the first thing you do is buy hardware and install ollama or lm studio without thinking about what powers it underneath and what actually runs it the first instinct should be learning the stack, use them but transition to the layer beneath with llama.cpp and try to understand it, then keep buying gpus, move to vllm and unlock your tp= do not settle there, even try to tweak the model and use mtp and all to speed it up, and try kernels along the way once you understand inference from the foundational level you will never have to worry about getting a job ever again learn inference

显示更多

709

转发到社区

热门用户