Ahmad
@TheAhmadOsman
Founder & CEO @OsmanticAI — Accelerating Opensource & Self-hosted / Local AI Adoption • I moderate GPUs on r/LocalLLaMA
가입 February 2011
464 팔로잉 중 78.2K 팬
YOU WANNA BOOKMARK THIS
The ultimate resource for running LLMs locally is now available online to read for free
Covers what to use on
- Laptop / edge / odd hardware
- Mac-first workflows
- Single RTX GPUs
- 2-4+ NVIDIA / CUDA GPUs
- General production serving
- Long-context / MoE / routing
- NVIDIA max performance
- Cluster orchestration
Software
- llama.cpp
- MLX / MLX-LM
- ExLlamaV2
- ExLlamaV3
- vLLM
- SGLang
- TensorRT-LLM
- NVIDIA Dynamo
You should read this, and if you cannot now then you most definitely wanna bookmark it for later
더 보기
