Ahmad
@TheAhmadOsman
Founder & CEO @OsmanticAI — Accelerating Opensource & Self-hosted / Local AI Adoption • I moderate GPUs on r/LocalLLaMA
参加 February 2011
464 フォロー中 78.2K ファン
YOU WANNA BOOKMARK THIS
The ultimate resource for running LLMs locally is now available online to read for free
Covers what to use on
- Laptop / edge / odd hardware
- Mac-first workflows
- Single RTX GPUs
- 2-4+ NVIDIA / CUDA GPUs
- General production serving
- Long-context / MoE / routing
- NVIDIA max performance
- Cluster orchestration
Software
- llama.cpp
- MLX / MLX-LM
- ExLlamaV2
- ExLlamaV3
- vLLM
- SGLang
- TensorRT-LLM
- NVIDIA Dynamo
You should read this, and if you cannot now then you most definitely wanna bookmark it for later
もっと見る
