註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | 🔔 Follow for AI & Vibe Coding Tips 👇
加入 July 2023
549 正在關注    11.2K 粉絲
The more I look at setups like this, the more I think OCuLink changes the perspective on Local AI mini-PCs. That old RTX 2080 Super doesn't need to hold your giant LLM to be useful. Give the NVIDIA GPU the CUDA-friendly jobs like 👁️ vision models and 🎨 image generation and ... mini-PC's CPU/iGPU/system RAM handles the larger model or another workload when 8GB isn't enough.
顯示更多
Loving this OCuLink setup with the GMKtec EVO-T1 paired with an RTX 2080 Super. An Intel mini-PC gets dedicated Nvidia CUDA + 8GB of VRAM over PCIe 4.0 ×4 OCuLink. Local AI possibilities are endless! Could run separate models/workloads across the iGPU/CPU and NVIDIA GPU or experiment with model offloading, or even play with disaggregated inference!
顯示更多