๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
๊ฐ€์ž… July 2023
549 ํŒ”๋กœ์ž‰ ์ค‘    11.2K ํŒฌ
The more I look at setups like this, the more I think OCuLink changes the perspective on Local AI mini-PCs. That old RTX 2080 Super doesn't need to hold your giant LLM to be useful. Give the NVIDIA GPU the CUDA-friendly jobs like ๐Ÿ‘๏ธ vision models and ๐ŸŽจ image generation and ... mini-PC's CPU/iGPU/system RAM handles the larger model or another workload when 8GB isn't enough.
๋” ๋ณด๊ธฐ
Loving this OCuLink setup with the GMKtec EVO-T1 paired with an RTX 2080 Super. An Intel mini-PC gets dedicated Nvidia CUDA + 8GB of VRAM over PCIe 4.0 ร—4 OCuLink. Local AI possibilities are endless! Could run separate models/workloads across the iGPU/CPU and NVIDIA GPU or experiment with model offloading, or even play with disaggregated inference!
๋” ๋ณด๊ธฐ