The more I look at setups like this, the more I think OCuLink changes the perspective on Local AI mini-PCs.
That old RTX 2080 Super doesn't need to hold your giant LLM to be useful.
Give the NVIDIA GPU the CUDA-friendly jobs like đī¸ vision models and đ¨ image generation and ...
mini-PC's CPU/iGPU/system RAM handles the larger model or another workload when 8GB isn't enough.
Loving this OCuLink setup with the GMKtec EVO-T1 paired with an RTX 2080 Super. An Intel mini-PC gets dedicated Nvidia CUDA + 8GB of VRAM over PCIe 4.0 Ã4 OCuLink. Local AI possibilities are endless!
Could run separate models/workloads across the iGPU/CPU and NVIDIA GPU or experiment with model offloading, or even play with disaggregated inference!