登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Ivan Fioravanti
@ivanfioravanti
LLM addicted, Apple MLX, NVIDIA DGX, Cloud, Kubernetes, Technology Advisor, Investor and Co-Founder / Board Member of CoreView. Local AI Ambassador.
参加 June 2009
1.5K フォロー中    45.3K ファン
The more I dig in the various inference engines out there, the more I see the shortcuts they take for speed. Don't assume that model layers are mapped 1-1 into your engine, some are quantizing before running, 8bit and even 4bit. I think I should post more on this, not to finger point anyone, but just to clarify that: more often than not, you bargain precision for speed. Kernel optimizations are great, but there is a physical limit (compute/memory bandwidth). There is no free lunch 🤷🏻‍♂️
もっと見る