GPU prices have actually gone up over the last handful of months.
As you scale inference aggressively, especially with open-source models, being more GPU-efficient is a big win.
This is an awesome post by our Decagon Labs team about how we achieved some of these gains!
もっと見る
