註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Benjamin Marie
@bnjmn_marie
Independent AI researcher (AI, LLM). Support my work:
加入 June 2019
227 正在關注    7.7K 粉絲
It's great to see custom inference engines becoming increasingly popular, but be careful about output quality. I ran some experiments last week and found how easy it is to get Astra or Opus to optimize an inference engine for a specific model or GPU, with impressive speedups. But LLMs can also take questionable shortcuts. Many of the speedups I observed came from much more aggressive arithmetic approximations. These approximations aren't necessarily bad, and output quality can remain excellent. Just make sure you validate it yourself.
顯示更多
0
12
72
4
轉發到社區