dealign.ai
@dealignai
i like breaking models -
参加 January 2024
64 フォロー中 5.5K ファン
Prefix caching now stores BY TOOL CALL instead of BY TURN allowing for dramatically speedier prefill speeds.
Memory allocator bug has been fixed to 8gb, compiled decode for hybrid and other models enabled by default. Will be focusing heavily in more optimizations where I can in the next few days.
もっと見る
