dealign.ai(@dealignai):Prefix caching now stores BY TOOL CALL instead of BY TURN allowing for dramatically speedier prefill speeds. Memory allocator bug has been fixed to 8gb, compiled decode for hybrid and other models enabled by default. Will be focusing heavily in more optimizations where I can in the next few days.

2026.09.01 04:53

Prefix caching now stores BY TOOL CALL instead of BY TURN allowing for dramatically speedier prefill speeds. Memory allocator bug has been fixed to 8gb, compiled decode for hybrid and other models enabled by default. Will be focusing heavily in more optimizations where I can in the next few days.

显示更多