註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

elie
@eliebakouch
research @PrimeIntellect (prev: @huggingface) anon feedback:
加入 January 2024
4.8K 正在關注    26.7K 粉絲
we replicated anthropic jspace analysis on @thinkymachines Inkling new 1T model! it seems to be an outlier: where other models split into near-orthogonal sensory/workspace/motor blocks, inkling keeps roughly one geometry across the whole stack (early-late CKA ~0.8 vs ~0.5 elsewhere) we also look computed the J-space of @poolsideai's laguna XS 2.1 in bf16 vs nvfp4 to test the impact of quantization. result: almost none. the quantized model has the same jlens space as the non-quantized one both jspace checkpoint are up on hugging face!
顯示更多
0
40
1.2K
106
轉發到社區