가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

GDP
@bookwormengr
AI model & hardware co-design, Inference economics Safe super intelligence for all All views strictly personal. No investment advice, dummies!
가입 July 2010
13.3K 팔로잉 중    19.8K 팬
RSI via architecture-algorithm innovation vs RSI via data quality Lot of research is combining lot of existing ideas and trial and error. AI is super human at that. However, ideas are cheap and compute is still very expensive. Hence, I do not see RSI contributing much on architecture other than AI assisting great human researchers. For example, I am yet to see anything as elegant as Causal Encoder-Decider architecture or KV reuse that US labs arrived at with RSI. They have access to great models from OpenAi and Anthropic. Case in point: - Just look at KV Cache size of Inkling 2 of Thinking machine: 8.2 GB, atrocious compared to DeepSeek v4.1’s 0.8 GB for million tokens. - Its prefill also is twice as expensive as DeepSeek 4.1 Flash (17B active parameters vs 8B active parameters) My hypothesis is that most quality gains are from data and RL environment and the goals based on human needs. AI can do a lot in data quality space given how much data needs to be reviewed for quality; but its help in architecture and algorithm is limited by how many experiments we can conduct. RSI is really researcher with good taste + great models to curate data and environments. Yes, can also help with scanning thousands of research papers for ideas and help tweak optimizers or right better kernels based on performance profile. So it is indeed RI and not RSI. @teortaxesTex @tszzl @_xjdr @Guangxuan_Xiao
더 보기