Yuandong Tian
@tydsh
Co-founder of @Recursive_SI. ex-Meta FAIR Director. ex-Google. Reasoning, Optimization and Understanding LLM. Novelist in spare time. PhD in @CMU_Robotics.
가입 December 2009
950 팔로잉 중 46.4K 팬
We introduce TAPA ( that decouples magnitude from angular contributions in position encoding. TAPA yields better OOD (long-context) performance than vanilla RoPE approach. We also provide theoretical analysis why it works.
Thanks @yusidwang and the colleagues for the great work!
더 보기
We’d like to introduce our paper on long-context positional encoding, centered on a simple principle:
