John Carmack(@ID_AA_Carmack):If the target market is real time robotics, the Jetson Thor system, with 128 GB of memory, but only 273 GB/s of bandwidth, seems over-provisioned with expensive memory. You want the model evaluating at tens of fps, so it can't use more than 10 GB of weights at most. More memory could be used if the model is a wide mixture of experts, or a large strategic planning model is operating in small time slices over a longer period of time, and more memory always makes development life easier, but for cost optimized systems, you should be able to get by with much less.

John Carmack

@ID_AA_Carmack

AGI at Keen Technologies, former CTO Oculus VR, Founder Id Software and Armadillo Aerospace

参加 August 2010

295 フォロー中 4.1M ファン

John Carmack@ID_AA_Carmack

2026.09.10 22:23

If the target market is real time robotics, the Jetson Thor system, with 128 GB of memory, but only 273 GB/s of bandwidth, seems over-provisioned with expensive memory. You want the model evaluating at tens of fps, so it can't use more than 10 GB of weights at most. More memory could be used if the model is a wide mixture of experts, or a large strategic planning model is operating in small time slices over a longer period of time, and more memory always makes development life easier, but for cost optimized systems, you should be able to get by with much less.

1.2K

コミュニティへ転送

人気のあるユーザー