A (Chinese) Hot Take on AI benchmarks and how by being effectively US "owned," (developed and maintained by US-based institutions) they're kind of a strategic advantage: "... The AA leaderboard is America’s Star Wars program, a tool designed to disrupt the training of Chinese open-source AI models. American companies have an ingrained habit of setting standards, using the banner of a greater cause to establish standards organizations and impose industry entry requirements. If you’re a follower, you’re forced to expend enormous effort and exhaust your resources reshaping yourself to conform to their standards, just to earn their recognition. In practice, this effectively eliminates the possibility of open-source models catching up." I’m not sure it makes sense to argue that this necessarily handicaps open-source models. Open source can absolutely become the standard itself. It just hasn’t in this particular case. The more interesting question obviously is not just who gets to define the benchmarks but whether optimizing for them is actually the best use of Chinese labs’ limited resources