Terence Tao says the math behind today’s LLMs is actually simple. Training and running them mostly uses linear algebra, matrix multiplication, and a bit of calculus, material an undergraduate can handle. We understand how to build and operate these models. The real mystery is why they work so well on some tasks and fail on others, and why we cannot predict that in advance. We lack good rules for forecasting performance across tasks, so progress is largely empirical. A key reason is the nature of real-world data. Pure noise is well understood, perfectly structured data is well understood, but natural text sits in between, partly structured and partly random. Mathematics for that middle regime is thin, similar to how physics struggles at meso-scales between atoms and continua. Because of this gap, we can describe the mechanisms but cannot yet explain capability jumps or give reliable task-level predictions. That mismatch, simple machinery versus hard-to-predict behavior, is the core puzzle. ---- Video from 'Dr Brian Keating' YT Channel (Link in comment)

Some other predictions / insider views from Wu, who was invited to the dinner banquet: On trade / investment: We have demonstrated the willingness to further open up the Chinese market. Of course, the United States also needs to expand its openness to Chinese investment in the U.S. It is foreseeable that, in the period ahead, economic and trade relations will stop declining and stabilize and may even gradually enter an upward phase. On tech: In technology, of course we mainly have to rely on our own technological progress and accelerate breakthroughs. But there is also a kind of anxiety within the U.S. tech community. They have found that U.S. tech suppression against China has not crushed China. China has continued to achieve breakthroughs, which could very likely cause the United States to lose the Chinese market. These dynamics will force the U.S. side to ease and relax its tech restrictions on China. Otherwise, China’s market will have nothing to do with them in the future, which in turn would actually affect the development of U.S. technology. In fact, there are also discussions within the Trump administration about how to balance national interests and market competition, and how to strike the right balance. On banquet: The atmosphere was relaxed and friendly. To be honest, I have not experienced this kind of atmosphere in China-U.S. relations for many years. The last time I attended a similar event was in 2016, during the final round of the U.S.-China Strategic and Economic Dialogue under Obama. On my left was John Hiller, deputy assistant to the president at White House, and on my right was Bret Baier at Fox News. We discussed many issues together. Overall, the atmosphere felt relatively relaxed and pleasant. People interacted very naturally. I felt this was a scene rarely seen in many years. They were quite surprised: how did a university professor end up at an occasion like this? In their minds, these kinds of banquets are usually attended by government officials and business execs. When they heard I was from Shanghai, Baier said Shanghai is a wonderful place. His mother had told him he absolutely had to visit Shanghai, and that it is an astonishing city. I asked Baier how the U.S. media would cover this meeting. He gave three keywords: first, strength; second, stability and cooperation; and third, friendship, especially the personal friendship between Xi and Trump. He said these would be the main points and focus of their coverage. After Hiller learned that I was familiar with the United States, he said his daughter works at the CSIS and has a podcast, and that she could invite me to participate. We also talked about the U.S. midterm elections and how to view Trump’s visit to China and his interactions with the Chinese side. From food and U.S. domestic politics to China-U.S. relations, we talked about many things. The Americans are warm, outgoing, and like to communicate. When the Chinese and American people sit together, it is still relatively easy for them to get along. ... I also noticed one detail at the scene. During the banquet, in addition to classic Chinese and American pieces the final piece on the program was the American song “Y.M.C.A.,” which many Chinese internet users are also familiar with and which is known as Trump’s campaign anthem. When the Americans heard the music, they became excited... They immediately picked up their phones to film the band and started imitating Trump’s signature dance moves. After Trump finished his remarks, American business representatives and others all stood up and applauded. This had not happened before, and it also reflects changes in American political culture...

Fascinating footage released by the IRGC of a class at the org's staff college in the 90s, where future IRGC leader Hossein Salami teaches a course on asymmetric warfare, teaching officers how to drag out a war with the US by driving up economic costs and political turmoil.