I am proud of the work my team did in Munich in 1991, when compute was millions of times more expensive. We published the roots of today's trillion-dollar AI boom: ★ 3/1991: the first kind of Transformer (see the T in ChatGPT) - now called the unnormalized linear Transformer: the predecessor of the normalized quadratic Transformer ★ 4/1991: Pre-Training (the P in ChatGPT) & Neural Net Distillation (see DeepSeek and many other LLMs) ★ 6/1991: Deep Residual Learning, basis of LSTM & Highway Net / ResNet (most-cited AIs of their centuries) ★ 8/1991: conference paper on GANs for World Models trained by Artificial Curiosity ★ Around the same time, Munich also was the origin of the first self-driving cars in traffic (Ernst Dickmanns et al.), going up to 175 km/h. The city was truly the epicenter of AI. Read the timeline with links to the original references, featuring a preface by @hardmaru: