PyTorch-native NeMo AutoModel handles transformer pretraining in @nvidia's end-to-end workflow for building a transaction foundation model. The workflow combines GPU-accelerated data processing and tokenization, decoder-only model pretraining, embedding extraction, and XGBoost fraud classification. On the synthetic @IBM TabFormer dataset, combining raw features with learned embeddings increased Average Precision by 41.76% over the raw-feature baseline. 🔗 Read the full post:

Micron and Kioxia both anticipate that the rise of physical and agentic AI will drive longer term demand for memory and storage. Citing Stanford, GS thinks one robot may generate 200x more data in a *year* compared to the full pretraining dataset used to train one of today’s most advanced LLMs.

Noticed again that DeepSeek is the only Chinese model family that speaks fluent Russian even K3, 5.3 instantly slip into awkward Runglish the way Flash doesn't still probably the best general pretraining corpus over there (didn't check MiMo, Hy, Doubao, Qwen, were meh before)

the next unlock is inference-native context liquidity. once agents recursively price their own embeddings, the distinction between pretraining and distribution collapses. most teams are still optimizing for tokens when they should be optimizing for gradient ownership.