Register and share your invite link to earn from video plays and referrals.

Search results for RecommenderSystems
RecommenderSystems community
One keyword maps to one global community path.
Create community
People
Not Found
Tweets including RecommenderSystems
📈 Bigger Recommender Transformers Do Not Automatically Scale Meta’s recent ads-ranking work has brought recommender scaling back into focus. Its central lesson is simple: predictable gains appear only when the model, user sequences, features, and serving system scale together. Zhihu contributor 九老师 explains why Transformer scaling is fundamentally a scale-matching problem, not a race to add more parameters. 1️⃣ Scaling changes the entire training system Increasing depth or width changes far more than model capacity. It also changes residual magnitudes, gradient flow, attention logits, optimizer states, and the best learning rate. A larger model can therefore appear to train normally while some layers contribute very little. The real question is not “How large is the Transformer?” but “Can every part of the system remain effective at this scale?” 2️⃣ Normalization determines whether depth is useful Post-Norm Transformers become increasingly fragile as depth grows. Gradient spikes can distort optimizer states, attention can collapse, and lower layers may gradually stop learning. Pre-Norm creates a cleaner gradient path and is generally more stable. But it has its own limitation: as residual signals accumulate, later layers may become dominated by the identity path. The network becomes deeper without learning proportionally richer representations. This is why residual scaling and initialization must evolve with model depth. 3️⃣ Attention and optimization must scale too Attention can fail in two opposite ways: 🔹 Entropy collapse: attention becomes extremely sharp and concentrates on very few positions. 🔹 Rank collapse: repeated mixing makes token representations increasingly similar. Techniques such as QK normalization can control attention sharpness, while residual scaling helps preserve token-specific information across layers. Optimizer settings also cannot be copied blindly from smaller models or LLM recipes. Recommendation data changes quickly, so historical gradients may become stale faster. Model scale, learning rate, initialization, and optimizer dynamics must be tuned as one system. 4️⃣ Loss alone cannot reveal silent failures A smooth training curve does not prove that the full Transformer is learning. Useful internal signals include: 🔹 Gradient strength across different layers 🔹 The size of parameter updates relative to parameter weights 🔹 Similarity between adjacent-layer representations 🔹 Attention-logit magnitude and attention entropy These diagnostics reveal whether lower layers are inactive, attention is collapsing, or updates have become poorly scaled. 5️⃣ Bigger models need richer inputs Scaling parameters alone may bring little improvement when the model still receives heavily compressed or low-complexity features. The author recalls that multimodal embeddings initially produced little gain in one recommender system. A manually designed distance feature worked better. After the model gained enough capacity, those raw embeddings became much more useful. The lesson is not that larger Transformers always win. It is that model capacity and input complexity must grow together. If the available information is simple, a smaller architecture may still be the better choice. ⚙ The core lesson A recommender Transformer is not a plug-and-play module. Features, sequences, tokenization, normalization, attention, optimization, training, and serving all interact. Scaling one component while freezing the others often creates cost without real capability. Using a Transformer is not the same as building a Transformer-native recommender system. A model truly scales only when the whole system scales with it. 🔗 Recent context: 🔗 Full Reading: #RecommenderSystems# #Transformers# #ScalingLaws# #MachineLearning# #AIInfra#
Show more
Generative recommender systems reformulate personalization as sequence modeling over user behavior, and NVIDIA recsys-examples now provides an end-to-end HSTU inference workflow with Dynamo-Triton. PyTorch AOTI compiles HSTU ranking models into native C++ artifacts, while FlexKV-backed KV caching stores reusable attention state to avoid recomputing long user histories. On an @nvidia RTX PRO 6000 Blackwell Workstation Edition GPU, the eight-layer HSTU model achieved up to 5.93x lower latency at batch size 8 with a 100% GPU KV-cache hit rate compared to the same AOTI configuration without caching. Read the full blog post:
Show more
What happens when you let AI agents autonomously run research on recommender models that take days to train? This paper finds out. Auto-RecSys: Harnessing Autonomous Research Agents for Industry-Scale Recommender Systems ❓ Why aren't existing autonomous research agents enough? 💡 Prior systems assume feedback loops of minutes to hours. Industry-scale recommender models take days per training run and involve thousands of lines of configuration, so serial iteration breaks down and you need a fundamentally different design that runs many ideas in parallel across distributed servers and recovers from failure. ❓ How does it juggle parallel experiments and recover from crashes? 💡 Each idea keeps its own independent state file moving through ideating, implementing, validating, training, and analyzing. When a session restarts, it rereads a global registry and trajectory logs on shared storage, letting work resume seamlessly across servers. ❓ Does the system actually get better over time? 💡 Experiment trajectories are distilled into model-specific playbooks that record dead ends as "do not" rules. Across 31 iterations, major fixes needed per iteration dropped from 4.0 to 0.5, with 83% of iterations completing with zero fixes. ❓ How autonomous is it in practice? 💡 One session executed 970 consecutive log entries with zero human intervention, and the system even switched its own training workflow after repeated publish failures. #AIAgents# #MachineLearning#
Show more
TL;DR: HiRank is a recommender ranking model that fuses sequential and non-sequential features two ways in a single backbone, and uses effective-rank-aware attention to keep representations diverse. In an online test on the Taobao homepage, orders per user rose 2.27%. Title: Unified Heterogeneous Feature Interaction via Effective-Rank Scaling for Recommender Systems URL: Key points ・🧩 Drops the encode-then-interact pipeline; event-level sequence states stay accessible across layers ・🚪 Models the main behavior sequence with HSTU-style gated attention ・🔎 Retrieves from long-term history memories with both interest queries and candidate queries for diverse retrieval ・🔀 Fuses sequence, retrieval outputs, features, and the candidate via token mixing ・🎯 Applies High-Rank Attention over scenario-aware compressed composite tokens ・📊 Offline: homepage GAUC 0.7801 (+0.0186 over the strongest baseline) ・🛒 Online: orders/user +2.27%, GMV/user +2.86% 💡 I like the takeaway that sequence length and representation diversity matter, not just model size. #RecSys# #MachineLearning#
Show more
TL;DR: Adding features to a recommender system is easy, but safely removing them is brutally hard. Pinterest built a production framework that automates that whole cleanup process. Title: FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems URL: Key points 🧩 Adding a feature just needs a metric win, but removing one must not regress any metric, a built-in asymmetry that stalls cleanup ⏱️ Daily perturbation experiments automatically track feature importance and rank stability over time 🧪 Grouped perturbation evaluates candidate feature sets together, catching interactions before costly retraining 🚀 Pooled execution resolves all experiments for a model in a single distributed inference pass, cutting runtime 54x for 300 experiments 🔗 Automated lineage tracking finds every downstream consumer model, making deprecation safe 📉 On the Pin Ranker model, removing 8 features cut parameters by 29.9%, boosted throughput 19.8-25.5%, and saved hundreds of thousands of dollars a year Feature removal infrastructure deserves as much engineering investment as feature addition, and this looks like a solid blueprint for it. #RecSys# #MLOps#
Show more
Head of Safety @michaelo will be at the Brand & AI Safety Summit next week speaking on why open-source recommender systems and controls people can see matter for safety and accountability.
Show more