Register and share your invite link to earn from video plays and referrals.

Search results for kmeans
kmeans community
One keyword maps to one global community path.
Create community
People
Not Found
Tweets including kmeans
A time-complexity cheat sheet of 10 ML algorithms: What's the inference time-complexity of KMeans?
TL;DR: e-means optimizes k-means with EMA and matches Lloyd's 25-epoch loss in about 1.4 to 2.6 epochs. No hyperparameter sweep needed. e-means: outperforming k-means (in one-tenth the epochs) Key points ・⚡ Reaches Lloyd's 25-epoch loss in roughly 1/10 of the epochs (as reported) ・🔁 EMA-updated counts and sums per cluster; centroid read out as S/N ・🌊 Cosine-annealed step size ・🩹 Dead clusters revived by splitting every 5 batches (the biggest effect) ・📐 Decay and batch size set by scaling laws, no sweep ・🚀 DINO10B (10B embeddings) split into 16M clusters in about 2.5 epochs ・🖥️ 9.5-45.5x wall-clock speedup over Flash-KMeans The scaling laws were mostly fitted on DEEP, so accuracy can drop in extreme settings. Clustering large embedding sets just got much cheaper. #MachineLearning# #kmeans#
Show more
In generative recommendation, everyone has assumed they know what makes a "good" semantic ID design. So what happens when you actually put that assumption to a large, controlled reproducibility test? 🔍 Generative recommenders turn items into discrete token sequences called semantic IDs, then generate the next item autoregressively. Designs like RQ-VAE, OPQ, and RQ-Kmeans have proliferated, but prior comparisons used different datasets and backbones, so nobody could say which design actually wins. So the authors reran 12 methods side by side under the exact same data splits and evaluation protocol, and the assumptions started falling apart. No single SID design dominates across datasets. Codebook utilization balance barely correlates with recommendation quality. Longer codes don't always help, and bigger backbones can even hurt performance. And when you measure how well an SID preserves an item's local semantic neighborhood, the winning method flips depending on whether you use Jaccard or RBO. The lesson from What Makes a Good Semantic ID for Generative Recommendation? A Reproducibility Study is that good SID design isn't about chasing one metric, it's about balancing several at once. URL: #RecSys# #Reproducibility#
Show more
Visualizing K-Means Clustering in action. Watch how the centroids (triangles) start at random, carve out their initial territories (Voronoi cells), and iteratively shift to the actual mathematical center of the data.
Show more
Meta just upgraded k-means after 70 years. (runs 10-45x faster on GPUs) the new variant, e-means, gets to the loss k-means reaches after 25 epochs in about 2, and it runs 10-45x faster on GPUs. classic k-means (Lloyd's algorithm) assigns every point to its nearest centroid, then moves each centroid to the mean of its points. one full pass over the data gives you exactly one update. mini-batch k-means updates from small batches instead. it moves fast early on but settles at a worse loss, which is why Lloyd stayed the default for decades. e-means keeps the mini-batches and fixes the reasons they fall short. it answers three questions: "how far should a centroid move after each batch?" each centroid keeps a running average of the points assigned to it. early batches move it a lot so it finds its region quickly, and later batches move it less so it settles. "what happens to a cluster nobody joins?" some centroids drift into empty regions and stop receiving points. every few batches, e-means splits a crowded cluster to give those centroids useful work again, and this step mattered more than any other in the paper's tests. "which batch size and step size should i pick?" earlier mini-batch methods needed a hyperparameter sweep for every new dataset. e-means sets both values directly from the number of points, clusters, and epochs, using scaling laws fitted over 75k training runs. each epoch costs about the same as a k-means epoch, and e-means simply needs far fewer of them. the chart in the image below shows its loss dropping under both k-means baselines within the first couple of epochs. at scale, it built 16 million clusters from 10 billion embeddings in under a day of GPU time. this matters well beyond clustering, since k-means builds vector database indexes, product quantization codebooks, and data curation pipelines. so before you throw another 25 epochs of Lloyd at your embeddings, consider e-means. it already ships with Faiss, and the paper is attached below if you want to read more. 🔗
Show more