Zyphra
@ZyphraAI
Full stack open superintelligence
加入 March 2021
39 正在關注 15.4K 粉絲
Zyphra Research releases PUFFER, a novel incremental fuzzy deduplication system for LLM-scale datasets.
PUFFER runs entirely on CPU and achieves 11-35x speedup over existing methods.
We use PUFFER internally for our own training dataset and release it under Apache 2.0 license.
顯示更多
