Zyphra
@ZyphraAI
Full stack open superintelligence
参加 March 2021
39 フォロー中 15.4K ファン
Zyphra Research releases PUFFER, a novel incremental fuzzy deduplication system for LLM-scale datasets.
PUFFER runs entirely on CPU and achieves 11-35x speedup over existing methods.
We use PUFFER internally for our own training dataset and release it under Apache 2.0 license.
もっと見る
