๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
๊ฐ€์ž… July 2023
552 ํŒ”๋กœ์ž‰ ์ค‘    12.5K ํŒฌ
Sipeed is taking direct shots at DGX Spark, and their latest benchmarks show why TOPS alone is a bad Local AI buying metric. Sipeed's new chip is rated at just 64 INT8 TOPS. In its latest INT8 test it just beat DGX Spark ResNet50 ๐ŸŸข AX8860 โ†’ 4,300 FPS ๐ŸŸฉ DGX Spark โ†’ 2,753 FPS YOLO26s ๐ŸŸข AX8860 โ†’ 1,450 FPS ๐ŸŸฉ DGX Spark โ†’ 617 FPS That's 56% faster on ResNet50 and 2.35ร— the throughput on YOLO26s. Sipeed's older 18-TOPS AX8850N manages 316 FPS on YOLO26s. DGX Spark โ†’ 617 FPS So an 18-TOPS edge NPU produced roughly half the FPS of Spark in this workload. Meanwhile, DGX Spark is marketed at up to 1,000 TOPS / 1 PFLOP FP4. โš ๏ธ These numbers are not directly comparable TOPS-for-TOPS. This benchmark is INT8. Nvidia's 1,000-TOPS is sparse FP4. Nvidia itself specifies that number as theoretical FP4 performance using sparsity. And these are Sipeed's own benchmarks, not independent testing. ๐ŸŽฏ The TOPS number on the box doesn't tell you how fast your workload will run. Precision, sparsity, kernels, architecture, memory bandwidth, and software utilization really matter. For Local AI, we would rather see model + quant + context + tps than a giant TOPS number. (which they have on the website) Sipeed says it used TensorRT INT8 on Spark with NVIDIA's recommended Docker environment and is inviting people to reproduce the results. Independent testing should be fun. ๐Ÿ‘€
๋” ๋ณด๊ธฐ