Sipeed is taking direct shots at DGX Spark, and their latest benchmarks show why TOPS alone is a bad Local AI buying metric.
Sipeed's new chip is rated at just 64 INT8 TOPS.
In its latest INT8 test it just beat DGX Spark
ResNet50
๐ข AX8860 โ 4,300 FPS
๐ฉ DGX Spark โ 2,753 FPS
YOLO26s
๐ข AX8860 โ 1,450 FPS
๐ฉ DGX Spark โ 617 FPS
That's 56% faster on ResNet50 and 2.35ร the throughput on YOLO26s.
Sipeed's older 18-TOPS AX8850N manages 316 FPS on YOLO26s.
DGX Spark โ 617 FPS
So an 18-TOPS edge NPU produced roughly half the FPS of Spark in this workload.
Meanwhile, DGX Spark is marketed at up to 1,000 TOPS / 1 PFLOP FP4.
โ ๏ธ These numbers are not directly comparable TOPS-for-TOPS.
This benchmark is INT8. Nvidia's 1,000-TOPS is sparse FP4. Nvidia itself specifies that number as theoretical FP4 performance using sparsity.
And these are Sipeed's own benchmarks, not independent testing.
๐ฏ The TOPS number on the box doesn't tell you how fast your workload will run.
Precision, sparsity, kernels, architecture, memory bandwidth, and software utilization really matter.
For Local AI, we would rather see model + quant + context + tps than a giant TOPS number. (which they have on the website)
Sipeed says it used TensorRT INT8 on Spark with NVIDIA's recommended Docker environment and is inviting people to reproduce the results.
Independent testing should be fun. ๐