DFlash drafts a whole block in parallel. DFlash 2 picks one path through the block instead of the top token at each position: over 20% more tokens per pass, about 1% more latency.
Our co-founder and chief scientist
@zhijianliu explains how at
@modal Runtime in SF on Oct 1.