Liquid AI LFM2.5-DSpark: 3.18x Peak GPU Throughput via Speculative Decoding

What Shipped and Architecture

Liquid AI released DSpark draft-model checkpoints for three target models in the LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. Each draft model is a 5-layer, attention-only speculative decoder with a block size of 9, trained for 15 epochs on a mix of SFT, chat, code, and function-calling data; the checkpoint was selected based on highest token acceptance rate rather than lowest loss. The architecture combines a DFlash-style parallel backbone conditioned on the target model's context features, a lightweight sequential Markov-chain head for inter-token dependency, and a confidence-scheduled verifier that prunes low-confidence suffixes when verification cost exceeds savings.

Draft Model Specs

The draft models add roughly 300 million parameters to each target model. LFM2.5-1.2B-Instruct uses a 295.7M-parameter draft (241.2M decoder stack, 21.0M hidden-state projection, 33.6M Markov head, and 27.5k norms plus confidence head), while the LFM2.5-2.6B and LFM2.5-8B-A1B drafts both weigh 327.7M parameters, differing only in the Markov head size (65.5M versus 33.6M). All three use the same 5-layer decoder stack and hidden-state projection, so the memory overhead is minimal relative to the target models.

Throughput Benchmarks

Vendor-reported throughput tests used batch size 1, temperature 0, block size 9, and FP16 or BF16 precision. On an H100 80GB with SGLang, the LFM2.5-8B-A1B draft reached a peak 3.18x speedup on MATH500 (428 -> 1362 tok/s), while the LFM2.5-2.6B draft averaged 2.67x across five datasets (323 -> 864 tok/s) with a mean acceptance rate of 4.81 out of 10 tokens. On an M4 Max MacBook Pro running llama.cpp with Metal, the LFM2.5-2.6B draft averaged 2.27x (61 -> 139 tok/s), the 1.2B draft peaked at 2.87x on HumanEval (136 -> 389 tok/s), and the 8B-A1B MoE draft only managed a 1.18x mean (90 -> 106 tok/s), which Liquid AI attributes to llama.cpp's current Metal backend and extra weight traffic during verification. In function-calling tests on the BFCL dataset, the 2.6B model cut average latency by 57 percent on the M4 Max.

Quality Guarantees and Availability

Because DSpark operates under greedy decoding and only accepts draft tokens that match the target model's distribution, the emitted sequence is identical to the baseline by construction; Liquid AI states that pass@1 and exact-match benchmark accuracy are therefore unchanged. The draft models are available on Hugging Face with day-one upstream integrations for llama.cpp and SGLang. Pricing for API access or commercial licensing was not disclosed in the available research material, and specific base-model accuracy scores on standard benchmarks were not provided.

Source: Hugging Face Blog