PANews|Oct 21, 2026 12:34
[Ant Group and vLLM Jointly Open Source Humming: High-Performance, Flexible Quantized Inference Kernel]
Ant Group and the vLLM community have jointly open-sourced Humming, a general-purpose, high-performance quantized inference operator library. Humming has the following features:
- Extensive device support, covering various GPUs and devices compatible with CUDA programming.
- Rich quantization support, enabling a unified architecture to support diverse combinations of weight and activation formats in Dense and MoE, covering multiple integer and floating-point types, bit widths, and scaling methods.
- Outstanding execution performance, significantly outperforming existing implementations such as vLLM Marlin, FlashInfer, and DeepGEMM across various typical workloads.
Share To
Timeline
HotFlash
APP
X
Telegram
CopyLink