Researchers in Huawei's huawei-csl group have open-sourced MACaQ (Model Agnostic Calibrated Quantization), a toolkit for quantizing and running text, image, video and audio models at low precision. The code is on GitHub under the Apache 2.0 license, with an accompanying arXiv paper titled "Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations."
According to the project README, MACaQ supports four backends. In llama.cpp it produces 2-, 3- and 4-bit GGUF models that can run on CPU. In vLLM it supports serving with 4-bit weights and activations (W4A4) or weight-only quantization. In Diffusers it handles image, video and audio generation at W4A4, and in PyTorch it covers speech generation at W4A4. Listed models include the Qwen3 and Qwen3.6 families, FLUX.2-klein-4B, Wan2.2-TI2V-5B, stable-audio-1.0 and Qwen3-TTS. The repository also includes W4A4 matrix-multiplication kernels for Huawei's Ascend accelerators, though full-model execution on Ascend still needs integration with an Ascend inference runtime.
The authors say MACaQ can quantize a 122-billion-parameter model in 64 minutes using 32.5 GB of device memory. They report lower perplexity than the MR-GPTQ and DuQuant++ methods, or the same best Llama-3-8B perplexity with 11 times less quantization time. On Qwen3.8-27B in vLLM, they report W4A4 perplexity of 2.8994 against 2.8463 for BF16, with prefill 2.33 times and decode 2.58 times faster than BF16.
The release also includes MACaQ-R, which refines the weight values inside an existing GGUF file while keeping its native format. On Qwen3.6-35B-A3B, the README reports a KL divergence from FP16 of 0.013367 for a MACaQ-R Q4_K_XL file, against 0.015731 for Unsloth's UD-Q4_K_XL at the same 22.36 GB size. Lower is better.
For people running models locally, MACaQ-R is the easiest part to adopt. It produces a standard GGUF file, so it works in existing llama.cpp setups without new runtime code. All benchmark figures come from the authors and have not yet been independently reproduced.