NVFP4 and MXFP4 quantization in MLX on Apple Silicon

Two four-bit floating-point formats, NVFP4 and MXFP4, are becoming the default way frontier models ship. Here's what actually separates them, where MLX and Apple Silicon stand today, and why the last bit below four is the hard one, grounded in our own low-bit measurements.

The formats, without the marketing

Both NVFP4 and MXFP4 are built on the same 4-bit element: E2M1. That's one sign bit, two exponent bits, one mantissa bit. Its sixteen codes span the values {0, ±0.5, ±1, ±1.5, ±2, ±3, ±4, ±6}. One mantissa bit is not a lot. On its own, E2M1 is far too coarse to hold a weight matrix without wrecking it.

What makes 4-bit floating point usable is microscaling: you split each tensor into small contiguous blocks and give every block its own scale factor. The E2M1 values are relative; the block scale restores their magnitude. The two formats differ in exactly two places, block size and the type of that scale:

• MXFP4 (the Open Compute Project microscaling format) uses a block of 32 elements and an E8M0 scale, an 8-bit exponent with no mantissa. The scale is therefore always a power of two. Cheap to store, coarse in what it can represent.
• NVFP4 (NVIDIA's format) uses a block of 16 elements and an E4M3 FP8 scale, a floating-point scale with an actual mantissa, usually paired with a second per-tensor FP32 scale. Smaller blocks and a finer scale mean it tracks the local weight distribution more accurately.

That's the whole trade. NVFP4's 16-element blocks and real-valued scales cost more metadata per parameter but preserve more signal. MXFP4's 32-element power-of-two scales are leaner and map cleanly onto simpler hardware. Both were designed with next-generation GPU tensor cores in mind, where 4-bit multiply-accumulate is a native operation.

Where MLX and Apple Silicon actually stand

This is where the honest answer matters more than the tidy one. MLX's built-in quantization, the thing mlx_lm.convert -q produces, is not NVFP4 or MXFP4. It's affine integer group quantization: each group of weights is mapped to n-bit integers with a stored scale and a stored zero-point (bias), at 2, 3, 4, 6, or 8 bits. It's a different scheme from microscaling FP4, even though both are "4-bit" in casual conversation.

Support for the microscaling FP4 formats themselves is real but uneven, and it's moving. MLX has been gaining the ability to load and run models that ship in MXFP4, because some open-weight releases now distribute in that format and there's no way around reading it. NVFP4 support across the Apple Silicon stack is thinner and evolving faster than any snapshot here would stay accurate. Treat "does MLX handle format X end to end today" as a question to check against your installed version, not a fixed fact.

There's also a hardware reality underneath. Formats like NVFP4 and MXFP4 pay off most on GPUs with dedicated FP4 tensor units, where the narrow format is also a compute speedup. Apple Silicon GPUs don't expose FP4 matmul hardware the way Blackwell-class parts do, so on a Mac these formats are mostly a storage and memory-bandwidth win: the weights are smaller and move faster, but the heavy math typically happens after dequantizing to a wider type. That's not a criticism. Model compression on Apple Silicon is memory-bound anyway, so a smaller footprint is exactly the lever that matters.

Why the drop below four bits is a cliff, not a slope

Here's the part people underestimate. Going from 8-bit to 4-bit is a gentle ramp for most weights. Going from 4-bit toward 2-bit is a cliff, and it's a cliff regardless of whether you're using integer quantization or microscaling FP4, because the problem is information, not encoding.

Our own numbers make the shape of that cliff concrete. These come from integer group quantization, not from an FP4 format, so read them as evidence about low-bit behaviour in general, not as an NVFP4 or MXFP4 benchmark. Measuring signal-to-quantization-noise ratio across models from 8B to 109B parameters, we found a clean gap: 2-bit configurations peak around 8.7 dB, while 3-bit configurations start around 10.4 dB. There is almost nothing in between. A safety threshold set at 9 dB sits in that empty band, and in our testing it blocked every catastrophic configuration while permitting every viable one. 2-bit quantization was almost always catastrophic; 3-bit was generally recoverable. We wrote about this and the rest of the case for budget-aware compression in the quantization bottleneck.

Two structural facts explain why that cliff exists and why block design fights it:

• The representable set gets sparse fast. At four bits you have sixteen levels per block; at two you have four. Weights that fall between levels get rounded, and the rounding error grows faster than the bit savings.
• Block size is a real quality knob, not a footnote. The whole reason NVFP4 uses 16-element blocks instead of MXFP4's 32 is that finer blocks track local weight statistics better. We see the same effect from the other direction: in one 30B model, the best allocation put 85% of tensors at group size 32 rather than the conventional 128, and that finer granularity closed most of the gap to full precision. Smaller blocks, whether you call them groups or microscaling blocks, buy accuracy at low bit widths.

What our own low-bit work does and doesn't claim

We compress aggressively and we publish the results, so it's worth being precise about which numbers are ours and which are format properties.

When the allocation is done well, roughly-4-bit compression can be effectively lossless. We took Gemma 4 31B down to a 31 GB footprint, about half its BF16 size, and it scored 85.2% on the full 12,032-question MMLU-Pro suite, matching Google's published BF16 baseline. That result is real, it's measured on the complete benchmark, and you can read the per-category breakdown yourself. It's also integer group quantization with per-tensor bit and group-size allocation, not an FP4 format.

What we have not done is publish NVFP4 or MXFP4 benchmarks, so we're not going to attach a number to those formats here. The honest summary is this: the format matters less than the allocation. A well-chosen mix of bit width and block size, applied per tensor, beats picking one global format and hoping. The FP4 formats give you a smaller, faster-loading building block; they don't decide, on their own, whether a specific tensor should be four bits or eight, or whether its block should be 16 wide or 32.

Where this leaves you on a Mac

If you're deploying quantized models on Apple Silicon today, a few things follow from all of the above:

1. Judge a 4-bit model by its behaviour, not its label. Integer 4-bit, MXFP4, and NVFP4 are different encodings that can land in very different quality places. Run the model.
2. Expect the wins on a Mac to be memory-shaped. Smaller weights and faster loading, not FP4-tensor-core speedups you don't have.
3. Don't reach below four bits casually. The 2-to-3-bit region is where models silently break. If you go there, gate it, our 9 dB signal-quality floor is a one-line check that catches the worst configurations before they ship.
4. Spend your effort on allocation, not on picking one format. Per-tensor bit width and block size decide more of the outcome than the choice of FP4 flavour.