MLX Quantization on Apple Silicon: How RAM Turns a Mac into a Model Compression Lab

Every result in the RAM paper was produced on a single Apple M2 Ultra. No NVIDIA GPUs. No cloud instances. No calibration data. Just Apple Silicon and unified memory. This article explains why that matters and what it means for anyone with a Mac.

The Hardware Nobody Expected

When quantization papers report results, the hardware section typically reads like a data centre shopping list: 8x A100 80 GB, 4x H100, a DGX cluster. The RAM paper's hardware section reads differently:

That's the entire compute stack. A Mac Studio sitting on a desk. The same machine ran every experiment, from an 8B dense model to a 109B parameter Mixture-of-Experts model with 16 experts. The largest model (Llama-4-Scout at ~203 GB in BF16) can't even fit in memory unquantized, yet RAM analysed, allocated, and quantized it to run on that same machine.

Why MLX Changes the Equation

Apple's MLX framework is purpose-built for Apple Silicon. Two properties make it a natural fit for RAM's proprietary compression pipeline:

This isn't a compromise. Unified memory is an advantage for quantization workloads. Traditional GPU setups are constrained by VRAM (24 GB on an RTX 4090, 80 GB on an A100). A 109B parameter model in BF16 needs ~203 GB. No single GPU can hold it. On Apple Silicon, the M2 Ultra's 192 GB is one contiguous memory space accessible by all compute units.

What RAM Actually Does on Apple Silicon

RAM's proprietary compression pipeline analyses every weight tensor in a model (18,867 in Qwen3-30B alone) and determines the optimal compression strategy for each one. The entire pipeline runs natively on Apple Silicon.

The key input is your memory budget: "fit this model in 30 GB" or "fit it in 64 GB". RAM's allocator finds the provably optimal per-tensor configuration for that budget in under 1 second. No manual tuning, no presets.

Timing: How Fast Is It?

From the RAM paper, Table 11, all timings on Apple M2 Ultra 192 GB:

A 109B parameter model, analysed, optimised, and allocated in under 50 minutes. On a Mac. Compare that to calibration-based methods like GPTQ, which need GPU clusters, representative datasets, and hours of Hessian computation for models this size.

Note the allocation time: under 1 second in every case. Once you've run the analysis pass, you can re-target the same model to any number of hardware budgets instantly. Analyse once, deploy everywhere.

Budget-Targeted Deployment: Name Your Hardware

RAM's defining feature is budget-targeted quantization. You tell it exactly how much memory you have, and it returns the optimal allocation. The paper demonstrates this with Qwen3-30B-A3B across specific Apple Silicon and GPU targets:

At the Mac M4 Pro budget of 30 GB, RAM produces a 30.75 GB quantized model with a mean perplexity of 8.657. That's within 1% of the full BF16 model (8.728) at 54% of the size. RAM's proprietary allocator puts bits where they matter most, closing 94% of the gap between uniform 4-bit and BF16 at the 19 GB budget point.

The 109B Model on a Mac Studio

The most striking demo is Llama-4-Scout, Meta's 109B parameter MoE model with 16 experts and 17B active parameters per token. In BF16 it needs ~203 GB. No consumer hardware can run it. Here's what RAM does with it on the M2 Ultra:

Scout in BF16 needs ~203 GB and can't run on any consumer hardware. At the M2 Ultra's 192 GB budget, RAM produces a 163 GB model that's 6.8% better than uniform 4-bit quantization (7.359 vs 7.899). At 64 GB (feasible on an M4 Max), it still beats uniform 4-bit by 2.5%. A model that was previously impossible to deploy on consumer hardware now runs with excellent quality.

RAM's built-in safety mechanisms prevent over-compression. Without them, aggressive 2-bit compression would produce a compact 34.6 GB model that's catastrophically broken (PPL 23.6, nearly triple the baseline). RAM's proprietary quality thresholds automatically block these dangerous configurations and redistribute capacity to where it actually helps.

Why Apple Silicon Is Ideal for This Workload

RAM's proprietary compression pipeline has computational characteristics that map perfectly to Apple Silicon's architecture:

The Apple Silicon Model Guide

Based on RAM's budget curves and the memory capacities of current Apple Silicon configurations, here's what you can realistically run:

*Usable budget accounts for OS, KV cache, and inference overhead. Actual available memory depends on workload.

The critical point: you don't guess which quantization preset to use. You tell RAM exactly how much memory you have, and it returns the mathematically optimal allocation for that budget. Different Mac? Different budget. Same pipeline, different optimal answer.

Group Size g=32: Why It Matters for MLX

RAM's most surprising finding is that 85.2% of tensors get allocated group size 32 rather than the conventional 128. This matters directly for Apple Silicon deployment:

• Apple Silicon natively supports g=32. No custom kernels or workarounds needed.
• Storage overhead is modest. Going from g=128 to g=32 adds ~0.125 bytes/parameter. For a 30B model, that's ~3.5 GB additional overhead.
• Quality improvement is substantial. The finer granularity of g=32 captures per-group weight distributions more accurately, closing up to 94% of the gap between uniform 4-bit and BF16.
• Performance cost is minimal. The 4× increase in groups from g=128 to g=32 adds negligible latency because the operation is memory-bound on Apple Silicon.

No Calibration Data: Why This Matters for On-Device

Calibration-based methods like GPTQ and AWQ need representative input data to compute sensitivity. For on-device deployment, that creates three problems:

1. Privacy. If you're quantizing a model for local use on a Mac, maybe for an enterprise deploying on-premises, sending proprietary data through a calibration pipeline on a cloud GPU defeats the purpose of local deployment.
2. Representativeness. A calibration set from English Wikipedia may not represent the Japanese legal documents your deployment processes. RAM sidesteps this by using only the weights themselves.
3. Compute requirements. Calibration needs forward passes through the model, which typically demand GPU clusters. RAM's data-free pipeline runs entirely on the same Mac that will serve the model.

What about quality? In matched-size comparisons against GPTQ across three MoE model families, RAM wins every time:

A data-free method, running on a Mac, producing better results than the gold-standard calibration method running on GPU clusters with representative data.

The Workflow: From Download to Deployment

For someone with a Mac and a model they want to deploy, the RAM workflow on Apple Silicon looks like this:

Step 4 is the key innovation. Because allocation takes under 1 second, you can target any number of hardware budgets from a single analysis pass. One analysis of Qwen3-30B produces optimal allocations for iPhone (15 GB), M4 Pro (30 GB), M4 Max (64 GB), and M2 Ultra (160 GB), all in seconds.

What This Means

RAM on MLX removes three barriers to deploying large language models on Apple Silicon:

A Mac Studio with an M2 Ultra is now a complete model compression lab. It can analyse, optimise, quantize, and serve models up to 109B parameters, producing results that beat calibration-based methods running on GPU clusters. The entire RAM paper is the proof.

For the full technical details, see the RAM paper on HuggingFace.

Read the Full Paper

The complete RAM paper, including formal derivations, benchmark results across 7 model families and 40,000+ questions, and the full optimal allocation framework, is available on our HuggingFace: