MLX Quantization on Apple Silicon: How RAM Turns a Mac into a Model Compression Lab

Every result in the RAM paper was produced on a single Apple M2 Ultra. No NVIDIA GPUs. No cloud instances. No calibration data. Just Apple Silicon and unified memory. This article explains why that matters and what it means for anyone with a Mac.

The Hardware Nobody Expected

When quantization papers report their results, the hardware section typically reads like a shopping list from a data centre: 8×A100 80 GB, 4×H100, a DGX cluster. The RAM paper’s hardware section reads differently:

That is the entire compute stack. A Mac Studio sitting on a desk. The same machine ran every experiment, from an 8B dense model to a 109B parameter Mixture-of-Experts model with 16 experts. The largest model (Llama-4-Scout at ~203 GB in BF16) cannot even fit in memory unquantized, yet RAM analysed, allocated, and quantized it to run on that same machine.

Why MLX Changes the Equation

Apple’s MLX framework is purpose-built for Apple Silicon. Two properties make it uniquely suited to RAM’s proprietary compression pipeline:

This is not a compromise. Unified memory is an advantage for quantization workloads. Traditional GPU setups are constrained by VRAM (24 GB on an RTX 4090, 80 GB on an A100). A 109B parameter model in BF16 requires ~203 GB, no single GPU can hold it. On Apple Silicon, the M2 Ultra’s 192 GB is one contiguous memory space accessible by all compute units.

What RAM Actually Does on Apple Silicon

RAM’s proprietary compression pipeline analyses every weight tensor in a model, for example there are 18,867 in Qwen3-30B alone, and determines the optimal compression strategy for each one. The entire pipeline runs natively on Apple Silicon.

The key input is your memory budget: “fit this model in 30 GB” or “fit it in 64 GB”. RAM’s allocator finds the provably optimal per-tensor configuration for that budget in under 1 second. No manual tuning, no presets.

Timing: How Fast Is It?

From the RAM paper, Table 11, all timings on Apple M2 Ultra 192 GB:

A 109B parameter model, analysed, optimised, and allocated in under 50 minutes. On a Mac. Compare this to calibration-based methods like GPTQ, which require GPU clusters, representative datasets, and hours of Hessian computation for models of this scale.

Note the allocation time: under 1 second in every case. This means once you’ve run the analysis pass, you can re-target the same model to any number of hardware budgets instantly. Analyse once, deploy everywhere.

Budget-Targeted Deployment: Name Your Hardware

RAM’s defining feature is budget-targeted quantization. You specify exactly how much memory you have, and RAM returns the optimal allocation. The paper demonstrates this with Qwen3-30B-A3B across specific Apple Silicon and GPU targets:

At the Mac M4 Pro budget of 30 GB, RAM produces a 30.75 GB quantized model with a mean perplexity of 8.657, within 1% of the full BF16 model (8.728) at 54% of the size. RAM’s proprietary allocator distributes bits where they matter most, closing 94% of the gap between uniform 4-bit and BF16 at the 19 GB budget point.

The 109B Model on a Mac Studio

The most striking demonstration is Llama-4-Scout, Meta’s 109B parameter MoE model with 16 experts and 17B active parameters per token. In BF16, it requires ~203 GB. No consumer hardware can run it. Here is what RAM does with it on the M2 Ultra:

The BF16 version of Scout requires ~203 GB and cannot run on any consumer hardware. At the M2 Ultra’s 192 GB budget, RAM produces a 163 GB model that is 6.8% better than uniform 4-bit quantization (7.359 vs 7.899). At 64 GB (feasible on an M4 Max), it still beats uniform 4-bit by 2.5%. A model that was previously impossible to deploy on consumer hardware now runs with excellent quality.

RAM’s built-in safety mechanisms prevent over-compression. Without them, aggressive 2-bit compression would produce a compact 34.6 GB model that is catastrophically broken (PPL 23.6, nearly triple the baseline). RAM’s proprietary quality thresholds automatically block these dangerous configurations and redistribute capacity to where it actually helps.

Why Apple Silicon Is Ideal for This Workload

RAM’s proprietary compression pipeline has computational characteristics that map perfectly to Apple Silicon’s architecture:

The Apple Silicon Model Guide

Based on RAM’s budget curves and the memory capacities of current Apple Silicon configurations, here is what you can realistically run:

*Usable budget accounts for OS, KV cache, and inference overhead. Actual available memory depends on workload.

The critical insight: RAM’s budget-targeted allocation means you don’t guess which quantization preset to use. You tell RAM exactly how much memory you have, and it returns the mathematically optimal allocation for that budget. Different Mac? Different budget. Same pipeline, different optimal answer.

Group Size g=32: Why It Matters for MLX

RAM’s most surprising finding is that 85.2% of tensors are allocated group size 32 rather than the conventional 128. This is directly relevant to Apple Silicon deployment:

• Apple Silicon natively supports g=32. No custom kernels or workarounds needed.
• Storage overhead is modest. Going from g=128 to g=32 adds ~0.125 bytes/parameter. For a 30B model, that’s ~3.5 GB additional overhead.
• Quality improvement is substantial. The finer granularity of g=32 captures per-group weight distributions more accurately, closing up to 94% of the gap between uniform 4-bit and BF16.
• Performance cost is minimal. The 4× increase in groups from g=128 to g=32 adds negligible latency because the operation is memory-bound on Apple Silicon.

No Calibration Data: Why This Matters for On-Device

Calibration-based methods like GPTQ and AWQ require representative input data to compute sensitivity. For on-device deployment, this creates three problems:

1. Privacy. If you’re quantizing a model for local use on a Mac, perhaps for an enterprise deploying on-premises, sending proprietary data through a calibration pipeline on a cloud GPU defeats the purpose of local deployment.
2. Representativeness. A calibration set from English Wikipedia may not represent the Japanese legal documents your deployment processes. RAM avoids this by using only the weights themselves.
3. Compute requirements. Calibration requires forward passes through the model, which typically demand GPU clusters. RAM’s data-free pipeline runs entirely on the same Mac that will serve the model.

And the quality? In matched-size comparisons against GPTQ across three MoE model families, RAM wins every time:

A data-free method, running on a Mac, producing better results than the gold-standard calibration method running on GPU clusters with representative data.

The Workflow: From Download to Deployment

For someone with a Mac and a model they want to deploy, the RAM workflow on Apple Silicon looks like this:

Step 4 is the key innovation. Because allocation takes under 1 second, you can target any number of hardware budgets from a single analysis pass. One analysis of Qwen3-30B produces optimal allocations for iPhone (15 GB), M4 Pro (30 GB), M4 Max (64 GB), and M2 Ultra (160 GB), all in seconds.

What This Means

RAM on MLX eliminates three barriers to deploying large language models on Apple Silicon:

A Mac Studio with an M2 Ultra is now a complete model compression laboratory. It can analyse, optimise, quantize, and serve models up to 109B parameters, producing results that beat calibration-based methods running on GPU clusters. The entire RAM paper is proof.

For the full technical details, see the RAM paper on HuggingFace.

Read the Full Paper

The complete RAM paper, including formal derivations, benchmark results across 7 model families and 40,000+ questions, and the full optimal allocation framework, is available on our HuggingFace: