Shepherd: enterprise model compression, explained

If you searched "ai shepherd" and landed here, this is the one. Shepherd is Black Sheep AI's platform for compressing frontier models and getting them into production. Here's what it does, what problem it solves, and why the compression underneath it holds up.

Most "compression" tools give you a smaller file and a shrug about what happened to the model inside it. You quantize a 31B model down to something that fits your hardware, run a couple of prompts, and hope the reasoning survived. It usually didn't, not all of it, and you find out in production when a legal summarization task starts inventing clauses.

Shepherd is our answer to that. It takes a frontier model, compresses it with our RAM engine, certifies what came out the other side, and deploys it to your infrastructure. Compression and capability assurance in one pipeline, not two tools duct-taped together.

First, the name

"ai shepherd" is a search people type when they've heard the name and want the actual product. So, to be unambiguous: Shepherd is Black Sheep AI's model compression platform. It's not a chatbot, not a prompt manager, not an agent framework. It compresses large language models and deploys them privately, in your VPC, on an Apple Silicon fleet, or in an air-gapped facility. If that's what you were looking for, you're in the right place.

The problem Shepherd solves

Running a frontier model privately used to mean one of two bad options. Either you paid for multi-GPU infrastructure to hold the full-precision weights, or you sent your data to an API provider and gave up on privacy. Both are expensive, just in different currencies.

RAM compression changes the arithmetic. It shrinks frontier models by 50 to 60 percent, which cuts your hardware requirement roughly in half while keeping the intelligence you deployed the model for in the first place. A 31B-parameter model that needed workstation-class hardware at full precision now fits on a single GPU instance, or on a Mac Studio sitting at someone's desk. Your data never leaves your infrastructure, because the model runs on hardware you control.

That's the claim. The reason to believe it is the benchmark.

The proof: Gemma 4 at half the size

We compressed Gemma 4 31B to 31GB with RAM and ran it against the complete 12,032-question MMLU-Pro suite, the most demanding multiple-choice academic benchmark in wide use. It scored 85.2 percent, matching Google's published BF16 baseline exactly, on the full suite with no subset selection and no scoring shortcuts.

The full-precision version weighs roughly 62GB. Ours is 31GB. Same published number, half the footprint. Six categories, including math at 94.4%, biology at 92.7%, and physics at 89.8%, cleared 89 percent, the reasoning-heavy categories where broken quantization usually collapses. There were no catastrophic drops. The lowest category, law, tracked a known Gemma-family soft spot rather than a compression artifact.

We wrote the whole result up, per-category tables and all, in our Gemma 4 research report. It's the single best answer to the question "does your compression actually preserve capability."

What Shepherd deploys

The Shepherd Enterprise platform is the production wrapper around RAM. What's in the box:

• The RAM compression engine, in production now. A data-free, post-training method built for both NVIDIA GPU and Apple Silicon targets. No calibration dataset required.
• CI/CD integration. Automated compression triggered on model release, with webhook triggers, CLI tooling, and API access. Compress once, deploy everywhere.
• Fleet and cluster deployment. Across Apple Silicon clusters, on-prem GPU infrastructure, or air-gapped and sovereign environments.
• Broad architecture support. Dense, Mixture-of-Experts, and multimodal models, on the same automated workflow.
• Batch compression for whole model catalogues, and multi-format export to MLX, GGUF, and custom targets.
• Dedicated engineering support and an SLA.

The hardware story is the part finance cares about. A RAM-compressed 31GB model fits comfortably on an AWS g7e.2xlarge (RTX PRO 6000, 96GB VRAM), an Azure NC24ads A100 80GB, or a GCP a2-highgpu-1g (A100 40GB).

*List pricing before corporate cloud discounts, based on a 10-hour workday. The Mac Studio runs RAM-compressed models natively via MLX with no cloud dependency at all.

Compression you can hand to an auditor

Here's where Shepherd stops being a convenience tool and starts being something a regulated organization can actually deploy. Every Shepherd Enterprise compression build is certified by Watchman, our model-provenance product, before it ships.

That certification is not a benchmark score. It's an evidence-grade provenance report plus a CycloneDX AI-BOM attestation confirming the compressed model is your declared base plus the declared quantization and nothing else, with pass/fail gating against your policy. Models that don't meet your standard never reach production. You get traceability from source model to deployed artifact, in one pipeline, with no separate tooling and no manual validation step.

This matters because compression is exactly the place a supply-chain problem can hide. A modified weight is easy to pass off as "just quantization noise" if nobody's checking. Watchman checks, on every build, which is why the two products ship together.

Where Shepherd fits

If you're weighing a private-AI deployment, the sequence Shepherd runs is the one you'd otherwise assemble by hand and get wrong somewhere: take the model, compress it without destroying capability, prove that's what you did, and push it to hardware you control. The RAM benchmark says the capability survives. The Watchman certification says you can prove it. The cost tables say your finance team will sign off.

A free individual edition, rebuilt on the same RAM engine, is coming for personal use on Apple Silicon. But the enterprise platform is available now, and it's the one that does compression, assurance, and fleet deployment as a single pipeline.