Frontier models are the default reach for every new feature. They shouldn't be. We moved three production workloads off frontier APIs to smaller domain-tuned models and the numbers were better than we expected.
Every AI feature we shipped between 2023 and 2025 started life as a call to a frontier model. It was the right default at the time - capabilities were moving fast, smaller models were not yet good enough at instruction-following and the engineering cost of self-hosting anything substantial outweighed the API bill. By late 2025 those assumptions had quietly stopped being true. Smaller models had crossed a competence threshold for a lot of bounded enterprise tasks, the tooling for fine-tuning and serving had matured and our frontier bill had grown faster than our usage. We had three workloads where the per-call economics no longer made sense and we ran an experiment on each of them.
This is the field report.
We didn't try to move everything. The pattern that made a workload a good candidate was specific: high call volume, bounded task shape, predictable input distribution and tolerance for a narrow domain. Open-ended reasoning, novel problem solving and anything where the input shape kept surprising us - those stayed on the frontier.
The three workloads we moved:
All three were paying frontier prices for tasks that didn't need frontier reasoning.
For each workload, the playbook was the same. Collect a representative dataset from production traces - anywhere from 5K to 20K examples. Generate ground-truth labels using a frontier model with a careful prompt, then have a human review a stratified sample. Fine-tune a small open-weights model on the dataset. Stand up an inference endpoint behind a feature flag. Shadow the frontier model in production for two weeks, comparing outputs. Cut over when the small model matched or beat the frontier on the eval suite.
The model choices varied - different workloads suited different base models - but the size class was consistent: in the 4B to 14B parameter range, with reasoning-tuned variants where available. Big enough to handle the task, small enough that a single mid-tier GPU served meaningful throughput.
# The shape of what we now think of as "the request"
result = small_model.complete(
prompt=domain_prompt,
input=document_text,
max_tokens=256,
)
# Same interface as the frontier call. The swap was a config change.
Quality, on the in-domain eval suite, was the surprise. Document classification: the small model beat the frontier by a small but consistent margin, because we'd trained it on our actual taxonomy rather than relying on the frontier model inferring it from a prompt. Structured extraction: parity within noise. Intent routing: parity within noise.
Latency dropped roughly four to seven times across the three workloads, with most of the improvement coming from removing the network round-trip to a hosted API. P99 became predictable in a way it never was on a shared API endpoint.
Cost per call dropped by between one and two orders of magnitude depending on the workload, once we'd amortised the fine-tuning and the inference infrastructure. The smaller the model and the higher the call volume, the better the math.
Out-of-distribution inputs got worse, not better. This was the headline risk and it played out exactly as predicted. The frontier model handled weird inputs gracefully - a malformed document, an unexpected language, an edge-case schema variant - by falling back on broad capability. The small model handled them poorly. We added an out-of-distribution detector in front of each small model that routes unfamiliar inputs back to the frontier. About 2% of traffic now takes that fallback path and it accounts for almost all our remaining frontier spend on these workloads.
Drift hit faster than expected. The frontier model's broad training masked changes in our input distribution. The small model had no such cushion. When the upstream document format changed in one of the compliance workloads, the small model's accuracy fell off within days. We now monitor input-distribution drift as a first-class signal, not just output quality.
Operational ownership shifted. Calling a frontier API has a clear ownership boundary - the vendor owns the model, you own the prompt. Self-hosting a fine-tuned small model means you own the model lifecycle: re-tuning when data shifts, evaluating new base models when they release, managing the inference infrastructure. This is real work and we didn't fully cost it into the original business case. It's still worth it at our volumes, but the savings calculation needs to include an engineer's time, not just GPU hours.
Anything where the task surface was open-ended. The customer-facing assistant that does multi-turn reasoning over arbitrary topics stayed on the frontier. The code-generation features stayed on the frontier. The deep-research synthesis stayed on the frontier. For those workloads, frontier capability is the product, not an implementation detail.
The principle that emerged: replace frontier calls when the task is narrow enough that capability headroom is being wasted. Keep frontier calls when capability headroom is what you're paying for.
The default reach for a frontier model on every new feature is a habit, not a strategy. For bounded, high-volume, domain-specific tasks, a fine-tuned small model now wins on quality, latency and cost - often by margins large enough to fund the engineering work several times over. The right architecture for most production AI stacks in 2026 is mixed: small models on the hot path for tasks they handle well, frontier models on the fallback path and on genuinely open-ended workloads. Building the routing and the drift monitoring is the work that makes that architecture viable.