An AI feature ships, usage grows, and a few weeks later someone in finance asks why the inference bill has a comma in it that wasn't there before. This is one of the most common — and most avoidable — failure modes of putting AI into production. Runaway AI costs almost always trace to the same short list of causes, and each one is fixable once you can see it.

Cost scales with tokens, not requests

The first thing to internalise: you're billed by tokens, not by requests. Every token in your prompt and every token in the response costs money. That has a non-obvious consequence — a feature can look cheap per call in testing and become alarming at scale, because the per-call cost was never the thing to watch. The thing to watch is tokens × volume, and volume is exactly what changes when a feature succeeds.

It also means cost is something you design, not just something that happens to you. The same task can cost wildly different amounts depending on how you build it.

The big levers

A handful of techniques deliver the overwhelming majority of savings. In rough order of impact:

Route to the right model. This is the single biggest lever. The most common waste is sending every request to an expensive frontier model when most of them are easy. A cheap, fast model handles summarisation, classification, extraction, and routine generation at a fraction of the cost and near-identical quality. Reserve the expensive model for the genuinely hard minority of requests, triggered by a classifier or a confidence check. Teams that adopt model-routing routinely cut costs by large multiples with no visible quality loss.

Trim the prompt. Bloated prompts are a quiet, constant tax. Every request that drags along a giant system prompt, redundant instructions, or far more retrieved context than the answer needs pays for those tokens on every single call. Audit what you're actually sending. Often a prompt can be cut substantially with no quality impact — pure savings, every request, forever.

Cache repeated work. A surprising amount of AI traffic is repetitive — the same or similar questions, the same documents processed again, the same large prompt prefix reused across calls. Caching answers to common queries, and using prompt-caching features that let you reuse a fixed prefix cheaply, eliminates paying full price for work you've already done.

Mind the reasoning tokens. Reasoning models can generate many times more internal tokens than the visible answer — and you pay for all of them. Using a thinking model for a task that doesn't need deliberation is a stealthy way to multiply your bill. Match the model's effort to the task's actual difficulty.

Cap output length. Letting models ramble costs money. Setting sensible output limits and asking for concise responses trims the most expensive tokens — the generated ones.

Measure first, or you're guessing

None of the above is actionable without visibility. The foundational discipline — the one that makes every other optimisation possible — is per-feature cost tracking. You need to know which features, which prompts, and which users are driving the bill. Without that, optimisation is guesswork and you'll spend effort shaving costs off things that barely matter while the real culprit runs unchecked.

Good cost observability tracks tokens and spend broken down by feature and request type, so you can see at a glance where the money goes. Almost always, a small number of features or prompts account for most of the cost — and that's where to aim. Tag your requests, log the token counts, and review the breakdown regularly.

A pragmatic cost checklist

• Track cost per feature — you can't manage what you can't see.
• Route aggressively — cheap model for the easy majority, expensive model for the hard minority.
• Audit and trim prompts — cut every token that doesn't earn its place.
• Cache repeated answers and reuse fixed prompt prefixes.
• Right-size the model's effort — don't pay for reasoning the task doesn't need.
• Cap output length and ask for concision.
• Set alerts so a cost spike is something you notice in hours, not in a monthly invoice.

The mindset

The teams that run AI affordably in production aren't using secret pricing. They treat cost as a first-class engineering concern from day one: they measure it, they route around the expensive paths, they trim relentlessly, and they cache what repeats. AI cost isn't an uncontrollable force of nature — it's a design variable. Build with that in mind and the comma in the invoice stays where you put it.