Almost every AI roadmap eventually collides with the same fork: do we build this ourselves, or buy it off the shelf? The answer matters more in AI than in traditional software, because the underlying technology is moving fast enough that a decision made today can look naive in two quarters.

The instinct to build is strong. Engineers like building, vendors are expensive, and there's a seductive story about "owning your AI stack." But that framing usually conflates three very different layers, and the right answer is different for each.

The three layers, and who should own them

The model layer — the foundation model itself — is now effectively a commodity input. Multiple capable frontier and open-weight models exist, prices have fallen sharply per token, and the gap between the best proprietary model and the best open model on most business tasks has narrowed. Almost no business should be pre-training its own foundation model. The economics only work at hyperscaler scale, and even then the depreciation curve is brutal. You buy (or rent) this layer.

The orchestration layer — retrieval, tool-calling, routing between models, guardrails, and the eval harness — is where the interesting build-vs-buy tension actually lives. There are credible platforms that sell this as a product. But this is also the layer most tightly coupled to your specific data and workflows, which is exactly why off-the-shelf orchestration often feels 80% right and maddeningly wrong on the last 20%.

The application layer — the thing a user or customer actually touches — is where your differentiation has to come from, if it comes from anywhere. This is almost always build, because it encodes your domain knowledge, your brand, and your particular sequence of steps.

The hidden cost of 'build'

The trap is judging build-vs-buy on the cost of version one. A competent team can stand up a retrieval-augmented chatbot over internal docs in a sprint. The bill arrives later.

Models get deprecated and replaced, and every migration means re-running and re-tuning your prompts and evals. Retrieval quality degrades silently as your document corpus grows and shifts. Someone has to own the on-call rotation when the thing hallucinates a wrong answer to a customer at 2am. And the eval suite — the only thing standing between you and silent regressions — is itself a product that needs continuous maintenance.

None of this shows up in the build-vs-buy spreadsheet on day one. All of it shows up by month six. When teams say a bought solution is "too expensive," they're often comparing a vendor's all-in price against their own initial build cost, not their own total cost of ownership.

A heuristic that holds up

The cleanest rule of thumb: build where the system touches proprietary data or a defensible workflow; buy everything that's a horizontal capability.

Transcription, generic document parsing, content moderation primitives, model hosting, observability — these are horizontal. Someone else can build them once and amortise the cost across a thousand customers, and they'll do it better than you because it's their whole company. Paying them is not a failure of ambition; it's leverage.

Conversely, the agent that knows how your claims process works, ranks your inventory, or reasons over your customer history — that's not something a vendor can sell you, because it's made of your data and your decisions. Build it.

The migration tax nobody budgets for

There's a second-order cost worth naming explicitly: the model-migration tax. If you've hard-wired your application to one model's quirks — its exact prompt format, its tendency to over-explain, its particular failure modes — you've created switching costs against yourself. When a cheaper or better model ships, you can't move without re-validating everything.

The defensible architecture treats the model as a swappable component behind an interface, with a strong eval suite as the gate. That way "buy" and "build" stop being a one-time decision and become a dial you can turn as the market moves. The teams that win the next two years won't be the ones who picked the right model — they'll be the ones who built so they never had to bet the company on a single one.