Spend any time around AI and you're soon swimming in numbers: a "70-billion-parameter" model, a "405B" flagship, a "3B" tiddler that runs on your laptop. They get quoted like horsepower figures, with the unspoken implication that bigger is better. So it's worth knowing what a parameter actually is — and why the count tells you a lot about a model's cost and very little, on its own, about how good it is.

What a parameter actually is

Inside a language model are billions of numbers, and a parameter is just one of them: a single adjustable value, set during training. Picture an enormous panel of tiny dials — billions of them — each of which can be tuned up or down. Training is the process of nudging every one of those dials, over and over, until the whole arrangement does a good job of predicting text. When that's finished, the settings are frozen. A "70-billion-parameter model" is simply a model with 70 billion of these learned dials.

What's strange and important is where the model's knowledge lives. It isn't a database of facts you could look up. Everything the model "knows" — grammar, facts, reasoning patterns, style — is smeared across those billions of dial settings, stored as a vast web of numerical relationships rather than as anything you could point to. More parameters means more dials, which means more raw capacity to capture detail and nuance.

Why the number matters — for two very different reasons

The parameter count drives two things, and it's worth keeping them separate.

The first is capacity. Broadly, more parameters give a model more room to store knowledge and handle complexity. This is the real reason the biggest models tend to be broadly knowledgeable: it takes a lot of dials to hold a lot of the world.

The second, and the one people forget, is cost. Every parameter has to be stored in memory and used in a calculation every time the model generates a word. So a model's size dictates the hardware it needs and the speed and price of running it. A 70-billion-parameter model wants serious, expensive kit; a 3-billion one can run on a decent laptop. Size isn't just a capability dial — it's a cost dial, and the two pull against each other.

Why bigger isn't simply better

Here's the trap in treating the number as a quality score. Parameter count measures capacity, not competence — and the two come apart all the time.

A smaller model trained on better data, with a smarter design, routinely beats a larger model that was trained carelessly. The architecture matters too: a mixture-of-experts model can carry a huge total parameter count but only use a fraction of it for any given word, getting big-model knowledge at small-model running cost. And the efficiency race is real: as we saw this month with a 3-billion-parameter model matching a frontier system on maths and coding, a compact model can rival a giant on narrow, well-defined tasks — even as it falls behind on broad knowledge, which genuinely does need scale. Comparing two models by parameter count alone, especially across different families, tells you very little about which is actually better at the thing you care about.

The takeaway

A parameter count is best read as a rough measure of a model's size and expense, not its intelligence. It answers "how heavy is this thing, and will it run on my hardware?" far better than "is it any good?" Bigger models genuinely tend to know more, because broad knowledge needs the room — but smaller, sharper, better-trained models keep proving that the headline number is only part of the story. When you next see "405B" or "8B" attached to a model, read it the way you'd read an engine size: useful context, not a verdict.