Why do the big AI labs keep spending fortunes to make their models bigger? The short answer is a set of empirical findings called scaling laws — and they are the closest thing modern AI has to a law of physics.

The finding

A scaling law is a measured relationship: as you increase a model's scale, its error falls in a smooth, predictable way. In 2020 a team at OpenAI published Scaling Laws for Neural Language Models, showing that a language model's test loss — roughly, how surprised it is by the next word — drops as a power law in three things: the number of parameters, the size of the training dataset, and the amount of compute used to train it. Crucially, the trends held across more than seven orders of magnitude, and they were smooth. No plateaus, no sudden jumps — just a steady, forecastable decline.

That predictability is the whole point. Before scaling laws, "make it bigger" was a hunch. After them, it became a forecast: you could train a series of small models, fit the curve, and predict — before spending the money — roughly how good a much larger model would be. That is what turned billion-dollar training runs from a gamble into a capital-expenditure decision.

The Chinchilla correction

The 2020 recipe had a flaw, and DeepMind found it. In 2022, in Training Compute-Optimal Large Language Models, its researchers showed that the industry had been building models that were too big and trained on too little data. For a fixed compute budget, they found, you should scale the number of parameters and the number of training tokens roughly in step — on the order of twenty training tokens for every parameter.

The demonstration was pointed. Their model, Chinchilla, had 70 billion parameters — a quarter the size of DeepMind's own earlier 280-billion-parameter Gopher — but was trained on far more data, and it beat the bigger model across a wide range of tests. The lesson reshaped how every lab since has spent its compute: data, not just parameters, is the lever, and a smaller model fed more tokens often wins.

The caveats that matter

Two things are worth holding onto. First, scaling laws describe loss, not capability directly. A lower next-word error correlates with a more capable model, but the link is loose — the specific abilities that emerge as models scale are much harder to predict than the loss curve itself, and that gap is where a lot of the field's surprises live.

Second, the pretraining version of scaling is running into a wall — a physical one. The recipe needs ever more high-quality training data, and there is only so much text in the world; the phrase researchers use is the data wall. That constraint is a big part of why the frontier has shifted. Labs now chase data quality and efficiency rather than raw size, which is how you get a capable 30-billion-parameter model built to run on a single machine. And they have opened a new axis entirely: spending compute at inference time rather than only in training — letting a model think for longer before it answers, which buys accuracy from a budget the original scaling laws never measured.

Why it still matters

Scaling laws explain the shape of the last five years: the relentless growth in model size, the arms race for compute and data, and the confidence with which labs committed to runs that cost more than most companies are worth. They also explain the present pivot. The smooth curve has not broken — but the cheapest inputs to it, especially data, are getting scarce, and the industry is now optimising the recipe rather than simply pouring more in. Understanding that one relationship between scale and error is the fastest way to understand why AI has looked the way it has, and why it is starting to look different.