How Much Compute Did Famous AI Models Take? 540 Models, 1950–2026

Shannon’s Theseus, from 1950, was a magnetic mouse that learned its way through a 25-square maze. Epoch AI estimates its training at 40 floating-point operations, the smallest figure among the 540 notable models in Epoch’s dataset that have an estimate. The largest estimate belongs to OpenAI’s GPT-6 Astra, published in September 2026, at about 1.0 × 10²⁷, though Grok 4 and GPT-4.5 sit inside its error bars. By Epoch’s estimates, that is about 25 septillion times as many operations as the smallest. Both ends also appear in Scale Guess, our daily game about AI training compute.

A floating-point operation, or FLOP, is one arithmetic step, such as a multiply or add, on a number stored in floating-point form. Training compute is the total number of those steps a model needed to learn. For Theseus, Epoch counts each relay switch, as the mouse checks the walls, as one operation.

Only a log scale holds both ends of that range. Each step up the axis multiplies rather than adds; on a linear axis, every model before 2020 would sit flat on the floor.

Two speeds

Epoch’s list of notable models has 1,072 entries, 540 of them with a training-compute estimate. We fitted a straight line through the logarithm of those estimates against publication date, separately before and after 2010.

The two lines have very different slopes. The 62 models before 2010 grow about 1.5 times per year, a doubling roughly every 20 months. The 478 models since 2010 grow about 4.3 times per year, a doubling roughly every 5.7 months. Start the window anywhere from 2016 to 2021 and the fit rises to about 5 to 6 times per year (5.8 from 2018, a doubling every 4.7 months), though a shorter window is noisier.

This is a simple fit on Epoch’s dataset, not a new finding, and it lands close to Epoch’s own figures. Their 2024 report Training compute of frontier AI models grows by 4-5x per year, which covers 2010 to May 2024, finds about 4.1 times per year for notable models and 5.3 for frontier models, which it summarises as 4 to 5 times. Their data insight on the same question is titled The training compute of notable AI models has been doubling roughly every six months. Epoch attributes most of that growth to bigger budgets, with better hardware playing a smaller part.

The slower pre-2010 pace matches earlier work. Compute Trends Across Three Eras of Machine Learning, a 2022 paper by Jaime Sevilla and colleagues, the group that went on to found Epoch, found that before 2010 training compute grew “in line with Moore’s law, doubling roughly every 20 months”.

The big jumps

Single steps from the 149-model Scale Guess pool show the same thing at closer range.

• LeNet-5 (1998) to AlexNet (2012): about 170,000 times more compute across 14 years. AlexNet won the 2012 ImageNet competition and is the usual starting point for the deep learning era. Both estimates are rated Confident.
• AlexNet to the Transformer (2017): only about 16 times. The Transformer paper introduced an architecture, not a record training run. AlphaGo Zero, published the same year, used about 88 times as much as the Transformer.
• GPT-1 (2018) to GPT-2 (2019): about 110 times. Epoch rates GPT-1 Likely and GPT-2 Speculative, so hold this one loosely.
• GPT-2 to GPT-3 (2020): about 160 times. GPT-3’s estimate is rated Confident.
• GPT-3 to GPT-4 (2023): about 67 times, with GPT-4 rated Likely. From GPT-1 to GPT-4 the multiple is about 1.2 million, in under five years.
• LLaMA 65B (February 2023) to Llama 3.1 405B (July 2024): about 69 times in roughly 17 months, both Confident. Meta stated the larger model’s compute in its own paper.

At the post-2010 trend of 4.3 times a year, 17 months would buy a factor of about 8. The LLaMA step was roughly nine times bigger than that.

Bigger is not the whole story

Compute tracks size, but not only size. DeepSeek-V3, released in December 2024, has 671 billion parameters, more than Llama 3.1 405B’s 405 billion. Epoch still puts its training compute at 3.3 × 10²⁴ FLOP, about one eleventh of Llama 3.1 405B’s 3.8 × 10²⁵. DeepSeek-V3 is a mixture-of-experts model: only 37 billion of its parameters are active for each token, and Epoch’s estimate is built on that active count, at six operations per active parameter per token over 14.8 trillion tokens. Both are rated Confident, within 3 times either way, and the 11.5-fold gap is larger than 3 × 3 = 9, so the ordering survives even if both errors run against it.

The AlphaGo pair is a different case. AlphaGo Zero (2017), which learned Go without human game records, comes out at 6.5 × 10²⁰ FLOP, about a third of the 1.9 × 10²¹ Epoch estimates for AlphaGo Lee, the version that played Lee Sedol in 2016. A tidy story about smarter methods, but the data cannot carry it. Epoch derived the AlphaGo Lee figure by multiplying its AlphaGo Fan estimate by five, and rates it Speculative. A 2.9-fold gap sits well inside that error. Epoch’s AlphaGo Zero figure also counts only the direct training, not the compute spent generating its self-play games. We cannot call this one.

Reading the estimates

Epoch gives every record a confidence label covering its compute, parameter and dataset figures. In Epoch’s words, ‘Confident’ records are ‘accurate within a factor of 3x’, Likely within 10x and Speculative within 30x. Of the 540 estimates, 321 are Confident, 152 Likely and 67 Speculative.

Newer models are not rated worse on that scale. Since the start of 2023, 21 of the 172 models with an estimate are Speculative, about 12 percent, almost the same share as before 2023 (46 of 368). What has changed is how many notable models get no estimate at all: 230 of the 402 published since 2023, or 57 percent, against 45 percent of the earlier ones. The file does not say why; thin public disclosure and a backlog of recent releases would both do it.

Error bars decide what Scale Guess will ask. Stable Diffusion (2022) and DALL-E (2021) are about 6 percent apart in Epoch’s estimates, 5.0 against 4.7 × 10²², with one rated Confident and the other Likely. Nobody can say which took more. Even GPT-4 and GPT-6 Astra, about 48 times apart, do not qualify, because both are Likely and 10 × 10 = 100. The game’s rule is that a pair only appears when the ratio between the two estimates is at least the two error factors multiplied together. Then the answer holds even if both estimates are off by their full stated error in opposite directions.

Data and method

• Source. Epoch AI, Data on AI models, released under CC BY 4.0. We used the notable-models file, downloaded on 28 September 2026: 1,072 models, 540 of them with a training-compute estimate.
• What notable means. Epoch flags a model as notable if it advanced the state of the art, was highly cited (over 1,000 citations), had over a million monthly users, was historically significant, or cost over a million dollars to develop. The file also tags some entries as discretionary picks.
• The fit. Ordinary least squares of log10 training compute on publication date, expressed as a decimal year, split at 1 January 2010. The pre-2010 fit explains about 69 percent of the variation in its period and the post-2010 fit about 66 percent, so plenty of models sit well off their line.
• The game pool. The 149 Scale Guess models are a subset of the same file that we picked by hand, each with a short note of our own.
• Limits. Some famous models have no estimate in the file, including Claude 3 Opus, GPT-4o, and Gemini 1.5 Pro and every later Gemini model. Dates are publication dates, not the dates training finished.

Try the gaps yourself

Scale Guess gives you ten pairs a day from the 149-model pool: three with gaps over 1,000 times, four between 30 and 1,000, and three under 30. Every answer shows both models’ estimated compute and flags the speculative ones. If the race interests you more than the numbers, Singularity Rush is our real-time strategy game about four fictional labs competing to finish a Gen 5 training run first. And Loss Curve puts you in charge of the runs themselves: an incremental game whose loss curve is the Chinchilla scaling law.