mybreathtaking

Articles › Physics Around Us

What's the Difference Between a CPU and a GPU?

· 阅读中文版

How Does a Photo Get Stored in Your Phone? explained how bits are stored. But storing them isn’t enough: photos have to be shown and touched up, videos played, AI questions answered, and all of that takes computing. In phones and computers, two chips do most of it: the CPU (central processing unit) and the GPU (graphics processing unit).

GPUs were made to draw pictures for games, yet today they are the workhorses of artificial intelligence. How are they different from CPUs? Both are made of the same transistors, so why are they good at such different things?

Two common ideas

There are two common ideas:

  • “A GPU is faster than a CPU.” Not quite. Some jobs are far faster on a GPU, others far faster on a CPU.
  • “A GPU is just for the display.” It started that way, but most of the world’s most powerful GPUs today aren’t connected to a screen at all: they sit in data centres training artificial intelligence.

Transistors: switches that get hot

CPUs and GPUs are both made of transistors, the little switches from How Does a Photo Get Stored in Your Phone?, minus the electron-storing “island”: put a voltage on the gate and the switch turns on; remove it and it turns off. Wire switches together and they can add numbers and compare them; any computation, however complex, comes down to a huge number of switches turning on and off.

A chip has a “clock,” and at each tick its switches flip. Three billion ticks a second is 3 GHz.

How short is a tick? At 3 GHz it lasts 0.33 ns, and in that time light travels only 10 cm. The computing pioneer Grace Hopper used to hand out pieces of wire 30 cm long at her talks: “a nanosecond,” the distance light travels in a billionth of a second. Signals on a chip are much slower than light. The electrons in a wire hardly move at all, creeping along at less than 1 mm a second; what travels along the wire is the electric field. In an ideal wire a change in the field travels at about half the speed of light; but the wires on a chip are only tens of nanometres wide and have a high resistance. To move forward, a signal must charge the “capacitor” formed by each stretch of wire and its surroundings through that resistance, like filling a long row of buckets through a very thin pipe. The longer the wire, the more there is to charge and the higher the resistance, so the delay grows with the square of the length. Long wires therefore get a “repeater” every so often to boost the signal; even so, a signal needs tens of picoseconds per millimetre, between a few hundredths and about a tenth of the speed of light.

In one tick, a signal has to leave one set of switches, pass through ten to twenty layers of switching logic, and arrive before the next tick. The faster the clock, the less time there is for that journey, so a core must be compact and its most-used data must sit right beside it. A chip is about 1 to 3 cm across, and a signal needs several ticks to cross it; fetching data from the memory sticks on the motherboard takes tens of nanoseconds to a hundred, two or three hundred ticks. That is why chips carry their own “cache,” keeping frequently used data close at hand.

Every time a switch flips, a tiny capacitor is charged and discharged: each charge takes an energy of about CV2CV^2 (where CC is the capacitance and VV the voltage), and on discharge that energy ends up as heat. So the heat a chip gives off is roughly

P≈αCV2fP \approx \alpha\, C V^2 f

where ff is the clock frequency and α\alpha the fraction of switches that actually flip on each tick. Faster ticks, more switches and higher voltage all mean more heat.

Smaller and smaller, yet the clock stopped rising

For decades engineers made transistors smaller and smaller. In 1974 Robert Dennard and colleagues at IBM pointed out that when a transistor shrinks, its capacitance falls and its voltage can be lowered too, so each switch makes much less heat. The same chip could then hold more transistors and run a faster clock with no more heat in total. This carried clock speeds from under 1 MHz in 1971 to over 3 GHz in the early 2000s.

More and more transistors, but the clock barely rises any more. Top: transistors on representative CPUs (log scale); the dotted line doubles every two years (Moore’s law). Bottom: clock frequency (log scale). From 1985 to 2004 it rose about 240 times; in the twenty years after 2004, the fastest chips only went from 3.8 GHz to 6.2 GHz.

But around 2004 the rise in clock speed stopped. When Intel launched the Pentium 4 in 2000, it said the design could reach 10 GHz; in October 2004 it cancelled the 4 GHz version, and the top speed stayed at 3.8 GHz. Intel gave no specific reason, saying only that it was moving engineers to dual-core chips; the common analysis at the time was that a 3.8 GHz Pentium 4 already drew about 115 W, and any faster it would make more heat than an ordinary PC could get rid of.

In the twenty years since, clock speeds have risen only a little: the fastest desktop chip of 2024, the Core i9-14900KS, reaches 6.2 GHz, and only on one or two cores for short “sprints”; phone chips mostly run around 4 GHz, and data-centre chips often at just 2 to 4 GHz. From 1985 to 2004, by contrast, clock speeds rose about 240 times.

The reason was that the voltage couldn’t come down any further. A transistor has a “threshold voltage”: the switch only turns on when the gate voltage is above it. The lower the threshold, the more easily the switch fails to close completely: electrons in silicon jiggle with heat, some faster, some slower, and a few always have enough energy to get over the threshold, giving a leakage current. By Boltzmann’s distribution, at room temperature every 0.06 V or so off the threshold makes the leakage at least 10 times larger (real transistors are somewhat worse). The threshold couldn’t fall, so the working voltage got stuck at about 1 V.

Why it stopped. Top: switches that don’t quite close; every 0.06 V or so lower threshold voltage makes the leakage when off 10 times larger (room temperature). Bottom: heat given off per square centimetre (log scale): human body about 0.006 W, stove burner about 6 W, the 2004 Pentium 4 chip about 100 W, the Sun’s surface about 6300 W.

With the voltage stuck, every increase in clock speed meant more heat. The chips themselves are small: a computer chip is about 1 to 3 square centimetres, the size of a fingernail; even the largest GPU chips are only about 8 square centimetres, the most a lithography machine can make in one exposure. The Pentium 4 chip was about 1.1 square centimetres and drew about 115 W at full load, about 100 W per square centimetre on average, more than ten times a stove burner; the 2022 H100 chip is about 8 square centimetres at 700 W, still about 86 W per square centimetre. The heat has to pass into a metal lid, then into a heat sink, and be carried away by fans or water. Any faster and it couldn’t get out.

Two roads: a few fast cores, or many slow ones?

The clock couldn’t go faster, but the number of transistors kept growing. How to use them? The answer was to turn one core into several, working together. Ideally the arithmetic is favourable: if the frequency is lowered by 20% and the voltage can be lowered by 20% too, each core makes only about half the heat (0.8×0.82≈0.510.8 \times 0.8^2 \approx 0.51), so two such cores are 60% faster than the original one with about the same heat. In practice the voltage can’t fall that much and leakage remains, so the sums are less neat, but the direction is right.

In 2005 both Intel and AMD launched dual-core chips for personal computers. The Core 2 Duo of 2006 had a lower clock than the Pentium 4, but each core did more work per tick, so it was faster than the Pentium 4 while drawing only 65 W.

But more cores aren’t always better. Every job has a part that must be done step by step, and extra cores can’t help with that part. If a fraction pp of the work can be shared out, then with NN cores the job gets at most

S=1(1−p)+p/NS = \frac{1}{(1-p) + p/N}

times faster; this is Amdahl’s law. Even if 95% of the work can be shared, no number of cores makes it more than 20 times faster: the remaining 5% becomes the bottleneck.

From here CPUs and GPUs took two different roads:

  • A CPU has a few to a few dozen cores, each fast and “clever”: it guesses which way a program will branch, fetches data in advance, and reorders instructions to run them in the fastest order. It is good at work that goes step by step with decisions everywhere, like running the operating system or opening a web page.
  • A GPU has thousands of simple cores, each slower and simpler, but all doing the same kind of job at once. It is good at “doing the same thing millions of times,” like working out the colour of each of millions of pixels.
A few fast cores, or many slow ones? (diagram) Task 1: colour 1024 independent squares. The CPU has only 4 cores, each colouring 1 square per step: 256 steps. The GPU has 64 cores, each taking 4 steps per square, but all 64 work at once: 64 steps. Task 2: 32 steps, each needing the previous result. More cores can only wait in line: the CPU finishes in 32 steps, the GPU needs 128.

So GPUs suit jobs where pp is almost exactly 1. A picture has millions of pixels, and each pixel’s colour can be worked out on its own: exactly what a GPU does best. In 1996 3dfx’s Voodoo card made 3D games popular on personal computers; in 1999 Nvidia put the steps of drawing a 3D scene onto a single chip, the GeForce 256, and called it “the first GPU.”

How GPUs moved into artificial intelligence

In 2006 Nvidia launched the GeForce 8800 GTX together with CUDA, which let programmers use the GPU to compute anything at all, not just pictures.

Most of the computing in artificial intelligence is matrix multiplication: multiplying the numbers in one big table by those in another, pair by pair, and adding them up. Each number in the result can be computed on its own, again just what a GPU does best. In 2012 Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton at the University of Toronto trained a neural network called AlexNet on two GTX 580 gaming cards for five or six days; in an image-recognition contest its error rate was only 15%, against 26% for second place. This is seen as the start of today’s wave of artificial intelligence.

In 2024 Hinton and John Hopfield won the Nobel Prize in Physics “for foundational discoveries and inventions that enable machine learning with artificial neural networks.” The “Boltzmann machine” that Hinton and colleagues proposed in 1985 uses the very Boltzmann distribution we met with leakage: a tool of statistical physics for describing huge numbers of particles became a foundation of artificial neural networks.

Starting with the V100 in 2017, GPUs gained “tensor cores” built just for matrix multiplication, which finish a small block of a matrix in one go. AI doesn’t need high precision either: ordinary computing stores a number in 32 bits (FP32), about 7 significant digits; AI often uses 16, 8 or even 4 bits (FP4), and 4 bits can only represent a dozen or so different numbers. Fewer bits means far fewer switches to flip in a multiplier, so the same energy buys many more operations.

A table: half a century of representative chips

CPUs

Year Chip Process Transistors Clock Cores Why it matters
1971 Intel 4004 10 µm 2.3×1032.3\times10^3 0.74 MHz 1 the first commercial single-chip microprocessor, designed for a Japanese calculator maker
1978 Intel 8086 3 µm 2.9×1042.9\times10^4 5–10 MHz 1 the origin of the x86 instructions in today’s PCs
1985 Intel 386 1.5 µm 2.75×1052.75\times10^5 16–33 MHz 1 32-bit
1993 Pentium 0.8 µm 3.1×1063.1\times10^6 60–66 MHz 1 could execute two instructions at once
2004 Pentium 4 (Prescott) 90 nm 1.25×1081.25\times10^8 3.8 GHz 1 the clock reached 3.8 GHz, about 115 W; the 4 GHz version was cancelled
2006 Core 2 Duo E6700 65 nm 2.91×1082.91\times10^8 2.66 GHz 2 lower clock, more cores, 65 W
2023 Apple A17 Pro 3 nm 1.9×10101.9\times10^{10} 3.78 GHz 6 a phone chip: CPU, GPU and neural engine on one chip
2024 Intel Core i9-14900KS Intel 7 not published 3.2–6.2 GHz 24 a desktop chip; the top speed is reached briefly on one or two cores; base power 150 W
2024 AMD EPYC 9965 3 nm not published 2.25–3.7 GHz 192 a data-centre server chip, 500 W

GPUs

Year Chip Transistors Operations per second (FP32) Low-precision AI (per second) Power Why it matters
1999 GeForce 256 1.7×1071.7\times10^7 — — — called “the first GPU” by Nvidia
2006 GeForce 8800 GTX 6.8×1086.8\times10^8 3.5×10113.5\times10^{11} — 155 W the first GPU for general computing with CUDA, 128 cores
2010 GTX 580 3.0×1093.0\times10^9 1.6×10121.6\times10^{12} — 244 W two of them trained AlexNet; 512 cores
2017 V100 2.1×10102.1\times10^{10} 1.6×10131.6\times10^{13} 1.25×10141.25\times10^{14} (FP16) 300 W the first tensor cores; 5120 cores
2022 H100 8.0×10108.0\times10^{10} 6.7×10136.7\times10^{13} 2.0×10152.0\times10^{15} (FP8) 700 W the workhorse of the large-language-model era
2024 B200 2.08×10112.08\times10^{11} 7.5×10137.5\times10^{13} 9×10159\times10^{15} (FP4) 1000 W two chips in one package
2026 Rubin 3.36×10113.36\times10^{11} 1.3×10141.3\times10^{14} 3.5×10163.5\times10^{16} (FP4) not published 3 nm, shipping from 2026
GPUs: transistors and computing speed (representative chips, log scale). Top: transistors on one GPU, from 17 million in 1999 to 336 billion in 2026. Bottom: operations per second. Blue is 32-bit precision, up about 400 times from 2006 to 2026; purple is low precision for AI, more than two orders of magnitude higher still.

A few numbers side by side:

  • The A17 Pro chip in your pocket has about 8 million times as many transistors as the 4004 of 1971.
  • At 4-bit precision a Rubin does about 3.5×10163.5\times10^{16} operations a second. If all 8.2 billion people on Earth each did one calculation per second, it would take them about 50 days to match one second of it.
  • A B200 draws about 1000 W, about as much as a microwave oven, but concentrated on chips of a dozen or so square centimetres; it needs huge heat sinks and strong airflow, and many data centres switch to liquid cooling instead.

What is the least energy a computation needs?

Almost all the electricity a chip uses ends up as heat: the answer it computes is just a few bits and doesn’t “contain” that energy. So must computing make heat at all? And how little could it make?

In 1961 Rolf Landauer, a physicist at IBM, worked this out: computing keeps erasing old bits, and erasing one bit must release at least kBTln⁡2k_BT\ln 2 of heat to the surroundings (kBk_B is Boltzmann’s constant, TT the temperature). The reason is the second law of thermodynamics: before erasure a bit could be 0 or 1; afterwards only one state remains, and that lost “uncertainty” can’t simply vanish; it has to be handed to the surroundings as heat. At room temperature this limit is about 3×10−213\times10^{-21} J. In 2012 physicists in Lyon, France, held a bead a micrometre across in laser “tweezers,” let it stand for one bit, and measured this limit in an experiment for the first time.

How much energy does one operation take? (log scale) The first four entries are power divided by operations per second. ENIAC in 1946: about 30 J per addition; the 8800 GTX in 2006: about 4.5\times10^{-10} J per 32-bit operation; the V100 in 2017: about 1.9\times10^{-11} J; the B200 in 2024: about 1.1\times10^{-13} J per 4-bit operation. One transistor flipping once is estimated at about 5\times10^{-17} J. At the bottom is Landauer’s limit: erasing 1 bit at room temperature takes at least about 2.9\times10^{-21} J.

In 1946 ENIAC, the first general-purpose electronic computer, used more than 17,000 vacuum tubes and about 150 kW of power to do 5000 additions a second, about 30 J per addition. Today’s GPUs use about 101410^{14} times less energy per operation. Yet they are still tens of millions of times above Landauer’s limit: a 4-bit operation takes about 10−1310^{-13} J, about 4×1074\times10^{7} times the limit, and a single transistor flip alone already costs about 20,000 times the limit.

The limit only applies to “erasing.” If every step could be run backwards and nothing were erased, computing could in principle make almost no heat; this is called “reversible computing.” It is far from practical, but it is a reminder that the heat isn’t the price of computing itself; it is the price of forgetting.

The chip in your phone

A phone has no room for several big chips, so the CPU, the GPU, a “neural engine” for neural-network calculations and an image processor for the camera all sit on one chip. Noise reduction when you take a photo and the blurred background of portrait mode are the GPU and the neural engine working through tens of millions of pixels, each on its own, in a few milliseconds.

In a sentence: CPUs and GPUs are both made of switches, and every flip of a switch makes heat; almost all the electricity ends up as heat. Around 2004 the voltage couldn’t fall any further, so the clock almost stopped rising and chips had to use more cores working together: a CPU has a few fast, clever cores, good at step-by-step decisions; a GPU has thousands of simple cores, good at doing the same thing millions of times at once. The pixels of a picture are like that, and so is the matrix multiplication of artificial intelligence, which is how GPUs went from games to today’s AI.

Try it yourself

  • See how many cores your computer has: on Windows open the “Performance” tab of Task Manager, on a Mac the “CPU” tab of Activity Monitor, and watch how busy each core is.
  • Feel it: play a big game or record video for a while and your phone warms up. That is the heat left behind by billions of switches turning on and off.

What we don’t know yet

How much smaller can transistors get? Today’s “3 nm” is just the name of a process; the narrowest parts of a transistor are only a few dozen atoms across, and electrons are starting to tunnel straight through. Engineers are trying to stack chips layer upon layer and to carry data with light instead of electricity. Another question is energy: a large AI data centre can use as much electricity as a city, and how long artificial intelligence can keep growing like this depends on where the electricity comes from and how the heat gets out.

Going further: numbers, formulas and sources

The energy of a switch. Charging a capacitance CC to a voltage VV draws CV2CV^2 from the supply: half is stored in the capacitor and half turns to heat during charging; the stored half becomes heat on discharge. Dynamic power is P=αCV2fP = \alpha C V^2 f. Dennard scaling (Dennard et al., 1974): shrink dimensions and voltage by 1/k1/k, and capacitance falls by 1/k1/k, frequency can rise by kk, power per transistor falls by 1/k21/k^2, and k2k^2 times more transistors fit in the same area, so power density stays constant.

Leakage. Below threshold, transistor current goes as I∝eqVGS/(nkBT)I \propto e^{qV_{GS}/(nk_BT)}, with n=1n = 1 ideally. At room temperature kBT/q=25.9k_BT/q = 25.9 mV, so a tenfold change in current takes ln⁡10×25.9≈60\ln 10 \times 25.9 \approx 60 mV, the limit of the “subthreshold swing”; real transistors have nn a little above 1, about 70 to 100 mV. The top of Figure 3 is drawn for the ideal case, as a diagram.

Clock frequency. The 386 of 1985 ran at 16 MHz and the Pentium 4 of 2004 at 3.8 GHz, about 240 times more; the Core i9-14900KS of 2024 reaches 6.2 GHz, only about 1.6 times more in twenty years. Intel gave no technical reason for cancelling the 4 GHz Pentium 4; the Prescott Pentium 4 of the time had a thermal design power of 115 W.

Heat per square centimetre. Human body: about 100 W at rest over about 1.8 m², about 0.0056 W per cm². Stove burner: 2000 W over 20 cm diameter, about 6.4. Pentium 4 Prescott: 115 W over a 112 mm² die, about 100. H100: 700 W over an 814 mm² die, about 86. Sun’s surface: σT4\sigma T^4 with T=5772T = 5772 K, about 6300. A lithography machine exposes at most about 26 × 33 mm, about 858 mm², in one go, so a single chip is at most about 8 cm²; B200 and Rubin each package two chips of about that size.

The multicore arithmetic. Multiply frequency and voltage by 0.8 and dynamic power is multiplied by 0.8×0.82=0.510.8 \times 0.8^2 = 0.51; two cores use about 1.02 times the power and deliver about 1.6 times the throughput. This assumes the voltage falls in proportion to the frequency, leakage is ignored and the work splits evenly: an ideal case. The first dual-core PC chips: Intel in April 2005 (Pentium Extreme Edition 840), AMD in May 2005 (Athlon 64 X2).

Figure 1. A diagram: the CPU’s 4 cores each finish 1 square per step, the GPU’s 64 cores each finish 1 square per 4 steps. For 1024 squares, the CPU takes 256 steps and the GPU 64; for a 32-step dependent chain, the CPU takes 32 and the GPU 128. Real GPUs have many more cores, and how a single GPU core compares with a CPU core depends on the task.

The numbers in the tables. Process: “3 nm,” “90 nm” and so on are the names of process generations; early on they roughly matched the smallest feature size, but for about twenty years they haven’t matched any actual dimension. Clock: the range of launch models or the top frequency. CPU power is the vendor’s thermal design power. GPU “operations per second” are vendor peak figures, counting a multiply-add as 2 operations; for FP32, the 8800 GTX is 128×1.35GHz×2128 \times 1.35\,\mathrm{GHz} \times 2 and the GTX 580 is 512×1.544GHz×2512 \times 1.544\,\mathrm{GHz} \times 2; the B200 figure is the HGX B200 eight-GPU system’s 600 TFLOPS divided by 8. The low-precision AI column gives “dense” figures: V100 is tensor-core FP16; H100 SXM is FP8 and B200 is FP4 for HGX B200 (1000 W), where vendors often quote figures with sparsity, twice as large; Rubin is FP4, using the training figure Nvidia labels dense; its advertised 5×10165\times10^{16} per second is an inference figure that relies on compressing the data. B200 and Rubin are each two chips packaged as one GPU. Nvidia has not published Rubin’s power. The A17 Pro transistor count is from Apple’s launch event; the makers have not published transistor counts for the Core i9-14900KS or the EPYC 9965. The i9-14900KS has 8 performance and 16 efficiency cores, and 6.2 GHz is the performance cores’ highest boost. The EPYC 9965’s compute dies use a 3 nm process and its input/output die 6 nm.

Distance per tick. At 3 GHz one tick is 0.333 ns, in which light travels 10.0 cm; in 1 ns light travels 29.98 cm (11.8 inches), the length of Hopper’s wires. Electron drift speed is v=J/(ne)v = J/(ne): with a high on-chip current density J=1010J = 10^{10} A/m² and copper’s free-electron density n=8.5×1028n = 8.5\times10^{28} per m³, v≈0.7v \approx 0.7 mm/s. On an ideal transmission line a signal travels at c/εrc/\sqrt{\varepsilon_r}; for silicon dioxide, εr≈3.9\varepsilon_r \approx 3.9, about 0.5c0.5c (about 6.6 ps per mm). Thin on-chip wires have a high resistance and act as resistor-capacitor (“RC”) lines: without repeaters the delay is about 12rcL2\tfrac{1}{2}rcL^2 (rr and cc per unit length), growing with the square of the length; with repeaters it becomes proportional to length, about 20 to 90 ps per mm in the literature, roughly 4% to 15% of the speed of light. Reading from memory sticks typically takes tens of nanoseconds to a hundred, mostly spent on operations inside the memory chips, not only on the distance.

Amdahl’s law. With p=0.95p = 0.95, N=64N = 64 gives about 15 times and N→∞N \to \infty at most 20 times; with p=0.99p = 0.99, about 39 and 100 times.

Landauer’s limit. kBTln⁡2k_BT\ln 2 is 2.87×10−212.87\times10^{-21} J at T=300T = 300 K. In Figure 5 a transistor flip is estimated as CV2CV^2 with C=0.1C = 0.1 fF and V=0.7V = 0.7 V, about 4.9×10−174.9\times10^{-17} J, about 2×1042\times10^4 times Landauer’s limit. GPU energy per operation is board power divided by peak operations per second, an order of magnitude only. ENIAC: 17,468 vacuum tubes, about 150 kW, 5000 ten-digit additions a second. A ten-digit decimal addition on ENIAC and a 4-bit operation on a GPU are not equivalent; only the orders of magnitude are compared.

Conversions. 1.9×1010/2.3×103≈8×1061.9\times10^{10}/2.3\times10^{3} \approx 8\times10^6. 3.5×1016/(8.2×109)≈4.3×1063.5\times10^{16}/(8.2\times10^9) \approx 4.3\times10^6 s, about 49 days. 1.1×10−13/2.9×10−21≈4×1071.1\times10^{-13}/2.9\times10^{-21} \approx 4\times10^{7}.

Sources. R. Landauer, IBM Journal of Research and Development 5, 183 (1961). A. Bérut et al., Nature 483, 187 (2012). ENIAC: H. Goldstine, Mathematical Tables and Other Aids to Computation 2, 97 (1946). Hopper’s “nanoseconds”: National Museum of American History collection. G. Amdahl, AFIPS 1967. R. Dennard et al., IEEE Journal of Solid-State Circuits SC-9, 256 (1974). Intel cancels the 4 GHz Pentium 4: Reuters, 14 October 2004. Intel Core i9-14900KS: Intel launch materials, 14 March 2024. 2024 Nobel Prize in Physics: nobelprize.org. Wire delay: the ITRS roadmap and research on on-chip global interconnects. AlexNet: Krizhevsky, Sutskever and Hinton, NeurIPS 2012. GPU specifications: Nvidia product information for GeForce 256, 8800 GTX and GTX 580, the V100 and H100 datasheets, and the HGX platform page (B200, Rubin). Rubin shipments: Nvidia’s quarterly results, November 2026. Apple A17 Pro: Apple’s event of 12 September 2023. AMD EPYC 9965: AMD launch materials, October 2024.