Your Eyes Only Count to Three: How Do We See Color?

After our last article, Why Do Autumn Leaves Turn Yellow and Red, Little by Little?, a friend asked a great question:

In the second chart, the leaf reflects a huge amount of light above 700 nm, far more than the green light. So why isn’t the leaf the color of that light?

The answer isn’t in the leaf. It’s in our eyes.

Take a guess

One guess: there just isn’t much sunlight up there. Not true; sunlight is plenty strong at those wavelengths. Another guess: that’s infrared, and our eyes can’t see it. That’s only half right. Light from 700 to 780 nm still counts as visible, but our eyes are already very dull to it.

To really explain it, we first need to know how the eye “sees” color.

Your eyes only keep three numbers

The cells in the eye that tell colors apart are cones, and there are three kinds. Each has its favorites: L cones lean toward red, M cones toward green, and S cones toward blue.

When light enters the eye, each kind of cone adds up all the colors in it, weighted by its own taste, and gets one number: how much this light excites it. So no matter how complicated the spectrum is, the eye keeps just three numbers and passes them to the brain. A color is the balance between those three numbers.

Back to the leaf

Multiply the light coming off the leaf by the sensitivity of each kind of cone, and you get the signal the eye actually receives:

In the top row, the big block of light above 700 nm is half of everything the leaf reflects. But look at the middle row: above 700 nm, all three kinds of cones have nearly flattened out. At 700 nm, even the most sensitive L cones are down to 0.6% of their peak; at 750 nm, to 0.015%. Multiply them, and in the bottom row the big block above 700 nm almost disappears. It is just 0.06% of the eye’s signal.

What really sets the leaf’s color is the modest bump between 500 and 600 nm. It excites the M cones a little more than the L cones, and the S cones only slightly, and the brain reads that as green.

A beam of light sweeps the rainbow

The edge of visible light isn’t a wall; it’s a slope that slowly fades away. Let a beam of single-color light of the same strength sweep from ultraviolet to infrared, and watch how the three kinds of cones respond:

In the middle, the cones take turns getting excited, and the color runs from violet through blue, green and yellow to red. At both ends the signals grow weaker, the color grows darker, and finally it goes black. A TV remote uses infrared light around 940 nm. None of the cones respond to it at all, which is why we see nothing when we press a button.

One yellow, three completely different lights

If the eye keeps only three numbers, something remarkable follows: any lights that produce the same three numbers look like the same color, however different they really are.

The first row is sunlight reflected by a yellow autumn leaf; its spectrum is wide and gentle. The second row is “yellow” on a phone screen. A screen has only tiny red, green and blue lights; here the red and green ones are lit, plus a little blue. There is no yellow light in it at all. The third row is a lamp that gives off only a narrow band of yellow light.

The three spectra look nothing alike, yet the three numbers are almost the same, so the eye can’t tell them apart. This is called metamerism. It’s why phones, TVs and computer screens can show thousands of colors with just three colors of tiny lights: they don’t have to copy the real spectrum, only fool the three kinds of cones.

Look closely, though: the lamp’s S signal is 0, a little lower than the other two, so its yellow looks a bit purer and more vivid. Change just one of the three numbers, and the color changes too.

In one sentence: color isn’t in the light; it’s in the eye. The eye squeezes a whole spectrum into three numbers, and however much infrared light there is, the three kinds of cones ignore it, so we don’t see it.

How do we know?

• Look at a TV remote with your phone. Try it at home: point a TV remote at your phone’s front camera and hold a button. On the screen you’ll see the remote’s light flicker, while your eyes see nothing. The phone’s sensor can “see” near-infrared light; our cones can’t. (Many phones have an infrared-blocking filter on the rear camera, so the front camera works better.)
• Look at a screen through a magnifier. Fill your phone screen with pure yellow and look at it through a magnifying glass or a macro lens. You’ll see only red and green dots.
• Color vision deficiency. If one kind of cone is missing, or two kinds have sensitivities that are too close, two colors that used to give different numbers can become impossible to tell apart. The most common kind is red–green color vision deficiency, which is not rare among men.
• Measuring the cones. In the 19th century, James Clerk Maxwell mixed three colors of light to match all sorts of colors. Later, scientists measured directly how single cone cells absorb each wavelength. The cone curves used here (Stockman and Sharpe, 2000) come from such measurements together with many color-matching experiments.

What we still don’t know

• How do three numbers become the feeling of “red”? We know a lot about how the eye and brain pass these numbers along and compare them. But why those numbers make us experience red is still an open question.
• Color constancy. A sheet of white paper sends very different spectra into the eye at noon and under a light bulb, yet we see it as white both times. How does the brain “subtract” the color of the light? The dress that some people saw as blue and black, and others as white and gold, is exactly this process disagreeing with itself.
• Other animals. Many birds have four kinds of cones and can see ultraviolet. Mantis shrimp have a dozen or so kinds of light-sensing cells, yet experiments show they are worse than we are at telling similar colors apart. There are more ways to “count” color than we might think.

Next time you look at a leaf or a screen, remember: right then, your eyes are quietly counting to three.

For the curious: how these figures were computed

The eye’s three numbers. If a light’s spectrum is S(λ) and the three cone sensitivities are l(λ), m(λ) and s(λ), the three signals the eye receives are

L = ∫ S(λ) l(λ) dλ, M = ∫ S(λ) m(λ) dλ, S = ∫ S(λ) s(λ) dλ.

A spectrum has infinitely many numbers (one per wavelength); after integration only three remain, and most of the information is thrown away. Any spectra with the same three integrals look identical to the eye; that is metamerism. All signals here are scaled so that daylight (CIE D65) gives 1.

The leaf’s light. Light from the leaf = daylight spectrum × leaf reflectance, with leaves from the PROSPECT-D model in the previous article (a summer green leaf, and a yellow leaf on day 28 of autumn).

The screen’s yellow. The red, green and blue sub-pixels are approximated by Gaussians peaking at 620, 530 and 462 nm with widths of 32, 36 and 24 nm. This is what a typical OLED screen looks like, not a measured screen. Making it match the yellow leaf’s three numbers exactly takes just solving three linear equations for how bright each sub-pixel should be. The yellow lamp is a 6 nm-wide band at the wavelength that gives the same L-to-M ratio as the yellow leaf.

Color and brightness. On-screen colors are computed with the CIE 1931 standard observer and converted to sRGB, with the same exposure as the previous article. Pure single-color light is more vivid than any screen can show; the excess is clipped, so the violet end of the animation looks somewhat brighter than it really is.

Data sources. Cone sensitivities: Stockman & Sharpe, Vision Research 40:1711–1737 (2000), via CVRL (University College London). CIE 1931 color-matching functions: CVRL. D65 spectrum: CIE 015. Leaf model: Féret et al., Remote Sensing of Environment 193:204–215 (2017).