WSJ on Wine: A Taste of Illusion

Image by Michele Eve via Flickr

We’ve covered how wine is basically bullshit here on Neuroworld before — for instance, here — at least when it comes to describing wine or determining which wines taste like “oak” or “cat pee.”

In this Saturday’s Wall Street Journal, the Weekend section (full disclosure: my wife is an editor at the section) takes a look at wine bullshit.

The opening anecdote is great:

Acting on an informant’s tip, in June 1973, French tax inspectors barged into the offices of the 155-year-old Cruse et Fils Frères wine shippers. Eighteen men were eventually prosecuted by the French government, accused, among other things, of passing off humble wines from the Languedoc region as the noble and five-times-as-costly wine of Bordeaux. During the trial it came out that the Bordeaux wine merchants regularly defrauded foreigners. One vat of wine considered extremely inferior, for example, was labeled “Salable as Beaujolais to Americans.”

Ah, the French and their contempt for the American palate. But here’s the question: Was selling Americans swill and telling them it was liquid gold really defrauding them?

Given the reliability and scientific validity of wine rating systems since devised, perhaps not:

In his first study, each year, for four years, Mr. Hodgson served actual panels of California State Fair Wine Competition judges—some 70 judges each year—about 100 wines over a two-day period. He employed the same blind tasting process as the actual competition. In Mr. Hodgson’s study, however, every wine was presented to each judge three different times, each time drawn from the same bottle.

The results astonished Mr. Hodgson. The judges’ wine ratings typically varied by ±4 points on a standard ratings scale running from 80 to 100. A wine rated 91 on one tasting would often be rated an 87 or 95 on the next. Some of the judges did much worse, and only about one in 10 regularly rated the same wine within a range of ±2 points.

Mr. Hodgson also found that the judges whose ratings were most consistent in any given year landed in the middle of the pack in other years, suggesting that their consistent performance that year had simply been due to chance.

…

This September, Mr. Hodgson dropped his other bombshell. This time, from a private newsletter called The California Grapevine, he obtained the complete records of wine competitions, listing not only which wines won medals, but which did not. Mr. Hodgson told me that when he started playing with the data he “noticed that the probability that a wine which won a gold medal in one competition would win nothing in others was high.” The medals seemed to be spread around at random, with each wine having about a 9% chance of winning a gold medal in any given competition.

To test that idea, Mr. Hodgson restricted his attention to wines entering a certain number of competitions, say five. Then he made a bar graph of the number of wines winning 0, 1, 2, etc. gold medals in those competitions. The graph was nearly identical to the one you’d get if you simply made five flips of a coin weighted to land on heads with a probability of 9%. The distribution of medals, he wrote, “mirrors what might be expected should a gold medal be awarded by chance alone.”

Prizes and point systems. It’s all pretty much hogwash when put under the microscope (oh man, my metaphors are getting away from me here).

The bottom line is that if people believe what they’re drinking is supposed to be good, most of them will think it tastes good. Even the most sophisticated oenophiles in the world are subject to their cognitive biases, just like the rest of us.

I’ll turn things over to Jonah Lehrer, writing about some of the same research covered in the Journal article:

Our brain has been designed to believe itself, wired so that our prejudices feel like facts, our opinions indistinguishable from the actual sensation. If we think a wine is cheap, it will taste cheap. And if we think we are tasting a grand cru, then we will taste a grand cru. And if we’re tasting 15 young and tannic wines, then we shouldn’t expert our poor olfactory cortex to be able to reliably assign an exact point score to the spoiled grape juice in our mouth. Our senses are vague in their instructions, and we parse their suggestions based upon whatever other knowledge we can summon to the surface.

The solution: If you like wine, drink a lot of it — but don’t pay extra for adjectives and medals.