Decanter, the self-proclaimed “World’s Best Wine Magazine,” organizes the mother of all wine competitions each year. The results of the 2011 judging are out — you can read them here.

I think that the Decanter World Wine Awards is the largest and most comprehensive wine competition in the world. The press release proclaims that “This year a staggering 12,252 wines from 44 countries were tasted in the DWWA, with 8,327 medals awarded.” Staggering is right! That’s a lot of wine from a lot of places and a lot of awards, too.

Can you imagine a wine competition with more than 8000 winners (two thirds of all wines entered)? What an incredible undertaking.

Suspicious Minds

Wine economists are suspicious of wine competitions. This is partly because economists are suspicious people in the first place, always looking for the dark dismal cloud whenever they spy a silver lining. But there are other reasons, too. De gustibus non est disputadum is the economists’ motto;  everyone is entitled to her own opinion on matters of taste. The idea that anyone, even experienced judges,  could objectively rank something as inherently subjective as wine runs against an economist’s nature, so you can imagine how suspicious we are about big competitions where thousands of wines are tasted and rated.

Richard Hodgson, a winemaker and retired statistics professor, was for many years a judge at the Mother of All American Wine Competitions, the California State Fair Commercial Wine Competition. California State Fair judges evaluated more than 3000 wines from 600 wineries in 2009. It’s a huge competition, although nothing compared to the Decanter contest. Hodgson’s analysis of raw data from wine judges suggests that they are only human after all and likely to suffer the sort of tasting inconsistencies that you would expect (if you are a suspicious-minded economist).

Hodgson and his colleague G.M. “Pooch” Pucilowski, California State Fair Wine Competition manager and chief wine judge discussed their findings at the 2010 meetings of the American Association of Wine Economists in Davis, California. Here’s a summary taken from the Wine Business Monthly report on the session.

Hodgson served as a judge in the California State Fair competition, and is now on the competition’s Wine Advisory Task Force working with Pucilowski to try to improve judging quality and consistency.

With Pucilowski’s assistance, Hodgson has been evaluating the competition judges since 2005 with trials that place three samples from the same wine bottle in one flight of judged wines to see if the judges ranked each sample consistently. Hodgson, who taught statistics at Humboldt State University, said, “Fewer than 10% of judges could judge the three wines and maintain consistency in their scores.” He added, “Some of the same wines received ratings that ranged from no award to gold.” When the study, “An Examination of Judge Reliabiity at a Major U.S. Wine Competition,” was published in the JWE, it received significant media attention and created a stir among wine judges and within the wine industry.

Pucilowski, who has managed the State Fair competition 25 years and often serves as a judge in other competitions, openly admits that his competition and all wine judging events are highly subjective. To his credit, he is constantly looking at ways to improve the competition and to help judges improve their abilities.

The Value of Wine Competitions

So it seems like there is good reason to be skeptical about wine competition results. Why, then,  do winemakers enter these competitions, given that they are the people who are most likely to know when their wines are scored too high or low compared with others? Ego may have something to do with it, but the obvious answer is that there is commercial value in a gold medal and the attention it receives, although I don’t know how much a medal is really worth — probably depends upon the circumstances. I noticed, for example, that the Achaval Ferrer Malbec that was the top wine last year in Decanter’s  big comparative tasting of Argentinean Malbec did not receive a medal at DWWA. I’ll bet that’s because it wasn’t entered.  Nothing to gain for this famous (and probably sold-out) wine.

Some wine producers probably enter competitions on the theory that they might win a medal in at least one of them, which gives them bragging rights. There has been a medal on the label of every bottle of Gallo’s value-priced Barefoot wines that I’ve ever seen, for example. A medal gives the cautious bargain-buyer some assurance of quality. Three non-vintage Barefoot wines — Merlot, Pinot Grigio and Moscato — earned “commended” medals in this year’s Decanter competition.

Even Two Buck Chuck wins gold medals, according to an article in the Napa Valley Register.

The Charles Shaw 2005 California chardonnay (yes, the $1.99 “Two Buck Chuck” made by Bronco Wine Company sold at Trader Joe’s)  was judged Best Chardonnay from California at California State Fair Commercial Wine Competition.

The chardonnay received 98 points, a double gold, with accolades of Best of California and Best of Class.

Decanter’s Value

As the video above shows, Decanter (like the California State Fair competition) goes to great lengths to overcome the inherently problematic elements of wine judging. This makes sense since there is so much at stake, both for the winemakers and for Decanter itself, which puts its reputation on the line. The Decanter awards probably have more commercial value than most because the Decanter name has credibility, especially in the U.K.  Decanter sells colorful foil medals to decorate winning bottles and the decorations sell the wine, the magazine and, well, the whole enterprise.

Winning a medal is good, but perhaps the biggest prize for many wineries is winning distribution. Making good wines is often easier than getting them into consumer hands, both here in the U.S. where our fragmented three-tier system creates many obstacles, and also in Great Britain, where the big supermarket chains dominate. Scrolling through the online winner lists I notice that a lot of the wines that are received medals in the competition  aren’t currently sold in the UK. Perhaps that’s the point of entering — to get distributor attention and break into the market.

Thick and Thin

Wine competitions are fun, but I admit that I don’t take the results too seriously since they depend on so many uncontrollable factors, including which particular wines are entered and which (like the Achaval Ferrer Malbec) are held out. I do, however, find the Decanter results worth careful study because  they have some important stories to tell.

The wine world is very broad but the world wine market surprisingly thin and uneven. Looking at the award list, it is interesting to see the large number of countries (44, including India, China and Thailand) that sent wine to London for the judging. As someone who writes about the globalization of wine, it is great to see evidence of the world wine web’s continuing expansion.

But the list of entries is also relatively thin and uneven in some respects, even with more than 12,000 entries, reflecting the fact that the British market is difficult to break into and so not everyone sees value in entering Decanter’s competition.

If you search for U.S. award winners, for example, I think you will be a bit puzzled by the long list of wineries that result, both in terms of the wines that appear and those that are missing, probably because they were not entered in the competition.  There are affordable wines from large scale producers (like Gallo’s Barefoot noted above) and some expensive boutique ones, too, but much of America’s vast middle kingdom of wine, which is in many ways the country’s great strength,  is under-represented. Not interested in the award because not represented in the British market, I suspect.

The U.S. Medal Count

This perhaps accounts for the odd showing of American wines on the Award league table. Only four U.S. wines earned top awards in 2011 (many more earned Silver, Bronze and Commended medals, however). The top four are:

Vina Robles Cabernet Sauvignon Huerhuero Estate 2008 (Paso Robles, San Louis Obispo County) earned a regional trophy (second only to an international trophy in Decanter’s galaxy of awards). It was the top U.S. wine. No U.S. wine earned an international trophy.

Three wines earned gold medals: Chateau Ste. Michelle Artist Series Meritage 2007 (Columbia Valley, Washington State), Justin Justification 2008 (Paso Robles. SLO) and the Silverado Vineyards Estate Cab 2008 (Napa Valley).

Are you surprised? I’ll bet this isn’t the list you were expecting. And it is interesting that none of the American wines made the highest level of awards.

Is four a good medal count? Not compared to Argentina, which received almost 20  gold medals and nine regional trophies. Why the big difference? Perhaps the judging panels applied different standards or maybe there just aren’t as many really good wines from the U.S. these days, but I think it has something to do with the intensity of Argentina’s export drive and the importance they attach to Decanter’s international reputation compared with producers from the United States.

>>><<<

I’ve been asked to chair the session on wine competitions at the annual meetings of the American Association of Wine Economists later in the month. It will be interesting to see shat new insights the panelists will provide. Watch this space for a report.