Judge & Jury Paradox: What Can Jury Duty Teach Us About Wine Markets?

juryI am out of the office for a week or two on jury duty and  the experience has unexpectedly reminded me of some important things about the world of wine.

A Jury Duty Paradox

I didn’t hesitate for a moment when the jury service summons arrived. I can find the time right now and besides it is a civic duty. I am not an expert on the law but that doesn’t seem to matter.

The idea of a jury trial is that the experts make their case and try to convince ordinary citizens like me that they are right. Some cases are decided by experts — the Supreme Court,  for example, is both judge and jury for the cases that come before it — but most justice starts (and often ends) with the judgment of a jury of peers.

I think it is a bit of a paradox, however, that I accepted the jury duty quickly but, about the same time, I declined an invitation to be on a different kind of jury — a jury of judges for an important regional wine competition. Thanks, I replied,  but I am a self-taught taster and inexperienced with these particular wines. This is a job for experts, not someone like me.

Each of these decisions makes sense on its own, but they are a bit puzzling when you put them together.Objectively, I might actually be better qualified to judge wine than a legal case. But for some reason I applied different standards in the two situations.

Why do I seem to believe that specialized knowledge is needed to serve on a large wine judging panel (where the stakes are actually very low), but expertise is not required for duty on a civil or criminal court jury where with potentially significant consequences? Seems like I’ve got my priorities backwards, doesn’t it?

The Jury of Public Opinion

But then I realized that I was missing something. Wine is actually a lot like the judicial system. While there are a few wine market cases that are decided mainly by the “Supreme Court” of experts (here I am thinking of the role of big-name critics in the en primeur market, for example), it is really the supply and demand “jury of peers” who render most verdicts.

At the end of the day for most wines, it is what the buying public thinks that matters more than the experts’ judgement. Is this a good thing? It is easy to point out that citizen juries have some disadvantages compared with expert panels, but there are advantages, too. It is important that arguments are persuasive enough to sway unbiased citizen peers. It sort of keeps us all honest, if you know what I mean.

In the same way, it is a good thing that critics don’t always reign supreme when it comes to wine markets and that most of us take their expert wine advice with a grain of salt. Wine’s most important job is to give us pleasure, as Jancis Robinson has said, and we amateurs are ourselves the best judge of that.

>><<<

(Did you notice the sneaky way I used Jancis Robinson as a kind of “expert witness” in the last paragraph?) The Wine Economist will take a brief recess until my court duties are complete. In the meantime, here’s a “judge and jury” scene from Eric Idle’s fun London Mikado production. Enjoy!

“Money, Taste & Wine” Honored as 2016 “Best in the World” Wine Writing

yantai

My new book Money, Taste, and Wine: It’s Complicated received this  year’s prize for “Best in the World” Wine Writing from Gourmand at the awards ceremony in Yantai, China on May 28, 2016.

My University of Puget Sound colleague Prof. Pierre Ly (shown above) accepted the award on my behalf, speaking in both English and Chinese. Pierre is in China lecturing and gathering material for the book that he and Cynthia Howson are writing about the Chinese wine industry.

Congratulations to all the winners, especially Gerard Bertrand (Le Vin à la belle étoile) and Andrea Zanfi (Prosecco on the Road) for best wine book and Suzanne Mustacich for best digital wine book (Thirsty Dragon).gourmand

Congratulations and a personal shout-out to Beate Joubert. Her book Taste of the Little Karoo won the “best in the world” bronze medal in the Local Cuisine cookbook category. We had a delicious lunch at Beate’s restaurant at the Joubert-Tradauw winery in Barrydale when we visited South Africa. Her husband Meyer and son Andreas are featured in the final chapter of Money, Taste, and Wine.

Thanks to Gourmand International for this honor. Thanks to my publisher Rowman & Littlefied and my editor Susan McEachern their valuable contributions to Money, Taste, and Wine. Special thanks to Pierre Ly for teaching me so much and representing me at the awards ceremony and to Édouard Cointreau for his encouragement and support.

Gourmand International Award for “Money, Taste, and Wine”

I’ve just learned that my new book Money, Taste, and Wine: It’s Complicated has won an international award for best drinks writing in a U.S. book this year.

The award comes from Gourmand International and will be presented in Yantai, China on  May 28, 2016 at the annual awards ceremony. As the U.S. winner, Money, Taste, and Wine is a finalist for the “Best in the World” award in this category, the winner of which will be revealed in Yantai. Very exciting!

The other national finalists and winners in other categories of the awards will be posted on Gourmand International website in February.

Sincere thanks to the Gourmand International  judges for this honor. Here is a video about the 2015 awards in Yantai. Enjoy!

It Takes a Village (and More): Washington State Wine Awards

wswa,jpgIt takes a village to raise a child, they say, but it takes a good deal more than just a village to make a regional wine industry a success. That was one of the take-away messages of the Washington State Wine Awards, although it might not have been the most important one. You be the judge!

And the Winner Is …

The  Washington State Wine Awards (organized by the Washington State Wine Commission) isn’t a wine competition, as you might expect from the name. There’s no Oscar-like presentation for “Best Bordeaux-style Blend”  or “Outstanding Un-oaked Chardonnay under $12.”

No wines receive awards at all! Instead, it is an opportunity for the wine industry in this state to recognize the villages of people who help get the wines into consumer hands.

This event used to be focused on restaurants and their wine programs, but this year they wisely decided to open it up and recognize people throughout the retail, wholesale, tourism and hospitality industries, giving them credit for what they do to support the cause of Washington wines.

What a great idea! I’ve pasted a few of the awards at the end of this post, click here to download a full list of the winners.

Surprise!

About 60 wineries set up tasting tables for the event — what’s a wine award without wine? — and it was fun to sample wines while talking to such a broad representation of the local “trade.”

I was lucky to run into food writer and cookbook author Cynthia Nims and I asked her what her take-away was from the event. She thought for a moment and then her eyes lit up — surprise!

Cynthia knows knows Washington wines very well, but she said that could always find something new, something surprising at events like this — a sign of the industry’s dynamism. I guess that’s what makes wine (and food, too) so eternally fascinating. We shared our new discoveries and surprises and returned to the tasting.

Hitting the Price Point

The third interesting reaction came from my guests at the event, colleagues Pierre Ly and Cynthia Howson, who frequently assist me with Wine Economist research projects. They grabbed the tasting menu and two glasses and headed out to do an experiment of their own design.

P1050258Taking turns, they would taste the wines “blind” — blind in a limited economic sense of not knowing how much they might cost in this case, since it was obviously impossible not to know something about the wines in this atmosphere.

They’d taste, think it over, and then give a “is it worth it?” score — how much would they be willing to pay for this wine?

It’s an interesting idea and challenging, I think, given that the wineries were pouring wines that ranged from $9 (Silver Lake Winery) to $120 (Côte Bonneville) per bottle!

Village, Value and Surprise

Their conclusion? Well obviously there were a few wines that cost far more than they would or could pay. But on the whole they found the wines to be excellent values, with prices falling well within their clearly subjective  “I’d buy that” range.

And, interestingly, they reported several wines that they’d now be willing to buy that are well outside their usual comfort zone — they never would have considered them without tasting them first.

The village, the value and the ability to surprise and delight. All of this reflects well on Washington wine today and bodes well for its future, too!

Partial list of Washington State Wine Awards

Restaurant of the YeaVisconti’s Italian Restaurant 
Sommelier of the Year Thomas Price, Metropolitan Grill
Retailer of the Year Metropolitan Market
Walter Clore Honorarium Doug Charles, Compass Wines
Independent Restaurant of the Year Copperleaf Restaurant at the Cedarbrook Lodge
Best Event Featuring Washington Wine Washington Wine Challenge, Urbane
Best Restaurant Group MacKay Restaurant Group
Independent Retailer of the Year Wine World & Spirits
Retailer Chain of the Year Yoke’s Fresh Market
Retailer Steward of the Year Doug King, Metropolitan Market
Tourism Champion of the Year Seattle’s Convention and Visitors Bureau
Tourism Concierge of the Year Anne Peavey, Seattle’s Convention and Visitor Bureau
Hotel of the Year Hotel Vintage Park
Distributor of the Year Young’s Market Company
Salesperson of the Year Kris Patten, Young’s Market Company

>>><<<

Images: This year’s beautiful grand award (top), Cynthia Nims (center) and Pierre and Cynthia (photo from a different tasting).

No One-Liners in Wine

King of One-Liners: Take my wine ... please!

Jon Fredrikson likes to say that there are no one-liners in wine. He isn’t saying that there aren’t any one-line jokes (take my White Zinfandel … please!) but rather that nothing in wine is cut and dry. Wine is always complicated — always this and that, too —  so generalizing is a dangerous practice.

I was reminded of this twice during our recent California expedition. The first time was by Jon Fredrickson himself, who stated the case very well in his talk at the Unified Wine and Grape Symposium in Sacramento (North America’s largest wine industry trade show and seminar series).  His dynamic analysis of how the wine market is evolving was widely reported in the press.

Winery of the Year

At the end of Jon’s report he always names a “winery of the year” and for 2011 it was DFV Wines of Manteca, California. DFV (for Delicato Family Vineyards) has its roots in the decision of Italian immigrant grape grower Gasparé Indelicato to try his hand at winemaking in early post-Prohibition California. His grandson, Chris Indelicato, has been CEO since 2004 and many other family members populate the company’s org-chart.

DFV sits in the #10 position in the Wine Business Monthly Top 30 American Wineries league table for 2011, producing more than 4.5 million cases. DFV owns more than 10,000 acres of vineyards (quite a change from Gasparé Indelicato’s first farm). But it is the business’s dramatic growth, not just its large size, that drew Jon Fredrikson’s attention and, well, everyone’s attention. “Delicato” was all that I heard in pre-announcement speculative conversations.

Gnarly and Twisted

You have probably seen Delicato wines on store shelves, but they are just the tip of the family business iceberg. Other DFV brands include Bota Box, Twisted, Gnarly Head and many more. I usually think of the DFV wine portfolio in terms of good value wines and I think this good value accounts for the company’s success.

But saying that a wine is a good value sometimes imposes a subconscious ceiling on perceived quality and distinctiveness. I admit that I tend to think of DFV wines as good, but not necessarily great. That’s because I sometimes forget Jon Fredrick’s line about one-liners. Good value doesn’t rule out distinctivenes — wine is too complicated for that.

On the Old Silverado Trail

This point was driven home to me for the second time as I stood at the tasting room bar at Black Stallion Estate Winery on Silverado Trail in Napa Valley — DFV’s newest venture, which it acquired just a couple of years ago. The winery itself resists being a one-liner as it is both historically significant (as an equestrian center) and an architectural beauty.

We drove by the winery a couple of years ago (on our way to a Stags Leap AVA event) but didn’t stop.  We were impressed with the BSEW Cab at a tasting back home (it is a larger production wine that is widely distributed), so we came back to try the small production (4000 total cases) wines sold only at the winery.

Imagine my surprise to learn that the same company that makes Botta Box also makes a $150 red blend called Bucephalus. I’m interested to see what happens as the Indelicato family’s winemaking knowledge and resources are focused on this relatively new enterprise — perhaps even more distinctive wines like the Rockpile Zinandel that was my tasting room favorite?

I expect there will be lots of interesting wines to taste and things to say as DFV and Black Stallion continue to develop. But don’t expect to hear any one-liners.

Wine Judges and their Discontents


“Do you trust me?” asks the hero in my favorite scene from the Disney cartoon Aladdin. The Princess hesitates (“What?)  as if trusting anyone is a radical idea. “Yes” she finally says and holds out her hand.

Who do you trust?

I think of this scene every time I read wine reviews or wine competition results. “Do you trust me?” is the obvious question when it comes to the scores and medals that wine critics and judges award. If we do trust we are more likely to reach out our hands to make a purchase. But trust does not always come easily with a product as ironically opaque as wine.

So do we trust wine critic Aladdins and should we trust them? I raised this question a few weeks ago in my report on “The Mother of All Wine Competitions,” which is the Decanter World Wine Awards and, after surveying the issues I promised an update from the meetings of the American Association of Wine Economists in Bolzano, Italy. This is the promised report.

The session on wine judging (see details below) was very interesting in terms of the research presented, not very encouraging from a trust standpoint. Previous studies that showed that wine judges at major competitions are not very consistent in their assessments were confirmed and attempts to improve their performance have not been very successful so far, according to expert analyst Robert Hodgson. The same wine can get very different ratings from the same experienced judge. It’s hard to “trust” a gold medal despite all the effort that goes into the judging process.

The Trouble with Economists

“If you put two economists in a room, you get two opinions, unless one of them is Lord Keynes, in which case you get three opinions.” according to Winston Churchill. Hard to trust any of them when they disagree so much.

Wine critics suffer the same problem as economists, according to research by Dom and Arnie Cicchetti, who compared ratings of the 2004 Bordeaux vintage by Jancis Robinson and Robert Parker and found a considerable lack of consensus. Two famous critics produced different opinions much of the time. Hard to know what to think or who to trust. Other presentations did little to increase the audience’s confidence in wine evaluators and their judgments.

Because tastes differ, wine enthusiasts are often advised to use good old trial and error methodology to find a critic with a similar palate — and then trust that critic’s recommendations. This conventional wisdom inspired Ömer Gökçekus and Dennis Nottebaum to compare ratings by major critics with “the peoples’ palate” as represented by CellarTracker ratings. CellarTracker lists almost 2 million individual wine reviews submitted by over 150,000 members.

Point / Counter-Point

Stephen Tanzer’s ratings correlate best to the CellarTracker crowd for the sample of 120 Bordeaux 2005 wines in the research database. But, as Ömer  suggested in his presentation, it is important to remember that the data can contain a lot of noise. Clearly the CellarTracker critics are well informed — they know what Parker, Tanzer, Robinson and the rest have written about these wines and their ratings may reflect positive and negative reactions to what the big names have to say.

The researchers detected a certain “in your face, Robert Parker” attitude, for example. In cases where Parker gave a disappointing score, CellarTracker users were likely to rate it just a bit higher while giving high-scoring Parker wines lower relative ratings. CellarTracker users apparently value their independence and, at least in some cases, use their wine scores to assert it. This is an interesting effect if it holds generally, but it also introduces certain perverse biases into the data stream.

Bottom line: The research presented in Bolzano suggests that there are limits to how much we do trust and how much we should trust wine critics and judges. The power of critics to shape the world of wine may be overstated or, as Andrew Jefford notes in the current issue of Decanter, simply over-generalized. “Opinion-formers are highly significant — for a tiny segment of the wine-drinking population.” he writes. “They remain irrelevant for most drinkers.”

>>><<<

AAWE Conference Session #1B: Wine Judging / Chair: Mike Veseth, University of Puget Sound

  • Robert T. Hodgson (Fieldbrook Winery), How to improve wine judge consistency using the ABS matrix
  • Dom Cicchetti (Yale U), Arnie Cicchetti (San Anselmo), As Wine Experts Disagree, Consumers’ Taste Buds Flourish: The 2004 Bordeaux Vintage
  • Ömer Gökçekus (Seton Hall U), Dennis Nottebaum (U of Münster), The buyer’s dilemma – Whose rating should a wine drinker pay attention to?
  • Jing Cao (Southern Methodist U), Lynne Stokes Southern Methodist U), What We Can Do to Improve Wine Tasting Results?
  • Giovanni Caggiano (U of Padova) Matteo Galizzi (London School of Economics, U of Brescia), Leone Leonida (Queen Mary U of London), Who is the Expert? On the Determinants Of Quality Awards to Italian Wines

The Mother of All Wine Competitions

Decanter, the self-proclaimed “World’s Best Wine Magazine,” organizes the mother of all wine competitions each year. The results of the 2011 judging are out — you can read them here.

I think that the Decanter World Wine Awards is the largest and most comprehensive wine competition in the world. The press release proclaims that “This year a staggering 12,252 wines from 44 countries were tasted in the DWWA, with 8,327 medals awarded.” Staggering is right! That’s a lot of wine from a lot of places and a lot of awards, too.

Can you imagine a wine competition with more than 8000 winners (two thirds of all wines entered)? What an incredible undertaking.

Suspicious Minds

Wine economists are suspicious of wine competitions. This is partly because economists are suspicious people in the first place, always looking for the dark dismal cloud whenever they spy a silver lining. But there are other reasons, too. De gustibus non est disputadum is the economists’ motto;  everyone is entitled to her own opinion on matters of taste. The idea that anyone, even experienced judges,  could objectively rank something as inherently subjective as wine runs against an economist’s nature, so you can imagine how suspicious we are about big competitions where thousands of wines are tasted and rated.

Richard Hodgson, a winemaker and retired statistics professor, was for many years a judge at the Mother of All American Wine Competitions, the California State Fair Commercial Wine Competition. California State Fair judges evaluated more than 3000 wines from 600 wineries in 2009. It’s a huge competition, although nothing compared to the Decanter contest. Hodgson’s analysis of raw data from wine judges suggests that they are only human after all and likely to suffer the sort of tasting inconsistencies that you would expect (if you are a suspicious-minded economist).

Hodgson and his colleague G.M. “Pooch” Pucilowski, California State Fair Wine Competition manager and chief wine judge discussed their findings at the 2010 meetings of the American Association of Wine Economists in Davis, California. Here’s a summary taken from the Wine Business Monthly report on the session.

Hodgson served as a judge in the California State Fair competition, and is now on the competition’s Wine Advisory Task Force working with Pucilowski to try to improve judging quality and consistency.

With Pucilowski’s assistance, Hodgson has been evaluating the competition judges since 2005 with trials that place three samples from the same wine bottle in one flight of judged wines to see if the judges ranked each sample consistently. Hodgson, who taught statistics at Humboldt State University, said, “Fewer than 10% of judges could judge the three wines and maintain consistency in their scores.” He added, “Some of the same wines received ratings that ranged from no award to gold.” When the study, “An Examination of Judge Reliabiity at a Major U.S. Wine Competition,” was published in the JWE, it received significant media attention and created a stir among wine judges and within the wine industry.

Pucilowski, who has managed the State Fair competition 25 years and often serves as a judge in other competitions, openly admits that his competition and all wine judging events are highly subjective. To his credit, he is constantly looking at ways to improve the competition and to help judges improve their abilities.

The Value of Wine Competitions

So it seems like there is good reason to be skeptical about wine competition results. Why, then,  do winemakers enter these competitions, given that they are the people who are most likely to know when their wines are scored too high or low compared with others? Ego may have something to do with it, but the obvious answer is that there is commercial value in a gold medal and the attention it receives, although I don’t know how much a medal is really worth — probably depends upon the circumstances. I noticed, for example, that the Achaval Ferrer Malbec that was the top wine last year in Decanter’s  big comparative tasting of Argentinean Malbec did not receive a medal at DWWA. I’ll bet that’s because it wasn’t entered.  Nothing to gain for this famous (and probably sold-out) wine.

Some wine producers probably enter competitions on the theory that they might win a medal in at least one of them, which gives them bragging rights. There has been a medal on the label of every bottle of Gallo’s value-priced Barefoot wines that I’ve ever seen, for example. A medal gives the cautious bargain-buyer some assurance of quality. Three non-vintage Barefoot wines — Merlot, Pinot Grigio and Moscato — earned “commended” medals in this year’s Decanter competition.

Even Two Buck Chuck wins gold medals, according to an article in the Napa Valley Register.

The Charles Shaw 2005 California chardonnay (yes, the $1.99 “Two Buck Chuck” made by Bronco Wine Company sold at Trader Joe’s)  was judged Best Chardonnay from California at California State Fair Commercial Wine Competition.

The chardonnay received 98 points, a double gold, with accolades of Best of California and Best of Class.


Decanter’s Value

As the video above shows, Decanter (like the California State Fair competition) goes to great lengths to overcome the inherently problematic elements of wine judging. This makes sense since there is so much at stake, both for the winemakers and for Decanter itself, which puts its reputation on the line. The Decanter awards probably have more commercial value than most because the Decanter name has credibility, especially in the U.K.  Decanter sells colorful foil medals to decorate winning bottles and the decorations sell the wine, the magazine and, well, the whole enterprise.

Winning a medal is good, but perhaps the biggest prize for many wineries is winning distribution. Making good wines is often easier than getting them into consumer hands, both here in the U.S. where our fragmented three-tier system creates many obstacles, and also in Great Britain, where the big supermarket chains dominate. Scrolling through the online winner lists I notice that a lot of the wines that are received medals in the competition  aren’t currently sold in the UK. Perhaps that’s the point of entering — to get distributor attention and break into the market.

Thick and Thin

Wine competitions are fun, but I admit that I don’t take the results too seriously since they depend on so many uncontrollable factors, including which particular wines are entered and which (like the Achaval Ferrer Malbec) are held out. I do, however, find the Decanter results worth careful study because  they have some important stories to tell.

The wine world is very broad but the world wine market surprisingly thin and uneven. Looking at the award list, it is interesting to see the large number of countries (44, including India, China and Thailand) that sent wine to London for the judging. As someone who writes about the globalization of wine, it is great to see evidence of the world wine web’s continuing expansion.

But the list of entries is also relatively thin and uneven in some respects, even with more than 12,000 entries, reflecting the fact that the British market is difficult to break into and so not everyone sees value in entering Decanter’s competition.

If you search for U.S. award winners, for example, I think you will be a bit puzzled by the long list of wineries that result, both in terms of the wines that appear and those that are missing, probably because they were not entered in the competition.  There are affordable wines from large scale producers (like Gallo’s Barefoot noted above) and some expensive boutique ones, too, but much of America’s vast middle kingdom of wine, which is in many ways the country’s great strength,  is under-represented. Not interested in the award because not represented in the British market, I suspect.

The U.S. Medal Count

This perhaps accounts for the odd showing of American wines on the Award league table. Only four U.S. wines earned top awards in 2011 (many more earned Silver, Bronze and Commended medals, however). The top four are:

Vina Robles Cabernet Sauvignon Huerhuero Estate 2008 (Paso Robles, San Louis Obispo County) earned a regional trophy (second only to an international trophy in Decanter’s galaxy of awards). It was the top U.S. wine. No U.S. wine earned an international trophy.

Three wines earned gold medals: Chateau Ste. Michelle Artist Series Meritage 2007 (Columbia Valley, Washington State), Justin Justification 2008 (Paso Robles. SLO) and the Silverado Vineyards Estate Cab 2008 (Napa Valley).

Are you surprised? I’ll bet this isn’t the list you were expecting. And it is interesting that none of the American wines made the highest level of awards.

Is four a good medal count? Not compared to Argentina, which received almost 20  gold medals and nine regional trophies. Why the big difference? Perhaps the judging panels applied different standards or maybe there just aren’t as many really good wines from the U.S. these days, but I think it has something to do with the intensity of Argentina’s export drive and the importance they attach to Decanter’s international reputation compared with producers from the United States.

>>><<<

I’ve been asked to chair the session on wine competitions at the annual meetings of the American Association of Wine Economists later in the month. It will be interesting to see shat new insights the panelists will provide. Watch this space for a report.