The Wine Spectator Award Hoax

It has been a couple of weeks now since the Wine Spectator hoax hit the news. Robin Goldstein (a.k.a. fearlesscritic.com) “blew the whistle” on Wine Spectator in a session that I happened to chair at the American Association of Wine Economists meetings in Portland. (Robin actually revealed his hoax as an unscheduled prelude to a completely different presentation at the meetings.)

The wine media quickly picked up the story and now it is everywhere. The story has generated a certain amount of embarrassment for Wine Spectator and given Robin and his new book a lot of  publicity.

What Robin did was to create a fake Italian restaurant (Osteria L’Intrepido di Milano) along with a made-up menu and wine list. Then, following directions on the Wine Spectator website, he applied for an Award of Excellence, which is the way that Wine Spectator recognizes and encourages restaurants with strong wine programs. Wine Spectator tried but was not able independently to confirm the facts about the fake restaurant; they took the application on trust as an honest entry and presented it with the appropriate award in the August 31, 2008 special restaurant issue (see page 181). You can read all about it on Robin’s website for the fake restaurant, http://osterialintrepido.wordpress.com/

Where is the Outrage?

How upset should we be to discover that Wine Spectator can be tricked into giving its wine award to a fake restaurant?  Michael Morrell, my chief cheap wine research assistant, was outraged.  Although price is the most important factor for him in choosing wine, he admits that he is also influenced by wine ratings. The award hoax undermines his trust in wine critics in general and the ratings and advice they produce.

I can understand Michael’s concern, so I consider this a very serious matter, but I don’t think the fact that Wine Spectator fell for a hoax is reason for us to doubt its integrity.  Here is my report.

The Wine Spectator Award of Excellence is given to restaurants to recognize their wine programs.  Although the actual criteria for receiving an award seem very modest to me (you can read them on page 97 of the special restaurant wine issue), it is a fact that about 30% of the new entries each year fail to meet them (the success rate is obviously higher for establishments who enter and receive an award year after year).

There are three levels of award.  3254 restaurants received the base level recommendation.  802 second tier “Best of Award” ratings were given to restaurants with more comprehensive wine lists. 73 top of the line “Grand Awards” were bestowed.  The people at Wine Spectator are proud of their award program and believe that it has encouraged restaurants to upgrade their wine programs.

Caution: Economics Content

I’m sure this is true, but I tend to view the matter in economic theory terms.  Consumers have lots of restaurant options and are uncertain which ones might have good wine choices. The restaurants know how good their wine selections are but have trouble effectively communicating this to potential customers. This is the classic economic problem of “asymmetric information” and the classic economic solution is “signaling” – where one side of a potential transaction finds a way to reveal key information to the other side to help seal the deal.

Restaurants that want to attract wine enthusiast customers need a way to “signal” them about their wine programs and the Award of Excellence is one way to do this.  Restaurants that think sending this signal is worth meeting the criteria and paying the entry fee do it and get on the list.  Others, even some that have strong  wine programs, don’t bother. They have other ways to send the message, I guess.

Wine Spectator fell for the Osteria L’Intrepido hoax because it relied upon the honesty of applicants, assuming, I suppose, that no one would go to the trouble and expense of applying without a conventional commercial purpose. This is another side of asymmetric information — Robin presumably knew his motives in setting the fake restaurant “sting” and Wine Spectator could only guess or assume.

In Vino Veritas

Truth is especially important in the wine world and, because of the problem of asymmetric information, it is particular difficult to know with confidence.  We depend upon the honesty of self-interested actors and the truthfulness of their signals. When we read wine ratings or see wine competition awards, for example, we assume that the judges and critics are tasting the same wines that we buy in the market. But it would be easy for a dishonest producer or distributor to put special wines in the bottles sent to the critics or wine award competitions. The easiest switch would be to put some of last year’s highly ranked wine in place of this year’s weak effort. Most wine critics rate products that are sent to them by makers and distributors and rely upon the honesty of the sender.  Only a few – Gaiter and Brecher at the Wall Street Journal come to mind – seek out and purchase their wines through normal retail channels.

Doctored “critic cuvee” wines are a potential hoax problem.  I am not aware of any wine publications that have been hoaxed in this manner, but I have read and heard speculation about special “award cuvee” wines being entered in competitions.  The nature of the situation makes us all vulnerable to hoaxes.

Wine Spectator fell for this hoax but it wasn’t because its editors are dishonest in giving their awards.  I think most of the criticism of Wine Spectator in this situation is a bum rap, especially since the magazine’s editors seem to be unusually careful in avoiding advertising conflicts of interest.  That’s the subject of my next post.

Globalization, Wine Value and the Two Buck Chuck Index

Has the globalization of the wine industry given us the best of wines, as many wine drinkers believe, or the worse of wines, as the film Mondovino suggests?

Two economists from the Whitehead School of Diplomacy at Seton Hall University address this question in the December 2007 issue of the Journal of Wine Economics (see full reference below). Their conclusion? Globalization has benefited American wine drinkers, who have a broader choice of quality wines at lower prices.

That’s pretty much what my supermarket empiricism leads me to conclude, but can it be proven scientifically? Here’s how the article’s authors arrived at their results.

First you need to define what it is that American wine drinkers are buying. The authors decided to focus on the Wine Spectator annual Top 100 list of wines. This has the advantage of limiting the study to a reasonable number of widely available wines. The Top 100 list is chosen each year on the basis of price, wine rating, availability and “excitement.” Many people use rankings like the WS 100 to guide their purchases, so I suspect that there really is some correlation between what is on the list and what is on store shelves and restaurant wine menus. The disadvantage of limiting the study to the Top 100 is of course that most of the wine sold in America — the inexpensive Gallo, Yellow Tail and Two Buck Chuck wine — does not make it to this or any other “top” list. If we want to know if globalization has improved choice at the middle and bottom of the market we will need more research.

The authors examined the WS 100 lists from 1988 – 2005 to determine (1) where the wines came from, (2) how much they cost and (3) their quality as measured by the WS ratings. They then calculated measures to determine changes in the geographical concentration of the wines (more or less choice in terms of countries of origin), the average quality rating and the relative value to consumers as measured by rating points per dollar.

What we learn from this is that the overall quality of the top wines has stayed relatively constant over the years, but the real price has fallen and the range of offerings has increased. It cost $4313 (in today’s dollars) to purchase the entire WS100 in 1988, for example, but just $2622 to buy the Top 100 wines in 2005. The cost per “point” of ratings in 1988 was 46 cents, so a hypothetical average 90-point wine cost $41.40. The per point cost was 28 cents in 2005 and so a hypothetical average 90-point wine cost just $25.20.

The top wines came from just six countries in 1998 versus 11 countries in 2005, an indication of the globalization effect. A great majority of WS100 over the years have come from four core wine countries: Australia, France, Italy and the U.S., but the proportion of non-core wines has increased, too, from just 5 percent in 1988 to 24 percent in 2005.

The authors divide the wine world of this study into Old World (France, Italy), New World (Australia and the U.S.) and “New-New World” (New Zealand, South Africa, Argentina and so on). Globalization has brought American wine drinkers more and more excellent New World and now especially New-New World wines that provide the same quality at lower average prices, according to the study.

Research like this is interesting both for the questions that it answers and for the new questions that are raised. It would be interesting, for example, to find how important the four criteria for selection are — price, rating, availability and “zing” — and if the relative weight they are given has changed. As the wine market has expanded, for example, greater emphasis may have been put on price and availability, leading to a Top 100 that leans more toward (global) good value wines.

It would also be interesting to see if the editors respond in any way to external forces. A lot of people read and study the Top 100 list, so perhaps they use it as a way to build the wine market (and thereby indirectly build their potential subscriber base). A focus on value would be consistent with this goal. A Top 100 list that you can’t find or can’t afford doesn’t build the wine market and won’t sell many magazines. The fact that there are more New-New World wines might reflect rising quality and availability of these wines or it could indicated that the WS editors desire to add these wines to keep costs down, value up and the market growing. In other words, the WS100 might show more choice and continuing good value because that’s what the WS editors want it to show. I suspect that the truth is that the market has evolved toward global good value and that WS has been part of that process, encouraging people to try New-New World wines by putting them on the Top 100 list.

Exchange rates could also play a role here. The dollar has fallen against most currencies (increasing the cost of imported wine), but the depreciation is not uniform. The Euro is much more expensive but the Argentine peso has not changed as much. If would be interesting to see to what extent the WS100‘s New-New World globalization has offset exchange rate driven increases in Old World wine costs.

Another interesting question relates to the idea of value in wine purchases. It does seem to me that people often find themselves buying WS points or Parker points more than the wine itself because they are unsure of their ability to judge quality. One local wine merchant had a sale of wines rated 90 points or more for $20 or less. The idea was that the wines must be good value because of the low cents per point ratio. But there is more to wine than rating scores, as anyone who has tasted high-scoring wines will tell you.

It might be interesting to try to put together a slightly more sophisticated wine value index using WS and other ratings. I don’t think that cents per point is a good measure because it assumes a linear relationship between money and quality — and we all know that is not the case. Very expensive wines frequently receive much lower ratings than their cheaper competitors. I understand that a $100 Chardonnay came in last at the tasting where Two Buck Chuck won the Gold Medal.

Even where price and quality are correlated, the relationship isn’t necessarily linear. The average price difference between an 86 point wine and a 88 point wine may be pretty small, for example, but it might cost a great deal to go from 92 to 94 points if the demand for the very best wines is particularly strong as is often the case in winner-take-all markets.

The price-quality relationship, even using imperfect wine scores as a measure of quality, is certainly non-linear. No wonder wine buyers are so confused — and depend so much on ratings and lists like the WS100.

Here is a simple alternative to cents per point as a measure of value. Let’s adjust price and quality for a baseline wine: Two Buck Chuck. You could call it the TBC index. Suppose that you can purchase a 70-point (to just make up a number) TBC Chardonnay for $2 (or $3 here in Washington State). The question we want to answer is how much does it cost to improve on TBC? A wine that gives you a lot of additional value for only a little additional money is a good deal.

In other words, the TBC index would be a relative index of value calculated by asking would be how many points in excess of 70 (or whatever the quality of the baseline wine you choose) you can buy for the dollars you spend in excess of the baseline cost. Here’s a numerical example. A 88 point wine for $20 would have a TBC rating of (88 – 70 points)/($20 – $2) = 18 point/$18 or a dollar a point. An 86 point wine for $10 would be a better value because (86 – 70)/(10-2) = 16 points /$8 = two points per dollar. It seems to me that this is a better (but still badly flawed) indicator of relative value. (Economics students have already realized that I am applying the principle of decision-making on the margin to this problem).

Perhaps I will find some students to work on the TBC index, perhaps using a different base wine for each varietal or wine type. I predict that their research would find that the “optimal” TBC point is being pretty close to the heart of the premium wine market — right on the center shelf in the supermarket — where so many wine brands compete for your wine dollars.

Wine ratings are very important in some parts of the wine market and very controversial, too, so I think I will see what I can learn about them. With this in mind I have subscribed to six different wine-rating publications: Wine Advocate (Robert Parker), Wine Spectator, Wine & Spirits, Wine Enthusiast, the British Decanter and Wine Press Northwest (for Washington/Oregon wine news and ratings). Watch this space for a comparative analysis of these influential publications.

References: Omer Gokcekus and Andrew Fargnoli, “Is Globalization Good for Wine Drinkers in the United States?” Journal of Wine Economics 2:2 (December 2007) pp. 187-195).