The Wine Spectator Award Hoax

It has been a couple of weeks now since the Wine Spectator hoax hit the news. Robin Goldstein (a.k.a. fearlesscritic.com) “blew the whistle” on Wine Spectator in a session that I happened to chair at the American Association of Wine Economists meetings in Portland. (Robin actually revealed his hoax as an unscheduled prelude to a completely different presentation at the meetings.)

The wine media quickly picked up the story and now it is everywhere. The story has generated a certain amount of embarrassment for Wine Spectator and given Robin and his new book a lot of  publicity.

What Robin did was to create a fake Italian restaurant (Osteria L’Intrepido di Milano) along with a made-up menu and wine list. Then, following directions on the Wine Spectator website, he applied for an Award of Excellence, which is the way that Wine Spectator recognizes and encourages restaurants with strong wine programs. Wine Spectator tried but was not able independently to confirm the facts about the fake restaurant; they took the application on trust as an honest entry and presented it with the appropriate award in the August 31, 2008 special restaurant issue (see page 181). You can read all about it on Robin’s website for the fake restaurant, http://osterialintrepido.wordpress.com/

Where is the Outrage?

How upset should we be to discover that Wine Spectator can be tricked into giving its wine award to a fake restaurant?  Michael Morrell, my chief cheap wine research assistant, was outraged.  Although price is the most important factor for him in choosing wine, he admits that he is also influenced by wine ratings. The award hoax undermines his trust in wine critics in general and the ratings and advice they produce.

I can understand Michael’s concern, so I consider this a very serious matter, but I don’t think the fact that Wine Spectator fell for a hoax is reason for us to doubt its integrity.  Here is my report.

The Wine Spectator Award of Excellence is given to restaurants to recognize their wine programs.  Although the actual criteria for receiving an award seem very modest to me (you can read them on page 97 of the special restaurant wine issue), it is a fact that about 30% of the new entries each year fail to meet them (the success rate is obviously higher for establishments who enter and receive an award year after year).

There are three levels of award.  3254 restaurants received the base level recommendation.  802 second tier “Best of Award” ratings were given to restaurants with more comprehensive wine lists. 73 top of the line “Grand Awards” were bestowed.  The people at Wine Spectator are proud of their award program and believe that it has encouraged restaurants to upgrade their wine programs.

Caution: Economics Content

I’m sure this is true, but I tend to view the matter in economic theory terms.  Consumers have lots of restaurant options and are uncertain which ones might have good wine choices. The restaurants know how good their wine selections are but have trouble effectively communicating this to potential customers. This is the classic economic problem of “asymmetric information” and the classic economic solution is “signaling” – where one side of a potential transaction finds a way to reveal key information to the other side to help seal the deal.

Restaurants that want to attract wine enthusiast customers need a way to “signal” them about their wine programs and the Award of Excellence is one way to do this.  Restaurants that think sending this signal is worth meeting the criteria and paying the entry fee do it and get on the list.  Others, even some that have strong  wine programs, don’t bother. They have other ways to send the message, I guess.

Wine Spectator fell for the Osteria L’Intrepido hoax because it relied upon the honesty of applicants, assuming, I suppose, that no one would go to the trouble and expense of applying without a conventional commercial purpose. This is another side of asymmetric information — Robin presumably knew his motives in setting the fake restaurant “sting” and Wine Spectator could only guess or assume.

In Vino Veritas

Truth is especially important in the wine world and, because of the problem of asymmetric information, it is particular difficult to know with confidence.  We depend upon the honesty of self-interested actors and the truthfulness of their signals. When we read wine ratings or see wine competition awards, for example, we assume that the judges and critics are tasting the same wines that we buy in the market. But it would be easy for a dishonest producer or distributor to put special wines in the bottles sent to the critics or wine award competitions. The easiest switch would be to put some of last year’s highly ranked wine in place of this year’s weak effort. Most wine critics rate products that are sent to them by makers and distributors and rely upon the honesty of the sender.  Only a few – Gaiter and Brecher at the Wall Street Journal come to mind – seek out and purchase their wines through normal retail channels.

Doctored “critic cuvee” wines are a potential hoax problem.  I am not aware of any wine publications that have been hoaxed in this manner, but I have read and heard speculation about special “award cuvee” wines being entered in competitions.  The nature of the situation makes us all vulnerable to hoaxes.

Wine Spectator fell for this hoax but it wasn’t because its editors are dishonest in giving their awards.  I think most of the criticism of Wine Spectator in this situation is a bum rap, especially since the magazine’s editors seem to be unusually careful in avoiding advertising conflicts of interest.  That’s the subject of my next post.

What are wine enthusiasts looking for?

The Search for Wine Drinker DNA

According to the data that WordPress collects about visitors to this website, the three most frequently viewed posts on The Wine Economist are

  • The World’s Best Wine Magazine?, an analysis of Decanter magazine, part of the ongoing series on wine critics and publications;
  • Costco and Global Wine, which examines Costco’s wine strategy in the context of the three most important global wine markets, the U.S., Great Britain and Germany, and
  • Masters of Wine (and Economics), which is about the prestigious Masters of Wine (MW) qualification and the importance of wine economics in its curriculum.

(Other popular posts include my discussions of global climate change, problems in Australia, rising wine prices, and the Hong Kong and Chinese markets.)

What can we learn from the fact that these three posts get the most hits? A closer examination of the WordPress data show that many visitors to this site are looking for information about the “Best” – the best wine, the best wine price, the best wine magazine and so forth. The search for the best and not just the good seems to be very important.

Wine enthusiasts also seem to be searching for credible authorities – people and publications that can guide them and tell them what to buy and drink.

Not unrelated to this is in the interest in Costco (and Trader Joe’s) and other retailers that seem to make the choice concerning good wine or good value wine a little simpler. Costco is now the largest wine retailer in the U.S., as the blog post explains, and it does this in an unexpected way – by giving consumers fewer choices than a typical upscale supermarket (about 120 different wines at typical Costco versus more than 1200 different wines at your supermarket), but also giving them more confidence in the choices that they make.

Project Genome

Visitors to The Wine Economist reflect many qualities that research by Constellations Brands (the largest wine company in the world) has uncovered. The study is called Project Genome, which suggests that it is an attempt to sequence wine drinker DNA. Wines and Vines reports that

The original 2005 study of 3,500 wine drinkers was one of the largest consumer research projects ever conducted by the wine industry. The new study examined the purchases of 10,000 premium-wine consumers–defined as those who purchased wine priced at $5 and higher–over an 18-month period. While the first Project Genome study asked online survey participants to recall their wine purchases during the last 30 days, the Home & Habits study tracked the actual purchases of Nielsen Co.’s Homescan® consumer purchase panel, which employs in-home bar code scanners and surveys to map consumer buying behavior across a demographically balance

Nielsen measured consumer attitudes and purchase behavior within multiple purchase channels, including warehouse clubs, supermarkets, mass merchandisers, drug stores, liquor stores and wine shops. The scan data were supplemented with online interviews to classify consumers by Project Genome consumer segments identified in Constellation’s original study: Enthusiasts, Image Seekers, Savvy Shoppers, Traditionalists, Satisfied Sippers and Overwhelmed.

The largest group of wine consumers are the Overwhelmed (23% of consumers). They are described as

  • Overwhelmed by sheer volume of choices on store shelves
  • Like to drink wine, but don’t know what kind to buy and may select by label
  • Looking for wine information in retail settings that’s easy to understand
  • Very open to advice, but frustrated when there is no one in the wine section to help
  • If information is confusing, they won’t buy anything at all.

The second largest group are Image Seekers (20% of consumers). They

  • View wine as a status symbol
  • Are just discovering wine and have a basic knowledge of it
  • Like to be the first to try a new wine, and are open to innovative packaging
  • Prefer Merlot as their No. 1 most-purchased variety; despite “Sideways,” Pinot Noir is not high on their list
  • Use the Internet as key information source, including checking restaurant wine lists before they dine out so they can research scores
  • Millennials and males often fall into this category.

Traditionalists (16% of consumers)

  • Enjoy wines from established wineries
  • Think wine makes an occasion more formal, and prefer entertaining friends and family at home to going out
  • Like to be offered a wide variety of well known national brands
  • Won’t often try new wine brands
  • Shop at retail locations that make it easy to find favorite brands.

The Savy Shoppers (16% of consumers)

  • Enjoy shopping for wine and discovering new varietal s on their own
  • Have a few favorite wines to supplement new discoveries
  • Shop in a variety of stores each week to find best deals, and like specials and discounts
  • Are heavy coupon users, and know what’s on sale before they walk into a store
  • Typically buy a glass of the house wine when dining out, due to the value.

Satisfied Sippers make up 14% of consumers. They

  • Don’t know much about wine, just know what they like to drink
  • Typically buy the same brand–usually domestic–and consider wine an everyday beverage
  • Don’t enjoy the wine-buying experience, so buy 1.5L bottles to have more wine on hand
  • Second-largest category of warehouse shoppers, buying 16% of their wine in club stores
  • Don’t worry about wine and food pairing
  • Don’t dine out often, but likely to order the house wine when they do.

And, finally, Wine Enthusiasts are the smallest group, accounting for just 12% of all wine buyers. They

  • Entertain at home with friends, and consider themselves knowledgeable about wine
  • Live in cosmopolitan centers, affluent suburban spreads or comfortable country settings
  • Like to browse the wine section, publications, and are influenced by wine ratings and reviews
  • 47% buy wine in 1.5L size as “everyday wine” to supplement their “weekend wine”
  • 98% buy wine over $6 per bottle, which accounts for 56% of what they buy on a volume basis.

The Fortune at the Bottom of the Pyramid

Not surprisingly, Wine Enthusiasts and Image Seekers account for nearly half of all wine sales while Overwhelmed consumers purchase disproportionately little wine. While wine magazines find a ready market at the top of the pyramid, retailers and wine companies probably view the Overwhlemed as the potential “fortune at the bottom of the pyramid.” There is a lot of money that can be made if wine can be simplified (or these consumers educated) so that they move up the wine buying ladder.

Visitors to The Wine Economists seem to fall into three of Constellation’s categories: Enthusiasts, Image Seekers and the Overwhelmed based upon the limited and superficial “most popular post” data reported here. It will be interesting to track further Project Genome results as they are released and to see how Constellation Brands uses this information in its wine market strategies.

The Big Vineyard Squeeze

Globalization and Wine Costs

wv_issues_2008-03_208.jpgThe March 2008 issue of Wine & Vines reports three stories of how global forces are pushing up the cost of making wine in the United States. They make interesting reading for anyone interested in how the wine industry is changing today.

The first article examines the rising cost of planting new vineyards. Environmental concerns are causing winegrowers to substitute steel posts and stakes for the old treated wood products that have been used for years. Steel makes the vineyard greener, but it is now pushing the vineyard bottom line into the red because the cost of posts and wire has been pushed up by surging demand in for steel in China.

Once the vineyards are planted the vines need to be pruned and the grapes harvested. This is normally a pretty labor-intensive process but machine alternatives exist and the technology has improved considerably over the last 20 years, according to a second article in the magazine. Winemakers are laying out vineyards with wider spacing to accommodate mechanical pruning and harvesting. This is partly because of rising labor costs in the United States and partly because unreliable federal immigration policy makes access to foreign vineyard workers uncertain. High end winemakers may be willing to pay up to $750 per acre for careful hand harvesting, but many others are willing to employ machinery for a cost per acre of $70 to $250 (plus the $150,000 – $250,000 cost of the machine). The cost difference is significant, of course, but my sense is that this trend would be much weaker if federal immigration policy were more stable and industry-friendly.

Bottle Top and Bottom Line

Winemakers are being squeezed everywhere — input costs, labor conditions and even on the cost of the capsules that protect the cork and decorate the bottle top. High end wines often use imported tin capsules for their sleek and beautiful appearance. But their cost is being pushed up in three ways.

First, the price of the raw material – tin – has soared due to high demand in China, where it is used to solder electronic connections. Tin sold for $3600 a ton in 2002 and $6600 in 2005. It costs $16,500 a ton today due to the China effect and speculation associated with it. This huge increase is necessarily reflected in rising capsule prices.

But wait, there’s more. Rising oil prices and demand for shipping container space has helped push up the cost of shipping tin capsules from Europe by 27 percent. The third and final factor is the falling dollar, which makes everything from Europe more expensive. Much more expensive, given the dollar’s recent collapse. Winemakers are having to consider how much a tin capsule adds to their wine’s image, since it is obviously taking from, not adding to, the bottom line.

Global market forces (steel, oil, tin, shipping costs, exchange rates) and the government policies that try to contain them (think immigration) are squeezing winemakers in many ways. It will be interesting to see how much the industry is transformed by these effects.

Wine in Restaurants: Recent Trends

mainpage_april08.jpgEach year Wine & Spirits publishes a special issue that reports the results of their annual survey of wine sales in restaurants — information of more use to trade professionals, I imagine, than to wine lifestyle readers. Although the sample is relatively small — 309 Zagat -ranked U.S. restaurants participated in the 2008 survey — and restaurant wine sales are probably unrepresentative of broader market sales, I still find the trends reported here to be of interest, especially since many of them reinforce data I have found elsewhere.

More and More.

Some of the trends are unsurprising to any restaurant wine-drinker. The importance of wine in restaurants continues to grow — over 70% of the restaurants reported that wine was a larger percentage of their total sales in 2007 compared with 2006. More restaurants are paying more attention to wine and wine-drinkers and increasing sales accordingly. A second non-surprise is this: restaurant wine costs more. More than 60 percent of the surveyed restaurants reported that the average price of the wine they sold increased in the last year. Personally I have been staggered at the price of wine in some restaurants recently. There are both demand and supply drivers behind this trend.

Restaurants have an incentive to raise wine prices, of course, but nobody forces diners to buy the stuff. Some of the price increase is demand-side — educated (or status-seeking) wine consumers choosing more prestigious and expensive bottles. Smart restauranteurs and their sommeliers take advantage of the wine boom by offering interesting and hard-to-find wines, which attract wine enthusiast diners and generate higher revenues. So higher prices are the result of education, enthusiasm and strategic behavior. We pay more because we are willing to pay more, up to a point at least.

Even Less is More

The falling dollar is another part of this trend. Cheap dollars mean that restaurants have to pay more for imported wines, which drives up costs and prices. The pass through effect of the exchange rate changes is not yet complete, however, so you can expect even higher prices in the future. Rising wine costs are a supply-side driver of higher wine prices generally. The recent trend to more wines from Argentina and Chile is partly a reflection of the fact that the dollar has not fallen quite so far relative to these currencies, so South American wine is a relative bargain.

The weak dollar also affects the demand side. Many of the surveyed restaurants are located in transnational hub cities where international travelers are a significant factor. Foreign tourists and business travelers take advantage of the weak dollar to treat themselves to otherwise more expensive wines when they dine in the U.S., thus driving up the price averages. This is not an insignificant factor for many of the upscale urban eateries that participate in the Wine & Spirits survey.

Prices continue to rise for even the most inexpensive restaurant wines. About 35 percent of the restaurants reported that they have increased the price of the least expensive wine on their list in the last year. In my experience, however, no one ever orders the least expensive bottle on a wine list. The real indicator would be the price of the second cheapest bottle. I imagine that it costs more now, too.With the price of wine edging up relentlessly it is not surprising to find that restaurants and wine drinkers are paying more attention to by-the-glass sales. More restaurants are offering more wines (and more interesting wines) by the glass as well as the bottle. The average price reported by the survey rose to a new high of $11.05.

The trend toward rising wine prices is not likely to slow very much in the future (see my previous post about The End of Cheap Wine), but this trend is not uniform across the entire wine list. Surveyed restaurants reported steep declines in sales of Merlot and Chardonnay, for example, and flat sales of Cabernet Sauvignon. Average sales prices actually declined for Cab and Merlot. Pinto Noir prices and sales have increased again, as you might expect.

Hot or Not?

No sales trend data were reported for two supposed “hot” wines: Riesling and Syrah. Riesling is the sommelier’s favorite, according the Wine & Spirits (and I don’t disagree), because it is so food-friendly, but it does not seem to be an important factor in restaurants sales. I have my own theories about this, but no facts, so I won’t speculate at this time. I’ll try to find out more at the Riesling Rendezvous that Ste Michelle Wine Estates is organizing this summer.

The case of Syrah is interesting, too. Wine & Spirits says that there was a Syrah/Shiraz boom a few years ago, but that it has faded and Syrah has now settled into a minor niche-role on the restaurant wine list. I suppose that this reflects the changing circumstances of Australian wine (see The Wizards of Oz) more than anything else since so many people identify Australian wine with Shiraz and vice versa.

The fact that Riesling and Syrah don’t figure prominently in restaurant sales suggests to me that restaurant buyers as a group are less adventurous than you might think. Rather than using an unfamiliar wine list as an open invitation to experimentation I think they might on average be looking to avoid making a faux pas, either in terms of the wine they choose or the social signals that they send to the others seated around the table with them. Wine trends in restaurants might, therefore, lag behind wine trends generally rather than leading them. Or it could be that restaurants believe that their patrons are unadventurous and wine lists reflect this, focusing mainly on old standbys rather than hot trends. The result would be the same in either case.

If this is true then Riesling and Syrah will move up on the restaurant wine lists, if they do at all, only after they have become more prominent in other wine venues. Or at least winegrowers in Washington State should hope that this will happen. Because, my goodness, we seem to be making a lot of Riesling and Syrah!

The Martha Stewart Wine System

march-2008-cover.jpgWine Enthusiast magazine celebrates its 20th anniversary with the March 2008 issue and editor and publisher Adam Strum reflects on the changing market in “The Enthusiast Corner” column. He writes that

“I’d like to think Wine Enthusiast played an important part in helping to bring wine to the attention of the American public at large, and not just the elite, over these 20 years. Wine magazines, books and the rise of food television have all undoubtedly played a role in making America a wine drinking nation. Other factors abound: American cuisine at home and in fine dining restaurants underwent a renaissance, and wine naturally became an important part of that. News of wine’s health benefits enlarged its consumer base. But most responsible for the growth of wine is the incredible leap in terms of overall quality at the same time that wine became more affordable. How often does that happen? Name me one consumer product that can compare.”

I think he is right in all this. Wine’s vigorous growth in the United States is a complex phenomenon. Many factors have contributed to the rise in per capita consumption in the United States and other New World markets at the same time that wine drinking has fallen dramatically in the Old World. The wine media’s role may be an under-appreciated element of this phenomenon.

The Supermarket as Home Depot with Wine

My friend Patrick works the wine aisle at a local upscale supermarket and he constantly delights me with his original insights into consumer behavior. He sees cable TV’s influence everywhere, for example. People watch Trading Spaces or the home remodeling network HGTV, he says, and run out to Home Depot for wallpaper and remodeling supplies. A huge industry has been built around their media-driven passion to renovate and restore. People watch the Food Network, he says, and run to supermarkets for exotic ingredients — and the wine to go with them. Wine is scattered throughout the store, not just in the wine aisle, to make the idea of a sophisticated meal (one that would please the Barefoot Contessa) a convenient choice.

Wine, in other words, is a lifestyle product that is promoted by lifestyle media like cable TV and lifestyle magazines that encourage and enable consumers to develop adventurous, sophisticated, consumption-driven identities. I don’t mean this in a bad way, although I know it sounds pretty bad. It’s just a fact. The magazine racks at Borders are filled with lifestyle magazines. You probably read a couple of them yourself. Even serious newspapers like the Wall Street Journal and Financial Times now have thinly disguised weekend “lifestyle” sections. Don’t pretend you don’t know what I mean!

Wine Enthusiast is a particularly good lifestyle magazine — there is a reason it has lasted 20 years. One factor in its success is that globalization has helped the wine market expand, providing more choice at affordable prices. Mr. Strum writes that

“New regions such as Australia, New Zealand, Chile, South Africa and others new to a global industry muscled their way on to the world stage. Competition drove improved methods in the winery and the vineyard. Competition also drove prices down at the middle and lower tiers.

“The world wine map has been redrawn so dramatically in the past 20 years it’s almost unrecognizable. Back then, it consisted of France, Italy, Spain and, way off in the margins, California. Now you must include Oregon and Washington State, not to mention the other New World countries I mentioned above. Every state in the union now produces wine. Countries like China and India are ramping up production in numbers that boggle the western mind.”

Martha Stewart Wine

Globalization has certainly made wine more interesting and wine drinkers can appreciate the value and variety. It would be a mistake to think that the wine media are passive observers of this phenomenon, however. It is in their interest to promote the industry that they cover and to try to profit from every aspect of it. You aren’t surprised when cable television networks expand outside the box, are you? They sell advertisements on their programs along with videos of the shows, books, lectures and assorted types of lifestyle paraphernalia. Think Martha Stewart! (And yes, there really is a Martha Stewart wine — made by Gallo).

Wine critic publications do the same thing — they have adapted the Martha Stewart System to lifestyle wine. I will focus on Wine Enthusiast here because it is their anniversary, but they are not an unusual example. Wine Spectator, Decanter, Gambero Rosso and most of the others have commercialized the wine experience in the spirit of Martha Stewart.

Mr. Strum describes Wine Enthusiast’s expansion this way

“Wine Enthusiast, as a company, has evolved dramatically over the past 20 years, too. In addition to the success of our catalog and our magazine, we have created an events division that is an astonishing success. We now annually produce four Toast of the Town events to introduce American consumers to wines that are available in their markets. These walk-around tastings, held in spectacular cultural venues, offer a sample of each city’s restaurants, accompanied by tastes of the portfolios of 70 wine companies. These events help educate and expand the palate of the American consumer, and to reinforce wine’s place at the table.”

Wine Enthusiast is more than a magazine, it is a lifestyle system. It sells magazines, of course, plus wine-related products through their catalog and website, produces wine events and so on. It informs, enlightens, educates and enables. A Wine Enthusiast cable network (or YouTube.com channel) would be the next logical step.

Even the magazine is commercialized in perhaps unexpected ways. Everyone knows that wine magazines sell lots of advertisements, of course. The editors always say that they don’t let advertising dollars influence their ratings, and I actually believe them — although market forces obviously do have some influence over the wines that they choose to consider for their reviews. National magazines need to pay attention to wines that are in national distribution. And these are the wines that are featured in the ads.

Wine Enthusiast takes one more step into commercial waters, however. The magazine includes a monthly Buying Guide that provides 100-point ratings and thumbnail reviews of dozens of wines. (I actually find their reviews to be very accurate, by the way.) But just before the long list of ratings there is section where a smaller number of wines are featured, with images of their labels for easy supermarket identification. These are the wines you will remember if you scan through the magazine quickly. I have always assumed that these were featured wines, selected by the editors for their good value or wide availability.

Imagine my surprise, then, when I started reading the fine print about how Wine Enthusiast rates wines and discovered that the labels are in fact “paid promotions.” Wineries can’t write the reviews or designate their products “best buys,” but they can pay to have them highlighted in the illustrated section! I wonder if that is true of other wine magazines? I’m going to be reading the fine print a lot more closely now so that I have a better idea of what is editorial content in the wine press and what is “paid promotion.”

Martha Stewart has only recently entered the wine business (with Paul Newman close behind), but it seems to me that the Martha Stewart system of total lifestyle marketing is already here. Hmmm. I wonder if that’s a good thing?

Wine by the Numbers

Rating the Wine Rating Systems

People turn to wine critics to tell them what’s really inside that expensive bottle (or that cheap one) and how various wines compare. Some critics are famous for their detailed wine tasting notes (Michael Broadbent comes to mind here) that provide comprehensive qualitative evaluation of wines, but with so many choices in today’s global market it is almost inevitable that quantitative rating scales would evolve. They simplify wine evaluation, which is what many consumers are looking for, but they have complicated matters, too, because there is no single accepted system to provide the rankings.

I’m interested in the variety of wine rating systems and scales that wine critics employ and the controversies that surround them. This blog entry is a intended to be a brief guide for the perplexed, an analysis of the practical and theoretical difficulties of making and using wine ranking systems.

Wine Rating Scales: 100-points, 20-points, Three Glasses and More

winescales.jpgThe first problem is that different wine critic publications use different techniques to evaluate wine and different rating scales to compare them. Click on this image to see a useful comparison of wine rating systems compiled by De Long Wine(click here to download the pdf version, which is easier to read).

Robert Parker’s Wine Advocate, the Wine Spectator and Wine Enthusiast all use a 100-point rating scale, although the qualitative meanings associated with the numbers are not exactly the same. It is perhaps not an accident that these are all American publications and that American wine readers are familiar with 100-point ratings from their high school and college classes.

In theory a 100-point system allows wine critics to be very precise in their relative ratings (a 85-point syrah really is better than an 84-point syrah) although in practice many consumers may not be able to appreciate the distinction. Significantly, it is not really a 100-point scale since 50 points is functionally the lowest grade and it is rare to see wines rated for scores lower than 70, so the scale is not really as precise as it might seem. ( Any professor or teacher will tell you, there has been both grade inflation and grade compression in recent years and this applies to wine critics too, I believe.)

The 100-point scale is far from universal. The enologists at the University of California at Davis use a 20-point rating scale, as does British wine critic Jancis Robinson and Decanter, the leading global wine magazine. The 20-point scale actually corresponds to how students are graded in French high schools and universities, so perhaps that says something about its origins.

The Davis 20-point scale gives up to 4 points for appearance, 6 points for smell, 8 points for taste and 2 for overall harmony, according to my copy of The Taste of Wine by Emile Peynaud. The Office International du Vin’s 20-point scale has different relative weights for wine qualities; it awards 4 points for appearance, 4 for smell and 12 for taste. Oz Clarke’s 20 point system assigns 2, 6 and 12 points for look, smell and taste. It’s easy to understand how the same wine can receive different scores when different critics used different criteria and different weights.

A 20-point scale (which is often really a 10-point scale) offers less precision in relative rankings, since only whole and half point ratings are available, but this may be appropriate depending upon how the ratings are to be used. Wines rated 85, 86 and 87 on a 100-point scale, for example, might all receive scores of about 16 on a 20 point scale. It’s up to you to decide if the finer evaluative grid provides useful information.

Decanter uses both a 20-point scale and as well as simple guide of zero to five stars to rate wines, where one star is “acceptable”, two is quite good, three is recommended, four is highly recommended and five is, well I suppose an American would say awesome, but the British are more reserved. Dorothy J. Gaiter and John Brecher (who write an influential wine column for the Wall Street Journal) also use a five point system; they rates wines from OK to Good, Very Good, Delicious and Delicious(!).

The five point system allows for less precision but it is still very useful – it is the system commonly used to rate hotels and resorts, for example. ViniD’Italia, the Italian wine guide published by Gambero Rosso, uses a three-glasses scale that will be familiar to European consumers who use the Michelin Guide’s three-star scale to rate restaurants.

Which System if Best?

It is natural to think that the best system is the one that provides the most information, so a 100-point scale must be best, but I’m not sure that’s true. Emile Peynaud makes the point that how you go about tasting and evaluating wine is different depending upon your purpose. Critical wine evaluation to uncover the flaws in wine (to advise a winemaker, for example) is different in his book from commercial tasting (as the basis for ordering wine for a restaurant or wine distributor or perhaps buying wine as an investment) which is different consumer tasting to see what you like.

Many will disagree, but it seems to me that the simple three or five stars/glasses/points systems are probably adequate for consumer tasting use while the 20- and 100-point scales are better suited for commercial purposes. I’m not sure that numbers or stars are useful at all for critical wine evaluation – for that you need Broadbent’s detailed qualitative notes. Wine critic publications often try to serve all three of these markets, which may explain why they use the most detailed systems or use a dual system like Decanter.

In any case, however, it seems to me that greater transparency would be useful. First, it is important that the criteria and weights are highlighted and not buried in footnotes. And I don’t see why a 20-point rating couldn’t be disaggregated like this: 15 (3/6/6) for a 20-point system that gives up to 4 points for appearance, 6 for smell and 10 for taste. That would tell me quickly how this wine differs from a 15 (4/3/8). Depending upon how much I value aroma in a wine and what type of wine it is, I might prefer the first “15” wine to the second.

Wine and Figure Skating?

So far I’ve focused on the practical problems associated with having different evaluation scales with different weights for different purposes, but there are even more serious difficulties in wine rating scales. In economics we learn that numerical measures are either cardinal or ordinal. Cardinal measures have constant units of measurment that can be compared and manipulated mathematically with ease. Weight (measured by a scale) and length (measured in feet or meters) are cardinal measures. Every kilogram or kilometer is the same.

Ordinal measures are different – they provide only a rank ordering. If I asked you to rate three wines from your most preferred to your least favorite, for example, that would be an ordinal ranking. You and I might agree about the order (rating wines A over C over B, for example), but we might disagree about how much better A was compared to C. I might think it was a little better, but for you the difference could be profound.

To use a familiar example from sports, they give the Olympic gold medal in the long jump based upon a cardinal measure of performance (length of jump) and they give the gold medal in figure skating based upon ordinal judges’ scores, which are relative not absolute measures of performance (in the U.S. they actually call the judges’ scores “ordinals”). Figure skating ratings are controversial for the same reason wine scores are.

So what kind of judgment do we make when we taste wine — do we evaluate against an absolute standard like in the long jump or a relative one like the figure skating judges? The answer is both, but in different proportions. An expert taster will have an exact idea of what a wine should be and can rate accordingly, but you and I might only be able to rank order different wines, since our abilities to make absolute judgements aren’t well developed.

This is one reason why multi-wine social blind tasting parties almost always produce unexpected winners or favorites. The wines we like better [relative] are not always the ones we like best [absolute] when evaluated on their own.

Ordinal and cardinal are just different, like apples and oranges (or Pinot Gris and Chardonnay). Imagine what the long jump would look like if ordinal “style points” were awarded? Imagine what figure skating would look like if the jumps and throws were rated by cardinal measures distance and hang time? No, it wouldn’t be a pretty sight.

Economists are taught that it is a mistake to treat ordinal rankings as if they are cardinal rankings, but that’s what I think we wine folks do sometimes. I’ve read than Jancis Robinson, who studied Mathematics at Oxford, isn’t entirely comfortable with numeric wine ratings. Perhaps it is because she appreciates this methodological difficulty.

Lessons of the Judgment of Paris

paris2.jpgOr maybe she’s just smart. Smart enough to know that your 18-point wine may be my 14-pointer. It’s clear that people approach wine with different tastes, tasting skills, expectations and even different taste buds, so relative rankings by one person need not be shared by others. This is true of even professional tasters, as the Judgment of Paris made clear.

The Judgment of Paris (the topic of a great book by George M. Taber – see below – and two questionable forthcoming films) was a 1976 blind tasting of French versus American wines organized (in Paris, of course) by Steven Spurrier. It became famous because a panel of French wine experts found to their surprise that American wines were as good as or even better than prestigious wines from French.

A recent article by Dennis Lindley (professor emeritus at University College London – see below) casts doubt on this conclusion, however. Read the article for the full analysis, but for now just click on the image above to see the actual scores of the 11 judges. It doesn’t take much effort to see that these experts disagreed as much as they agreed about the quality of the wines they tasted. The 1971 Mayacamas Cabernet, for example, received scores as low as 3 and 5 on a 20-point scale along with ratings as high as 12, 13 and 14. It was simultaneous undrinkable (according to a famous sommelier) and pretty darn good (according to the owner of a famous wine property). If the experts don’t agree with each other, what is the chance that you will agree with them?

Does this mean that wine critics and their rating systems are useless and should disappear? Not likely. Wine ratings are useful to consumers, who face an enormous range of choices and desperately need information, even if it is practically problematic and theoretically suspect. Wine ratings are useful commercially, too. Winemakers need to find ways to reduce consumer uncertainty and therefore increase sales and wine ratings serve that purpose.

And then, of course, there is the wine critic industry itself, which knows that ratings sell magazines and drive advertising. Wine ratings are here to stay. We just need to understand them better and use them more effectively.

References:

Dennis V. Lindley, “Analysis of a Wine Tasting.” Journal of Wine Economics 1:1 (May 2006) 33-41.

George M. Taber, Judgment of Paris: California vs. France and the History 1976 Paris Tasting that Revolutionized Wine. Scribner, 2005.

Taste and Power

Jonathan Nossiter’s new book inspires a research project.

legout.jpg

Taste and Power — that’s what the wine business is all about. Or at least that is the thesis of the new book (Le goût et le pouvoir) by Mondovino director Jonathan Nossiter. Those with power can influence taste or even dictate it. The English translation will appear later this year. For now I am working my way through the original French. I’ll post a review when I finish.

A recent conversation with Tyler Colman, a.k.a. Dr. Vino, started me thinking about the influence of wine critics and how globalization is increasing their importance. Think about the number of wines you have to choose from. My local upscale supermarket stocks more than 1500 different wines from at least 15 different countries. This is an enormous and complex choice space that ranges from the familiar to the exotic and from inexpensive to near investment-grade. And the number of choices is unexceptional. A local farm store, with acres of space to fill, actually stocks more than 3500 wines from more than 25 countries.

 

An Embarassment of Riches

The simple fact of this embarrassment of riches makes wine critics useful and influential. How can you make an intelligent choice from among so many different wines? It is difficult to know what’s in the bottle without tasting and there are too many to taste. Wine critics are middlemen who do the tasting for us and arbitrage that information, reducing the uncertainty that is both the joy of wine and its curse. (Wine brands are another way to reduce uncertainty and increase sales, as I have discussed in earlier posts).

Wine critics and their descriptions and ratings would be useful if you have only 100 wines to choose from. With thousands of wines available, they become practically indispensable. The global expansion of wine trade increases choice and uncertainty and magnifies the value (and power?) of wine critics.

 

Wine critics are important for other reasons, too. Wine can be an investment as well as a consumable product and wine critics provide information to this forward-looking market. The question here is not what wine tastes like today, but what it will taste like in several years and, most importantly, what a buyer will pay for it in the future. Here, because there are so many unknowns, wine critics can have great influence. Some wines probably have investment value because the critics say they do, so much is the market driven by critic-inspired perceptions of value.

winemags.jpg

Critics are also important because wine is increasingly an identity investment, not just a financial investment. Individuals invest in both wines and in specialized knowledge about wine both for their own pleasure and to make a statement about their identities. To be very knowledgeable about wine is to display a specialized cultural sophistication. It isn’t the same as owning a Ferrari or a Renoir (it might be more expensive than owning a Ferrari or Renoir) but it makes a statement in the same way. The very best wine, not just good wine, can be an object of obsession, hence the outrageous prices that are paid. Wine critics both enable and encourage the quest.

Wine Critics and Wine Markets

The influence of wine critics are everywhere in the wine market, from budget buyers seeking good value to elite wine collectors. You can complain about their power and dispute their taste, as critics of Robert Parker frequently do, but they are here to stay. So I think we need to learn more about them.

Hence my current project. I’ve collected a number of wine publications (click on the photo above to see them) that evaluate and rate wines as well as provide other information about wine, including investment reports, wine tourism guides, winemaker biographies, food and wine pairing tips and so on. Rather than criticize their numerical ranking scales or bemoan their philistine tastes, I want to compare and contrast them, to try to figure out what wines they are rating, how, why and for whom? My working hypothesis is that wine critics are simultaneously influenced by the market segments they inhabit and shape them, too. If I’m right, then these publications should be very different, even when the wines they evaluate are the same.

The publications I’m studying are Wine Spectator, the best-selling American wine magazine and Robert Parkers Wine Advocate, which is said to be the most influential. I am also examining two other national publications, The Wine Enthusiast and Wine & Spirits, Wine Press Northwest, a regional publication, and two very important foreign journals, Decanter (Great Britain) and Gambero Rosso (Italy). Watch this space for upcoming reports!

Follow

Get every new post delivered to your Inbox.

Join 2,521 other followers