Wine Economist Top 100

February 22, 2009 By Mike Veseth in Mondovino, Shameless Self-Promotion, Supply and Demand, wine awards, Wine Competitions, wine critics, wine markets, wine ratings, Yellow Tail 2 Comments

This is the Wine Economist‘s 100th post. The idea of a Wine Economist Top 100 — my 100 best blog posts — is therefore kinda ridiculous.

But my wine enthusiast friends hungrily devour Top 50 and Top 100 wine lists even though the idea that it is possible to identify and rank the Top X [fill in the number] wines is kinda ridiculous, too, although in a different way. This provokes a digression on wine rankings and a brief report on what I’ve learned so far from writing this blog.

Supply and Demand

Ranking wines from 1 to 100 is certainly not an exact science; there are literally thousands of wines on the market, so narrowing down the list to 100 and then actually ranking them from bottom to top (with no ties) is necessarily a problematic exercise when examined closely.

Individual tastes differ significantly and consumers are not uniformly able to detect even objective qualities in wine (much less make comparable subjective judgments), so it is hard to see why so many people take these ratings so very seriously. But they do. It’s a matter of demand and supply.

Consumers demand wine rankings. They use Top X lists as guides to shopping (or investing) and sometimes as a means to establish status or credibility with other wine enthusiasts. This makes top wine lists a really useful tool for wine merchants and distributors, who supply what consumers demand (and sometimes try to help the demand along a little, too).

Wine critics must feel some pressure to supply what buyers and merchants want. The Top X lists get so much attention that any critic who fails to issue a ranking must be a little bit concerned about the effect of this action in the crowded wine opinion marketplace. If I ranked wines, which I don’t, I’d sure want to publish a Top X list of some sort if only to draw attention to my other work. Everyone has an interest in these lists, so it’s no wonder they are so popular.

Winner-Take-All

It is interesting to consider how Top X lists and the attention they receive may have invisibly shaped the wine world. Cornell economist Robert H. Frank has written two books that are worth reading in this regard. The Winner-Take-All Society (co-authored by Philip J. Cook, 1995) looks at what happens when market attention is focused on a few top-rated products. The result, not surprisingly, is that everyone wants the best (or what is rated the best) and the nearly-as-good and really-quite-pleasant are left behind. Who wants to drink pretty good Chardonnay when you can get a 90+ bottle for the same price (even if you cannot really taste the difference yourself)?

The book’s subtitle tells you where the argument goes: “How more and more Americans compete for fewer and bigger prizes, encouraging economic waste, income inequality and an impoverished cultural life.” If you’ve seen Mondovino, you know what Frank and Cook are talking about.

Frank’s 1999 book Luxury Fever continues the argument, looking more deeply at the impact of a world where status, identity and satisfaction are linked to money and the purchase of top-rated products. Frank talks about the high price that some consumers will pay for goods that are just a little bit better or harder to get. He calls it the “charm premium.” Unsurprisingly, he cites the “charm premium” that highly rated ultra-premium wines receive as an important example (pp. 29-30).

Elite winemakers can mine the charm premium effect by offering increasing expensive variations on their main product: regular bottling, reserve, single-vineyard and so on. Each increase in perceived quality (or decrease in general availability) produces a disproportionate increase in price. Or at least that’s how it is supposed to work.

Some wine merchants and producers see the charm premium in a different light. Wines that get 95+ points sell out immediately — they essentially don’t exist. Ultra-premium wines that receive less than 90 points are hard to sell, because no one wants a merely very good wine when they can get an apparently excellent one. (I understand that there is at least one wine store that automatically discounts any wine that is cursed with an otherwise unsellable 89-point rating.) That just leaves the 90-94 point wines and large charm premiums are sometimes paid for what must be impossibly small absolute quality differences within this range.

Wine buyers are a diverse group and so it is dangerous to generalize, but a lot of them search not just for good wine, but for the best wine (or the best wine value). For better or worse, Top 100 lists have evolved to satisfy that demand and have therefore helped spread luxury fever and create the winner-take-all wine market segments we see today.

Lessons Learned

I’ve been writing this blog for about a year and a half and it has been a great experience — I’ve met a lot of thoughtful, interesting people and had some great wine conversations. Because my posts are a bit longer than most — about 900 words on average — the total 100-post output is equivalent to a short book. What have I learned from this process? Well, I know a little more about what internet wine readers are looking for.

The most popular Wine Economist article in its 100-post history is my piece on Decanter magazine (The World’s Best Wine Magazine?), part of an occasional series on wine critics. This post gets a lot of hits because the web is crawling with people searching for “best wine,” “best wine magazine” and “world’s best wine.” The winner-take-all dynamic this represents shows up everywhere, even in my blog stats.

Almost as many readers are searching for the best wine value, which explains why my posts on[Yellow Tail] Tales and Costco and Global Wine are the second and third most read articles on this blog.

Wine industry readers are worried about the future, as most of us are in this economic environment. This helps explain why How will the Economic Crisis affect Wine? and Big Trouble Down Under: Crisis in Australian Wine receive so many hits.

Finally, many readers come here looking to unlock the mysteries of the wine buying experience. What do the ratings mean? Who are the most credible wine authorities? This search leads them to posts onWine by the Numbers and Masters of Wine (and Economics), which get dozens of hits each week.

Thanks for reading The Wine Economist. I’ll give an update on trends in reader interests and concerns in a few months, when I published the Wine Economist Top 150.

Attack of the Super-Cuvées

June 28, 2008 By Mike Veseth in critics, decanter, wine critics, wine markets, wine price, wine ratings 1 Comment

I’ve been thinking a lot recently about the different strategies winemakers use to sell their products and I find myself coming back again and again to a particularly good column by Andrew Jefford in the June 2008 issue of Decanter magazine. Jefford demonstrates how important economic factors are in shaping what gets poured into your glass.

Bordeaux, Burgundy and Super-Cuvée

Winemakers generally find themselves working with variable quantities and qualities of fruit and the individual wines made from that fruit. What should be done if some of the raw materials are much better than others as is typically the case? Blend them all together? That’s what happens sometimes, especially in the less sophisticated cooperatives in Southern Europe. The result is often a whole that does not exceed the sum of its parts. Historically, Jefford explains, there are two dominant approaches to the problem of wine quality to take into account variable wine quality.

The first is the Bordeaux strategy. The best lots are blended into the grand vin — the one that sells for a high price (in good years at least). Good wines that just don’t make the cut for the grand vin go into a second label. Lesser wines are sold off on the bulk market or even a third label. The system is transparent, relatively consumer friendly and the wines are as good as the vintage allows. In The Wine Advocate’s report on 2005 Bordeaux, for example, the grand vin Latour (96+ points, 12,000 cases) costs $1125 while the second wine Les Forts de Latour (92 points, 10,000 cases) is less than $200. A third wine, designated simply Paulliac (89 points) sells for about $60. Something for everyone, I guess, and a pretty clear hierarchy of wines, although not every Bordeaux producer listed in Parker’s guide displays such a clear link between price and apparent quality.

Burgundy provides a second model, according to Jefford. The top wines are released as individual vineyard-designated wines and the remaining wines are blended together in to an appellation-designated wine. I am not an expert on Burgundies, but I have seen this in Oregon, where some wineries release a one Willamette Valley Pinot Noir along with several vineyard designated wines. This makes the wines very interesting if you are able to taste them side-by-side, but it can otherwise be confusing. I think it shifts a bit of power to the wine critics and specialists. But that’s fine if the terroir really comes through and the wines are significantly different.

The third strategy, which Jefford links to the Rhône, is to create a tiny amount of a sort of super-wine that is made, more or less, to gain high scores from wine critics. I know that wine makers always say that they don’t make wines to get high scores, but a few have privately told me that big Parker numbers are so valuable that they don’t hesitate to make some wines to try to impress wine critics like RP. Most customers won’t have a chance to buy the Parker wine, it is true, because the production is so small, but the benefits to the winery’s reputation may extend down the line and boost prices and sales of the other wines.

Money, Taste and Power

“Let’s set aside the question of income,” Jefford writes, because he is concerned with the effect of the three systems on the quality of the wine. The Burgundy system, with its stress on terroir, makes sense when there really is terroir, but otherwise he argues in favor of the Bordeaux plan because it produces more complete wines, wines of good “disposition,” as he puts it. Wines that are good to drink.

The Super-Cuvée strategy, he fears, makes wines that are good to taste (and rate), but not to drink because they are wines of “accumulation,” monster wines, where winemakers seek high scores by adding more and more layers of identifiable attributes to the super-wine at the expense of the quality, complexity and completeness of the wines that make up the bulk of production.

On the face of it, Jefford’s critique of super-wines and the rising power of critics associated with them is based upon taste, but as the column continues it is quickly apparent that he cannot leave the wine economics out of it, because it is at the heart of the problem. The Bordeaux model doesn’t just make better wines, he suggests, but also better incomes in the long run for all producers. The simplicity of the single Grand vin strategy makes wines more understandable and a “perfect building block,” as he says for a reputation. He credits Bordeaux’s use of this system in part for their consistently strong reputation and ability to attract investment and maintain strong prices. He contrasts this with the “chaos and frenzy” of the Super-Cuvées and the “false intellectual challenge” of the many single-vineyard offerings. Pretty strong words, I would say.

The Burgundy and super-wine strategies depend in different ways on the power of wine critics (like some of those who write for Decanter). Wine critics validate the legitimacy of single-vineyard offerings and create super-markets for the Super-Cuvées. It would be interesting to study what factors lead winemakers, especially New World producers, to choose one approach over another. Perhaps it is terroir. Perhaps it is scale, training or philosophy. Or perhaps it is access to wine critics and the power they can have in the marketplace. Watch this space for more research on this interesting question.

Tyranny of the 100 Point Wine Scale

May 27, 2008 By Mike Veseth in book reviews, critics, Paul Gregutt, Washington wine, wine critics, wine ratings Leave a comment

Wine rating systems are like the weather — everyone complains about them but no one does anything. Paul Gregutt thinks he knows why.

Wine by the Numbers

There are many ways to rate or rank different wines and consumers are very interested in trying to understand what they mean and how to use them. That’s why my column on wine rating systems, Wine by the Numbers, is one of the most popular posts in this blog’s brief history.

Twenty-point rating systems are popular in Europe in part, I understand, because that is how papers are graded in French high schools. Here in the United States the 100 point system that Robert Parker popularized and many others use dominates in part, I suppose, because that’s how our papers were graded in school. Any simple wine scoring system is problematic, however, since a good deal of information is necessarily lost when the attributes of a multidimensional product like wine are reduced to a single number. Wine isn’t like gasoline, where the critical components can be captured, like octane, in a single number.

But there are other problems, too. Paul Gregutt (PG), wine critic for the Seattle Times and Wine Enthusiast magazine, explained the limitations of the 100 point scale and the tendency toward “grade inflation” in his recent book, Washington Wines & Wineries:

This practice of promoting wine, a multi-faceted, subjective sensory experience, simply by broadcasting numbers has gradually devalued the numbers while shrinking the original 100-point scale. At first blush, such a rating system sounds generous, allowing room for a lot of subtlety in the grading curve. But in actual practice it’s not a hundred point scale at all, nor even close.

It has become a ten point scale. Wines rated under 85 are ignored completely. Wines rated 85 to 89 must be marketed as value wines – those numbers only work for wines priced at the low end of the scale. If your wine is going to sell for $15 or more, it must hit 90 points at least. One prominent retailer even makes a point of selling (at discount) wines that have scored the “dreaded” 89. Once a wine moves up the ladder from there, it becomes increasingly rare and expensive. As a result, wines scoring 95 or above are virtually unobtainable for the average consumer.

Although the ratings are “devalued” in terms of their utility, they are also “inflated” in terms of their commercial importance. It would seem that consumers might be better served if someone would re-center the scale so that it uses more of the 100 point range and is therefore potentially (and only potentially) a more accurate guide to quality. This would obviously lower the average score, however. Who is going to be the first to break the pattern and give merely good wines average scores, 70 or 75 instead of 85 so that 85 (B+) means something?

Revising the 100-Point Scale

PG decided to try in his book on Washignton wine. Instead of rating individual wines, he rated the wineries themselves in terms of a modified 100-point scale that gives marks for style (30 points), consistency (30 points), value (30 points) and the winery’s contribution to the development and improvement of the Washington wine industry (10 points).

How did the wineries rate? Quilceda Creek, with its perfect 100 Robert Parker point Cabernet Sauvignon, also got a perfect PG score (30/30/30/10). Leonetti ranked second with 98 points (30/28/30/10). So far so good. The trouble comes further down the list where some prominent wineries get scores in the 70s, 60s, and 50s. The scores make sense when you break them down into the four factors that PG evaluates, but the raw numbers are sort of shocking when you see them for the first time or out of context. (My university students know this feeling, I suspect. It happens when they get their first college papers back after several years of high school grade inflation.)

PG writes about the reaction to his winery rating on his website in a column titled “Time to Dump my 100 Point System?”

Comments from readers and reviewers have been largely positive. Some have embraced my scoring; others have simply accepted it and moved on to the book’s other assets. But within the ranks of the industry itself – wineries and distributors in particular – there has been an awkward silence.

There is little doubt that this book’s sales have been seriously impacted by 1) the decision not to include 3/4 of this state’s wineries and 2) the scoring system itself. One winery veteran, after some prompting, took the trouble to explain why his winery wouldn’t sell or promote the book, even though I had given them one my highest scores.

“Our problem with promoting your book,” the winemaker said, “is that, in spite of the wonderful written praise, we’d spend all out time explaining our B+ grade.” PG is trying to decide if he should scrap his ratings for the next edition of his book (if you have an opinion you can contact Paul through his website PaulGregutt.com).

The First Mover Disadvantage

It is hard to know what to say about this. On one hand I admit that my first reaction to PG’s winery rating scale was neutral to negative, but then with some encouragement from Karen Wade I looked at it more closely and decided that it was actually pretty useful to me as a consumer — so I guess I am the source of some of the positive feedback PG received. On the other hand, I understand that no winemaker is going to take out an ad that boasts “Rated 72 by Critics!” — even if that’s a very good and appropriate rating by the scale being used.

I think we have to admit that re-centering the 100 point scale is a hopeless task and move on. The first mover in point reform will suffer the sort of criticism that PG reports. The only way to do it would be for everyone to switch scales at once. I don’t see that happening.

So what should we do about the ratings? For my part I’m going to try to get my students, who are very much into wine ratings by the time they come to me, to use the UC Davis 20-point scale. I think it might work because (1) they aren’t used to thinking in terms of 20 points and so they will have more open minds about what scores mean and (2) I like the way it breaks down elements of sensory perception: 4 points for appearance, 6 points for smell, 8 points for taste and 2 for overall harmony, according to my copy of The Taste of Wine by Emile Peynaud. Using the Davis scale will encourage them to make up their own minds about what they see, smell and taste. That’s a good thing.

In the meantime I think Paul Gregutt’s experience suggests both why the 100 point system should die and why it probably never will. My advice to Paul: keep the analysis in your book, which is terrific, but kiss the 100-point ratings goodbye.

They Always Buy the Ten Cent Wine

April 22, 2008 By Mike Veseth in wine markets, wine ratings Leave a comment

Ernest and Julio Gallo built their business in the years after the repeal of Prohibition according to a strict division of labor: Julio made the wines and Ernest sold them. I don’t know if Ernest Gallo really appreciated the spiritual element of wine, but he did know that selling wine is not just about selling what is in the bottle. Selling wine, like selling anything else, is also about selling image, mystique or terroir.

The story is told of a sales call that Ernest Gallo made to a New York customer in the dark days of the depression. He offered sample glasses of two red wines – one costing five cents per bottle and the other ten cents. The buyer tasted both and pronounced, “I’ll take the ten-cent one.” The wine in the two glasses was exactly the same. Clearly, the customer wanted to buy an identity – the image of someone who wouldn’t drink that five-cent rotgut- even if he couldn’t actually taste the difference.

They always buy the ten cent wine, Ernest Gallo said.

I wonder how much things have changed since the days when Ernest Gallo made his calls? Two recent studies provoke this question.

Price and Taste

The first, which has been widely reported, is a study that was published in the Proceedings of the National Academy of Sciences that showed that test subjects displayed the Ernest Gallo effect. Their ratings of wines changed when they were given price information — even bogus price information. Identical wines received different ratings depending upon price information provided. “Expensive” wines, naturally, were rated higher than their inexpensive twins.

A second study, just released by the American Association of Wine Economists, answers the question Do more expensive wines taste better? The answer, based on a large sample of blind tastings, is that there is no correlation between price and wine evaluation (or perhaps a modestly negative one). This will be no surprise to readers of wine publications like Wine Spectator. Sure, the top wines are usually expensive, but there are also a lot of costly wines that get low ratings.

These findings will give bring great satisfaction to my friend and part-time research assistant Michael Morrell, who prides himself on drinking cheap wines, trusting his own tastes not ratings or price “signals” of quality. Michael would buy the five cent bottle every time.

But ask any wine distributor or retailer and you will find that price is the critical factor in retail wine sales. Although wine enthusiasts like to think of themselves in complicated ways — favoring red versus white, old world versus new world, merlot versus pinot noir, fruit bomb versus barrique reserve — the dirty little secret of wine retailing is that price is the key to most wine buying decisions. When push comes to shove, buyers are really looking for an $8 wine or a $10 wine and and make their purchases within a relatively narrow price range, regardless of other factors.

Evidence from the Wine Aisle

The wine aisle in your grocery store is probably organized this way. Yes, I know there is a California section and an Import section and even a jug/box wine spot, but look within each wine display and you’ll see the clear price stratification effect. The wines you have come to buy are probably on the shelf just below your natural eye level, so that you cannot help but see those special occasion wines just above them (and the higher priced wines above them on the top shelf). Cheaper wines are down below, near the floor, so that you have to stoop down to choose them.

The physical act of taking the wine from the shelf mirrors the psychological choice you make — reach up for better (more expensive) wines, stoop down for the cheaper products. The principle will be the same in upscale supermarkets and discount stores but the choices (what price wine will be at the bottom, middle and top) will differ as you might expect.

Studies suggest that people establish a wine price comfort zone (and corresponding wine shelf) and stay there, moving up a rank for special occasions and down a shelf for parties and other higher-volume purchases. A lot of factors drive this behavior, including fear. I have some $8 wine friends who are afraid to start drinking $12 wines for fear that they will be able to taste the difference — and have to upgrade their wine budgets.

One local supermarket has taken this principle to its logical extreme. It is a discount store that counts a lot of immigrants and retired people among its customers. The wine aisle is not organized as you might expect — by country of origin or wine varietal. It’s logic is simple and clear. This rack has wines that cost $3 and less. The next rack has $3 – $5 wines. And so on up to the $15+ wine rack. Large signs efficiently guide buyers directly to their target zone. Do they sell a lot of wine? You bet they do!

The relationship between the price of wine and our evaluation of it is complex. These recent studies indicate that we shouldn’t let price information influence our decisions, but marketing experience shows that most of us do.

The Martha Stewart Wine System

March 4, 2008 By Mike Veseth in Martha Stewart, wine awards, wine critics, wine magazines, wine ratings Leave a comment

Wine Enthusiast magazine celebrates its 20th anniversary with the March 2008 issue and editor and publisher Adam Strum reflects on the changing market in “The Enthusiast Corner” column. He writes that

“I’d like to think Wine Enthusiast played an important part in helping to bring wine to the attention of the American public at large, and not just the elite, over these 20 years. Wine magazines, books and the rise of food television have all undoubtedly played a role in making America a wine drinking nation. Other factors abound: American cuisine at home and in fine dining restaurants underwent a renaissance, and wine naturally became an important part of that. News of wine’s health benefits enlarged its consumer base. But most responsible for the growth of wine is the incredible leap in terms of overall quality at the same time that wine became more affordable. How often does that happen? Name me one consumer product that can compare.”

I think he is right in all this. Wine’s vigorous growth in the United States is a complex phenomenon. Many factors have contributed to the rise in per capita consumption in the United States and other New World markets at the same time that wine drinking has fallen dramatically in the Old World. The wine media’s role may be an under-appreciated element of this phenomenon.

The Supermarket as Home Depot with Wine

My friend Patrick works the wine aisle at a local upscale supermarket and he constantly delights me with his original insights into consumer behavior. He sees cable TV’s influence everywhere, for example. People watch Trading Spaces or the home remodeling network HGTV, he says, and run out to Home Depot for wallpaper and remodeling supplies. A huge industry has been built around their media-driven passion to renovate and restore. People watch the Food Network, he says, and run to supermarkets for exotic ingredients — and the wine to go with them. Wine is scattered throughout the store, not just in the wine aisle, to make the idea of a sophisticated meal (one that would please the Barefoot Contessa) a convenient choice.

Wine, in other words, is a lifestyle product that is promoted by lifestyle media like cable TV and lifestyle magazines that encourage and enable consumers to develop adventurous, sophisticated, consumption-driven identities. I don’t mean this in a bad way, although I know it sounds pretty bad. It’s just a fact. The magazine racks at Borders are filled with lifestyle magazines. You probably read a couple of them yourself. Even serious newspapers like the Wall Street Journal and Financial Times now have thinly disguised weekend “lifestyle” sections. Don’t pretend you don’t know what I mean!

Wine Enthusiast is a particularly good lifestyle magazine — there is a reason it has lasted 20 years. One factor in its success is that globalization has helped the wine market expand, providing more choice at affordable prices. Mr. Strum writes that

“New regions such as Australia, New Zealand, Chile, South Africa and others new to a global industry muscled their way on to the world stage. Competition drove improved methods in the winery and the vineyard. Competition also drove prices down at the middle and lower tiers.

“The world wine map has been redrawn so dramatically in the past 20 years it’s almost unrecognizable. Back then, it consisted of France, Italy, Spain and, way off in the margins, California. Now you must include Oregon and Washington State, not to mention the other New World countries I mentioned above. Every state in the union now produces wine. Countries like China and India are ramping up production in numbers that boggle the western mind.”

Martha Stewart Wine

Globalization has certainly made wine more interesting and wine drinkers can appreciate the value and variety. It would be a mistake to think that the wine media are passive observers of this phenomenon, however. It is in their interest to promote the industry that they cover and to try to profit from every aspect of it. You aren’t surprised when cable television networks expand outside the box, are you? They sell advertisements on their programs along with videos of the shows, books, lectures and assorted types of lifestyle paraphernalia. Think Martha Stewart! (And yes, there really is a Martha Stewart wine — made by Gallo).

Wine critic publications do the same thing — they have adapted the Martha Stewart System to lifestyle wine. I will focus on Wine Enthusiast here because it is their anniversary, but they are not an unusual example. Wine Spectator, Decanter, Gambero Rosso and most of the others have commercialized the wine experience in the spirit of Martha Stewart.

Mr. Strum describes Wine Enthusiast’s expansion this way

“Wine Enthusiast, as a company, has evolved dramatically over the past 20 years, too. In addition to the success of our catalog and our magazine, we have created an events division that is an astonishing success. We now annually produce four Toast of the Town events to introduce American consumers to wines that are available in their markets. These walk-around tastings, held in spectacular cultural venues, offer a sample of each city’s restaurants, accompanied by tastes of the portfolios of 70 wine companies. These events help educate and expand the palate of the American consumer, and to reinforce wine’s place at the table.”

Wine Enthusiast is more than a magazine, it is a lifestyle system. It sells magazines, of course, plus wine-related products through their catalog and website, produces wine events and so on. It informs, enlightens, educates and enables. A Wine Enthusiast cable network (or YouTube.com channel) would be the next logical step.

Even the magazine is commercialized in perhaps unexpected ways. Everyone knows that wine magazines sell lots of advertisements, of course. The editors always say that they don’t let advertising dollars influence their ratings, and I actually believe them — although market forces obviously do have some influence over the wines that they choose to consider for their reviews. National magazines need to pay attention to wines that are in national distribution. And these are the wines that are featured in the ads.

Wine Enthusiast takes one more step into commercial waters, however. The magazine includes a monthly Buying Guide that provides 100-point ratings and thumbnail reviews of dozens of wines. (I actually find their reviews to be very accurate, by the way.) But just before the long list of ratings there is section where a smaller number of wines are featured, with images of their labels for easy supermarket identification. These are the wines you will remember if you scan through the magazine quickly. I have always assumed that these were featured wines, selected by the editors for their good value or wide availability.

Imagine my surprise, then, when I started reading the fine print about how Wine Enthusiast rates wines and discovered that the labels are in fact “paid promotions.” Wineries can’t write the reviews or designate their products “best buys,” but they can pay to have them highlighted in the illustrated section! I wonder if that is true of other wine magazines? I’m going to be reading the fine print a lot more closely now so that I have a better idea of what is editorial content in the wine press and what is “paid promotion.”

Martha Stewart has only recently entered the wine business (with Paul Newman close behind), but it seems to me that the Martha Stewart system of total lifestyle marketing is already here. Hmmm. I wonder if that’s a good thing?

Wine by the Numbers

February 9, 2008 By Mike Veseth in critics, decanter, Jancis Robinson, journal of wine economics, Michael Broadbent, robert parker, Uncategorized, wine critics, wine magazines, wine ratings 1 Comment

Rating the Wine Rating Systems

People turn to wine critics to tell them what’s really inside that expensive bottle (or that cheap one) and how various wines compare. Some critics are famous for their detailed wine tasting notes (Michael Broadbent comes to mind here) that provide comprehensive qualitative evaluation of wines, but with so many choices in today’s global market it is almost inevitable that quantitative rating scales would evolve. They simplify wine evaluation, which is what many consumers are looking for, but they have complicated matters, too, because there is no single accepted system to provide the rankings.

I’m interested in the variety of wine rating systems and scales that wine critics employ and the controversies that surround them. This blog entry is a intended to be a brief guide for the perplexed, an analysis of the practical and theoretical difficulties of making and using wine ranking systems.

Wine Rating Scales: 100-points, 20-points, Three Glasses and More

The first problem is that different wine critic publications use different techniques to evaluate wine and different rating scales to compare them. Click on this image to see a useful comparison of wine rating systems compiled by De Long Wine(click here to download the pdf version, which is easier to read).

Robert Parker’s Wine Advocate, the Wine Spectator and Wine Enthusiast all use a 100-point rating scale, although the qualitative meanings associated with the numbers are not exactly the same. It is perhaps not an accident that these are all American publications and that American wine readers are familiar with 100-point ratings from their high school and college classes.

In theory a 100-point system allows wine critics to be very precise in their relative ratings (a 85-point syrah really is better than an 84-point syrah) although in practice many consumers may not be able to appreciate the distinction. Significantly, it is not really a 100-point scale since 50 points is functionally the lowest grade and it is rare to see wines rated for scores lower than 70, so the scale is not really as precise as it might seem. ( Any professor or teacher will tell you, there has been both grade inflation and grade compression in recent years and this applies to wine critics too, I believe.)

The 100-point scale is far from universal. The enologists at the University of California at Davis use a 20-point rating scale, as does British wine critic Jancis Robinson and Decanter, the leading global wine magazine. The 20-point scale actually corresponds to how students are graded in French high schools and universities, so perhaps that says something about its origins.

The Davis 20-point scale gives up to 4 points for appearance, 6 points for smell, 8 points for taste and 2 for overall harmony, according to my copy of The Taste of Wine by Emile Peynaud. The Office International du Vin’s 20-point scale has different relative weights for wine qualities; it awards 4 points for appearance, 4 for smell and 12 for taste. Oz Clarke’s 20 point system assigns 2, 6 and 12 points for look, smell and taste. It’s easy to understand how the same wine can receive different scores when different critics used different criteria and different weights.

A 20-point scale (which is often really a 10-point scale) offers less precision in relative rankings, since only whole and half point ratings are available, but this may be appropriate depending upon how the ratings are to be used. Wines rated 85, 86 and 87 on a 100-point scale, for example, might all receive scores of about 16 on a 20 point scale. It’s up to you to decide if the finer evaluative grid provides useful information.

Decanter uses both a 20-point scale and as well as simple guide of zero to five stars to rate wines, where one star is “acceptable”, two is quite good, three is recommended, four is highly recommended and five is, well I suppose an American would say awesome, but the British are more reserved. Dorothy J. Gaiter and John Brecher (who write an influential wine column for the Wall Street Journal) also use a five point system; they rates wines from OK to Good, Very Good, Delicious and Delicious(!).

The five point system allows for less precision but it is still very useful – it is the system commonly used to rate hotels and resorts, for example. ViniD’Italia, the Italian wine guide published by Gambero Rosso, uses a three-glasses scale that will be familiar to European consumers who use the Michelin Guide’s three-star scale to rate restaurants.

Which System if Best?

It is natural to think that the best system is the one that provides the most information, so a 100-point scale must be best, but I’m not sure that’s true. Emile Peynaud makes the point that how you go about tasting and evaluating wine is different depending upon your purpose. Critical wine evaluation to uncover the flaws in wine (to advise a winemaker, for example) is different in his book from commercial tasting (as the basis for ordering wine for a restaurant or wine distributor or perhaps buying wine as an investment) which is different consumer tasting to see what you like.

Many will disagree, but it seems to me that the simple three or five stars/glasses/points systems are probably adequate for consumer tasting use while the 20- and 100-point scales are better suited for commercial purposes. I’m not sure that numbers or stars are useful at all for critical wine evaluation – for that you need Broadbent’s detailed qualitative notes. Wine critic publications often try to serve all three of these markets, which may explain why they use the most detailed systems or use a dual system like Decanter.

In any case, however, it seems to me that greater transparency would be useful. First, it is important that the criteria and weights are highlighted and not buried in footnotes. And I don’t see why a 20-point rating couldn’t be disaggregated like this: 15 (3/6/6) for a 20-point system that gives up to 4 points for appearance, 6 for smell and 10 for taste. That would tell me quickly how this wine differs from a 15 (4/3/8). Depending upon how much I value aroma in a wine and what type of wine it is, I might prefer the first “15” wine to the second.

Wine and Figure Skating?

So far I’ve focused on the practical problems associated with having different evaluation scales with different weights for different purposes, but there are even more serious difficulties in wine rating scales. In economics we learn that numerical measures are either cardinal or ordinal. Cardinal measures have constant units of measurment that can be compared and manipulated mathematically with ease. Weight (measured by a scale) and length (measured in feet or meters) are cardinal measures. Every kilogram or kilometer is the same.

Ordinal measures are different – they provide only a rank ordering. If I asked you to rate three wines from your most preferred to your least favorite, for example, that would be an ordinal ranking. You and I might agree about the order (rating wines A over C over B, for example), but we might disagree about how much better A was compared to C. I might think it was a little better, but for you the difference could be profound.

To use a familiar example from sports, they give the Olympic gold medal in the long jump based upon a cardinal measure of performance (length of jump) and they give the gold medal in figure skating based upon ordinal judges’ scores, which are relative not absolute measures of performance (in the U.S. they actually call the judges’ scores “ordinals”). Figure skating ratings are controversial for the same reason wine scores are.

So what kind of judgment do we make when we taste wine — do we evaluate against an absolute standard like in the long jump or a relative one like the figure skating judges? The answer is both, but in different proportions. An expert taster will have an exact idea of what a wine should be and can rate accordingly, but you and I might only be able to rank order different wines, since our abilities to make absolute judgements aren’t well developed.

This is one reason why multi-wine social blind tasting parties almost always produce unexpected winners or favorites. The wines we like better [relative] are not always the ones we like best [absolute] when evaluated on their own.

Ordinal and cardinal are just different, like apples and oranges (or Pinot Gris and Chardonnay). Imagine what the long jump would look like if ordinal “style points” were awarded? Imagine what figure skating would look like if the jumps and throws were rated by cardinal measures distance and hang time? No, it wouldn’t be a pretty sight.

Economists are taught that it is a mistake to treat ordinal rankings as if they are cardinal rankings, but that’s what I think we wine folks do sometimes. I’ve read than Jancis Robinson, who studied Mathematics at Oxford, isn’t entirely comfortable with numeric wine ratings. Perhaps it is because she appreciates this methodological difficulty.

Lessons of the Judgment of Paris

Or maybe she’s just smart. Smart enough to know that your 18-point wine may be my 14-pointer. It’s clear that people approach wine with different tastes, tasting skills, expectations and even different taste buds, so relative rankings by one person need not be shared by others. This is true of even professional tasters, as the Judgment of Paris made clear.

The Judgment of Paris (the topic of a great book by George M. Taber – see below – and two questionable forthcoming films) was a 1976 blind tasting of French versus American wines organized (in Paris, of course) by Steven Spurrier. It became famous because a panel of French wine experts found to their surprise that American wines were as good as or even better than prestigious wines from French.

A recent article by Dennis Lindley (professor emeritus at University College London – see below) casts doubt on this conclusion, however. Read the article for the full analysis, but for now just click on the image above to see the actual scores of the 11 judges. It doesn’t take much effort to see that these experts disagreed as much as they agreed about the quality of the wines they tasted. The 1971 Mayacamas Cabernet, for example, received scores as low as 3 and 5 on a 20-point scale along with ratings as high as 12, 13 and 14. It was simultaneous undrinkable (according to a famous sommelier) and pretty darn good (according to the owner of a famous wine property). If the experts don’t agree with each other, what is the chance that you will agree with them?

Does this mean that wine critics and their rating systems are useless and should disappear? Not likely. Wine ratings are useful to consumers, who face an enormous range of choices and desperately need information, even if it is practically problematic and theoretically suspect. Wine ratings are useful commercially, too. Winemakers need to find ways to reduce consumer uncertainty and therefore increase sales and wine ratings serve that purpose.

And then, of course, there is the wine critic industry itself, which knows that ratings sell magazines and drive advertising. Wine ratings are here to stay. We just need to understand them better and use them more effectively.

References:

Dennis V. Lindley, “Analysis of a Wine Tasting.” Journal of Wine Economics 1:1 (May 2006) 33-41.

George M. Taber, Judgment of Paris: California vs. France and the History 1976 Paris Tasting that Revolutionized Wine. Scribner, 2005.

Wine Critics Reviewed: Gambero Rosso

February 3, 2008 By Mike Veseth in critics, gambero rosso, Italian wine, Paul Gregutt, wine critics, wine ratings Leave a comment

The first in a series on the actors and institutions that shape the wine world.

I’m interested in the role of wine critics (and their publications) in shaping the global wine market. This is the first of several entries that will examine a few of the most influential publications. I’m not entirely sure where this process will take me, but I’m pretty interested to find out what I will learn. So let’s begin.

I want to start with Gambero Rosso, which is a very influential Italian publication. It is the Italian equivalent of Wine Spectator, but it goes well beyond WS in its coverage of food and wine culture and tourism. It publishes a number of important guides to Italian regional restaurants, food and wine that I have found to be extremely useful when I have lived in or visited Italy. The Italian monthly magazine is physically imposing – the issue that appears in the photo above weighs in at more than 400 pages and was filled with ads. I subscribe to the much thinner abridged quarterly English edition, but my comments here concern the Italian original.

Birth of the Red Prawn

The name Gambero Rosso means red prawn, so it is easy to imagine that the magazine started out with a focus on seafood and expanded from there. The true explanation is a bit more Italian and a bit more political (I wrote about it in Chapter 7 of my book, Globaloney.) A group of young revolutionaries found themselves working on a Northern Italian newspaper called il manifesto in the 1970s. They formed a social club, which later evolved into the Slow Food movement, on the theory that political revolution needed to have deep social roots and to draw upon important economic sources in order to succeed (the status quo they opposed, after all, was political, social and economic, too). It is hard to imagine a better way to bring people into a political movement than through food and wine. And so their project thrived.

Gambero Rosso first appeared as a slim culinary supplement to il manifesto, so I imagine the name made reference to both its “red” politics and its shrimpy size. The magazine grew to become an independent publication, spawned a number of books and guides, and now even includes a television network. It is no shrimp any more.

Gambero Rosso is unique among the wine critic publications I am examining here in that it is almost entirely focused on the wines of a single nation, Italy. Although other publications may emphasize one country or region over others, all attempt some form of international coverage (this is even true of Wine Press Northwest, which covers the Washington, Oregon and British Columbia wine industries). In this as in many other things Gambero Rosso is different.

Mario Batali has said that there is no such thing as Italian cuisine, there are only the regional cuisines of Italy. Gambero Rosso takes this view of Italian wines, locating each wine within its particular region and quietly seeking to reinforce regional wine identities and to resist the industrialization and homogenization of wine. It thus holds true to its Slow Food roots. Gambero Rosso has an agenda, as its il manifesto origins might suggest, and it promotes it effectively by identifying the very best regional products and local producers. Other wine critics have agendas, too, I think, but are not always so transparent as Gambero Rosso.

The Italian Way of Wine

Each monthly issue of Gambero Rosso features a major section on wine, usually devoted to a particular regional wine. Winemaker profiles and interviews are accompanied by ratings and tasting notes. The ratings use the usual 100-point scale, although it is applied a bit more rigorously than in U.S. publications, I think, judging by the relatively small number of 90+ wines that appear. A surprisingly small number of wines are rated each month given the thousands and thousands of Italian wines produced. This is because the most important ratings appear in an annual guide called ViniD’Italia (one word), which is the bible of Italian wine. You can purchase an English translation, ItalianWines. The 2004 edition that I have here ran 864 pages and rated 14,208 wines from 1,937 producers. The new 2008 edition (due out later this month) is probably even bigger.

Several features of ViniD’Italia are noteworthy. First, it eschews the 100-point rating system in favor a simple scale of zero, one, two or three wine glasses. The three glass or tre bicchieri wines are the best in Italy. There were only 254 of them in 2004, so this really is a high distinction and I am sure it is a tremendous market advantage to receive the highest rating. I admit that I think the wineglass rating system is refreshing and usefully informative. Tasting wine is not really rocket science at my level of (un)sophistication. Robert Parker probably really can tell the difference between an 88-point merlot and one rated 87, but I’m not sure I can. Good, better and best ratings meet my needs pretty well.

ViniD’Italia is noteworthy in other ways in comparison with other wine guides I have seen. It is organized by wine region (and then by wine town) rather than wine type, producer or some other system. The idea seems to be that regional identity really counts and so you should be thinking about comparing wines based on their terroir and the qualities of the people who make them rather than just grape varietal or brand name (there is no emphasis on brands here at all). Within regions, each important producer gets a single page. Terse but data-packed text evaluates the winemaker’s progress since the last publication and surveys briefly the changes the tasters perceive. Then a small number of wines are rated and compared to some of the same wines from the same producer in previous years, with relative cost noted to provide a quick index of value.

(The focus on identifying quality producers (and not just exceptional wines) reminds me of the winery rating system that Paul Gregutt developed in his recent book about Washington wines. Click here to read my book review.)

If you want to know about the wines of Venica & Venica, for example, you turn to the chapter on the Friuli Venezia Giulia province, find their town, Dolegna del Collio, and learn that their reserve Tocai Friulano, the Ronco delle Cima, received three glasses for the 2002 vintage. Venica & Venica are quality producers, according to the text, and their wines are reliably good — you really cannot go wrong. One or another of their wines usually receives the highest rating and they generally receive several two glass ratings as well. This is useful information. (By the way, because of its geographical organization, this guide is a great resource if you plan to do any wine touring in Italy.)

A Critical Agenda

The book is frankly hard to use if you are standing in a wine store, flipping back and forth trying to find out how different wines (from different provinces and cities, made in different vintages) come out in the ratings. But I doubt that the editors really care. You are supposed to learn to use it their way, not your way. They want you to stop thinking about wines except in terms of their regional and local identities and to make more disciplined and informed choices, taking the long term development of quality producers into consideration.

The Gambero Rosso publications want you to think about wine in a particular way that reflects its original political agenda and distinctive Italian roots. In many ways it is exactly the opposite of the wine guide I’ll consider next, Decanter.

Globalization, Wine Value and the Two Buck Chuck Index

January 11, 2008 By Mike Veseth in globalization, journal of wine economics, robert parker, Two Buck Chuck, wine magazines, wine ratings, wine spectator 1 Comment

Has the globalization of the wine industry given us the best of wines, as many wine drinkers believe, or the worse of wines, as the film Mondovino suggests?

Two economists from the Whitehead School of Diplomacy at Seton Hall University address this question in the December 2007 issue of the Journal of Wine Economics (see full reference below). Their conclusion? Globalization has benefited American wine drinkers, who have a broader choice of quality wines at lower prices.

That’s pretty much what my supermarket empiricism leads me to conclude, but can it be proven scientifically? Here’s how the article’s authors arrived at their results.

First you need to define what it is that American wine drinkers are buying. The authors decided to focus on the Wine Spectator annual Top 100 list of wines. This has the advantage of limiting the study to a reasonable number of widely available wines. The Top 100 list is chosen each year on the basis of price, wine rating, availability and “excitement.” Many people use rankings like the WS 100 to guide their purchases, so I suspect that there really is some correlation between what is on the list and what is on store shelves and restaurant wine menus. The disadvantage of limiting the study to the Top 100 is of course that most of the wine sold in America — the inexpensive Gallo, Yellow Tail and Two Buck Chuck wine — does not make it to this or any other “top” list. If we want to know if globalization has improved choice at the middle and bottom of the market we will need more research.

The authors examined the WS 100 lists from 1988 – 2005 to determine (1) where the wines came from, (2) how much they cost and (3) their quality as measured by the WS ratings. They then calculated measures to determine changes in the geographical concentration of the wines (more or less choice in terms of countries of origin), the average quality rating and the relative value to consumers as measured by rating points per dollar.

What we learn from this is that the overall quality of the top wines has stayed relatively constant over the years, but the real price has fallen and the range of offerings has increased. It cost $4313 (in today’s dollars) to purchase the entire WS100 in 1988, for example, but just $2622 to buy the Top 100 wines in 2005. The cost per “point” of ratings in 1988 was 46 cents, so a hypothetical average 90-point wine cost $41.40. The per point cost was 28 cents in 2005 and so a hypothetical average 90-point wine cost just $25.20.

The top wines came from just six countries in 1998 versus 11 countries in 2005, an indication of the globalization effect. A great majority of WS100 over the years have come from four core wine countries: Australia, France, Italy and the U.S., but the proportion of non-core wines has increased, too, from just 5 percent in 1988 to 24 percent in 2005.

The authors divide the wine world of this study into Old World (France, Italy), New World (Australia and the U.S.) and “New-New World” (New Zealand, South Africa, Argentina and so on). Globalization has brought American wine drinkers more and more excellent New World and now especially New-New World wines that provide the same quality at lower average prices, according to the study.

Research like this is interesting both for the questions that it answers and for the new questions that are raised. It would be interesting, for example, to find how important the four criteria for selection are — price, rating, availability and “zing” — and if the relative weight they are given has changed. As the wine market has expanded, for example, greater emphasis may have been put on price and availability, leading to a Top 100 that leans more toward (global) good value wines.

It would also be interesting to see if the editors respond in any way to external forces. A lot of people read and study the Top 100 list, so perhaps they use it as a way to build the wine market (and thereby indirectly build their potential subscriber base). A focus on value would be consistent with this goal. A Top 100 list that you can’t find or can’t afford doesn’t build the wine market and won’t sell many magazines. The fact that there are more New-New World wines might reflect rising quality and availability of these wines or it could indicated that the WS editors desire to add these wines to keep costs down, value up and the market growing. In other words, the WS100 might show more choice and continuing good value because that’s what the WS editors want it to show. I suspect that the truth is that the market has evolved toward global good value and that WS has been part of that process, encouraging people to try New-New World wines by putting them on the Top 100 list.

Exchange rates could also play a role here. The dollar has fallen against most currencies (increasing the cost of imported wine), but the depreciation is not uniform. The Euro is much more expensive but the Argentine peso has not changed as much. If would be interesting to see to what extent the WS100‘s New-New World globalization has offset exchange rate driven increases in Old World wine costs.

Another interesting question relates to the idea of value in wine purchases. It does seem to me that people often find themselves buying WS points or Parker points more than the wine itself because they are unsure of their ability to judge quality. One local wine merchant had a sale of wines rated 90 points or more for $20 or less. The idea was that the wines must be good value because of the low cents per point ratio. But there is more to wine than rating scores, as anyone who has tasted high-scoring wines will tell you.

It might be interesting to try to put together a slightly more sophisticated wine value index using WS and other ratings. I don’t think that cents per point is a good measure because it assumes a linear relationship between money and quality — and we all know that is not the case. Very expensive wines frequently receive much lower ratings than their cheaper competitors. I understand that a $100 Chardonnay came in last at the tasting where Two Buck Chuck won the Gold Medal.

Even where price and quality are correlated, the relationship isn’t necessarily linear. The average price difference between an 86 point wine and a 88 point wine may be pretty small, for example, but it might cost a great deal to go from 92 to 94 points if the demand for the very best wines is particularly strong as is often the case in winner-take-all markets.

The price-quality relationship, even using imperfect wine scores as a measure of quality, is certainly non-linear. No wonder wine buyers are so confused — and depend so much on ratings and lists like the WS100.

Here is a simple alternative to cents per point as a measure of value. Let’s adjust price and quality for a baseline wine: Two Buck Chuck. You could call it the TBC index. Suppose that you can purchase a 70-point (to just make up a number) TBC Chardonnay for $2 (or $3 here in Washington State). The question we want to answer is how much does it cost to improve on TBC? A wine that gives you a lot of additional value for only a little additional money is a good deal.

In other words, the TBC index would be a relative index of value calculated by asking would be how many points in excess of 70 (or whatever the quality of the baseline wine you choose) you can buy for the dollars you spend in excess of the baseline cost. Here’s a numerical example. A 88 point wine for $20 would have a TBC rating of (88 – 70 points)/($20 – $2) = 18 point/$18 or a dollar a point. An 86 point wine for $10 would be a better value because (86 – 70)/(10-2) = 16 points /$8 = two points per dollar. It seems to me that this is a better (but still badly flawed) indicator of relative value. (Economics students have already realized that I am applying the principle of decision-making on the margin to this problem).

Perhaps I will find some students to work on the TBC index, perhaps using a different base wine for each varietal or wine type. I predict that their research would find that the “optimal” TBC point is being pretty close to the heart of the premium wine market — right on the center shelf in the supermarket — where so many wine brands compete for your wine dollars.

Wine ratings are very important in some parts of the wine market and very controversial, too, so I think I will see what I can learn about them. With this in mind I have subscribed to six different wine-rating publications: Wine Advocate (Robert Parker), Wine Spectator, Wine & Spirits, Wine Enthusiast, the British Decanter and Wine Press Northwest (for Washington/Oregon wine news and ratings). Watch this space for a comparative analysis of these influential publications.

References: Omer Gokcekus and Andrew Fargnoli, “Is Globalization Good for Wine Drinkers in the United States?” Journal of Wine Economics 2:2 (December 2007) pp. 187-195).