Wine rating systems are like the weather — everyone complains about them but no one does anything. Paul Gregutt thinks he knows why.

Wine by the Numbers

There are many ways to rate or rank different wines and consumers are very interested in trying to understand what they mean and how to use them. That’s why my column on wine rating systems, Wine by the Numbers, is one of the most popular posts in this blog’s brief history.

Twenty-point rating systems are popular in Europe in part, I understand, because that is how papers are graded in French high schools. Here in the United States the 100 point system that Robert Parker popularized and many others use dominates in part, I suppose, because that’s how our papers were graded in school. Any simple wine scoring system is problematic, however, since a good deal of information is necessarily lost when the attributes of a multidimensional product like wine are reduced to a single number. Wine isn’t like gasoline, where the critical components can be captured, like octane, in a single number.

But there are other problems, too. Paul Gregutt (PG), wine critic for the Seattle Times and Wine Enthusiast magazine, explained the limitations of the 100 point scale and the tendency toward “grade inflation” in his recent book, Washington Wines & Wineries:

This practice of promoting wine, a multi-faceted, subjective sensory experience, simply by broadcasting numbers has gradually devalued the numbers while shrinking the original 100-point scale. At first blush, such a rating system sounds generous, allowing room for a lot of subtlety in the grading curve. But in actual practice it’s not a hundred point scale at all, nor even close.

It has become a ten point scale. Wines rated under 85 are ignored completely. Wines rated 85 to 89 must be marketed as value wines – those numbers only work for wines priced at the low end of the scale. If your wine is going to sell for $15 or more, it must hit 90 points at least. One prominent retailer even makes a point of selling (at discount) wines that have scored the “dreaded” 89. Once a wine moves up the ladder from there, it becomes increasingly rare and expensive. As a result, wines scoring 95 or above are virtually unobtainable for the average consumer.

Although the ratings are “devalued” in terms of their utility, they are also “inflated” in terms of their commercial importance. It would seem that consumers might be better served if someone would re-center the scale so that it uses more of the 100 point range and is therefore potentially (and only potentially) a more accurate guide to quality. This would obviously lower the average score, however. Who is going to be the first to break the pattern and give merely good wines average scores, 70 or 75 instead of 85 so that 85 (B+) means something?

Revising the 100-Point Scale

PG decided to try in his book on Washignton wine. Instead of rating individual wines, he rated the wineries themselves in terms of a modified 100-point scale that gives marks for style (30 points), consistency (30 points), value (30 points) and the winery’s contribution to the development and improvement of the Washington wine industry (10 points).

How did the wineries rate? Quilceda Creek, with its perfect 100 Robert Parker point Cabernet Sauvignon, also got a perfect PG score (30/30/30/10). Leonetti ranked second with 98 points (30/28/30/10). So far so good. The trouble comes further down the list where some prominent wineries get scores in the 70s, 60s, and 50s. The scores make sense when you break them down into the four factors that PG evaluates, but the raw numbers are sort of shocking when you see them for the first time or out of context. (My university students know this feeling, I suspect. It happens when they get their first college papers back after several years of high school grade inflation.)

PG writes about the reaction to his winery rating on his website in a column titled “Time to Dump my 100 Point System?”

Comments from readers and reviewers have been largely positive. Some have embraced my scoring; others have simply accepted it and moved on to the book’s other assets. But within the ranks of the industry itself – wineries and distributors in particular – there has been an awkward silence.

There is little doubt that this book’s sales have been seriously impacted by 1) the decision not to include 3/4 of this state’s wineries and 2) the scoring system itself. One winery veteran, after some prompting, took the trouble to explain why his winery wouldn’t sell or promote the book, even though I had given them one my highest scores.

“Our problem with promoting your book,” the winemaker said, “is that, in spite of the wonderful written praise, we’d spend all out time explaining our B+ grade.” PG is trying to decide if he should scrap his ratings for the next edition of his book (if you have an opinion you can contact Paul through his website PaulGregutt.com).

The First Mover Disadvantage

It is hard to know what to say about this. On one hand I admit that my first reaction to PG’s winery rating scale was neutral to negative, but then with some encouragement from Karen Wade I looked at it more closely and decided that it was actually pretty useful to me as a consumer — so I guess I am the source of some of the positive feedback PG received. On the other hand, I understand that no winemaker is going to take out an ad that boasts “Rated 72 by Critics!” — even if that’s a very good and appropriate rating by the scale being used.

I think we have to admit that re-centering the 100 point scale is a hopeless task and move on. The first mover in point reform will suffer the sort of criticism that PG reports. The only way to do it would be for everyone to switch scales at once. I don’t see that happening.

So what should we do about the ratings? For my part I’m going to try to get my students, who are very much into wine ratings by the time they come to me, to use the UC Davis 20-point scale. I think it might work because (1) they aren’t used to thinking in terms of 20 points and so they will have more open minds about what scores mean and (2) I like the way it breaks down elements of sensory perception: 4 points for appearance, 6 points for smell, 8 points for taste and 2 for overall harmony, according to my copy of The Taste of Wine by Emile Peynaud. Using the Davis scale will encourage them to make up their own minds about what they see, smell and taste. That’s a good thing.

In the meantime I think Paul Gregutt’s experience suggests both why the 100 point system should die and why it probably never will. My advice to Paul: keep the analysis in your book, which is terrific, but kiss the 100-point ratings goodbye.