We gathered at my house around 4pm. Of those in attendance, five of us have over 1000 ratings, one has about 600, another has about 300, and the other two were relatively new to rating. We knew which beers would be included in the tasting, but that was it. In deciding what beers to include, we specifically excluded barrel-aged impys, as well as beers with ABVs above 14%. We conducted the tasting using five flights of five beers each. Before each flight, Ali (wife to kmweaver) and Catherine (girlfriend of wickedpete) graciously poured the samples while the tasters were in the front room or outside, calling us in when all the bottles/cans/growlers were hidden, leaving only the roughly 2-ounce samples on the counter, arranged in cups numbered 1 through 5. After three flights (15 beers), we took a dinner break. While in the fourth flight, I discovered that I was no longer felt confident in picking up enough of the nuances that distinguish each beer, and decided that I would not be entering any of my last 10 ratings onto the site (though I did continue to rate).


I’m not sure what we expected to get out of this tasting, but I think we learned a lot. First off, we didn’t enjoy ourselves that much - it was a lot of work attempting to "judge" each beer, separating out the impressions that the preceding beer left. My respect for beer judges shot up enormously after our tasting - I have no idea how the judges were able to discern the necessary nuances among the 44 Imperial Stouts entered in this year’s GABF, when I felt myself inadequate to the task after sampling fewer than half that many. It wasn’t like trying multiple beers of different styles, when you can mix up the styles as a way to cleanse your palate.


More significantly, our average scores when rating blindly were much lower than the ratings we have for these beers on the site. I think that there are several reasons for this. First, most of these beers are seasonals, and it’s certainly possible that they vary from year to year, though in most instances probably not by enough to account for the dramatic change. Second, many of the ratings we collectively have placed on the site are old. Speaking only for myself, I think it’s only natural that my ratings and rating style have changed somewhat since those early ratings. In fact, the 12 beers with scores from the blind tasting that dropped the most from my rating on RateBeer were all rated in March 2004 or earlier. Maybe it’s no coincidence that I gave my highest score in the blind tasting (4.1) to an imperial stout that I had recently rated highly, Foothills Sexual Chocolate. The third reason goes back to the "work" feeling of trying to judge these beers - we drink beer to enjoy, and if you’re focused too much on "getting it done," the enjoyment is diminished. If we were to have performed this tasting in a leisurely manner, one rating a night over 25 nights, with a bottle (or more) of each beer for each of us, I think there’s a very good chance that the scores would be noticeably higher, even though we would have lost the ability to compare the beers directly.


Click to Zoom
Then again, a "direct comparison" is difficult to pull off with this many beers. The beer that you’ve tried first isn’t being directly compared with the one you have near the end. It might be the same evening, but there have been many intervening flavors on your tongue, palate fatigue has likely set in, and your approach has undoubtedly shifted as well. Also, I think there’s a good argument to be made that the score of one beer is affected by the score of the one that preceded it. In other words, if you just tried a lousy beer, a decent beer is going to taste even better than it normally would because you’re subconsciously comparing it to the lousy one.

Another explanation for the drop in blind ratings from the ratings we have on RateBeer is the possibility that we’re biased raters who have given high scores to these imperial stouts because we were affected by the knowledge of what we were drinking. When these beers stand alone without their labels, suddenly they’re not as appealing. I’d like to think that to the extent this played a role, it wasn’t significant, but I honestly don’t know.

All that being said, I suppose I can’t end this article without telling you the results.

Rank	Name	Avg Score
1	Rogue Imperial Stout	3.87
2	Oscar Blues Ten FIDY	3.83
3	Bells Batch 7000	3.80
4	Alesmith Speedway Stout	3.80
5	Bells Expedition Stout (Batch 7251)	3.78
6	Great Divide Yeti Imperial Stout	3.76
7	Bells Expedition Stout (Batch 7740)	3.74
8	Kunhenns Road Rash Imperial Stout	3.72
9	Leinenkugels Big Eddy Imperial Stout	3.69
10	Stone Imperial Russian Stout (2007 Vintage)	3.68
11	Otto’s Jolly Roger Imperial Stout	3.68
12	Raasted Imperial Stout	3.66
13	Stone Imperial Russian Stout (2004 Vintage)	3.66
14	Nøgne Ø Imperial Stout	3.59
15	Southern Tier OAT	3.56
16	North Coast Old Rasputin Russian Imperial Stout	3.56
17	Foothills Sexual Chocolate Imperial Stout	3.54
18	Ølfabrikken Winter Stout	3.53
19	Thirsty Dog Siberian Night Imperial Stout	3.52
20	Victory Storm King Imperial Stout	3.52
21	Mikkeller Black Hole	3.52
22	Three Floyds Dark Lord Russian Imperial Stout	3.36
23	Brooklyn Black Chocolate Stout	3.27
24	Lagunitas Imperial Stout	2.73
25	Sweetwater High Desert Imperial Stout	2.68

As you can see, all but four of the beers scored within 0.35 of each other, suggesting that our collective appreciation for most of these beers didn’t vary very much. Two people selected Bells Batch 7000 as the best beer of the night, two did the same for Leinenkugels Big Eddy, and the following beers were selected as the best by one person each: Rogue Imperial Stout, Oscar Blues Ten FIDY, Otto’s Jolly Roger Imperial Stout, and Foothills Sexual Chocolate. The biggest disappointment of the night was Sweetwater High Desert Imperial Stout, but Ken (kmweaver) later noticed residue that appeared to be fungal around the growler gasket, which likely accounted for its highly disappointing performance (Sweetwater is giving Ken a free growler to make it up to him). If you disregard that beer, the beers with lowest scores were Lagunitas Imperial Stout (5.5 votes), Three Floyds Dark Lord Imperial Stout (2 votes), Brooklyn Black Chocolate Stout (1 vote), and Mikkeller Black Hole (0.5 vote).

Participants: argo0, artusory, jasonp, jeffin7, kmweaver, mj, Peregrinari, pjweaver, and wickedpete
Photo credits: Michael James