A Going Blind Tasting

Sampling 25 Imperial Stouts in One Evening

Features February 14, 2008
Written by argo0

, WASHINGTON DC -

This past Saturday, nine of us gathered to conduct a blind tasting of 23 Imperial Stouts involving 25 separate samples (two of the beers had two vintages among the samples). I had never participated in a blind tasting before, and I admit that I was a bit nervous about what I would learn. Would I pan my favorites, or give kudos to a beer I had previously slammed? One thing I was sure of is that there is no way I would unerringly give the same marks to beers I had previously rated (I had previously tried all but three of the beers in the tasting).
<style type=text/css></style>
<div class=post>
We gathered at my house around 4pm. Of those in attendance, five of us have over 1000 ratings, one has about 600, another has about 300, and the other two were relatively new to rating. We knew which beers would be included in the tasting, but that was it. In deciding what beers to include, we specifically excluded barrel-aged impys, as well as beers with ABVs above 14%. We conducted the tasting using five flights of five beers each. Before each flight, Ali (wife to kmweaver) and Catherine (girlfriend of wickedpete) graciously poured the samples while the tasters were in the front room or outside, calling us in when all the bottles/cans/growlers were hidden, leaving only the roughly 2-ounce samples on the counter, arranged in cups numbered 1 through 5. After three flights (15 beers), we took a dinner break. While in the fourth flight, I discovered that I was no longer felt confident in picking up enough of the nuances that distinguish each beer, and decided that I would not be entering any of my last 10 ratings onto the site (though I did continue to rate).

<center><img border=0 src=/images/features/02142008-bis-01s.jpg></center>
I’m not sure what we expected to get out of this tasting, but I think we learned a lot. First off, we didn’t enjoy ourselves that much - it was a lot of work attempting to "judge" each beer, separating out the impressions that the preceding beer left. My respect for beer judges shot up enormously after our tasting - I have no idea how the judges were able to discern the necessary nuances among the 44 Imperial Stouts entered in this year’s GABF, when I felt myself inadequate to the task after sampling fewer than half that many. It wasn’t like trying multiple beers of different styles, when you can mix up the styles as a way to cleanse your palate.

<center><img border=0 src=/images/features/02142008-bis-02s.jpg></center>
More significantly, our average scores when rating blindly were much lower than the ratings we have for these beers on the site. I think that there are several reasons for this. First, most of these beers are seasonals, and it’s certainly possible that they vary from year to year, though in most instances probably not by enough to account for the dramatic change. Second, many of the ratings we collectively have placed on the site are old. Speaking only for myself, I think it’s only natural that my ratings and rating style have changed somewhat since those early ratings. In fact, the 12 beers with scores from the blind tasting that dropped the most from my rating on RateBeer were all rated in March 2004 or earlier. Maybe it’s no coincidence that I gave my highest score in the blind tasting (4.1) to an imperial stout that I had recently rated highly, Foothills Sexual Chocolate. The third reason goes back to the "work" feeling of trying to judge these beers - we drink beer to enjoy, and if you’re focused too much on "getting it done," the enjoyment is diminished. If we were to have performed this tasting in a leisurely manner, one rating a night over 25 nights, with a bottle (or more) of each beer for each of us, I think there’s a very good chance that the scores would be noticeably higher, even though we would have lost the ability to compare the beers directly.

<center><a href=/images/features/02142008-bis-03l.jpg><img border=0 src=/images/features/02142008-bis-03s.jpg>
<font size=2><a href=/images/features/02142008-bis-03l.jpg>Click to Zoom</font></center>
Then again, a "direct comparison" is difficult to pull off with this many beers. The beer that you’ve tried first isn’t being directly compared with the one you have near the end. It might be the same evening, but there have been many intervening flavors on your tongue, palate fatigue has likely set in, and your approach has undoubtedly shifted as well. Also, I think there’s a good argument to be made that the score of one beer is affected by the score of the one that preceded it. In other words, if you just tried a lousy beer, a decent beer is going to taste even better than it normally would because you’re subconsciously comparing it to the lousy one.

Another explanation for the drop in blind ratings from the ratings we have on RateBeer is the possibility that we’re biased raters who have given high scores to these imperial stouts because we were affected by the knowledge of what we were drinking. When these beers stand alone without their labels, suddenly they’re not as appealing. I’d like to think that to the extent this played a role, it wasn’t significant, but I honestly don’t know.

All that being said, I suppose I can’t end this article without telling you the results.

<table cellpadding=2 cellspacing=0 border=0 bgcolor=f7f7f7 align=center>
<tr><td class=beerfoot>Rank</td><td class=beerfoot>Name</td><td align=center class=beerfoot>Avg Score</td></tr>
<tr><td class=ratingsListRank>1</td><td class=beerfoot>Rogue Imperial Stout</td><td class=beerfoot align=center> 3.87</td></tr>
<tr><td class=ratingsListRank>2</td><td class=beerfoot>Oscar Blues Ten FIDY</td><td class=beerfoot align=center> 3.83</td></tr>
<tr><td class=ratingsListRank>3</td><td class=beerfoot>Bells Batch 7000</td><td class=beerfoot align=center> 3.80</td></tr>
<tr><td class=ratingsListRank>4</td><td class=beerfoot>Alesmith Speedway Stout</td><td class=beerfoot align=center> 3.80</td></tr>
<tr><td class=ratingsListRank>5</td><td class=beerfoot>Bells Expedition Stout (Batch 7251)</td><td class=beerfoot align=center> 3.78</td></tr>
<tr><td class=ratingsListRank>6</td><td class=beerfoot>Great Divide Yeti Imperial Stout</td><td class=beerfoot align=center> 3.76</td></tr>
<tr><td class=ratingsListRank>7</td><td class=beerfoot>Bells Expedition Stout (Batch 7740)</td><td class=beerfoot align=center> 3.74</td></tr>
<tr><td class=ratingsListRank>8</td><td class=beerfoot>Kunhenns Road Rash Imperial Stout</td><td class=beerfoot align=center> 3.72</td></tr>
<tr><td class=ratingsListRank>9</td><td class=beerfoot>Leinenkugels Big Eddy Imperial Stout</td><td class=beerfoot align=center> 3.69</td></tr>
<tr><td class=ratingsListRank>10</td><td class=beerfoot>Stone Imperial Russian Stout (2007 Vintage)</td><td class=beerfoot align=center> 3.68</td></tr>
<tr><td class=ratingsListRank>11</td><td class=beerfoot>Otto’s Jolly Roger Imperial Stout</td><td class=beerfoot align=center> 3.68</td></tr>
<tr><td class=ratingsListRank>12</td><td class=beerfoot>Raasted Imperial Stout</td><td class=beerfoot align=center> 3.66</td></tr>
<tr><td class=ratingsListRank>13</td><td class=beerfoot>Stone Imperial Russian Stout (2004 Vintage)</td><td class=beerfoot align=center> 3.66</td></tr>
<tr><td class=ratingsListRank>14</td><td class=beerfoot>Nøgne Ø Imperial Stout</td><td class=beerfoot align=center> 3.59</td></tr>
<tr><td class=ratingsListRank>15</td><td class=beerfoot>Southern Tier OAT</td><td class=beerfoot align=center> 3.56</td></tr>
<tr><td class=ratingsListRank>16</td><td class=beerfoot>North Coast Old Rasputin Russian Imperial Stout</td><td class=beerfoot align=center> 3.56</td></tr>
<tr><td class=ratingsListRank>17</td><td class=beerfoot>Foothills Sexual Chocolate Imperial Stout</td><td class=beerfoot align=center> 3.54</td></tr>
<tr><td class=ratingsListRank>18</td><td class=beerfoot>Ølfabrikken Winter Stout</td><td class=beerfoot align=center> 3.53</td></tr>
<tr><td class=ratingsListRank>19</td><td class=beerfoot>Thirsty Dog Siberian Night Imperial Stout</td><td class=beerfoot align=center> 3.52</td></tr>
<tr><td class=ratingsListRank>20</td><td class=beerfoot>Victory Storm King Imperial Stout</td><td class=beerfoot align=center> 3.52</td></tr>
<tr><td class=ratingsListRank>21</td><td class=beerfoot>Mikkeller Black Hole</td><td class=beerfoot align=center> 3.52</td></tr>
<tr><td class=ratingsListRank>22</td><td class=beerfoot>Three Floyds Dark Lord Russian Imperial Stout</td><td class=beerfoot align=center> 3.36</td></tr>
<tr><td class=ratingsListRank>23</td><td class=beerfoot>Brooklyn Black Chocolate Stout</td><td class=beerfoot align=center> 3.27</td></tr>
<tr><td class=ratingsListRank>24</td><td class=beerfoot>Lagunitas Imperial Stout</td><td class=beerfoot align=center>2.73</td></tr>
<tr><td class=ratingsListRank>25</td><td class=beerfoot>Sweetwater High Desert Imperial Stout</td><td class=beerfoot align=center>2.68</td></tr>
</table>

As you can see, all but four of the beers scored within 0.35 of each other, suggesting that our collective appreciation for most of these beers didn’t vary very much. Two people selected Bells Batch 7000 as the best beer of the night, two did the same for Leinenkugels Big Eddy, and the following beers were selected as the best by one person each: Rogue Imperial Stout, Oscar Blues Ten FIDY, Otto’s Jolly Roger Imperial Stout, and Foothills Sexual Chocolate. The biggest disappointment of the night was Sweetwater High Desert Imperial Stout, but Ken (kmweaver) later noticed residue that appeared to be fungal around the growler gasket, which likely accounted for its highly disappointing performance (Sweetwater is giving Ken a free growler to make it up to him). If you disregard that beer, the beers with lowest scores were Lagunitas Imperial Stout (5.5 votes), Three Floyds Dark Lord Imperial Stout (2 votes), Brooklyn Black Chocolate Stout (1 vote), and Mikkeller Black Hole (0.5 vote).

Participants: argo0, artusory, jasonp, jeffin7, kmweaver, mj, Peregrinari, pjweaver, and wickedpete
Photo credits: Michael James
</div>

................................................................

Comments

No comments added yet

You must be logged in to post comments

................................................................

Anyone can submit an article to RateBeer. Send your edited, HTML formatted article to our Editor-In-Chief.

We didn’t enjoy ourselves that much - it was a lot of work attempting to judge each beer, separating out the impressions that the preceding beer left

Copyright © 2000-2024,
RateBeer LLC. All rights
reserved.

about us
About RateBeer
FAQ
Contact/Feedback
New Beers
add
Advanced Search
Add A Beer
Add A Brewer
Add A Place
Events
membership
Log In
Edit Personal Info
Buy Premium Membership
Your Messages
the best
RateBeer Best
100 Beer Club
The Top 50