Comments in this post are not directed at CBoS exclusively, rather at the overall structure of the competition process and tasting panels throughout CAMRA. Neither are they an attack on the Judges who sat on CBoS 2011. From the bus journey home after a days drinking I made some ill-judged remarks via Twitter that went over the top. Apologies for those, especially to @TheBeerCast who bravely identified himself publicly as part of the judging process which produced some contentious results. Clearly @TheBeerCast is qualified to taste and discuss beer. Indeed I look forward to his insider's view of the most controversial CBoS results in living memory and I'll be keeping my eyes peeled for his next Blog on the subject. That said the results coming from the CBoS judging process as a whole remain a concern and must be considered further if the prize is to retain its integrity.
Winning a CAMRA title like Champion Beer of Britain (CBoB) or its Scottish equivalent CBoS can make a huge difference to a brewery in terms of sales. Suddenly every pub and regional festival wants the winning beer on their bar with the coveted badge attached to the pumpclip. It is a valuable prize that can accelerate, and in some cases distort, a brewery's development. For CAMRA there is a great deal of credibility at stake: awarding prizes to ordinary beers whilst ignoring demonstratively better ones on the market undermines that organisations authority. This has a knock on effect in terms of weakening the impact of every public pronouncement it makes. Clearly then CAMRA has a duty to ensure that the nominated beer is worthy of the title Champion.
The long road to winning a gong starts at regional, branch and individual member level. CBoB for example categorises beer into one of 11 styles to which individual CAMRA members can make nominations via the pages of What's Brewing. Generally for a beer to make it to a National Final it would need to have won a prize at a local festival or be nominated from a regional tasting panel. When it comes to prize-giving at beer festivals a blind tasting system is used whereby the competition organizer alone knows the identity of the beers being sampled. Judges mark, discuss and ideally come to a consensus on the best beer being presented to them. Appearance, Aroma, Taste, Mouth-feel, After-taste should all be factors informing their deliberation. This process is repeated all the way up the chain until the top prize is eventually settled on. Then the arguments start.
Two immediate problems centred on the tasting panels jump out at me.
Firstly the competition organizers have to find approximately 20 to 30 willing participants on the first full afternoon of a festival to sit on the tasting panels. No problem, right. Just walk into the main hall, make a public announcement and a queue forms. But wait how many in that queue could explain the five factors above? And will they be at the head of that queue? Fair play to CAMRA, they've thought of this and have arranged training sessions where a basic course on tasting sets out the finer points of beer appreciation equipping the participant to put into words what we intrinsically already sense. Unfortunately most of the people who've been on these courses are CAMRA activist types and on the first full afternoon of a festival are likely to be serving beer, running product stalls, working the door, etc. Consequently a panel forms where there may be some experienced tasters sat beside someone roped in at the last minute who fancies a free drink, knows the organizer or just wants to help out. Absolutely nothing wrong with a mix of experience but too often I've seen the person with the vocabulary dominate discussions and skew the result towards their preferred taste. Verdict then becomes the choice of one or two individuals rather than a group consensus.
By having a basic entry level test along the lines of:
- Can you describe the taste of this solution (Bitter, Sweet, Sour, Salt)?
- Are you comfortable arguing your corner in a group discussion?
Secondly there is the setting of the tasting panel: the beer festival. A one-off, annual event with a wide choice of beer. The aspect that lends itself to blind tasting also results in the introduction chance events. Even the most experienced drinker would struggle to identify all 100+ beers from the programme. But within that hall or on the way to the stillage the single cask of beer which determines competition success anything can go wrong. A sample size of one is not a reasonable basis for determining such an important outcome.
The Good Beer Guide doesn't rate pubs on the basis of one visit. CBoS and CBoB need to move more towards year round, more intensive measuring. Blind tasting in pubs isn't feasible, but with properly trained tasters there surely should be no need for it. The sole reason for blind tasting is to give the impression of fairness but this is largely a myth. No-one believes the sampling is 100% genuinely blind because distinctive beers stand out. There is even an argument for saying that if a taster can't identify a beer they drink on a regular basis then they shouldn't be on the panel in the first place. Moving away from the festival competition towards sampling in pubs brings the competition closer to where most beer is drunk.
The local Whetherspoons in Paisely runs a great competition called Battle of the Brewers. Over a weekend two Brewers have a selection of their range pitched head to head in sampling panels. Between Thursday and Sunday ordinary drinkers give their verdict. A cup style competition sees the winner through to subsequent rounds. By the end of the year an overall champion brewer is crowned and they'll have earned it by consistently producing quality throughout that time across their whole range.
Here I have identified two problems and hopefully two suggestions for a way forward. Using only taste course trained testers is something I see as being a short term measure that could be phased in. Moving to a scheme sampling beer year round is more of a long term proposition with the professionalising of beer judging something that could feed into its development producing more representative results. We must protect and strengthen the integrity of the awards we hand out.
Hi, I have indeed written a post about the various issues that came out of this year’s SRAF. I’ll try and respond directly here instead to some of the points you raised. Firstly, apology accepted – I’ve been called worse things in the past! Secondly, I’ll go into some of the details about what went on at the tasting and how the events unfolded.
ReplyDeleteI’m a CAMRA member, but first and foremost a beer blogger. I was asked to judge last year’s CBoS after one of the SRAF organisers emailed us. We’re local, and have been writing about beer for over four years, during which time I’ve sampled everything from alcohol-free lagers to Sam Adams Utopias and BrewDog End of History, so I think I know what I’m talking about.
The beers are indeed brought to us blind – we have no say in how they make it through to the final, which is decided a long time previously. It is literally the best beer on the day, out of the nine style winners, drunk in ascending order of gravity. In 2010, the strong bitter we sampled was head and shoulders above the rest – and true enough once the scores had been totalled it had won.
That was Highland Orkney Blast, which for me was a worthy winner. This time around, there was no clear cut victor – indeed the panel felt (and expressed this) that the quality was not where it should be for some of the beers, and the others were very similar in terms of merit. I haven’t seen the individual scores, but I know mine were grouped quite closely.
Aside from myself, on the panel were Steve Stewart of Stewart Brewing and Helen McLoughlan from the Bow Bar, plus three experienced CAMRA tasters. There were no members of the public involved. I have no doubt that any ordinary drinkers would have come to pretty similar conclusions as we did. For me, beer C9 had the best depth of flavour and the best condition, so I scored it the highest.
That was Isle of Skye Cuillin Beast, and for me that was a worthy winner too. Cairngorm Black Gold came second – fair enough, it’s a solid porter. The third placed beer turned out to be Houston Peter’s Well, which was way down on my personal list, but I can see why the other judges liked it. With no disrespect to the brewer, is this the third best beer in Scotland? No, of course not (I’d actually never heard of it before Thursday).
The problem is – with any kind of judging – is that it’s all subjective. Before the end, someone has to state an opinion. If, as you say, CAMRA are or have ‘awarded prizes to ordinary beers whilst ignoring demonstratively better ones’ then when was this superiority demonstrated? How was it recorded? Surely from the basis of personal opinion.
(cont - a long comment, sorry!)
ReplyDeleteThat said, I agree totally with a number of your points – the system could definitely do with tightening. The almost twelve month lag between a beer making the final and that judging taking place is too long. Beers can only be entered after having been on sale for twelve months – presumably to stop a one-off seasonal from winning and then not being available.So that means a beer has to be going for two years before it has a crack at CBoS - so no Fyne Ales Jarl in the 2011 process, for example.
Festivals are never the best places to drink and sample beer, by their very nature. Brewers I’ve spoken to about this hate them for that fact. On the Wednesday of this year’s SRAF I spoke with an extremely angry brewer who had taken some of his beer off, due to the temperature of the room. Indeed, the beer I scored lowest on Thursday was one of my favourite Scottish beers – when I found out later, I was shocked – but the condition was completely off.
CBoS is a snapshot – a beery It’s a Knockout – after all the nominations and area tastings, eventually the nine champs are sampled and the best one on the day wins. I was as surprised as anyone that Cuillin Beast won – but good on them, it could be the making of an island brewer of their size (and when was the last time a mainlander won?). I’m also pleased that a barley wine won, an under-reflected style here in Scotland.
I’m not defending CAMRA here, just voicing my opinion. Clearly, I get on well with Ian, Bob and the others who organise CBoS or I wouldn’t have been invited back – but it’s something I take seriously. I do think it’s a shame that such a prize is decided on a single tasting, when there are so many variables. The Battle of the Brewers is a great idea, if it were feasible for CBoS – but it could end up being too complicated.
Hopefully discussions like these will lead to CAMRA thinking about a more modified way of doing things, but I don’t think Isle of Skye should be any less proud of their CBoS award for this year.
Cheers!
Richard
The BeerCast