Aabc Feedback

[Andyd]

Folks,

As I suggested in another thread, I'm starting this one to collect some feedback on how things went at the AABC this year.

I don't want this to turn into a slanging match, so only constructive feedback is welcome here... the point is to capture what went well and what could be improved, to help the organisers of future competitions know how to do things to make it all run smoothly.

Discussion on points people raise is welcome, as long as (again) it doesn't turn into a slanging match.

I'll kick it off with this. I thought that the stewards didn't have enough opportunity to develop their judging skills at this years event. While I appreciate that constant conversation with the judges can protract the time taken to complete a flight, I think that we can make room for stewards to learn while they're on the job. I also believe that a certain level of involvement also helps the stewards gain an appreciation for how the judges are going, and can actually help to keep the pace of judging up.

On a different note, having the judging occur at the accomodation venue worked really well - judges and stewards were on time and ready to commence when needed, and other people at the event could easily drop back to their rooms to rest.

Other thoughts? Competition, transport, accomodation etc?

Regards,

Andy

 

[haysie]

I found at Vicbrew 09, not much thought had gone into clean cold/room temp glasses, come the 4th 5th flight on Saturday, the glasses were dishwasher hot. Do you stop all proceedings and rectify? my answer YES, but that wasnt the case.
Was it conducive to have a dozen people tasting the slops and getting sloppier and noisy err? Immediatly behind a judging table. It wasnt.
Matching of judges/experience could be improved, how? I dont know but there was clearly a judge or 2 whom wasnt happy on that style, and the spread showed many times.
Cheers

 

[jayse]

I thought the stewarding was very well done and a nice tight strict ship operation, to a degree the stewarding was very transparent in that it all got done seemlessly and quietly.

However personally I would like to beable to communicate with stewards a little and get them more involved.

One thing I would mention is in the case of not being affraid to score very high scores for beers you think are excellent examples every judge has to keep that in mind or it does not work, you could end up getting highest scoring beers of show being all from flights done by the judges not affraid to score higher then other judges proberly would. A beer of show round would make it possibly better rather then having one or two judges being able to overly influence which beer gets beer of show simply on points.

As is I think it was run perfectly and they are the only two minor things I can think of.

 

[Barry]

Given that the stewards are mostly/all judges at a state level they did seem to be a bit too isolated from the judging process. I appreciate why this was so and the judging process went exceptionally well because the stewards were so efficient. However the stewards would/could gain more from the experience if they spent more time observing the judging at the tables.
The calibration beer is an excellent idea for setting standards.
Maintaining sealed bottles of the entries was excellent. In one class I judged it helped to establish 1st, 2nd and 3rd by retrying the beers again side by side. In an other class it reinforce the position of 3rd. This is especially useful when you judging 15+ beers over more than 3 hours.
Using running sheets to record the scores for each beer is also an essential which should be done in all comps (it is probably done in most comps already).
I hope this helps and generates some views and comments.

 

[Maxt]

I can understand the idea of stewards getting experience (I think having a 4th non scoring judge (judge in training) is a great exercise as well, and would be great at a national level as the standard of discussion (should be) higher.
As for the Nats themselves, I could also see the argument that as the top event of the amateur ranks, then the stewards should be there just to do the job as efficiently and inconspicuously as possible. This is the homebrewers 'olympics', and perhaps the state and club comps are a better place for stewards to get experience.

Beer of show is a contentious one. Personally I would like to see the 2 500ml bottle idea implemented. One for regular flight, one for BOS round if the winner of the flight. Would both bottles need to be 500ml? Could the BOS one be 250/325/375ml?

As for judging inconsistencies, in both my flights I am pretty sure there would have only been about 4 beers that needed adjustment (ie judges out by more than 5 pts), in the 30+ beers we did. And when they were out it was usually by 6 or 7.
Callibration beers helped this, but perhaps when changing styles within a big class (ie AMBER & DARK LAGER where you go from a California common to a schwarzbier) perhaps the idea of another callibration may be looked at.

Andy I like the idea, doing this while it's all fresh will help make the event all the better next year. Good luck


Last idea. I didn't realise club night was fancy dress, so perhaps advertising that would help with the spirit of things....can't let the boys and girls from WA hog all the limelight!

 

[ham2k]

maybe this file could help with some insights.

I copied all the AABC data out of the pdf with some formulae and the state qualification scores i entered by hand so there is scope for error there, apologies in advance. For states such as SA & QLD which did not have scores out of 150, i have adjusted them.

The Brown Ale category is an interesting one. Not one Brown Ale scored the same or more at AABC than at state. Every single one got judged less at AABC... something to work on probably at state and national level.

Of course there are many pinches of salt about the data in this file - the time between the state comps and AABC, the beer entered might not even be the same beer and the inherent differences between judges. i just hope this file can help the process.

View attachment AABC_vs_State.xls

 

[haysie]

nice file ham, the brown ale was terrible in spreads between the states then the nats, before some blockhead says the nat judges are better than state, hmmmmm read the spreadsheet. The stewarding could be improved by asking NOT allocating styles, I would not be the brightiest re. belgian wheat beers german wits etc bocks or their brother, give me a style I understand no worries.
Forgetting about stewards, do we have enough resources re judging? Some judges in a nutshell are useless, whats the criteria besides BJCP, none.
Useless= me judging IPA`S vers PA`S, dont take it personal, commenting.

 

[Maxt]

Certainly puts paid to the idea that the ACT beers had an advantage.

 

[lczaban]

Thanks for the spreadsheet Ham2k - this is a great analytical tool for assessing the outcomes of all the different beers and how they scored!

It's interesting looking at the breakdown of how different beers compared score-wise from their State scores to their Nat scores. I think that the differences are probably not too far out of what you would expect. The scores allocated at Nats should be tougher given the improved level of expected competition and a more informed standard of judging. I'd say that you want to ensure there is a level of consistency between State and National results (which the BJCP provides), although trying to provide this at a State level is hard due to having a smaller number of qualified people to call on to judge and steward. This is also true wrt the wider HB community and making sure that brewers are making beers that are of a high quality and conform to the BJCP guidelines. Maybe there is a need to expand the number of BJCP courses being run to expand the judging pool and to ensure the current judges maintain their experience and skills at a consistent level.

Whatever happens, the systems that are used need to ensure the best beers win. Barry's suggestion of having running scoring sheets is a great idea, and I think that judges should have the opportunity to resample beers at the conclusion of the first tasting run to re-check their scores and make sure that a consistent approach is used for scoring a class of beers.

By and large I don't think there is too much wrong with how things are being conducted. Any changes that might be contemplated should be minor IMHO.

I think the obvious question is about why the majority of Stout/Porter scores were scored higher at nationals than compared with their state comp scores? Improving education amongst brewers and making sure judges understand the guidelines and interpret them correctly is part of this process in rectifying this little statistical anomaly.

 

[matho]

does anybody think that the main advantage the act had was how close the dates of their comp was and the nationals

just a thought.

cheer's steve

 

[Andyd]

maybe this file could help with some insights.

I copied all the AABC data out of the pdf with some formulae and the state qualification scores i entered by hand so there is scope for error there, apologies in advance. For states such as SA & QLD which did not have scores out of 150, i have adjusted them.

The Brown Ale category is an interesting one. Not one Brown Ale scored the same or more at AABC than at state. Every single one got judged less at AABC... something to work on probably at state and national level.

Of course there are many pinches of salt about the data in this file - the time between the state comps and AABC, the beer entered might not even be the same beer and the inherent differences between judges. i just hope this file can help the process.

View attachment 32745

Great work - I've been hoping that someone could pull something like this together! I'll take a good look at this over the next couple of weeks...

Cheers!

Andy

 

[Bribie G]

I found at Vicbrew 09, not much thought had gone into clean cold/room temp glasses, come the 4th 5th flight on Saturday, the glasses were dishwasher hot. Do you stop all proceedings and rectify? my answer YES, but that wasnt the case.
Was it conducive to have a dozen people tasting the slops and getting sloppier and noisy err? Immediatly behind a judging table. It wasnt.
Matching of judges/experience could be improved, how? I dont know but there was clearly a judge or 2 whom wasnt happy on that style, and the spread showed many times.
Cheers

I don't know about the AABC but at the QLD state champs, the glasses on return from the judging tables were rinsed, placed in the glass washer in trays (it was held in a bar) and as each fresh tray of glasses were brought out of the dishwasher they were filled with ice water from the Postmix guns and allowed to sit until they were due to be filled from the comp bottles and taken to the judging tables. Worked like clockwork and Chappo was on his feet for hours working the gun.

 

[browndog]

does anybody think that the main advantage the act had was how close the dates of their comp was and the nationals

just a thought.

cheer's steve

I would hesitate to say that the ACT had a main advantage, but I will be lobbying the QLD reps to have the QABC either closer to the date of the AABC or further away so that you can submit the scoring beer still fresh to the AABC or have ample time to rebrew it.

cheers

Browndog

 

[///]

Certainly puts paid to the idea that the ACT beers had an advantage.

Orr, and I will certainly trivulise, they are dodgy b*stards in Kanbeera .... there ... I said it ...

Dodgey ... <like foxy lady ....>

Scotty

ps. I speel my Dodgey with an e, but does anyone notice speel check takes it without an e .... ???

pps. you guys did good with the Nats, judged hard ...

 

[starkesbier]

Hi all,

Just had a look at the spreadsheet. I did some analysis that I think is interesting. While the mean differences certainly appear similar when you look at the spread of the data it shows that ACT certainly had a much tighter range of differences (see attached file). Now this doesn't say anything about judging or home ground advantage because of course it could just be that the ACT brewers are brewing beers that are more 'normal' to a wider range of judges. Ot it could be that this chart is totally meaningless .

Anyway I thought i would put it out there.

Starkesbier

View attachment aabc_spread.bmp

 

[Bribie G]

I would hesitate to say that the ACT had a main advantage, but I will be lobbying the QLD reps to have the QABC either closer to the date of the AABC or further away so that you can submit the scoring beer still fresh to the AABC or have ample time to rebrew it.

cheers

Browndog

+1 Browndog, especially in the case of beers such as UK Bitters and Milds that are often at their best drunk young. The three week difference isn't really enough time to rebrew, but is too long for some styles to sit there - Hefes also spring to mind as well. Possibly a fortnight difference would sit well with all styles. Anyway I thought the AABC was normally held in November or have I got that wrong?

 

[dgilks]

I don't know about the AABC but at the QLD state champs, the glasses on return from the judging tables were rinsed, placed in the glass washer in trays (it was held in a bar) and as each fresh tray of glasses were brought out of the dishwasher they were filled with ice water from the Postmix guns and allowed to sit until they were due to be filled from the comp bottles and taken to the judging tables. Worked like clockwork and Chappo was on his feet for hours working the gun.

AABC and the ACT Territory comp were held at a hotel. After the glasses came out of the dishwasher, they were put into the glass chillers. Those that were still too hot were soaked in ice water to chill further.

 

[randyrob]

Keep up the good work Andy - but clearly the best beer won!

 

[Guest Lurker]

Keep up the good work Andy - but clearly the best beer won!

 

[Andyd]

Keep up the good work Andy - but clearly the best beer won!

Indeed!