Difficulty

Old college threads.
Locked
User avatar
Mike Bentley
Sin
Posts: 6465
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Difficulty

Post by Mike Bentley »

I set out to make this tournament less difficult than the standard collegiate national tournament. A quick glance at the stats indicate that this was certainly true. It’s hard to make a direct comparison to a tournament like TRASHionals because I don’t think any teams (save those attending UTC’s mirror) attended both tournaments in full. However (without having actually seen the TRASHionals set), I suspect that if the same team played both tournaments the team playing CULT would probably have put up at least 1-2 additional PPB.

I’m curious to see how teams felt about the difficulty, especially on the bonuses. The 3rd through 8th place teams at this tournament all ended up having bonus conversions that were pretty similar to each other. I’m not sure if this implies that the teams were all pretty close in skill or if the bonuses didn’t do the best job of distinguishing between knowledge—I suspect it’s a little bit of both. I think overall the tournament could have done a better job with the difficulty of the medium part. Middle parts I thought would be reasonably challenging to teams often ended up being converted pretty easily. This probably served to compress the bonus scores a bit more than I would have liked. If I were to run this tournament again, I might also think about upping the difficulty on easy parts. I’m not sure how much utility there is at having these really easy parts at a national event—it seems teams naturally self-selected in such a way that they’d be able to answer these parts without any trouble.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
adosreme
Rikku
Posts: 330
Joined: Sat Jul 18, 2009 11:14 pm

Re: Difficulty

Post by adosreme »

I personally really liked the difficulty of this tournament. I found nearly every tossup to be quite accessible, even if only by the end of the question, while the bonuses were fairly balanced difficulty-wise across the board.
Aaron Dos Remedios
Formerly of the Waterloo School of Abstruse Canadian Inside Jokes (University of Waterloo)
User avatar
stevebahnaman
Wakka
Posts: 110
Joined: Thu Sep 17, 2009 1:25 pm

Re: Difficulty

Post by stevebahnaman »

Some of the bonuses felt easy on the 'hard part' for a "nationals-level" tournament, such as Blink-182's "I Miss You."
Steve Bahnaman, Campbell University
Commissioner, Online Quiz League USA (quizcentral.net)
NC Wesleyan College, Librarian and Quiz Bowl Advisor/Coach 2009-2011
Emory Academic Team, 1999-2004
Pretty trashy
User avatar
Mike Bentley
Sin
Posts: 6465
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Difficulty

Post by Mike Bentley »

stevebahnaman wrote:Some of the bonuses felt easy on the 'hard part' for a "nationals-level" tournament, such as Blink-182's "I Miss You."
The third part of this bonus was originally on the much more difficult song, "Aliens Exist". I toned this down after no one got this in the online or in-person playtesting sessions. In general, I think this approach helped me avoid many (but not all) of the "impossible 3rd parts" in this set, but had the consequence of making the bonuses a bit easier than I wanted. Once the final site finishes running the set, I may post the 10 or so packets I have from about 2 months ago before I did the difficulty corrections (and some additional editing). It may be useful to compare some of the changed parts to examine my bonus leveling philosophy in-depth a bit more.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Cheynem
Sin
Posts: 7222
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Difficulty

Post by Cheynem »

I thought difficulty was pretty impressively done. Tossup answer lines were almost to a fault pretty okay, with even some seemingly eccentric things (The Skulls! John Q! Johnny Pesky!) all being something that could be converted at some point. Bonuses' biggest issue to me were the occasional "hard part is MUCH harder" than the rest, which at least for me, seemed to be mostly prevalent in the sports (particularly football) and comic books.

EDIT: I wasn't overly thrilled with the occasional spurts of academia that made its way into the tournament. I'm not talking about art films, which are a legitimate trash topic, but there were a few bonus parts that it seemed like you only needed academic knowledge of art or literature to pick up 10-20 points. I'm not sure if this was intended to make certain parts easier (like, for example, Elmer Gantry in the Burt Lancaster bonus), but I wondered if it gave a further boost to teams playing this with academic backgrounds.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Mike Bentley
Sin
Posts: 6465
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Difficulty

Post by Mike Bentley »

Cheynem wrote:EDIT: I wasn't overly thrilled with the occasional spurts of academia that made its way into the tournament. I'm not talking about art films, which are a legitimate trash topic, but there were a few bonus parts that it seemed like you only needed academic knowledge of art or literature to pick up 10-20 points. I'm not sure if this was intended to make certain parts easier (like, for example, Elmer Gantry in the Burt Lancaster bonus), but I wondered if it gave a further boost to teams playing this with academic backgrounds.
This is probably something I should have mentioned before the tournament, but I felt it wouldn't be a bad idea to include a few "reverse academic" clues in the set. My main reason for doing this is to blur the lines a little bit between academic and pop culture. It strikes me as unusual for someone to be completely interested in strictly pop culture or strictly academic stuff. Thus, I decided to have something of an academic distribution in much the same way that many academic tournaments have a trash distribution.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
Cheynem
Sin
Posts: 7222
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Difficulty

Post by Cheynem »

This is okay, I guess, although using the ACF Nats = no trash rule this year, I felt it a little out of place at an ostensible national title contest. For stuff like CO Trash, it's all part of the game. That said, it certainly wasn't that big of a deal.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
stevebahnaman
Wakka
Posts: 110
Joined: Thu Sep 17, 2009 1:25 pm

Re: Difficulty

Post by stevebahnaman »

The only bonus I felt hazy about in the academic-vs-trash regard was that one that had the Camus answer. I think it's because I distinguish between clues and answers being academic...but it wasn't like a huge problem or anything.
Steve Bahnaman, Campbell University
Commissioner, Online Quiz League USA (quizcentral.net)
NC Wesleyan College, Librarian and Quiz Bowl Advisor/Coach 2009-2011
Emory Academic Team, 1999-2004
Pretty trashy
User avatar
Demonic Leftovers
Wakka
Posts: 248
Joined: Tue Aug 07, 2007 2:26 pm
Location: Wilmington, DE

Re: Difficulty

Post by Demonic Leftovers »

Overall I though the difficulty for this tournament was spot on. There were almost no dead tossups in our rounds, including numerous rounds where all 20 tossups were converted. Although part of this might have been the fact that we were the highest scoring team, there were far fewer tossups going dead than at other trash tournaments (including TRASH regionals and TRASHionals). At the same time I thought this was done without only asking about really easy answers. So it seems that this tournament really hit a sweet spot of asking about challenging things that were still answerable. The one thing I thought that was off regarding difficulty were the power marks. Numerous times I saw really impressive buzzes that weren't in power. However this at least seemed to be pretty consistent (with a few notable exceptions like the John Fogerty tu).
TWO TIME NATIONAL CHAMPION David A. Seal, Esq.
QUINTUPLE CROWN WINNER: ICT/ICT/NHB/NHB/CULT
Coach of 2014 and 2015 ICT Champion UVA
University of Chicago '10
University of Virginia School of Law '13
Not Every Team Can Afford Me
Coattail Rider
User avatar
Cheynem
Sin
Posts: 7222
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Difficulty

Post by Cheynem »

Yeah, powers seemed a touch stingy at times--one noted example is perhaps the tossup on Rush Hour 3, which we as a team had literally seen the night before, but failed to power (in fairness, we did miss about a third of it).

I'll have to say a lot of the answer lines really were A1 though: this tournament had a nice knack for taking answer lines that defy obvious expectations (none on big blockbusters of the last few years, for example) but still being stuff that would be converted by a lot of people.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
Locked