Fuck it, let's rank the sets

Tell your tales of bygone days and rank historical things here.
Post Reply
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Fuck it, let's rank the sets

Post by vinteuil »

As Jonathan Suh recently pointed out, we haven't done this in forever, and it's actually a highly valuable exercise. (How else will posterity know what to read first?)

I'm not going to produce a numerical ranking here, and I'm not going to plug any sets I've worked on (although I think 2014 Fall and 2016 MYSTERIUM were truly great sets, especially the latter; I'm very proud of the 2017 and 2018 SCTs, especially the former; and I think 2014 CO was excellent despite my contributions; 2018 is a little too fresh to rank). But in ascending order of difficulty/chronology, I think the best all-subject sets I've heard or played since the last thread were:
  • 2013 DRAGOON
  • 2015 Regionals
  • 2016 MLK
  • 2016 Terrapin
  • 2017 Regionals
  • 2017 ICT
  • 2018 CMST
  • 2017 Nationals
  • 2018 Nationals
  • 2015 CO
I mainly picked sets that I thought nailed their difficulty target (sorry 2018 Regionals or 2016 CO! I like you a lot more than several of these sets!) in addition to having rock-solid questions and innovative or exciting content. I apologize for not having more easy sets; for all I know 2016 MUT and SMT really might belong up here (I do think that none of the ACF Falls since 2014 have measured up to these sets, for a variety of reasons).

I could also be convinced that this kind of ranking/discussion is best done in difficulty tiers. In that case, we could also open the discussion up for subject tournaments; obviously I'm a big fan of VICO, but I think that It's Lit, Jordaens, 2015 CO Visual Arts, and SHEIKH really set a high standard for "standard quizbowl fare" side events, and the Mike Bentley and Will Nediger events have been just awesome for their well-built questions on not-at-all-standard quizbowl fare.

Finally: how do the older sets match up? In practice, I've been stunned at how well both Regionals and Fall from 2008 have stood up. I think MAGNI, and 2011/12 Regionals all would play pretty well now. (At least for me, 2010 Regionals may have been a victim of its own success—too many of the tossup clues seem to have become hyper-canonical, so it's hard for me to judge. The same might be true of the 2011 VCU Open.) 2010 MO was probably the best tournament of its kind until CMST, and I could be easily be convinced that it tops the latter. And 2012 Nationals could hold its own against a tournament as good as 2015, I think. (2005 Nationals remains humanities-impossible.)

Some honorable mentions:
  • 2014 Nationals. Aside from some wonkiness in the lit and visual arts, and some problems in the music, the rest of this tournament really was stellar.
  • 2015 VCU Open. Mostly fantastic, but the science could be all over the place.
EDIT: I'm hoping to generate discussion with this thread, not to impose my opinion. I've almost definitely forgotten something or had some blind spots! So please point those out.
Jacob R., ex-Chicago
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Fuck it, let's rank the sets

Post by Cheynem »

I really enjoyed Kenji's GEODUCK set for one-subject events.

I feel like This Tournament is a Crime might deserve some respect. I liked that better than CMST I think.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
naan/steak-holding toll
Auron
Posts: 2514
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: Fuck it, let's rank the sets

Post by naan/steak-holding toll »

TTIAC is the best open tournament I've had the pleasure of playing. Bruce did a fantastic job for his first time trying at that level, and Ike, Auroni, and Eddie all wrote delightful questions. Auroni in particular deserves credit for learning from his (well-done, but far too hard) previous execution of George Oppen and leading the production of an innovative tournament that hit an al dente balance of canonical and underasked content. I can't comment on Wang's stuff bcz I can't science, but I'm sure it was great as well.

"stanford housewrite" also deserves credit as a tournament that was a bit rough around the edges, but nonetheless had a ton of great ideas. It did a good job proving that you can write an open set that's not oppressive in difficulty, and is probably my second-favorite open tournament. I think having more sets like "stanford housewrite" would be great.

I'd second the selection of DRAGOON and Terrapin as ideals for the upper end of the new paradigm of "medium difficulty" that has been promoted recently.
Last edited by naan/steak-holding toll on Wed Aug 01, 2018 4:45 pm, edited 3 times in total.
Will Alston
Dartmouth College '16
Columbia Business School '21
Ewan MacAulay
Lulu
Posts: 68
Joined: Sun Nov 04, 2012 10:15 am

Re: Fuck it, let's rank the sets

Post by Ewan MacAulay »

I'll jump on the Terrapin 2016 train - a really solid regular-difficulty set which still managed to feel fresh. Penn Bowl 2015 was another fun set, although alas the last PB set that have remembered that time difference means that UK sites need packets earlier than American ones.

On either side of regular difficulty, EFT 2016 and Stanford Housewrite stuck in my mind as being very good fun to play.*

As for side events, I say it all the time, but GEODUCK and Stock Clues were some of the best questions I've ever played and showed how rich the geo/food distributions can be. Would be very excited to see sequels for them.

*my opinions on all-subject sets should be read while remembering that I only understand like 3-4 questions per packet
Last edited by Ewan MacAulay on Wed Aug 01, 2018 4:48 pm, edited 2 times in total.
Ewan MacAulay
Oxford 2015
Cambridge 2018
User avatar
Dantooine is Big!
Rikku
Posts: 294
Joined: Mon Nov 02, 2015 9:33 am
Location: Buffalo Grove, Illinois
Contact:

Re: Fuck it, let's rank the sets

Post by Dantooine is Big! »

vinteuil wrote:I apologize for not having more easy sets; for all I know SMT really might belong up here
:mad: :mad: :mad:

On the other hand, if you're talking about "hitting their difficulty target," it might be less fair to talk about SMT or MUT, since I think they both tended harder than what their ideal audience would have liked. SMT definitely improved a lot as the mirrors progressed, but I can't really be bothered to go through a retrospective analysis of the set. I don't think anyone cares too much, anyway.
Young Fenimore Lee (they/them)
Stevenson High School 2017
Stanford 2021
The New School 2023
Ohio University 2028
User avatar
Red Panda Cub
Wakka
Posts: 211
Joined: Thu Dec 22, 2011 9:59 pm

Re: Fuck it, let's rank the sets

Post by Red Panda Cub »

Stock Clues remains the best quizbowl tournament.
Joey Goldman
Oxford '17
City, University of London '19
User avatar
ryanrosenberg
Auron
Posts: 1890
Joined: Thu May 05, 2011 5:48 pm
Location: Palo Alto, California

Re: Fuck it, let's rank the sets

Post by ryanrosenberg »

Red Panda Cub wrote:Stock Clues remains the best quizbowl tournament.
Yes, this.

CO Trash 2016 is the best full tournament I've played, and ACF Nationals 2018 the best I've read.
Ryan Rosenberg
North Carolina '16
ACF
touchpack
Rikku
Posts: 450
Joined: Tue Sep 20, 2011 12:25 am

Re: Fuck it, let's rank the sets

Post by touchpack »

I know this is tooting my own horn a bit, but for easier tournaments, I was really proud of the work my co-editors and I did on MUT 2014, and I remember that set being very well-received as well.
Billy Busse
University of Illinois, B.S. '14
Rosalind Franklin University, M.S. '21, M.D. Candidate '25
Emeritus Member, ACF
Writer/Subject Editor/Set Editor, NAQT
User avatar
merv1618
Tidus
Posts: 719
Joined: Tue Aug 02, 2011 11:43 pm

Re: Fuck it, let's rank the sets

Post by merv1618 »

touchpack wrote:I know this is tooting my own horn a bit, but for easier tournaments, I was really proud of the work my co-editors and I did on MUT 2014, and I remember that set being very well-received as well.
It was a very fun and good set.
Adam Sperber
Hickman '10
Northwestern B '14
Loyola (inactive) '21

" 'Yay, more Adam Sperber' --Nobody " --Cody Voight
Banana Stand
Wakka
Posts: 158
Joined: Thu Apr 26, 2012 4:38 pm

Re: Fuck it, let's rank the sets

Post by Banana Stand »

TTIAC and Stanford Housewrite were both better than CMST.
Jack Mehr
St. Joe's NJ '14
UVA '19
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Re: Fuck it, let's rank the sets

Post by vinteuil »

Banana Stand wrote:TTIAC and Stanford Housewrite were both better than CMST.
The science in Stanford Housewrite had a lot of problems; TTIAC and CMST were both more consistent in that regard.
Jacob R., ex-Chicago
User avatar
Doga (Dog Yoga)
Lulu
Posts: 48
Joined: Wed Jun 04, 2014 7:12 pm

Re: Fuck it, let's rank the sets

Post by Doga (Dog Yoga) »

1) 2018 CO
2) 2016 ACF Fall
Rohith Nagari
Brown '17
User avatar
Bosa of York
Rikku
Posts: 362
Joined: Sun Jun 15, 2014 9:16 am

Re: Fuck it, let's rank the sets

Post by Bosa of York »

IMO 2017 ACF Fall was quite a bit better than 2016 (or at least didn’t have any bonuses on impossibly hard Tibetan rivers). I’d tentatively say without having thought about it much that 2017 Fall was one of the best recent easy sets.
Eric Wolfsberg
Bethlehem Central High School 2016
University of Delaware 2020
Stanford 2025 or whatever
User avatar
Corry
Rikku
Posts: 331
Joined: Fri Feb 10, 2012 11:54 pm

Re: Fuck it, let's rank the sets

Post by Corry »

Relatively few college sets have stood out in my memory, but Terrapin 2016 was definitely one of them - the questions just felt "fresh" in a way that I can't quite articulate. Still have no idea how Jordan and co. pulled it off.

Regarding side events, I too have recently enjoyed GEODUCK and Stock Clues.
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor
User avatar
ThisIsMyUsername
Auron
Posts: 1005
Joined: Wed Jul 15, 2009 11:36 am
Location: New York, NY

Re: Fuck it, let's rank the sets

Post by ThisIsMyUsername »

I agree with a lot of the tournaments that have been praised so far. To this I might add a couple of things:
- I think Jordaens is one of the best academic side events ever written, from a technical standpoint. It is not merely the model of how to write a good academic side event, but also of how to include a healthy amount of canon expansion while still keeping questions very playable, using smart common-links and good sub-distribution. Even if we isolate individual best-edited categories in most all-subject tournaments of the past few years, very few of them would surpass Jordaens.
- I think the past few ICTs (2016-2018) are the best ICTs I've played. 2017 was maybe the strongest, but the general trend is what I find more noteworthy. At the time of the past list, I think the general impression was that NAQT still often operated below the collegiate circuit norms. I don't think that is the case any longer.
- I think the best Nats since 2012 is actually ACF Nats 2015. 2014, 2017, and 2018 attract a lot of attention because of the assertive philosophical underpinnings of their editors' questions. But I thought the general execution of 2015 across the tournament displayed the best control (finals packet excepted), while still having interesting answerlines.
- Perhaps we're deliberately limiting ourselves to academic or academic-proximate sets, but CO Trash 2016 needs to be up here among the top. (EDIT: Ah, I see Ryan mentioned it.)

Stepping back, though: I feel like we've undergone a big shift since the time of the previous list. All four Regionals from 2010 to 2013 were well received, and all of the most praised hard tournaments from the previous era (MO 2010, CO 2011, and ACF Nats 2012) were packet submission. MAGNI is one of the few housewritten tournaments that is mentioned alongside the other great tournaments of that era.

Now, the situation has reversed. I think the best regular-difficulty tournament (Terrapin 2016) and the best hard tournaments (stanford housewrite, TTIAC) have been housewrites. (It seems like many others agree with this general assessment.) And I think that these are on the whole better than the flagship core events (ACF Regionals, ACF Nats, and CO) have been over the past few years. This should make us wonder whether we've gotten to the point where the community at large is markedly better at writing than it is at editing, and whether we should self-examine why that is and how to alter that.

A fun exercise to do, perhaps, in addition to listing events that are great as a whole: Can one imagine great "Frankensets" that could be made out of recents sets (i.e. the Lit from Tournament X, the Science from Tournament Y, etc.)? That is, are there also standout individual categories within sets that otherwise were not as strong across the board?
John Lawrence
Yale University '12
King's College London '13
University of Chicago '20

“I am not absentminded. It is the presence of mind that makes me unaware of everything else.” - G.K. Chesterton
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Re: Fuck it, let's rank the sets

Post by vinteuil »

ThisIsMyUsername wrote:This should make us wonder whether we've gotten to the point where the community at large is markedly better at writing than it is at editing, and whether we should self-examine why that is and how to alter that.
I'd love to see you expand on this. I'd think that the difference comes from housewrites' opportunity to avoid the compounded problems of "unknown topics" and "have to write/edit quickly" that arise from the inevitable late packets.
Jacob R., ex-Chicago
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Fuck it, let's rank the sets

Post by Cheynem »

Some of this is also to blame on the submissions too, no?
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
rylltraka
Tidus
Posts: 568
Joined: Mon Oct 31, 2005 10:55 pm
Location: Los Angeles

Re: Fuck it, let's rank the sets

Post by rylltraka »

As someone who doesn't really have a lot to add to this discussion, but was interested, I found it rather difficult to find the previous thread on this topic (addressing 2010-2012 or so; I did eventually find it). Might be just me.

What might be useful to the average player or enthusiast would perhaps be an easily accessible "trophy room" section of HSQB or perhaps through the forums, where anyone could take a look at the "best sets of all time", perhaps with a blurb describing difficulty, or at least some expert feedback. While it's fun to rank them, I wouldn't advise doing it there, but to simply have a collection of 20 or so sets where the community has said "this are examples of what excellent quizbowl sets look like".

Because, in this day and age, we have an embarrassment of riches when it comes to choice. Even subject tournaments are in such abundance that it's hard to keep track; when I was a wee lad, history-only tournaments were a thing only of the imagination - now there are tons. So perhaps subsections of "best subject-only sets" or "best specialty tournaments", with explanation, would be useful for the wider community.
Mik Larsen
USC '08, UCLA '14
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Re: Fuck it, let's rank the sets

Post by vinteuil »

rylltraka wrote:As someone who doesn't really have a lot to add to this discussion, but was interested, I found it rather difficult to find the previous thread on this topic (addressing 2010-2012 or so; I did eventually find it). Might be just me.
I literally linked to it in the first clause of my post.
Jacob R., ex-Chicago
User avatar
rylltraka
Tidus
Posts: 568
Joined: Mon Oct 31, 2005 10:55 pm
Location: Los Angeles

Re: Fuck it, let's rank the sets

Post by rylltraka »

Yes, you did. I didn't realize it and tried to use the site's search function.
Mik Larsen
USC '08, UCLA '14
User avatar
UlyssesInvictus
Yuna
Posts: 845
Joined: Thu Feb 10, 2011 7:38 pm

Re: Fuck it, let's rank the sets

Post by UlyssesInvictus »

rylltraka wrote:What might be useful to the average player or enthusiast would perhaps be an easily accessible "trophy room" section of HSQB or perhaps through the forums, where anyone could take a look at the "best sets of all time", perhaps with a blurb describing difficulty, or at least some expert feedback. While it's fun to rank them, I wouldn't advise doing it there, but to simply have a collection of 20 or so sets where the community has said "this are examples of what excellent quizbowl sets look like".
QuizDB does have a "quality" attribute on the sets in its database, and I've long contemplated outsourcing this to a Google Poll where people can simply rank sets and I'll take that data and apply it to the database. (Thus making this, and prior threads, in effect a nominating thread.) It's a 5 point scale right now, but I'll probably change that to a 7 point scale and reset all the current values I have (since they're just subjectively being set by the uploaders right now i.e. mainly me).

If people have recommendations on how to conduct such a poll (the most glaring problem is that ranking 50+ sets over the past 5 years would just turn into a name recognition slogfest, as often happens in similar polls), or don't think such a venture would be helpful, I'd love feedback.
Raynor Kuang
quizdb.org
Harvard 2017, TJHSST 2013
I wrote GRAPHIC and FILM
User avatar
UlyssesInvictus
Yuna
Posts: 845
Joined: Thu Feb 10, 2011 7:38 pm

Re: Fuck it, let's rank the sets

Post by UlyssesInvictus »

Oh, and I'd probably run separate HS and College/Open polls.
Raynor Kuang
quizdb.org
Harvard 2017, TJHSST 2013
I wrote GRAPHIC and FILM
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: Fuck it, let's rank the sets

Post by theMoMA »

Of tournaments not yet mentioned, Cane Ridge Revival was an excellent set with innovative and interesting humanities and social science questions (with the caveat that I don't remember the science and wouldn't be able to comment on it even if I could). That more than any other set influenced how I look at clues, especially drawn from contemporary scholarship, in history and social science questions.
Andrew Hart
Minnesota alum
User avatar
ErikC
Rikku
Posts: 288
Joined: Sat Sep 24, 2016 12:44 pm

Re: Fuck it, let's rank the sets

Post by ErikC »

I'd like to throw my lot in with 2016 EFT for the easier side of collegiate quizbowl. I was a bit outside the target demographic when I played it but there was still a good amount of challenging clues in power for the regionals/regionals+ crowd. It also had some really fresh things I was excited to hear clued in Quizbowl.
Erik Christensen
University of Waterloo - School of Planning Class of '18
Defending VETO top scorer
User avatar
hydrocephalitic listlessness
Forums Staff: Moderator
Posts: 167
Joined: Sun Feb 27, 2011 1:26 pm

Re: Fuck it, let's rank the sets

Post by hydrocephalitic listlessness »

theMoMA wrote:Of tournaments not yet mentioned, Cane Ridge Revival was an excellent set with innovative and interesting humanities and social science questions (with the caveat that I don't remember the science and wouldn't be able to comment on it even if I could). That more than any other set influenced how I look at clues, especially drawn from contemporary scholarship, in history and social science questions.
Seconded!
Will H-M
Harvard '16
Princeton ~'26
ACF
gyre and gimble
Yuna
Posts: 765
Joined: Tue Feb 17, 2009 2:45 am

Re: Fuck it, let's rank the sets

Post by gyre and gimble »

ThisIsMyUsername wrote:Now, the situation has reversed. I think the best regular-difficulty tournament (Terrapin 2016) and the best hard tournaments (stanford housewrite, TTIAC) have been housewrites. (It seems like many others agree with this general assessment.) And I think that these are on the whole better than the flagship core events (ACF Regionals, ACF Nats, and CO) have been over the past few years. This should make us wonder whether we've gotten to the point where the community at large is markedly better at writing than it is at editing, and whether we should self-examine why that is and how to alter that.
I'm not so sure about this. Just because the best housewrites have been better than the best ACF events doesn't mean that on the whole the community is better at writing than editing. (For example, how many regular-difficulty events besides Terrapin 2016 have been better than, say 2015 or 2017 ACF Regionals? I can't really think of any clearly better ones.)

The question you raise is pretty interesting, though. Are writing and editing skills very distinct? I ask this as an editor who tends to just write all my questions from scratch rather than editing submissions. The reason I do so is often not because the submissions are poor, but because I am adhering to a carefully-curated subdistribution that makes most submitted question topics unworkable. By choosing my own topics I can also manage how much novel/"fresh" content I can put into a set. I wonder whether the perceived difference in quality between the best housewrites and the best packet-sub tournaments can be attributed to the lack of control an editor has over distributional feng shui, relative to a subject writer? Has enough of the community become competent at writing, such that it's this difference--one of distribution, novelty, freshness--rather than a difference in average question quality, that stands out to us now?
Stephen Liu
Torrey Pines '10
Harvard '14
Stanford '17
User avatar
ErikC
Rikku
Posts: 288
Joined: Sat Sep 24, 2016 12:44 pm

Re: Fuck it, let's rank the sets

Post by ErikC »

gyre and gimble wrote:The question you raise is pretty interesting, though. Are writing and editing skills very distinct? I ask this as an editor who tends to just write all my questions from scratch rather than editing submissions. The reason I do so is often not because the submissions are poor, but because I am adhering to a carefully-curated subdistribution that makes most submitted question topics unworkable. By choosing my own topics I can also manage how much novel/"fresh" content I can put into a set. I wonder whether the perceived difference in quality between the best housewrites and the best packet-sub tournaments can be attributed to the lack of control an editor has over distributional feng shui, relative to a subject writer? Has enough of the community become competent at writing, such that it's this difference--one of distribution, novelty, freshness--rather than a difference in average question quality, that stands out to us now?
I've noticed that at submission tournaments, especially the ones that use most submitted questions, that my individual performance spikes. I think that's because the approach taken by people to the same subjects or even answerlines can be very different. Sometimes the packet is written by people who like the exact same things you like about certain subjects, sometimes they don't - I'm sure a least a few people have noticed this.

At housewrites, I generally find myself having a much more uniform experience because it's only written by a few writers. Good writers will usually write questions using a variety of clues and answerlines (like tossing up a balanced mix of places, people, and things in history, for example) over the course of the tournament - which I think most non-sub tournament that I've played have done. I think this helps their reception, besides just the distributional feng shui.
Erik Christensen
University of Waterloo - School of Planning Class of '18
Defending VETO top scorer
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Re: Fuck it, let's rank the sets

Post by vinteuil »

ThisIsMyUsername wrote:
- I think the best Nats since 2012 is actually ACF Nats 2015. 2014, 2017, and 2018 attract a lot of attention because of the assertive philosophical underpinnings of their editors' questions. But I thought the general execution of 2015 across the tournament displayed the best control (finals packet excepted), while still having interesting answerlines.
I can't say that I'm initially inclined to agree with this characterization of either 2015 or 2017: I don't remember the former as having great "control" (the range of easy parts, some of the whackier tossup answerlines. I was under the impression that, by consensus, 2016 was widely agreed to be better "controlled" in many ways---not that that consensus is necessarily correct), while the latter didn't strike me at all as having a major divide between the submitted and editors' questions. I'd be happy to be convinced of the opposite about either though.

On the other hand, I completely agree with you that the trend toward really phenomenal ICTs has been a really heartening phenomenon; I look forward to this year's, and I can't say I did even in 2016.
Jacob R., ex-Chicago
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Fuck it, let's rank the sets

Post by Cheynem »

The decline in the numbers of packet submission tournaments probably plays some role in lowering people's experience in editing submissions. In many years, the ACF tournaments, Chicago Open, and maybe a random tournament here and there are the only packet submission sets, so people turning in submissions and the people editing them don't have a lot of experience at doing both.

Now, I am not of the Matt Weiner school which asserts that almost all tournaments should be packet submission, and I don't think *inherently* this is a problem, but certainly submitting packets and editing them are becoming less frequent, and I think that plays a role in what happens in packet submission tournaments (for the record, I think most of the last few COs have been fine and I don't think the problems of them can be blamed on submissions).
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
Post Reply