So I have some reactions to the tournament and to the discussion surrounding it that I wanted to share with the forums.
First of all, I want to say that I was genuinely surprised by the results of ACF Nationals. Not, mind you, the result of Yale's victory, which did not surprise me at all, but rather how hard the teams found this tournament. It's pretty clear that in writing the editor questions I overestimated the knowledge base of even the top teams at this tournament. That's not me saying you're all dummies and I'm a genius; it's just that I genuinely thought the top bracket teams would be able to convert the hard questions, and that turned out not to be the case.
I am curious as to why that happened. I'm a year out of playing collegiate quizbowl but I have been to multiple tournaments this year so it's not like I'm "out of touch" with what the kids are doing. So I find it weird that I overestimated the difficulty to such a great extent, and what's even stranger to me is that when I look at the submitted material, I find a lot of things that I would consider to be very difficult, being written by the top-bracket teams. And so I want to move from a discussion of individual questions to a larger context.
I'd like to first offer up a possible explanation of why people found many of these questions so difficult, and that explanation is pretty simple: they were on things you'd never seen before. When you're pretty sure that an answer is something you recognize, you're already half-preparing yourself for that key clue that you're hoping is going to give you the information you need to buzz. But there's no way to "prepare" for a tossup on "preface to Cromwell," because you've never seen that question before and you have no idea what to look for. So if you're waiting for when you're going to hear that one clue that's going to put you over the top, you'll never get it (except maybe at the giveaway) because those clues just don't exist for that question. Instead you pretty much have to accumulate information throughout and reduce your answer space (e.g. "oh, this is French. It's about aesthetic theory. About Romantic literature. About drama...") and then of course you have to actually know the answer to buzz. The way many of these questions were written, that was the only shot you had before the giveaway of converting on the answer. And for many people, I am guessing it's hard to maintain that level of concentration through what was a pretty demanding schedule.
Next, I want to focus on how submissions affected this tournament. You can find the answer sheet that shows what was submitted
here. Let me explain the color coding. The green parts correspond to stuff that was written for the editor packets. Anything marked yellow was written to replace a question in the submissions without reusing the answer line. Anything in blue was submitted and was either used as-is or was rewritten in some way; the spreadsheet doesn't indicate which, but that's not terribly important for my purposes.
I want to draw your attention to how very blue that matrix is. So very much of what was in this tournament was stuff that people submitted and we either let through or brought up to par. In the vast majority of cases, we edited the bonuses downwards to make them easier. We threw out what we thought were truly egregiously difficult questions and replaced them with something more reasonable. A large number of tossups got rewritten completely from scratch because even if the answer lines were good, we couldn't use the questions because they were bad. And so on.
But what's really important here for my point is how much of the material that people submitted was just really fucking hard. My policy with editing this tournament was to use good questions even if they were hard, and to try and salvage the answer lines if I could; otherwise, why would we even need packet submissions if we had to write the whole thing ourselves? But what that ends up meaning is that this tournament has answer lines like "deterministic finite state machines," and "Khorasan." The issue here is not just one of aesthetic preferences; it's also an issue of time. We just don't have the time to replace every single question that we think is either not the right difficulty.
When I look at the range of submitted answers, I draw the following conclusion: when you are sending me questions on the holographic principle and "Walking Around",
The Three Cornered World and
Woe from Wit, "On the Improvement of the Understanding", Nelson Goodman, Lysimachus, hesychasm, and Coyolxauqui, what I'm going to assume is that you're big kids who are ready to play tossups on: locality,
Berlin, Alexanderplatz, hypergeometric functions, the Majapahit empire, Shah Abbas, and
Empiricism and the Philosophy of Mind. I may be guilty of making a faulty inference from this data, but I can only judge from the data I have, and the data that I have tells me that you wanted this to be a really difficult tournament. If you didn't, why would you have written many of the questions that you did? Why would you have made your bonuses so hard that most teams would struggle to get 10 points on them?
I'm obviously not saying that anything about the submissions made me write the editor questions one way or the other. But they certainly made me think that the field was ready for these questions. I get the strong impression that a lot of people don't seem to mind when they are the ones writing the hard stuff, but once they have to play it themselves, they've got a wholly different view on the matter.
In the end, a large part of the tournament you get is the tournament you write. Nothing about the submitted material made me think that anyone was interested in a substantially easier tournament, and indeed, indicated to me that the majority of the writers thought the tournament should, literally, be harder than it ended up being. So while on the one hand I'm sympathetic to the complaints about difficulty in this tournament, the revealed preferences of question writers indicate something entirely different from what is being said post hoc.