A Not-Quite Essay on Criticism

Old college threads.
Locked
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

A Not-Quite Essay on Criticism

Post by naan/steak-holding toll »

“Words are like Leaves; and where they most abound,
Much Fruit of Sense beneath is rarely found.”

-- Alexander Pope

I've had a number of discussions recently with people who have said the state of tournament discussion proves discouraging to new writers. I wanted to call out some annoying things that happen in tournament discussion, and suggest some types of feedback that are useful. I do not intend this post to absolve me of any dumbass complaining that I've done before.

--- Bad Things ---

1) Stop generically complaining about bonus variance. It is literally impossible to make all bonuses the same. Unless you have precedent from other tournaments to back you up (i.e. don't make Methods of Ethics your ACF Fall easy part when it's a hard part at This Tournament Is A Crime) or a lot of good examples, please don't just post "man there was an issue with bonus variance" without any specifics. Inevitably, some bonus parts will be artifically easier than others because they've come up in more packets before - I think in this case the more constructive thing to do would be to suggest using a fresher bonus part. The same generally goes for tossup clues as well.

To digress a bit - I will content that what matters on a bonus is the outcome and not necessarily the difficult structure of its parts. To achieve a distribution of 90 / 50 / 10 (as is often idealized), one need not make the easy part have 90% conversion, the middle part have 50% conversion, and the hard part have 10% conversion (such that almost everyone who gets the hard gets the easy as well). Granted, this is probably the simplest bonus structure to execute. However, in some cases, I think a model where you have an easy part, then two parts with about 30% conversion each can work. The issue is that these 30% parts can't be of the "if you know one, you always know the other" - they have to be orthogonal in some way. Assuming a high conversion for the easy part and the probability of converting one of the 30% parts to be independent of the probability of converting the other 30% part, then you have a 9% chance of getting a 30 on the bonus and 49% chance of getting 20 - which is pretty close to a 90 / 50 / 10 structure. I will note, though - it's hard to judge when you've pulled this structure off correctly and I think the traditional easy / middle / hard structure is what should be aimed for in the vast majority of cases.

2) Back your assertions up when you complain about easy powers. Focus on clue ordering and improving consistency / feel. If one clue is harder than the other, then make sure they're in the right order. If a clue is misplaced, then it should be moved around, or maybe replaced if the tossup is noticeably off. If everyone is buzzing on the second clue, then that's not good unless it's a music mafia shootout or something. But I find that most people consistently over-estimate the amount of knowledge that other people have, and that this is especially pronounced for people who don't have as much experience as an editor. It's better to err on the side of generosity in powers. Multiple times at the EFT site, I was asked "why is that in power?" or "how is that not the last clue?" and the stats we had showed that only two or three teams had powered that tossup.

I'll make a side note here that one of the most impressive things about Jordan's writing is that he managed to consistently find fresh new clues AND provide people who may not know these with a reasonable shot at getting 15 points on his questions - and that he does this despite possessing more knowledge than most people playing his questions on almost every subject!

3) Tone down complaints about "category mistakes" and lack of category purity. So if someone writes a philosophy distribution for a major tournament that massively overdoses on anecdotes and secondary clues at the expense of primary material, or insists on filling the religion distribution with a bunch of "myth-y" legends and stories to the exclusion of actual practice, that's not good. But category boundaries are fluid, and there is always going to be material that can fit into multiple areas. In these cases, I like to take the attitude that my classical myth professor from sophomore summer took - he had us watch Pan's Labyrinth and draw connections to mythology from it, and said that we should think about studying mythology in terms of the impact it has on the everyday culture around us.

Here, I would like to define a "plausibility criterion". This means asking yourself whether, when somebody goes to learn about a given topic, it's plausible that they would encounter some form of material that would, by "traditional" quizbowl standards, not be included in the category in question. An example might be a clue about Rene Descartes and Queen Christina of Sweden. Is that more a "history" clue than philosophy one? Sure. But is it something you'd plausibly learn about in a philosophy class that covers Rene Descartes? Or in a lecture on his life? Absolutely! And the same for a history question on Queen Christina (or, if you're ambitious, on Descartes). In all cases, I'd still try to keep the amount of that sort of "non-traditional" content below 50% of the total meat of a tossup - but the boundaries of real-world learning aren't as constrained by categories as quizbowl is, and I see no reason to impose too harsh of a restriction on individual questions as long as we don't lose all semblance of standards.

--- Good Things ---

1) Hey, this is kind of stale / has been repeated X times in past year. It's easy to miss out on what's trendy and what's not, especially if you're not as in tune with the circuit. To some extent, experience is inevitably going to help you in quizbowl (like it does in most activities) but it's always good to suggest ways to avoid people immediately getting a question because they heard it at the previous tournament or whatnot.

2) Is anyone converting this bonus part / does this clue help anyone? In a long tossup I don't think you can always have leadins of the same difficulty, but every clue should at least be hypothetically helpful to some portion of the field (or at least be entertaining if it isn't). For bonuses - nothing is more frustrating than to possess a lot of knowledge on a bonus, then have somebody throw something utterly impossible / way out of difficulty whack at you and deny you a chance at 30 points. This should speak for itself, but in general you should write bonuses with the goal of actually rewarding knowledge you believe to exist, or which people will find educational and rewarding to listen to even if a hard part falls in quizbowl's collective blind spot. I personally haven't always been the best at this and it's something I hope to keep working on.

As a side note, I think this is why clues about specific notes generally aren't great in music tossups. Sure, there are some famous opening passages, chords, motifs, etc. that you can describe, and that people will buzz on - but for the casual listener audience, these clues will often be worthless. This is especially true if it's something that wouldn't be called out in a program score. (EDIT: I do think they can be helpful with secondary context though)

3) Hey - I think tossup X was good / bad for Y specific content-related reason. Calling out specific things you liked or disliked for specific reasons is generally more helpful than saying "I wasn't a fan of X." How the heck am I going to improve if you're not telling me why X was bad? Conversely, it's very rewarding to get positive feedback for content selection reasons, because (at least I think) most writers and editors put time and thought into their clue selections, and receiving appreciation for the time-consuming process of crafting a question is a nice reward.

4) This tournament did a good job emphasizing Z area / totally missed on Z area. These things are easy to overlook when crafting a set sometimes - you may just straight up forget to have Russian history in your tournament. Unlikely, but possible! Similarly, most times that someone puts a lot of questions in a traditionally under-emphasized area, they're doing it on purpose, and giving feedback on this helps signal to other people good areas to explore with future question writing.
Will Alston
Dartmouth College '16
Columbia Business School '21
Locked