Quizbowl: More Impossible than ACF?

[DumbJaques]

I have granted Jerry's wish and created a thread based on this post from CO discussion:

This is something that also deserves its own thread, but "this has been a clue before" does not imply that it can now be a bonus or tossup answer. Just because Bohemund of Otronto is mentioned in tossups on the First Crusade doesn't mean he's now fair game for any bonus answer. This is the kind of thinking that leads to unanswerable bonus parts because shockingly people don't remember every instance of whatever it is that comes up as a clue.

I vehemently agree with this; I was just looking over the bonus conversion statistics and no team at CO broke 18.5 ppb, and only three cracked 15. I simply don't believe there's a reason to aim for anything like this. Unless people think there's something about a set in which some of the best squads ever assembled average in the neighborhood of 22-23 ppb that somehow skews results, I think we need a bit of a collective reality check about just how hard we've been making some of these tournaments. Chicago Open this year featured a number of intensely amazing teams; the top four teams had anywhere from 2-3 people who would rank pretty highly on a greatest all-time player list. Yeah, such a tournament should naturally push the bar, include some excitingly difficult stuff, and generally reflect the high level of competition. But it's just so beyond unnecessary to write sets that run roughshod over even these extraordinary squads.

Not only do you make the event almost prohibitive for most teams that don't have a lights out player or two (which was certainly most of the squads at this event), but I really do think you hit a difficulty point at which you're actually making the outcome less legitimate because you've so limited the opportunities to score points. A bonus that ends up being a bit more reasonable can swing a set, and if lots of tossups are essentially unanswerable until the giveaway (merely because 90% of the field knows only one thing about them), well, that's in many ways just as bad as having lots of apyramidal questions.

I suppose this is getting kind of far abreast from Jerry's post, which I think correctly deals with the kind of mindset that leads people to write these kind of questions, but I'm saying right now that we've just got to reel it in. There's such an attitude now that because someone (or even a group of someones) would 30 a bonus or find a question surprisingly accessible, then man, we've got to amp up the impossible. I found the Chicago Open lit tournament quite refreshing, as it featured plenty of hard questions and surely got the job done even with such a strong field, but also managed to cover a representative swath of the literature canon. I frankly don't think you can argue that CO did a very good job of representing the body of the quizbowl canon, and I know some people think it doesn't need to. Well, I think that's nonsense. An experimental tournament is experimental for a reason; its primary purpose is not to produce legitimate results. The primary purpose of a regular (in nature, not difficulty) event like CO or Nationals or whatever is to resolve games in the most legitimate matter on the way to crowning a legitimate champion. I'm submitting that if some of the greatest quizbowl squads ever assembled, at the present apex of each member's ability, aren't even approaching 20 ppb, then we've lost the mission.

I admit that even as strongly as I believe all this, I still sometimes end up abusing my quizbowl peers with bonus parts on Nabil Kanso. But that answer, which I can't imagine would turn up in any archive searches, had about as much of a chance of being answered (read: 0%) as lots of things people justified as askable because they showed up in random bonus from 2005 or whatever. I guess this is a different discussion as well, but I find few things more frustrating than people justifying a question based on its packet archive prevalence. Those of us who played the Designate Bowl tournament got an object lesson in why something showing up a handful of times in an older packets is not a justification for evaluating its relative ease today.

[Matt Weiner]

I'm somewhat reluctant to personalize or ad-hominem this discussion, but I earnestly believe, based on empirical trends, that this is the reason for random difficulty spikes in certain tournaments, so here goes.

You do not have to prove your intellectual worthiness by writing impossible questions! Time and time again, when people who are not "elite" players and/or are new to the game take on writing projects or opine about ideal difficulty on the board, they overshoot the mark by a long shot. No one is going to think less of you if you write a solid tossup on Ficciones instead of Don Segundo Sombra. We won't think you don't know what Don Segundo Sombra is, because we don't care whether you know that or not. Writing good questions that good teams can play on using easier answers is, in fact, the way to impress people in quizbowl.

I don't want to name the names of people who do this, but it's interesting to look at people who don't--people like Seth Teitler, Jerry Vinokurov, and Andrew Yaphe, who are really in that elite class of players, rarely fall victim to the temptation of writing on impossible things. Be like them, don't be that guy who writes a twelve line tossup on Mary Barton for ACF Regionals and then defends it by saying it "comes up all the time."

The rush by competent-but-not-great players to fall over themselves writing tossups on the _semi-operas of John Eccles_, call anything that someone can buzz on before line 7 "transparent," and show off how hard they want quizbowl to be is just intellectual insecurity. Be confident that no one is judging your intelligence or character by your quizbowl skill or the difficulty of the questions you write, and stop doing this.

[Important Bird Area]

unanswerable bonus parts

I vehemently agree with this; I was just looking over the bonus conversion statistics and no team at CO broke 18.5 ppb, and only three cracked 15. I simply don't believe there's a reason to aim for anything like this. Unless people think there's something about a set in which some of the best squads ever assembled average in the neighborhood of 22-23 ppb that somehow skews results, I think we need a bit of a collective reality check about just how hard we've been making some of these tournaments.

I think the mechanism for this is pretty simple: question writers like hearing outrageous bonus parts in their own fields more than they like raising the average conversion rates of their own teams. That is: there's a fine line between "this bonus part will be an enjoyable challenge for the 4-5 best players in the country on category X" and "this bonus part is unanswerable by any player anywhere, and thus largely a waste of time," which is easy for writers and editors to miscalculate.

I know one of the things I enjoy most about difficult tournaments like CO is the presence of extreme bonus answers (in this set: say, Lake Yssyk Kul, King Penda, or Martin Bucer). Now, the downside of that is that science or music bonuses are either unanswered or answered by my teammates? Sure, but that's also true of regionals-level questions.

Maybe this is different for players with a broader range of knowledge?

[DumbJaques]

I think the mechanism for this is pretty simple: question writers like hearing outrageous bonus parts in their own fields more than they like raising the average conversion rates of their own teams. That is: there's a fine line between "this bonus part will be an enjoyable challenge for the 4-5 best players in the country on category X" and "this bonus part is unanswerable by any player anywhere, and thus largely a waste of time," which is easy for writers and editors to miscalculate.

I know one of the things I enjoy most about difficult tournaments like CO is the presence of extreme bonus answers (in this set: say, Lake Yssyk Kul, King Penda, or Martin Bucer). Now, the downside of that is that science or music bonuses are either unanswered or answered by my teammates? Sure, but that's also true of regionals-level questions.

Maybe this is different for players with a broader range of knowledge?

I agree that we want to be asking about exciting things that really test the full range of the considerable knowledge base that's annually arrayed for Chicago Open. But I think you're missing my point. I'm saying that the set should be constructed like a normal tournament, whose goal is to resolve games between teams and produce a legitimate champion by testing from the broad base of quizbowl knowledge. The extension of that paradigm to an event like CO is that the base of knowledge expands more than it does at other events, but it's important to remember that it doesn't shift. John Hay and everything he did is not somehow no longer a valid part of history, creating a need for ten-line tossups on John M. Clayton. There are ample experimental tournaments whose goal IS just getting to ask about and answer questions on amazingly exciting stuff from deep within a certain field, but that can't be the basis on which we write mainstream events. In other words, the opportunity cost of you hearing those questions is that probably nobody else converted more than one of them (if that). The opportunity cost for asking slightly more reasonable things about the same basic stuff (is Mercia such an easy answer that it couldn't be an equally legitimate hard part) is, as far as I can tell, that it becomes slightly less personally awesome when you actually luck into something you know. It doesn't seem like much of a contest to me.

Don Segundo Sombra

Well, so much for my ACF fall tossup on Don Segundo Sombra's Sombrero.

[Mechanical Beasts]

Yeah; I like the points that people are making here. Having made the mistakes Matt predicted in writing HI too hard (I think it's not just "first writing project;" it's "first writing project at difficulty x," because trying to hit any given difficulty is a new experience, though I'm sure HFT participants from last year would have rather my Barthes bonus found itself in HI instead), and having resolved not to do that again, I think it's very important that as you get more difficult, you also get much more specific, and as such have poorer coverage of the breadth of a subcategory. If I have sixteen biology tossups to work with, and I write about seven little-known pathways, five different cytochromes, a common link on opsonins and on teichoic acids, the red-headed cousin of the least-well-known form of speciation, and a taxonomy tossup on a specific coral, then think of all the things I've left untouched even though it might be possible to write those questions so that they don't intersect with each other too much. If I don't do that, then I've given fair treatment to everyone who studies or is interested in any subarea of biology.

[Cheynem]

I way overshot the difficulty in several of my questions for CO. I agree with Andy that this happened due to mostly writing inexperience combined with a little bit of vanity/misplaced fears of transparency. I was not intentionally trying to impress anyone. I think I probably fell victim the most in this area in the topics I like the most (Euro history, American history, and American lit--operations of Skorzeny, Heed Their Rising Voices, The Contrast) where I wrote some ill-advised and mostly unused tossups, partly because I felt (wrongly) CO was probably the only tournament of the year where some of these answers could come up. In retrospect, I should have saved them for experimental tournaments or my own vanity stuff. I got the impression, actually, that the stuff I didn't know a whole lot about (Mythology and Social Science) produced, quality of questions themselves aside, far more accessible answers (Kracauer and Talos). In the future, I will be sure to dial down the difficulty in my vanity niches.

[Skepticism and Animal Feed]

Matt Weiner,

I want to thank you for making that post. I agree that it is a major factor. Also I think that players like you are exactly the crowd these writers are trying to impress. So it's good for them to hear it from you.

[grapesmoker]

Matt Weiner,

I want to thank you for making that post. I agree that it is a major factor. Also I think that players like you are exactly the crowd these writers are trying to impress. So it's good for them to hear it from you.

One of the things people seem to miss is that what makes good players good isn't that they know one obscure thing. It's their dominance on all questions across the board. Seth Teitler isn't an awesome player because he can power a tossup on Tlazolteotl; he's an awesome player because even if you beat him to a tossup on Tlazolteotl, he'll still answer a whole bunch of tossups anyway because he has a broad, dominant knowledge base. Great players are defined by their ability to answer questions on a wide range of topics, and writing on crazy things that no one else knows isn't going to impress those players, it's going to irritate them. As editors, we'll be way more impressed by a competent tossup on a solidly accessible answer than we will be on a rambling effort on something no one has heard of, and as players, we'll be more impressed by knowledgeable buzzes on important and reasonably known things than we are by knowledge of diverse obscurata and claims that Vox Clamantis is totally famous and easy. In the end, the former knowledge base is what wins games and titles; the latter might give you a chance to preen a little bit, but the last stop for that train is Notwinningville.

[Skepticism and Animal Feed]

If anything, it is more impressive to be able to write a good "hard tossup on easy answer" than to write a good tossup about something really hard.

[SnookerUSF]

Hello,

No one is going to think less of you if you write a solid tossup on Ficciones instead of Don Segundo Sombra.

Yeah, I don't know about this. What will inevitably happen is someone will equivocate on the "solidness" of the tossup rather than its ease, suggesting that while a particular answer space was "okay" there was something pathetically flawed about the tossup. This kind of comment generally relies on a silent or tacit acceptance of those who are unwilling to post that they :gasp: didn't know this clearly superfamous fact, and thus it becomes quickly filed away as "stock." While sometimes those types of complaints are legitimate, more often, they are a feature of the intellectual grandstanding you go on to talk about.

Generally, I would claim that for an inexperienced writer, or for one who is inexperienced in some subcategory, the refuge from these kinds of complaints is generally a tossup on a more challenging answer. It seems clear that the backlash has begun in earnest in this regard, so perhaps there will be some kind of self-moderation not only in the question difficulty, but in the corrupting type of comments discussed above in response to this "kinder, gentler" quizbowl of the future. Forgive me though if I am a bit skeptical of the latter and thus the former.

Moreover, I totally admit to being "intellectually insecure" about my questions, and it showed in the quirky non-distribution of SnF. Though this does not exculpate me in anyway; it behooves me to write tossups in areas I think I can write in, while criminally neglecting others. This should change.

Also, I wonder if there are other reasons, if you believe quizbowl as gotten more impossible than ACF, besides the kind of "ad-hominem" reasons Matt points out. Could it be that there is a kind of category arms race as well? What I mean is that the transition from SCIENCE! to science, has prompted the exponents or devotees of other fields such as: Music, Social Science, Literature to respond in kind, and thus accelerating the canon solifluction into a full-on avalanche of impossibility. You might argue that this is a one-off of the ad-hominem claim, but I am inclined on quick reflection to believe that there is some legitimacy to this process anyway. If science, literature etc. insist on a "reality principle," that is, those who study the field either as a major, vocation (in the traditional sense), or hardcore enthusiast, ought to be rewarded first and most often, then all of the devotees in the other categories in the distribution feel like they ought to keep up in some fashion. The major problem in this becomes, given that few of us are qualified to judge relative difficulty, is that we are inclined to overshoot the mean in some hilariously difficult ways.

In the end, the former knowledge base is what wins games and titles; the latter might give you a chance to preen a little bit, but the last stop for that train is Notwinningville.

Of course you are right, but the last stop for most of us on this train of quizbowl is indeed Notwinningville, opportunities for preening or not. That does sound defeatist, but it is the nature of the conveyance. So, I guess I am disinclined to be especially critical of that natural, albeit, selfish sentiment. It seems that there should be enough room in quizbowl for both camps to be satisfied, and I think the subject/experimental tournaments are the best and perhaps ONLY place to do that.

Finally, I ask the community-at-large, what can be done about this if anything. Is this a "genie-in-a-bottle" situation, can we really undo this in any meaningful way? Moreover, I am sure there are those out there who disagree with the initial premise of this topic, I hope they come out to play as well.

[TheKingInYellow]

Being perhaps a semi-competent player who wrote for this tournament (indeed, on Tlazolteotl), I can tell you that I didn't write that question to impress, but rather, because I really liked the idea of a Tlazolteotl toss-up. I think I can relate to Mike Cheyne, when I say that I wanted this sort of thing to come up, and the Chicago Open seemed the place to do it.

[Mechanical Beasts]

Being perhaps a semi-competent player who wrote for this tournament (indeed, on Tlazolteotl), I can tell you that I didn't write that question to impress, but rather, because I really liked the idea of a Tlazolteotl toss-up. I think I can relate to Mike Cheyne, when I say that I wanted this sort of thing to come up, and the Chicago Open seemed the place to do it.

Well, there are other ways to do it, aren't there? To take one extreme, you could write a tossup on "Aztec myth" where most of the clues are about that dude's antics; to take a more moderate position, you could have written a tossup on a better-known god with many clues having to do with antics including Tlazolteotl, or at the other extreme, you could have really kicked our asses with a tossup on Tlaelquani, her coprophage aspect. Each option involves Tlazolteotl coming up more; the one you choose may depend on difficulty level. This is not to say that I've never done exactly what you have, and will never do it again: I just mean to say that it's quite easy to, if you choose to, achieve the effect of a Tlazolteotl tossup without the drawbacks.

[grapesmoker]

Yeah, I don't know about this. What will inevitably happen is someone will equivocate on the "solidness" of the tossup rather than its ease, suggesting that while a particular answer space was "okay" there was something pathetically flawed about the tossup. This kind of comment generally relies on a silent or tacit acceptance of those who are unwilling to post that they :gasp: didn't know this clearly superfamous fact, and thus it becomes quickly filed away as "stock." While sometimes those types of complaints are legitimate, more often, they are a feature of the intellectual grandstanding you go on to talk about.

If people do this, they should stop doing so. There's nothing wrong with a good Ficciones tossup; the problem is that often such questions end up not being solid at all because wacky postmodern happenings + Spanish surnames = Borges so let's buzz and say that. People in general should stop engaging in such grandstanding of course.

Also, I wonder if there are other reasons, if you believe quizbowl as gotten more impossible than ACF, besides the kind of "ad-hominem" reasons Matt points out. Could it be that there is a kind of category arms race as well? What I mean is that the transition from SCIENCE! to science, has prompted the exponents or devotees of other fields such as: Music, Social Science, Literature to respond in kind, and thus accelerating the canon solifluction into a full-on avalanche of impossibility. You might argue that this is a one-off of the ad-hominem claim, but I am inclined on quick reflection to believe that there is some legitimacy to this process anyway. If science, literature etc. insist on a "reality principle," that is, those who study the field either as a major, vocation (in the traditional sense), or hardcore enthusiast, ought to be rewarded first and most often, then all of the devotees in the other categories in the distribution feel like they ought to keep up in some fashion. The major problem in this becomes, given that few of us are qualified to judge relative difficulty, is that we are inclined to overshoot the mean in some hilariously difficult ways.

But we do privilege the knowledge of experts in all categories, or at least try to do so. If you're an English major who has read a lot about books, you might well get a tossup on a critical clue before anyone who has read the book. Barring that, if you've read the work, you will get it before those who haven't and so on. I think rather the literature distribution actually got there first and the reworking of the way science questions were written around 2001 were a result of trying to emulate that.

Of course you are right, but the last stop for most of us on this train of quizbowl is indeed Notwinningville, opportunities for preening or not. That does sound defeatist, but it is the nature of the conveyance. So, I guess I am disinclined to be especially critical of that natural, albeit, selfish sentiment. It seems that there should be enough room in quizbowl for both camps to be satisfied, and I think the subject/experimental tournaments are the best and perhaps ONLY place to do that.

Well, ok, that's fair enough. I'm certainly happy to restrict wacky things to novelty/experimental tournaments.

Finally, I ask the community-at-large, what can be done about this if anything. Is this a "genie-in-a-bottle" situation, can we really undo this in any meaningful way? Moreover, I am sure there are those out there who disagree with the initial premise of this topic, I hope they come out to play as well.

People need to ask themselves who their questions are directed towards. It's something many of us fail to do when writing, myself included. But over the years I've consciously tried to bring down the difficulty of the things I write because as I've played more and more quizbowl and got a better idea of what people know, I've realized that in most cases we can distinguish between good and not so good teams by means of deep clues rather than demanding that they identify really obscure things and then risking having those things go dead across the field.

[cvdwightw]

Yeah, I don't know about this. What will inevitably happen is someone will equivocate on the "solidness" of the tossup rather than its ease, suggesting that while a particular answer space was "okay" there was something pathetically flawed about the tossup. This kind of comment generally relies on a silent or tacit acceptance of those who are unwilling to post that they :gasp: didn't know this clearly superfamous fact, and thus it becomes quickly filed away as "stock." While sometimes those types of complaints are legitimate, more often, they are a feature of the intellectual grandstanding you go on to talk about.

There are two reasons that someone would cram a whole bunch of way-too-hard stuff into a hard tossup on an easy answer:

1. The question author has some kind of perverse need to prove his intellectual prowess by including a bunch of clues that no one is expected to know.
2. The question author has serious doubts about "transparency" or fears that the middle clues that should be included will be attacked as "too easy" or showing up "too early."

I think both (1) and (2) are much more prevalent at the high school level than at the college level. For a lot of people, writing high school questions has devolved from "what can we reasonably expect the average high schooler to know?" to "how can I write this question so that I still hit 80% conversion but I don't have high school players and coaches (or worse, college players) complaining on the board that the question was too easy?". So what ends up happening is that we get canon expansion in the leadins of high school questions, we get tossup answers and third bonus parts that are just not appropriate for many teams, and we get caught in the middle between your average local team complaining that the questions are too hard and the elite national team complaining that the questions are too easy.

Realistically, the reason that this gets better at the college level is that the truly elite players tend to complain only when clues are either (a) useless, (b) way too hard, or (c) rewards having no actual knowledge of the answer. But the fact remains that we have a lot of players who haven't outgrown the same mentality they had in high school. It's not just the young ones; heck, I'm not as guilty as a lot of people, but I'm still prone to the Fundamental Difficulty Error.

Generally, I would claim that for an inexperienced writer, or for one who is inexperienced in some subcategory, the refuge from these kinds of complaints is generally a tossup on a more challenging answer. It seems clear that the backlash has begun in earnest in this regard, so perhaps there will be some kind of self-moderation not only in the question difficulty, but in the corrupting type of comments discussed above in response to this "kinder, gentler" quizbowl of the future. Forgive me though if I am a bit skeptical of the latter and thus the former.

Multiple choice question: you're a new editor who just signed onto the ACF Fall team after being identified as a young writer with editor potential, and you get some really good packets that you don't have to worry about, and you get some poor packets that you edit up to snuff, and you think, "hey, I did a pretty good job." And then you see the ACF Fall discussion and there's some freshman from a top-10 high school team last year complaining about "why the heck is the Sunyaev-Zeldovich effect in the second line? And taq polymerase [which you put just before FTP]? That's so stock.". What are you thinking?

A. "This person probably knows as much or more than me about this. I should readjust my difficulty views accordingly."
B. "This person probably knows as much or more than me about this. I should get a second opinion from an older, elite player."
C. "I stand by my clue placement and this person needs to readjust his difficulty views accordingly."
D. "This person is a disciple of Ryan Westbrook and I should train myself to ignore all of his views on difficulty."

I would conjecture that your average new editor is probably answering A; a few lucky ones at programs driven by elite players might be thinking B.

Moreover, I totally admit to being "intellectually insecure" about my questions, and it showed in the quirky non-distribution of SnF. Though this does not exculpate me in anyway; it behooves me to write tossups in areas I think I can write in, while criminally neglecting others. This should change.

First off, I think this whole "intellectual insecurity" is the reason why a lot of people don't refute challenges to the difficulty of the question. Most of the time, person A will say "Bonus X was the easiest bonus of the tournament" and person B will look at the bonus and think "What are you talking about? I haven't even heard of the third part, and I might pull the second part on a good day after taking a final in that class." 99 times out of 100 person B will not post any such thing, because person B thinks posting such a thing implies person B's "intellectual inferiority" to person A. Really, it is only the elite players who really can make this claim, because they've already found their intellectual security. Because they have the respect of just about everyone on the circuit, and because they have an excellent command of the subject, they can post that "I've never heard of Bonus X's third part" or that "Clue Y is not, in fact, too early" and the fault appears to lie with the question (and by extension, the person attacking it as too easy) rather than the person posting the rebuttal.

Second, I think most of us are intellectually insecure when it comes to our questions, especially when we're writing in areas we don't really know that well. My first two years of college I was responsible for just about every science question from those UCLA teams because even if I didn't know science all that well, I sure knew it better than the other people on the team. The point of this is that a lot of times you don't have the luxury of writing only in categories you can write competently; you have to write absolute crap in some subjects and ask for constructive criticism. It's much harder to take "that's too easy" complaints when you don't, in fact, know if whatever middle clue you randomly grabbed from Wikipedia and cross-checked with three ACF Regionals packets was really too easy.

Also, I wonder if there are other reasons, if you believe quizbowl as gotten more impossible than ACF, besides the kind of "ad-hominem" reasons Matt points out. Could it be that there is a kind of category arms race as well? What I mean is that the transition from SCIENCE! to science, has prompted the exponents or devotees of other fields such as: Music, Social Science, Literature to respond in kind, and thus accelerating the canon solifluction into a full-on avalanche of impossibility. You might argue that this is a one-off of the ad-hominem claim, but I am inclined on quick reflection to believe that there is some legitimacy to this process anyway. If science, literature etc. insist on a "reality principle," that is, those who study the field either as a major, vocation (in the traditional sense), or hardcore enthusiast, ought to be rewarded first and most often, then all of the devotees in the other categories in the distribution feel like they ought to keep up in some fashion. The major problem in this becomes, given that few of us are qualified to judge relative difficulty, is that we are inclined to overshoot the mean in some hilariously difficult ways.

The reality of it is that we no longer have true "history" players, or "science" players, or whatever. We have people who have strangleholds on specific subcategories, and oftentimes are writing their leadins/third parts/hard-tournament questions for people who are of similar strengths in those categories. It's not so much a category arms race, as it is that the canon for each subcategory is simultaneously expanding at a rate roughly proportional to (# quizbowlers at an elite playing level), with minor corrections for (average writing ability of the elite players). Elite players will generally write their high-end questions to challenge other elite players, because "hey there's this cool thing that other elite players might have heard of and if they haven't they might want to know about it," while everyone else is going to stick to "hey there's this thing that has come up before, so even if it's unimportant, I can point to this other question as evidence that people might know it." There is now a critical mass of elite CS players, so the CS canon is expanding quite significantly given that there's < 1 CS question per packet. There aren't a lot of elite earth science players, so expansion in that subcategory is much slower than it is in CS, because people are sticking to the "tried and true" answers and clues in earth science and there aren't as many people at the elite level or the fringe of the category who can (and want to) write for other people at that level.

[magin]

Personally, I'm much more impressed when someone writes a tossup of appropriate length and difficulty for its audience, with many buzzable middle clues that people playing the tournament can buzz on, than when that same person selects a really hard tossup answer or writes a tossup with 6 lines of leadin clues that no one at the tournament has a chance of knowing.

Writing bonuses with an easy part that most teams can answer, a middle part that around half the teams can answer, and a hard part that a player with reasonably good knowledge can answer is similarly more impressive than writing a bonus which the majority of teams in a tournament get 0 or 10 points on.

If writers/editors are worried about being excoriated for using clues that people can answer (on this board, or elsewhere), please do not fret. Among all the experienced editors I know, you will gain much more respect for writing difficulty-appropriate questions with solid middle clues and bonus parts that you expect many people at a tournament to know than by adventuring into really hard clues that basically no one has a reasonable chance of buzzing on.

[No Rules Westbrook]

Okay, I'm going to be very measured and temperate in commenting here, because I know some people often throw my opinion out the window poste haste when it comes to these matters.

As Graham's post evidences, I really don't think that impressing elite players is a motivation of most (or maybe even any) writers. I don't know where the empirical proof comes from, but I've never heard any such writer say something to me that even suggests that their primary motive is "showing off how smart they are." I also think a great majority of players/writers do not really believe in what everyone calls the fundamental difficulty error, as it's usually stated - most people know that something isn't easy just because they think it's easy - most people naturally understand that there must be criteria outside of their own experiences or opinions that inform how "easy" or "hard" some question is. Sorry, but those are two theories that I hear get floated around every time this comes up, and I just think they're pretty unsubstantiated. Heck, quizbowl in general may be built on intellectual insecurity...but I just don't see it as prime motivation for the kinds of effects people are talking about here.

So, what is at work here? Well, let's think about back in the day - relatively inexperienced writers (those with an iffy gauge of qb canon, etc.) would routinely and predictably write on insanely hard stuff, stuff that was clearly unacceptable by any standard - because it had come up in their class or discussion section or because it was the first thing they found in some online search or for arbitrary personal reasons like that. Now, relatively inexperienced writers have the advantage of having lots of packets online - it's much easier for them to get at least a superficial grasp of the canon. So, they don't write on clearly crazy wacko stuff anymore, but often they do write on things that are very hard - on the outermost reaches of acceptable canon material - because they make some logical leaps based on what is freely available, and they form some biased opinions. Might we be well-advised to push them toward not making such blind leaps? Sure. But, the current situation is a lot better than the previous situation.

[grapesmoker]

Just to be clear, I think a tournament like Chicago Open does accept a lot of difficult things as possible answers. To use the example of the Tlazolteotl tossup, I actually thought that was a cool idea and enjoyed that question. It's hard but it's not so hard that good players can't get it. The things I have objections to are questions that ask you to identify something magically obscure that even most experts can't get. I don't think most of the things at CO fell into this category though some did.

[Strongside]

With Chicago Open being generally regarded as the premier open quiz bowl tournament of the year, I think writing questions on things that are extremely difficult/challenging is the way to go.

Looking at the stats from the main event, every team averaged at least 7 points a bonus. If a team were to average 22-23 points per bonus at Chicago Open, the questions would be too easy, but that's just my opinion.

[Mechanical Beasts]

Looking at the stats from the main event, every team averaged at least 7 points a bonus. If a team were to average 22-23 points per bonus at Chicago Open, the questions would be too easy, but that's just my opinion.

It seems silly to cluster teams any more than is necessary (i.e. I guess we could have four-part bonuses, but those would probably be hard to write); it only degrades our ability to distinguish between teams. I certainly don't think my own team should have been getting many more hard parts than it was, but I think the top two teams certainly should have been.

[cvdwightw]

As Graham's post evidences, I really don't think that impressing elite players is a motivation of most (or maybe even any) writers. I don't know where the empirical proof comes from, but I've never heard any such writer say something to me that even suggests that their primary motive is "showing off how smart they are."

I will believe that this is true, for the majority of good writers and/or the majority of people who want to be good writers. I do believe that there is a group of people that does not necessarily adhere to the tenets of good writing and derives some sense of satisfaction from writing a question that even the best players can't answer.

I also think a great majority of players/writers do not really believe in what everyone calls the fundamental difficulty error, as it's usually stated - most people know that something isn't easy just because they think it's easy - most people naturally understand that there must be criteria outside of their own experiences or opinions that inform how "easy" or "hard" some question is.

Whether or not the fundamental difficulty error (during post-tournament construction) is real or imagined, the fact remains that people write their questions with the fundamental difficulty error in mind. How else do you explain the dearth of good, solid middle clues in a substantial proportion of submitted science (and other categories) tossups? I'm pretty sure it's not a function of "inexperienced people write poor science questions," because it sometimes happens with decent but not great science writers. I posit that it's partially due to writers finding several lines of awesome but ridiculously difficult clues and then realizing that they better wrap up the question quick, and partially due underestimating the point at which they should be getting the question (that is, the point at which they believe they should get the question is later than the point at which they should get the question). I would call the latter problem the fundamental difficulty error. Perhaps both of these problems are unique to my writing, but I seriously doubt that.

So, what is at work here? Well, let's think about back in the day - relatively inexperienced writers (those with an iffy gauge of qb canon, etc.) would routinely and predictably write on insanely hard stuff, stuff that was clearly unacceptable by any standard - because it had come up in their class or discussion section or because it was the first thing they found in some online search or for arbitrary personal reasons like that. Now, relatively inexperienced writers have the advantage of having lots of packets online - it's much easier for them to get at least a superficial grasp of the canon. So, they don't write on clearly crazy wacko stuff anymore, but often they do write on things that are very hard - on the outermost reaches of acceptable canon material - because they make some logical leaps based on what is freely available, and they form some biased opinions. Might we be well-advised to push them toward not making such blind leaps? Sure. But, the current situation is a lot better than the previous situation.

I agree with everything written here (I know, amazing). However, at the high-end areas of the canon, it becomes much more difficult to say whether or not something is canonical or not. In this case, someone who does indeed have a good grasp of the canon might suppose that something that isn't actually important or something that no one actually knows would make for a fine third part of an upper-difficulty tournament (cf. Jerry's railing against Eponym Bowl). There are also specialists who do know that something is important and do attempt to justify their selections because (a) the topic is important in that field and other specialists in that field either will know it or will want to know about it and (b) there's a nebulous "post-Nationals difficulty" at which the degree and frequency of canon expansion is ill-defined (at least, less well-defined than at lower levels). I would find it very difficult to believe that canon expansion at the far reaches of the canon is largely due to inexperienced writers; it seems much more plausible that it's done by either people trying to bridge a gap between academic importance and quizbowl importance or people who have an otherwise good grasp of the (lower-level) canon and have difficulty selecting an appropriate high-difficulty third party.

[DumbJaques]

Looking at the stats from the main event, every team averaged at least 7 points a bonus. If a team were to average 22-23 points per bonus at Chicago Open, the questions would be too easy, but that's just my opinion.

This argument doesn't make a whole lot of sense to me. I'm not sure what your logical basis is for saying that, if ANY conceivable team averaged 22 ppb at CO, then the questions would be too easy. The premise that you could really even speculate on an arbitrary cutoff like that also seems inherently flawed.

What do you even mean by "too easy?" The only reason I can think of that you could really say a CO was "to easy" would be if the material covered in the questions was so narrow that it didn't adequately test that true knowledge bases of all the participating teams. I don't see how you could possibly argue that this would be the case if the top 1-2 teams are hitting 22 ppb. Unless you're prepared to argue that deciding games on lower point totals is somehow fairer than deciding games on higher point totals (and I submit that the opposite is usually true, ceteris peribus), what happens at this point that messes up the event?

[theMoMA]

I wouldn't want Chicago Open to become a tournament where the top teams expected to get 22+ points per bonus. In case people are forgetting, you need to make bonuses quite a bit easier than you'd expect to ensure that kind of bonus performance, even when there are superteams playing. Chicago Open is the place for pushing the limits and testing players on the most rigorous questions of the year, and I hope this continues to mean that the bonuses will have an easy part, a tough-but-doable middle part, and a difficult part that is gettable by the most knowledgeable players. A really good, well-rounded team would be able to hit twenty on bonuses like that.

A more noble goal that I'm sure we can all agree on is to get past the frustrating minority of bonuses that have an incredibly difficult third part, or have no true easy parts. If people rein in the most difficult fifth of their hard parts, and take the time to ensure that their bonuses have a true easy part, it would improve everyone's conversion by a noticeable amount without running counter to the conception of CO as a tournament for testing deeper knowledge.

[Important Bird Area]

I endorse Andrew's post; with the note that I looked up the history of CO bonus conversion stats.

The hardest parts of this year's bonuses were more challenging than those from past years: the top bonus conversion of 18.26 is the lowest such figure on record.

But the low rate of teams converting middle parts to get to 15 ppb is perfectly normal. Only 3 of 14 teams converted half of the bonus points this year; but only 3 of 13 did so in 2006.

Finally, it's worth noting that bringing the top teams to 22-23 ppb will require a substantial decrease in bonus difficulty from future writers and editors. Note here that no team in history has ever put up 22 ppb on Chicago Open (and only one team even cracked 21: the Yaphe/Teitler/Kemezis/Matthews team from 2005).

[grapesmoker]

Chicago Open is the place for pushing the limits and testing players on the most rigorous questions of the year, and I hope this continues to mean that the bonuses will have an easy part, a tough-but-doable middle part, and a difficult part that is gettable by the most knowledgeable players. A really good, well-rounded team would be able to hit twenty on bonuses like that.

I think the problem is exactly the fact that many bonuses do not fit this description. In other words, when the most knowledgeable players are those near to a Ph.D. in the subject and are still not able to answer the 3rd bonus part, something is wrong.

A more noble goal that I'm sure we can all agree on is to get past the frustrating minority of bonuses that have an incredibly difficult third part, or have no true easy parts. If people rein in the most difficult fifth of their hard parts, and take the time to ensure that their bonuses have a true easy part, it would improve everyone's conversion by a noticeable amount without running counter to the conception of CO as a tournament for testing deeper knowledge.

This is a good suggestion and I will definitely be working towards this next year.

[MLafer]

I also noticed that while most bonuses did have an easy part, that easy part was often made deliberately vague and made certain bonuses very frustrating. I will try to find some examples when the set is posted.

[No Rules Westbrook]

Yeah, Hart's post is the most eminently reasonable here. I find that when bonuses have reasonable easy parts (reas within the context of CO, that is) and don't often have impossible hard parts, most if not all of the complaints go away. I think a lot of the frustration people feel here is related more to inconsistency and roller-coaster problems in packets, where people start to feel like the questions aren't being all that fair, and less about ideal notions of difficulty.

[Strongside]

Looking at the stats from the main event, every team averaged at least 7 points a bonus. If a team were to average 22-23 points per bonus at Chicago Open, the questions would be too easy, but that's just my opinion.

This argument doesn't make a whole lot of sense to me. I'm not sure what your logical basis is for saying that, if ANY conceivable team averaged 22 ppb at CO, then the questions would be too easy. The premise that you could really even speculate on an arbitrary cutoff like that also seems inherently flawed.

What do you even mean by "too easy?" The only reason I can think of that you could really say a CO was "to easy" would be if the material covered in the questions was so narrow that it didn't adequately test that true knowledge bases of all the participating teams. I don't see how you could possibly argue that this would be the case if the top 1-2 teams are hitting 22 ppb. Unless you're prepared to argue that deciding games on lower point totals is somehow fairer than deciding games on higher point totals (and I submit that the opposite is usually true, ceteris peribus), what happens at this point that messes up the event?

I probably should have been clearer in my previous post. As Jeff Mentioned, the 2005 team of Yaphe/Teitler/Kemezis/Matthews had a 21.03 bonus conversion, which was the highest between 2004 and 2009.

I see Chicago Open as a tournament that should be difficult and challenging. People who come to Chicago Open generally know what they are getting into when they sign up, and they spend their own money to play it. There aren't any prizes except for maybe a book or two, and most people play for fun and the prestige/bragging rights that come with winning or doing well. I know I want to hear difficult questions and be challenged on bonuses.

One thing that I thought was interesting when looking at the stats was the bonus conversion. Although it can't be empirically proved, it seems pretty obvious that there is not a single person (at least among active players) would have been able to average 15 points per bonus at this tournament had they played solo (i.e. if you asked the same 240 or so bonuses to everybody).

I think this stats is what makes quiz bowl interesting, and difficult. We have people who are among the all time greats, and have been playing tournaments like these for years, and still no individual can even convert a simple majority of the potential bonus points at a tournament like this. This demonstrates that even the elite players have a lot of room for improvement.

This is why I think converting 22 points per bonus which is 73.33 percent would be difficult. Even last year's 2 "superteams" didn't break 21, and I felt the bonuses last year were easier than this year. This year's bonuses were definitely harder than the bonuses at ACF Nationals. Hopefully this makes sense.

[grapesmoker]

One thing that I thought was interesting when looking at the stats was the bonus conversion. Although it can't be empirically proved, it seems pretty obvious that there is not a single person (at least among active players) would have been able to average 15 points per bonus at this tournament had they played solo (i.e. if you asked the same 240 or so bonuses to everybody).

I think this stats is what makes quiz bowl interesting, and difficult. We have people who are among the all time greats, and have been playing tournaments like these for years, and still no individual can even convert a simple majority of the potential bonus points at a tournament like this. This demonstrates that even the elite players have a lot of room for improvement.

I don't see how you can use this in support of anything really. Yes, no one playing solo at this tournament would break 15 PPB, but so what? No one has argued that bonuses should not be challenging, but I think people are right to protest an ever-increasing ramping up of difficulty for no reason other to make the tournament harder. Here's my suggestion: if you do it right, even good teams will have to work really hard to break 20 PPG. But they can do it if they know a lot. At this tournament, often even knowing a lot (or as pointed out above, having two of the best science players in the game on your team) wasn't enough to get you more than 10 points. That's absurd and dumb and sounds to me like difficulty not for the sake of challenging people but for its own sake. We don't need to ask about the 15th best known character from Israel Potter or whatever wacky thing people are trying to write about to challenge good players.

[Cheynem]

I guess the problem is what is meant by "challenging." Is challenging encountering a tossup or a bonus with deep clues about moderate difficulty stuff? Is challenging running into a tossup or a bonus about something you've never heard of before aside from memorizing a list? I mean, I suck, I understand that CO isn't meant for me, but my feeling is that if a tournament is meant to be written in which, say, Jerry and Eric flail at science bonuses, then I wonder if it's even worth it to play such a tournament (Note: I don't regret attending CO and I'll be back there as often as I can--I just am concerned about ever-escalating difficulty).

[grapesmoker]

I guess the problem is what is meant by "challenging." Is challenging encountering a tossup or a bonus with deep clues about moderate difficulty stuff? Is challenging running into a tossup or a bonus about something you've never heard of before aside from memorizing a list? I mean, I suck, I understand that CO isn't meant for me, but my feeling is that if a tournament is meant to be written in which, say, Jerry and Eric flail at science bonuses, then I wonder if it's even worth it to play such a tournament (Note: I don't regret attending CO and I'll be back there as often as I can--I just am concerned about ever-escalating difficulty).

The way I read it is that "challenging" should force you to dig deep into your arsenal of knowledge. Sometimes you don't know something or you don't come up with the answer, and that's fine, but we should be avoiding questions that are basically "name this thing you've never heard of, have no reason to know, and couldn't possibly care about." The answers themselves can be of moderate difficulty and they can also stretch the boundaries of the canon, but they should do so in a sensible fashion with regard to the kinds of things people (certainly good players) can answer. Here's the thing: if you write a science bonus on which Eric and I get 10 points, chances are you've also written a bonus on which most everyone else got zero points. Those kinds of questions are bad because they don't allow you to differentiate between teams with rather different knowledge levels in the subject, a fact I've harped on many times and which I believe to still be fundamentally correct.

[at your pleasure]

The way I read it is that "challenging" should force you to dig deep into your arsenal of knowledge.

I realize that this has probably been said,but does this even remotely require tossups on things that graduate students in the subject may not have heard of? If a writer wants to "force people to dig deep into their arsenal of knowledge", that writer can just as well test super-deep knowledge of canonical thing as knowledge of exceptionally obscure things. For that matter,I'd be more impressed by someone who writes a good a post-nationals level tossup on Michelangelo or Bach than someone who writes a tossup on the fourth most famous Estonian composer.

they can also stretch the boundaries of the canon,

I realize that this is probably repeating someone else as well, but it might help if the canon is expanded primarily to introduce "things that don't come up even though they are objectively important" and not so much so people can fulfill a dubious obligation to write on unecessarily obscure things for CO.

[grapesmoker]

I realize that this has probably been said,but does this even remotely require tossups on things that graduate students in the subject may not have heard of?

No, of course not. I'm trying to suggest the opposite, in fact.

If a writer wants to "force people to dig deep into their arsenal of knowledge", that writer can just as well test super-deep knowledge of canonical thing as knowledge of exceptionally obscure things. For that matter,I'd be more impressed by someone who writes a good a post-nationals level tossup on Michelangelo or Bach than someone who writes a tossup on the fourth most famous Estonian composer.

Well, my opinion is that a tossup on Michelangelo would have to search far and wide to challenge good players because just so much is known about him. That's the kind of tossup I'd rather see at ACF Regionals, say, than Chicago Open. But that doesn't mean that CO is your chance to write a tossup on the Master of the Naumberg Madonna.

I realize that this is probably repeating someone else as well, but it might help if the canon is expanded primarily to introduce "things that don't come up even though they are objectively important" and not so much so people can fulfill a dubious obligation to write on unecessarily obscure things for CO.

Sure; that's a difficult task and it has to be done appropriately.

[at your pleasure]

the Master of the Naumberg Madonna

If this ever comes up, I will eat my hat. Anyhow, I agree that a CO tossup on Michelangelo would be exceptionally difficult to execute well, which is why it would be so impressive. It could be done, however, if people would get it off problematic or destroyed works.

[Sargon]

I think the current emphasis on rewarding several years of graduate study is part of the problem. Many quizbowlers aren't graduate students in anything, and even advanced graduate students will only have that specialized knowledge in a very small area. If you happen to study modern literature this could be a few tossups a round, if as in my case you study Assyriology it amounts to maybe one tossup a tournament, but as pointed out earlier in the forum, it will not win you matches. Most quizbowlers are playing most questions on things they will not have done serious academic work in. Hence, I think the aim of the question writer, especially in answer selection, should be to keep answers to what a learned amateur might reasonably know, which incidentally is much easier to gauge since most people are reasonably learned in a lot more fields than the ones they know at an expert level. If you've heard of something and its not in a field you study professionally, then odds are a decent number of other people in quizbowl have as well. If some who had read a half dozen introductory books on the subject would not be helped by a clue (at least thinking "The X effect, I should know where this going but am blanking") or could not produce an answer, then it is probably too hard at all bust the highest levels of quizbowl, if then. Also, in my experience, it is also not very fun to play on questions where you don't have much of a chance of knowing the answer outside you specialty or getting anything useful out of in the first 5 linesl; I for one would much rather sacrifice my one free tossup a round of insane deep knowledge for having a fighting chance at the majority of questions in the round.

[grapesmoker]

Most quizbowlers are playing most questions on things they will not have done serious academic work in.

If you've taken history classes, you've done some academic work in that area. Obviously, a lot more of what people know just comes from outside reading, reading packets, and so on, but I don't think that's too much of a problem. Science tends to be different because unlike literature and history, which science players read on their own time, non-scientists tend not to know much about science (with some exceptions).

Hence, I think the aim of the question writer, especially in answer selection, should be to keep answers to what a learned amateur might reasonably know, which incidentally is much easier to gauge since most people are reasonably learned in a lot more fields than the ones they know at an expert level.

When you say "learned amateur," do you just mean people who play quizbowl and know things they're not experts in? Because if so, then you're essentially saying, "pick things players will know," which I agree with. But if "learned amateur," is meant to refer to some standard outside of quizbowl, you're going to have a hard time figuring out what that actually is.

If you've heard of something and its not in a field you study professionally, then odds are a decent number of other people in quizbowl have as well. If some who had read a half dozen introductory books on the subject would not be helped by a clue (at least thinking "The X effect, I should know where this going but am blanking") or could not produce an answer, then it is probably too hard at all bust the highest levels of quizbowl, if then.

People who read a dozen introductory science books and are not actually studying science are quite rare.

Also, in my experience, it is also not very fun to play on questions where you don't have much of a chance of knowing the answer outside you specialty or getting anything useful out of in the first 5 linesl; I for one would much rather sacrifice my one free tossup a round of insane deep knowledge for having a fighting chance at the majority of questions in the round.

It doesn't have to be an either-or thing; you can have your deep Assyrian knowledge buzz on something that someone else might get later in the question.

[Important Bird Area]

I think the right move is to reward people who have taken advanced undergraduate courses in a subject (we've seen this a lot lately with the use of class notes and syllabi to find clues for science questions). There aren't enough grad students around in any given subject area to make actual grad-school content worthwhile, except maybe as tossup leadins at the very hardest tournaments.

[No Rules Westbrook]

It's all nice and well to try to reward people who've taken undergrad classes, but outside of science...you have to take into account that undergrad classes are a pretty poor source of qb information. Unfortunately, university courses often do not do a very good job of catering to the demands of playing qb - at best, a lot of times, they may introduce you to the name of something that gets asked about but often they won't even do that. Lots of people here, I'm sure, can attest to the fact that they've taken some undergrad course on point and never been introduced to quite a few of the clues and/or answers that appear in QB - that's just a reflection of the fact that qb is a unique game that doesn't mirror academic study.

I think it's a good idea to look at course syllabi and notes for guidance on clues and answers. But, I want to qualify attempts to use "what undergraduates learn" as any kind of touching stone...because, while the basic subject matter of qb may be introduced in some undergrad courses, there is relatively little chance that most of what it would help to know for qb purposes is covered.

[Skepticism and Animal Feed]

Also there is no one standard thing that undergraduates learn.

[Sargon]

Most quizbowlers are playing most questions on things they will not have done serious academic work in.

If you've taken history classes, you've done some academic work in that area. Obviously, a lot more of what people know just comes from outside reading, reading packets, and so on, but I don't think that's too much of a problem. Science tends to be different because unlike literature and history, which science players read on their own time, non-scientists tend not to know much about science (with some exceptions).

I should clarify, by "serious academic work" I mean several upper level classes, not one or two intro courses. My intro astronomy or Economics Statistics classes ought not to count as "serious" study of either field.

I agree science is its own critter and works differently; my comments are meant largely for non-science questions. However, it is probably worth discussion why quizbowl accepts a significant science distribution even though there are high barriers to entry for non-science people, but rejects a large distributions in other fields, such as language, on what seems to be the same grounds. It strikes me that not just linguistic theory, but concrete things like the Latin third declension, the French passee sample or the Arabic alphabet could be askable if people put their minds to it. Non language people would groan as non science people groan at a functional group tossup, but there are enough people who know foreign languages for appropriately written questions to not to go dead in most rooms, and once they started coming up, people would learn to pick things up at the giveaway (I hear consonant stems and Latin, I say "third declension" and get ten points). I'm not saying I support removing science or adding 3/3 languages, but it is something that probably needs to be worked out on a philosophical level.

Hence, I think the aim of the question writer, especially in answer selection, should be to keep answers to what a learned amateur might reasonably know, which incidentally is much easier to gauge since most people are reasonably learned in a lot more fields than the ones they know at an expert level.

When you say "learned amateur," do you just mean people who play quizbowl and know things they're not experts in? Because if so, then you're essentially saying, "pick things players will know," which I agree with. But if "learned amateur," is meant to refer to some standard outside of quizbowl, you're going to have a hard time figuring out what that actually is.

What I mean is mostly the type of knowledge. People without formal training tend to have little knowledge of secondary literature. If you read a few books on Mesopotamia, you probably won't know that names of any of the great Assyriologists of the past century; I only started learning the names when I was writing my B.A. theis, and then only in the area my thesis was on. Likewise, if you read a book like the Georgics, you will probably not know who are the famous commentators on it are, unless you are taking a class and someone is guiding you as to which articles and books to read. I even read the whole thing in Latin for a class and wrote a paper on it and still could not name a single Georgics secondary author of note.

However, primary things that you probably would not have heard of without reading a journal article or a technical publication would probably also qualify unknowable by an amateur. Thus, including information about Sargon II of Assyria from his letter to Ashur, which is famous and important in the field, but is only obtainable in a German translation, would probably be a bad idea. One would be better advised to mention that he wrote the text or its basic plot, which comes up even in intro level books, than details that would require actually having read it. Similarly the extant text of the Tiglath-Pileser I epic, while translated into English, is only available in a journal article, so non-assyriologists almost certainly would not have read it.

If you've heard of something and its not in a field you study professionally, then odds are a decent number of other people in quizbowl have as well. If some who had read a half dozen introductory books on the subject would not be helped by a clue (at least thinking "The X effect, I should know where this going but am blanking") or could not produce an answer, then it is probably too hard at all bust the highest levels of quizbowl, if then.

People who read a dozen introductory science books and are not actually studying science are quite rare.

Agreed. A different standard is probably in order for science questions (though I have read at least one intro science book for fun, and a sizable number of wikepedia articles ).

Also, in my experience, it is also not very fun to play on questions where you don't have much of a chance of knowing the answer outside you specialty or getting anything useful out of in the first 5 linesl; I for one would much rather sacrifice my one free tossup a round of insane deep knowledge for having a fighting chance at the majority of questions in the round.

It doesn't have to be an either-or thing; you can have your deep Assyrian knowledge buzz on something that someone else might get later in the question.

What I mean is that I don't think it is worthwhile to fill one or two lines with clues about famous articles, Assyriologists or technical linguistic things that would only be helpful if you have a few years of Akkadian under your belt. Putting those things in will ensure I get the question, but I would rather get the question based on direct knowledge, even if this means I might occasionally lose out to a non-Assyriologist who happens to know a lot about the particular topic. Thus I would rather have a tossup on Hammurabi mention his defeat of an Elamite-Eshnunnan coalition, or his dealings with Zimri-lim of Mari and Rim-Sin I of Larsa (both of whom, incidentally, probably are in the top 10 or so most famous Mesoptoamian rulers), rather than technical details about his code, or his canal inscription, both of which would be intimately well known to anyone who had passed first year Akkadian.

[Captain Sinico]

It's all nice and well to try to reward people who've taken undergrad classes, but outside of science...you have to take into account that undergrad classes are a pretty poor source of qb information. Unfortunately, university courses often do not do a very good job of catering to the demands of playing qb - at best, a lot of times, they may introduce you to the name of something that gets asked about but often they won't even do that. Lots of people here, I'm sure, can attest to the fact that they've taken some undergrad course on point and never been introduced to quite a few of the clues and/or answers that appear in QB - that's just a reflection of the fact that qb is a unique game that doesn't mirror academic study.

I think it's a good idea to look at course syllabi and notes for guidance on clues and answers. But, I want to qualify attempts to use "what undergraduates learn" as any kind of touching stone...because, while the basic subject matter of qb may be introduced in some undergrad courses, there is relatively little chance that most of what it would help to know for qb purposes is covered.

Unless I've misunderstood this, Ryan Westbrook's argument is: "We shouldn't/can't use the content of undergraduate courses to determine what comes up in quizbowl because what comes up in quizbowl is currently different from what comes up in undergraduate courses." I hope a better argument than that can be made.

MaS

[Cheynem]

I'll bite here. One of the difficulties with using college courses for purposes of quizbowl writing is that depending on the course, different professors have different mindsets for what their courses should entail. Let's take history, the disciplines I have the most experience in. I took one undergrad history course which was very conducive to quizbowl--lots of laws, lots of names, lots of dates. I took another where the professor was more concerned about "big picture themes," so we didn't really have those sorts of lectures and would read stuff like monographs about what the average day in the life of a French peasant was like. The latter is a perfectly legitimate way to teach and study history, but it's a little difficult to write quizbowl questions from (aside from maybe some concept questions).

[No Rules Westbrook]

The latter Mike is right. My argument is that trying to draw connections between the undergraduate curriculum and quizbowl, in just about any subject but science maybe, is a fool's errand. You can't write questions about the lives of 15th century French peasants or child mortality demographics in Puritan New England - at least I damn well don't want to see them. Yeah, there are a handful of undergrad classes and textbooks that do a good job of being a complete survey of important names, dates, people, events, etc. - but even in those relatively rare cases, you're probably not going to be taught many of the clues you need to know to do well at QB.

Generally, I think people are too obsessed with trying to endow quizbowl with some "larger meaning" by relating the canon to some silly standard like "what educated undergraduates know" or whatever. Said standards are always either blatantly inaccurate or can't possibly be applied across the board, and in any event they're not very relevant to the actual game of quizbowl. I'd much rather see people just put their heads down and be professional quizbowl players, for its own sake - for the sake of the game, without any made-up external justifications.

[Skepticism and Animal Feed]

Again, one of the advantages of Ryan's approach here is that it makes it possible to honestly tell players "if you want to be good, just (i) show up to practice; (ii) play tournaments; and (iii) read packets". It gives them a simple, easy-to-follow route to sure quizbowl success. Without this approach, a new player must bear tremendous information costs; before he can begin learning things he must ascertain by what standard importance is judged, and then hope that this does not change with turnover among the editorial class.

Nor does Ryan's approach invariably lead to a completely arbitrary canon made up solely of what prolific writers and editors adore. The canon is constantly expanding; new lead-ins, bonus prompts, and bonus third parts must of necessity be born every tournament. The canon does not expand by itself, but rather by the hand of experienced writers. They can certainly choose to expand the canon in a way that brings new focus on what is "objectively important", if they believe in such a thing.

[Mechanical Beasts]

They can certainly choose to expand the canon in a way that brings new focus on what is "objectively important", if they believe in such a thing.

Okay; if they don't believe in objective importance (somehow!) then by what rationale does the canon expand save their whim? Perhaps what's most likely to be converted [even though not in the canon]? Well, I'd say that the things that aren't in the canon already that are more likely converted are things covered in curricula. What's the alternative--things covered in the Uncle Scrooge Adventures about the Kalevala? I'm pretty sure more people have undergraduate classes in common than that comic. Indeed, undergraduate classes on x might be the best common point we have.

Ryan, I don't think anyone who suggests undergraduate curricula should play a role believes that they should be a sufficient condition--that could, perhaps, lead to scary who-am-I tossups where the answer is "a peasant in Genoa who needs salt" or something. (I don't think anyone would actually do this, but I don't think we need to make that however tiny leap of faith in order to have an argument here.) Rather, I think we're saying that, except in some reasonable instances, it should be a necessary one. Perhaps a pretty-close-to-necessary condition.

[Skepticism and Animal Feed]

Okay; if they don't believe in objective importance (somehow!) then by what rationale [i|does[/i] the canon expand save their whim? Perhaps what's most likely to be converted [even though not in the canon]?

Sure, clues likely to be useful are to be favored over clues less likely to be useful. I don't think even the most hardcore Westbrookian would disagree with you on that.

Rather, I think we're saying that, except in some reasonable instances, it should be a necessary one. Perhaps a pretty-close-to-necessary condition.

I must condemn this suggestion. Again, my belief is that there is less commonality in the curricula than the Weinerists believe there to be, and the result will be either an asphyxiatingly small canon, or a canon that unfairly and arbitrarily favors people raised in certain academic traditions, certain states, certain types of school, etc.

Alternatively, this could be more politely phrased as saying that your "reasonable instances" exception would have to be expanded quite widely.

[Matt Weiner]

Again, my belief is that there is less commonality in the curricula than the Weinerists believe there to be

Paul's or Andy's opinions are not mine. I think that trying to argue for a limiting of clues based on what isn't taught at a hypothetical standard curriculum is just the inverse of arguing that some unanswered tossup is really important. Clues should be things that are worth knowing that produce the appropriate frequency of correct buzzes for where they are used in a question of a certain difficulty level. One becomes able to predict this, while writing, by participating in games as a player or moderator and developing a sense of what people know. To diverge from this approach by constantly checking for things on a syllabus seems as silly to me as the opposite extreme of bean-counting how many times something has come up in quizbowl before.

[No Rules Westbrook]

Look, whenever I decide to introduce new answers or clues to the canon, I always consult my personal gauge on how "objectively important" those things are in whatever field of study you're talking about. But, that's different than arguing that such a standard should or can be intelligently applied to qb writing as a whole.

[Cheynem]

Yeah, Matt and Ryan are really sort of expressing the same sort of common sense approach (at least in terms of rhetoric). Good quizbowl topics are academically important and thus for the most part appear in college courses because they are indeed important. But quizbowl and academia are not the same thing--different topics and different themes are rewarded and thus it is not a perfect marriage. I don't see anyone here advocating for a rigid "This must appear in five undergrad course syllabi!" approach, but I do see a lot of people saying "This has to have some importance in an academic field--it probably will be mentioned or referenced in a few major texts that you read an undergrad."

[Mechanical Beasts]

Paul's or Andy's opinions are not mine. I think that trying to argue for a limiting of clues based on what isn't taught at a hypothetical standard curriculum is just the inverse of arguing that some unanswered tossup is really important. Clues should be things that are worth knowing that produce the appropriate frequency of correct buzzes for where they are used in a question of a certain difficulty level. One becomes able to predict this, while writing, by participating in games as a player or moderator and developing a sense of what people know. To diverge from this approach by constantly checking for things on a syllabus seems as silly to me as the opposite extreme of bean-counting how many times something has come up in quizbowl before.

Note that I'm not trying to make suppositions about a standard curriculum. To drag out the poor, tired Autler-Townes effect--you'd get a vastly disproportionate idea of the "worth knowing" part of your "Clues should be..." standard given the rate at which people buzz on that clue.

To make it clear what I do not believe: I do not believe that the phrase "Autler-Townes effect" should be placed really early because of its low (or, perhaps, "peripheral," to note that AT might be damn important to the small population of people who are really up on it) academic importance. That's dumb because that would probably fail to "produce the appropriate frequency of correct buzzes" etc.

The place where this is important, rather, lies in saying "is this clue, which has a property that makes it inherently memorable [for example, a funny name, or being named at all, or featuring Tim Tebow exploding], better suited for the canon than something less memorable but much more academically important?" The former is as good a quizbowl clue as the latter, but for a non-academic reason. Neither would produce substantially higher initial conversion rates--since people would only know them from previous exposure to things with funny names or to meaningful academic study, or something.

[grapesmoker]

Is quizbowl a game in which topics of academic importance come up or isn't it? Because if it is, you're not going to be able to divorce it from the undergraduate curriculum, which, despite what Bruce may think, has a lot of commonalities in it. If the only reason we're asking about the Treaty of Tilsit is because it produces the correct frequency of buzzes, then let's just pack the whole thing in and go home. Trying to guess what people will know without knowing how important those topics are and where they are studied is a fool's errand.