Back to Basics

[magin]

"Quizbowl, thou art sick!
The invisible worm,
That flies in the night,
In the howling storm,

Has found out thy bed
Of crimson joy;
And his packet archive
Does thy life destroy."

- from Songs of Quizbowl Experience

Other threads have alluded to this, but I wanted to start an explicit thread about where questions should come from and the siren song of the packet archive. Question writing is a kind of writing, and all good writing is written for a specific audience. However, I feel that the packet archive is tempting people to write questions for the audience of players who study old packets instead of the audience of people who understand a topic. I'd like to argue that the audience for quizbowl questions should be the latter, and that writing for the former audience makes difficulty shoot up wildly and does not reward understanding as well as it should.

The more understanding a writer has of a topic, the easier time he or she has of writing a tossup that rewards deep understanding of it. However, most of us don't have understanding of many areas that come up in quizbowl, but we want to write good questions in those areas. So, where do we turn? It's very tempting to go look at past questions written by people we believe to be good writers, and emulate them. However, if we don't understand the clues in the past questions (on a deeper level than word associations), attempting to use similar clues will often reward knowledge of old packets, not necessarily understanding of the answer.

Moreover, this causes difficulty to start escalating past what people are likely to know. I'd like to use the tossup on The Exhumation of the Mastodon at MO as an example. I used it as a clue for a bonus part on CW Peale in my MO packet last year, and then it came up in a tossup Gautam wrote on the Peales for Gaddis II (it was also a clue for a tossup on Peale at the 2006 ACF Nationals). To my mind, it can really only be defended as an answer by saying "it's come up a couple of times before recently, and therefore people will/should know it." However, let's look at whether people can possibly have encountered this painting outside of quizbowl; I highly doubt any player at MO has (unless qb is chock-full of Peale scholars, which I doubt). Since it's unlikely anyone has any sort of non-quizbowl understanding of that painting, it's highly dubious as a tossup answer, to my mind. Also, it's just incredibly hard to people who don't study old packets, which is a lot of people. In that vein, your easy part should be easy independent of knowing old packets (saying "Machado de Assis comes up all the time" does not make him an easy part for anything but very hard tournaments; it just ensures that plenty of smart people will not get points on your bonus asking for him and two of his works).

Thus, relying on previous packets to write questions creates an artificial difficulty barrier for new players/players who don't study those packets, and unduly privileges those who do. "But Jonathan," you may ask. "How are we to tell players to improve as players and writers if not by sending them to the packet archive?" An excellent question. Here are some ways:

For writers: attempt to understand every clue you use in a question and attempt to write questions that reflect this understanding. That usually means finding good primary or secondary sources: textbooks, online class notes, Google Books, JSTOR and other databases, the Grove dictionary of music, and so on (there's a stickied thread somewhere with many links to good sources. Take advantage of them).

For players: Read a book. Read about history and RMP, and attempt to understand science. Look at/listen to some art. Read about art/science/social science. Use those good sources to learn interesting things. It's pretty fun.

The best questions come from people who understand what they write about; if you don't attempt to understand something, you'll have trouble writing questions that are accessible to people who have understanding. So, let's try to get back to the basics of good quizbowl; it makes for a much healthier game.

[Cheynem]

Heh, I actually studied Peale in my public history class. Before quizbowl, I was unaware that he was actually notable as a painter--we studied him more for his quasi-cootish museum/anthropology proclivities.

[Strongside]

I think this is an interesting thread, and I know some people on the IRC were discussing this recently. I have a few things to say.

I will admit that when I write questions, I use a question archive. I feel question archives are good for writing because you can search for how many times, and what context whatever you are writing on has come up. I know that this helps me a lot with clue selection and pyramidality. I do agree that writing entire questions straight out of the archive is a poor idea, and doing a little bit of research/searching is necessary to write good questions.

One thing that ought to be mentioned is that even the best quiz bowl players know a small percentage of the clues that come up in a given regular or difficult tournament. One just has to look at a tournament like Chicago Open, and see how many of the clues in power that they know. I mentioned this after Chicago Open, but the stats from that tournament seemed to indicate that very few, if any people in quiz bowl could average over 15 points (a simple majority) per bonus playing solo at Chicago Open this year.

There was discussion on the IRC that quiz bowl is very fake in some ways, and I agree. Quiz bowl is basically stuffing as much knowledge into your head, and then applying that knowledge when you play a tournament. You don't really need deep knowledge or a conceptual understanding of things to become a good player.

There is such a wide array of things that come up in quiz bowl, that one can only learn so much by reading books, and learning stuff in class. Studying old questions and packet archives is the most efficient way to get better in my experience.

Things change in difficulty over time, but I do agree we should be careful not to make something that otherwise might be hard an easy part at an easy tournament, just because it has come up a bunch of times.

Also, some people may argue that individuals who don't look at old packets and packet archives are at a disadvantage, and I agree. I don't feel that quiz bowl should go out of its way to avoid rewarding people who look at question archives.

People who tend to look at archives in their free time are serious players who want to get better at quiz bowl, and I see no problem in rewarding them. There are many free archives and question available to look at on the internet, and one can also create their own archive if they want.

[Cheynem]

I admit to being a packet archive old fogey, but I really don't see how many times "something has come up" is that important. If it's worth writing on, write on it. If it isn't, then don't (or write on it less). I agree that the packet archive can provide important information about pyramidality (so you don't reuse lead-ins too many times) or difficulty.

[Skepticism and Animal Feed]

For writers: attempt to understand every clue you use in a question and attempt to write questions that reflect this understanding. That usually means finding good primary or secondary sources: textbooks, online class notes, Google Books, JSTOR and other databases, the Grove dictionary of music, and so on (there's a stickied thread somewhere with many links to good sources. Take advantage of them).

For players: Read a book. Read about history and RMP, and attempt to understand science. Look at/listen to some art. Read about art/science/social science. Use those good sources to learn interesting things. It's pretty fun.

The best questions come from people who understand what they write about; if you don't attempt to understand something, you'll have trouble writing questions that are accessible to people who have understanding. So, let's try to get back to the basics of good quizbowl; it makes for a much healthier game.

You are imposing high information costs on players and writers. Now, not only do they have to incur the cost of learning things (which is inherent to quizbowl and unavoidable), but they also have to incur the cost of knowing WHAT to learn. That is, they need to perform the following tasks:

(1) Determine the standards by which various editors determine what is or isn't "academically important";
(2) Apply those standards;
(3) Seek out the results of applying those standards; and
(4) learn things from the results

This is a cumbersome process and basically privileges people with:

(1) similar academic backgrounds to editors;
(2) similar interests to editors;
(3) people with more resources (i.e., who went to better schools, or schools that were biased in the same way that editors or editor's schools are biased).

Whereas with the "read old packets" approach, we minimize people's costs to just
(1) find old packet (conveniently on the internet in multiple places); and
(2) learn things from old packet.

So that's my basic "the Westbrookian approach is the fairest approach for new players" speech. I've made it loads of times.

But in addition, I will also accuse you of making a straw man. You make several faulty assumptions, including:

(1) that people who learn things from a packet won't say "oh, that's cool", and then go look it up in non-packet sources;
(2) that writers won't add clues from outside of the canon

[Skepticism and Animal Feed]

I admit to being a packet archive old fogey, but I really don't see how many times "something has come up" is that important. If it's worth writing on, write on it. .

Because the "importance" of something is often not objectively determinable.

[grapesmoker]

There's a lot that should really be responded to in Brendan's post, but for now I'll just confine myself to saying that I know that I'm personally someone who has not gained all that much knowledge from studying packets. In fact, I almost never read old packets outside of practice unless it's to comment on questions. I'm not saying this to toot my own horn but rather to point out that it's not at all necessary to read archives in order to improve. In fact, back in around aught-one (that's like the quizbowl neolithic era, for some of you), there were no packet archives as such (certainly not online ones) and yet somehow good players still became good, despite the fact that the game was much more prone to packet-memorization back then than it is now (although we're going in the wrong direction for sure).

[Cheynem]

Well...okay, yeah, I see what you mean by the "importance" of it not being objectively determinable. But this is a weaselling out. Surely someone had to decide that Blank X was important enough to write about in the first place right? That explains the other Blank X and Blank X tossups. Surely we haven't lost that ability and must only rely on old packets, right?

Like I am not a physicist and I'd surely write crappy physics tossups, but I'd like to think I could at least nail down what's important by consulting a physics textbook or website without having to delve into the packet archive.

[Skepticism and Animal Feed]

Like I am not a physicist and I'd surely write crappy physics tossups, but I'd like to think I could at least nail down what's important by consulting a physics textbook or website without having to delve into the packet archive.

Maybe it's different in science, but my feeling is that in a lot of non-science subjects, what's "important" or what is "taught in classes" varries widely depending on the institution or instructor. Some people read x in class, some people read y. The concept of a unified obejctive canon of things that can sustain a large answer space is, in my view, a fairy tale, and basing quizbowl around this concept would be like basing a tournament around the assumption that a unicorn will bring teams to the tournament site.

EDIT: And, of course, some (probably most, for any given subject) people don't major in that subject and take no classes at all, but rather learn through self-directed independent study.

[Theory Of The Leisure Flask]

back in around aught-one (that's like the quizbowl neolithic era, for some of you), there were no packet archives as such (certainly not online ones)

Um, what about the Stanford Archives? They were around back then.

[cvdwightw]

Back when I wrote my half of Science Non-Strosity, the only thing I used the packet archive for was "how often has this clue come up?" The idea being that people with primary knowledge of the topic (i.e. the clue has shown up not at all or only a few times) should get it before people who learned the topic from old packets. Obviously this has some inherent flaws, but I think it's a reasonable way to order two clues that you think are roughly equal in difficulty. The main message here is that you shouldn't use the packet archive for old clues; you should use the packet archive to make sure that people with more fraudulent ("old-packet") knowledge aren't outbuzzing the people with non-old-packet knowledge, if two clues are roughly equal in terms of "amount of non-old-packet knowledge needed to buzz."

I've probably written this elsewhere, but one thing that I've noticed in the science and literature questions especially is an ironically accelerated trickle-down effect when something gets mentioned as being "ridiculously difficult." That is, the mere fact that it was ridiculously difficult at higher levels makes it somehow more acceptable for lower levels than things that are less difficult at higher levels. As one example, electrospray ionization is not a novice-level third part on mass spectrometry, no matter how over-discussed it was after ACF Nationals 2007.

[theMoMA]

I think Jonathan is misguided when he says that going to the archive for harder answers is not a good idea. It's actually a really good idea, when done right. Take Jonathan's example of The Exhumation of the Mastodon, which is the most important painting of an artist who actually does not come up in quizbowl as often as he is encountered in academia (for example, there are roughly as many JSTOR hits for "Charles Willson Peale" as there are for "Thomas Cole"). That painting is one of the most famous and important American paintings of the 1800s, believe it or not. For example, it's one of the representative paintings in the glossy pages of the B volume of the early American Norton Anthology. It's a legitimately cool and notable thing that is now also askable because of the diffusion of knowledge from old tournaments. To me, that's a rousing success for the archive.

Contrast this with something that I wrote for MO, the tossup on John Gay's most studied poem, Trivia, or the Art of Walking the Streets of London. Now, I have encountered this in classes three times, and I wanted to make sure my experience was not uncommon, so I checked to see if it pops up on Brit poetry syllabuses. It does. It's also a legitimately cool thing by a notable author who hasn't come up very much lately. From how the tossup played, maybe the poem is just not as real-world known as I thought, and should have been confined to a bonus part. Based on the reaction I heard from this tossup, most people didn't like it. Jonathan was actually very annoyed, which I find a bit puzzling. If we're trying to figure out what's important based on our experiences outside the archive, we're going to make a few mistakes. That's just the nature of writing outside the canon.

I really appreciate when people try to be innovative in their answers, and write on things that they think people know from outside of quizbowl. But I guess what I'm trying to say is that it cuts both ways. What's come up before represents a huge and disparate mass of facts, and some of those things are legitimately underasked and should surface via the Westbrook method. There are also things that should come up that just never have, and it's important to find them and introduce them as well. Just keep in mind that you can never trickle down answers from clues to bonus parts to tossup answers (i.e. responsibly expand the canon, by most people's definition) without writing on what's come up.

[women, fire and dangerous things]

For what it's worth, I've encountered both Exhuming the First American Mastodon and Gay's Trivia outside of Quizbowl, and I'm not nearly of the calibre of many of the players at MO. I don't study art or literature at school, but I get the sense that both of these are fairly important within the academy. While I agree with Jonathan's general point, it's also possible to hypercorrect, as it were, by avoiding asking about things that are legitimately important and deserve to come up just because one gets the impression that people are asking about them merely because they've seen them in old packets (which may be the case, but to assume that it is begs the question; maybe they saw them in an old packet, looked them up, and discovered that they're interesting and important and worthy of being asked about). Nevertheless, the fact that this discussion is happening is a good thing, since it raises awareness of the problem that Jonathan mentions, which is, I think, far more toxic than the problem I just mentioned.

[magin]

My problem with the tossups on The Exhumation of the Mastodon and Gay's Trivia is not that those answers are unimportant. I think they suffered from the problem of the field not knowing more than one or two clues about them. I sympathize with wanting to include important answers in the canon, but I don't think writing a tossup on something that most people only know one or two things about (if that; I suspect that the vast majority of players in MO knew nothing about either answer).

I would have preferred to see tossups about Peale and Gay (perhaps devoting the first half or two-thirds to clues about Trivia). That way, the tossups would better test differences in knowledge that players are likely to have. In any case, my point is that academic importance isn't the only criteria to consider; writers should also consider "how many clues will players realistically know about this thing?", and if the answer is small, then that answer just won't make a good tossup, pragmatically.

[women, fire and dangerous things]

My problem with the tossups on The Exhumation of the Mastodon and Gay's Trivia is not that those answers are unimportant. I think they suffered from the problem of the field not knowing more than one or two clues about them. I sympathize with wanting to include important answers in the canon, but I don't think writing a tossup on something that most people only know one or two things about (if that; I suspect that the vast majority of players in MO knew nothing about either answer).

I would have preferred to see tossups about Peale and Gay (perhaps devoting the first half or two-thirds to clues about Trivia). That way, the tossups would better test differences in knowledge that players are likely to have. In any case, my point is that academic importance isn't the only criteria to consider; writers should also consider "how many clues will players realistically know about this thing?", and if the answer is small, then that answer just won't make a good tossup, pragmatically.

If that's what you're arguing, then I agree. (Though it's often hard to determine how many players will know about something, particularly from a vantage point outside of the academy; as long as question writers make a good-faith attempt to do this, I'm happy.)

[magin]

It's a legitimately cool and notable thing that is now also askable because of the diffusion of knowledge from old tournaments. To me, that's a rousing success for the archive.

I'd like to address this point, which I don't agree with. To me, this is a little too close to the idea that once something has come up enough times, no matter how difficult it may actually be, it's askable because players who read old packets will be able to convert it. I don't know if that's what Andrew means, but I don't think that's a good model for several reasons.

One, it presupposes that a question being answered is a good thing, even if it's only answered because it's come up before. I worry that that mindset can lead to really hard triply eponymous equations from obscure physics that even advanced physicists have never heard of being deemed acceptable answers because they have come up X times in previous packets, even if no one had any idea what they were in past tournaments.

Two, it dramatically increases the difficulty for teams that don't study old packets. I can easily imagine teams consisting of four smart, well-read, intellectually curious players being baffled by tossups and bonuses full of answers that they have a very slight chance of knowing (or even having heard of) through undergraduate study and reasonable intellectual curiosity.

Surely, people like Bruce will argue that telling everyone to go to the packet archive is an equalizing force, but it's also a fairly artificial way to learn about things. I would much rather people pick tossup answers based on the principle of "what are people reasonably likely to know?" For an example, I'm a big fan of Henry Ossawa Tanner, and wrote a tossup on him for Gaddis I. However, I wouldn't write a tossup on him for anything but a very hard experimental tournament, because even at ACF Nationals or NAQT ICT or even Chicago Open, I don't think enough players at those tournaments are likely to know him (or, if they've heard of him, know enough about him to have the tossup differentiate knowledge effectively). Whereas, it's likely that enough players are likely to know Thomas Eakins from non-packet archive knowledge to select him as a tossup answer for most college tournaments. I strongly believe that people can and must consider what players are likely to know from non-packet archive sources when choosing their answers and clues.

[theMoMA]

Actually, I was arguing that things that are legitimately important become notable to enough people in quizbowl that they can be tossup answers. For a long time, we've considered introducing things in bonus parts and as clues, and eventually writing on them if they're important enough, to be the way to expand the canon responsibly. I'm not of the mind that everything that can and should be asked about has already come up, so I think this is the best mechanism for expanding our horizons.

I don't think anyone disagrees that we need to limit writing about things just because they've come up before. And things that have come up at recent hard tournaments are trickling down into regular tournaments at a rate that suggests laziness in picking novel answers. Both are trends that can be combated without claiming that quizbowl can never build upon what's already come up. The problem is not that there is an archive, and the problem is not that people write on what's come up. The problem is that writers are using the archive as a crutch for writing easy and stale questions instead of as a tool to find interesting and fresh topics.

[grapesmoker]

The much-anticipated response:

I will admit that when I write questions, I use a question archive. I feel question archives are good for writing because you can search for how many times, and what context whatever you are writing on has come up. I know that this helps me a lot with clue selection and pyramidality. I do agree that writing entire questions straight out of the archive is a poor idea, and doing a little bit of research/searching is necessary to write good questions.

Writing good questions requires more than just a little bit of research. I don't want to pick on you, but I've read the questions you've posted in various threads, and they're terrible. They demonstrate a complete absence of understanding of the concept of useful clues; for the most part they are just lists of things in a vaguely descending order of difficulty of association. Sure, you can use the archives to give yourself some idea about what kinds of clues other people have used and their context, but for the most part, the archives are useless for writing things that aren't just rehashes of previous information.

One thing that ought to be mentioned is that even the best quiz bowl players know a small percentage of the clues that come up in a given regular or difficult tournament. One just has to look at a tournament like Chicago Open, and see how many of the clues in power that they know. I mentioned this after Chicago Open, but the stats from that tournament seemed to indicate that very few, if any people in quiz bowl could average over 15 points (a simple majority) per bonus playing solo at Chicago Open this year.

So? I can't see what that has to do with anything or what it proves about question writing. I have my own theories about what Chicago Open should and should not be, but that's neither here nor there.

There was discussion on the IRC that quiz bowl is very fake in some ways, and I agree. Quiz bowl is basically stuffing as much knowledge into your head, and then applying that knowledge when you play a tournament. You don't really need deep knowledge or a conceptual understanding of things to become a good player.

Ok, that blows. You should have to have some kind of knowledge of stuff beyond word associations in order to be a good player. I think good quizbowl should actively reward people's understanding. It may be the case that today one can become good simply by perusing old packets, but this is a problem and needs to be fixed.

There is such a wide array of things that come up in quiz bowl, that one can only learn so much by reading books, and learning stuff in class. Studying old questions and packet archives is the most efficient way to get better in my experience.

Then let's just fold the whole thing up and go home. When you're rewarded more for memorizing lists of Murakami works than for reading Faulkner, we're no longer engaged in anything other than masturbatory regurgitation. That's not an academic game worth playing.

Also, some people may argue that individuals who don't look at old packets and packet archives are at a disadvantage, and I agree. I don't feel that quiz bowl should go out of its way to avoid rewarding people who look at question archives.

Actually, I think quizbowl should actively punish people whose knowledge comes from reading packet archives. People whose knowledge of physics comes from associating names with second-order effects have no business getting physics questions before people who study physics. Enjoy ACF Regionals!

People who tend to look at archives in their free time are serious players who want to get better at quiz bowl, and I see no problem in rewarding them. There are many free archives and question available to look at on the internet, and one can also create their own archive if they want.

This is just awful circular reasoning. If the only measure of how good one is is one's recall of stuff that has come up before, then this game is stupid and we should all stop doing it. By rewarding this kind of "knowledge" we are incentivizing the wrong kind of behavior that leads to getting better, and I won't have anything to do with it, and neither will any tournament that I'm a part of.

[magin]

Actually, I was arguing that things that are legitimately important become notable to enough people in quizbowl that they can be tossup answers. For a long time, we've considered introducing things in bonus parts and as clues, and eventually writing on them if they're important enough, to be the way to expand the canon responsibly. I'm not of the mind that everything that can and should be asked about has already come up, so I think this is the best mechanism for expanding our horizons.

I don't think anyone disagrees that we need to limit writing about things just because they've come up before. And things that have come up at recent hard tournaments are trickling down into regular tournaments at a rate that suggests laziness in picking novel answers. Both are trends that can be combated without claiming that quizbowl can never build upon what's already come up. The problem is not that there is an archive, and the problem is not that people write on what's come up. The problem is that writers are using the archive as a crutch for writing easy and stale questions instead of as a tool to find interesting and fresh topics.

Certainly, my position is not that we can never ask about new things. However, to ask about something new, we should try to determine whether people know or understand clues about it besides what they know from the packet archive. If enough people playing the tournament know something about a underasked topic besides the clues from the packet archive, then sure, write a question about it. Similarly, it's generally a good idea to include underasked clues about an important topic in a tossup on something that people will be able to answer (and know more than one or two clues about). For instance, instead of writing a tossup on Intruder in the Dust, it's almost certainly a better idea for the vast majority of college tournaments to write a tossup on Faulkner that uses good clues about Intruder in the Dust. And so on.

[grapesmoker]

In case it wasn't clear already: difficulty creep on lower-level tournaments based on stuff that comes up at harder events is an enormous problem that even ACF Fall did not entirely steer clear of (how many people answered that bonus part on VDJ recombination?). It doesn't matter how many times it comes up in Crazy Literature Singles of Chicago Open, The Forty Days of Musa Dagh will never be an acceptable answer at lower levels. People really, really need to internalize this because this percolation has the potential to ruin novice and regular events.

[millionwaves]

For writers: attempt to understand every clue you use in a question and attempt to write questions that reflect this understanding.

I think this is the most important part of what Jonathan wrote above, putting to the side his extremely important point about the packet archives. This game isn't about memorizing words and what goes with them with strength s, it's about learning and understanding things and being rewarded for that, at least in some abstract way. I think that that is the single most important dimension of quizbowl, and our first priority as writers should be focused around making sure that this remains the case.

[theMoMA]

I have a bit of a problem with the idea that the important thing is knowledge independent of quizbowl. This seems like a particularly poor substitution for an abstract idea of "importance." A lot of the stuff that I know is based on things I learned reading packets or writing questions, but that doesn't make the understanding any less real, or the subject any less important.

I think our standard for asking things should always be twofold: is it gettable, and is it academically important. Even if most players were introduced to a topic through quizbowl, if people think the topic is important, it should come up. We need to write our questions on subjects that don't privilege playing or studying recent packets over understanding, and our questions should strive to reward comprehension of a subject over word association. But like I said before, word association and rote knowledge is still knowledge, and we all depend on it in every game of quizbowl we play. We should make sure that people with understanding have the advantage, but that doesn't mean that we have to denigrate buzzes on less thorough knowledge; we just need to write questions so those buzzes come at an appropriate time.

[Sir Thopas]

Without delving too much into the arguments in this thread, which others have argued far better than I could, I'd just like to remind everyone that choosing academically important topics can be difficult for those who haven't been steeped in academia for some amount of time. While I (for one) of course aim to choose important topics, sometimes this is very difficult without consulting the packet archive.

[Auroni]

I'm going to agree with Guy here: when writing non-biology sciences, I have to use the packet archive to pick answers and sometimes even clues. If I did the alternative, which was to crack open a textbook, then you would receive terrible tossups on "force" and "resonance"

[grapesmoker]

I'm going to agree with Guy here: when writing non-biology sciences, I have to use the packet archive to pick answers and sometimes even clues. If I did the alternative, which was to crack open a textbook, then you would receive terrible tossups on "force" and "resonance"

There's no reason why tossups on "resonance" should be awful! In fact, I think I wrote a tossup on such a thing a long time ago. Resonances of all kinds are hugely important in physics and studied in virtually every class. I understand that it's hard to pick important topics in areas you are not an expert in; when I have to do it for bio and chem, I still look in textbooks though and not old packets, because a textbook will give me some rough idea of whether something is important or not. The worst you can do with a textbook is write a slightly awkward question on something you don't understand too well (unless you are just picking random enzymes out of a table of contents or some similar thing).

[Cheynem]

I agree with Jerry. Looking at a textbook is one way to determine what at least some practitioners in the field judge to be important. Using the archive in tandem with it is a way just to determine if you're not writing on something perhaps important but unanswerable to most quizbowlers.

[grapesmoker]

I agree with Jerry. Looking at a textbook is one way to determine what at least some practitioners in the field judge to be important. Using the archive in tandem with it is a way just to determine if you're not writing on something perhaps important but unanswerable to most quizbowlers.

It's more than that. Assume that there has never been a question on resonances ever before; resonances are still very important and a tossup on resonances is answerable at the Regionals level. So right there is one example of something that you'd never think of to write on if all you did was check archives but which would be appropriate for a wide range of tournaments. If you opened a standard physics text to the table of contents and simply picked the thing that comes up most often, you almost certainly do better than browsing archives.

[Cheynem]

I see your point. I was just trying to ward off folks that picked an author who had a nice spread, for example, in the Norton Anthology of American Literature to write a tossup on without perhaps checking to see if the author had even been mentioned in quizbowl before.

[Strongside]

The much-anticipated response:

I will admit that when I write questions, I use a question archive. I feel question archives are good for writing because you can search for how many times, and what context whatever you are writing on has come up. I know that this helps me a lot with clue selection and pyramidality. I do agree that writing entire questions straight out of the archive is a poor idea, and doing a little bit of research/searching is necessary to write good questions.

Writing good questions requires more than just a little bit of research. I don't want to pick on you, but I've read the questions you've posted in various threads, and they're terrible. They demonstrate a complete absence of understanding of the concept of useful clues; for the most part they are just lists of things in a vaguely descending order of difficulty of association. Sure, you can use the archives to give yourself some idea about what kinds of clues other people have used and their context, but for the most part, the archives are useless for writing things that aren't just rehashes of previous information.

You don't want to pick on me, so you decide to post on the forums about how my questions are terrible. Whatever. I did not spend a lot of time on those CATO/TACO questions. I didn't have a lot of time to work on them, and I wanted to give Tommy something because the tournament was in need of questions. I will admit that the physics bonus I wrote for Chicago Open wasn't very good.

I will admit I am not a great question writer, but I have definitely gotten better as a question writer though over the past year or two. Also, part of the reason for this is fundamental differences in how I feel questions should be written.

As for my writing being lists of things in a decreasing order of difficulty of association, isn't that what quiz bowl should do? Maybe I should consider paying more attention to how my clues flow, and decrease the amount of word association present.

One thing that ought to be mentioned is that even the best quiz bowl players know a small percentage of the clues that come up in a given regular or difficult tournament. One just has to look at a tournament like Chicago Open, and see how many of the clues in power that they know. I mentioned this after Chicago Open, but the stats from that tournament seemed to indicate that very few, if any people in quiz bowl could average over 15 points (a simple majority) per bonus playing solo at Chicago Open this year.

So? I can't see what that has to do with anything or what it proves about question writing. I have my own theories about what Chicago Open should and should not be, but that's neither here nor there.

This may not pertain specifically to question writing, but it shows how even the top players can't answer most of the bonuses at a hard tournament like Chicago Open. I suppose it isn't extremely relevant to this thread.

There was discussion on the IRC that quiz bowl is very fake in some ways, and I agree. Quiz bowl is basically stuffing as much knowledge into your head, and then applying that knowledge when you play a tournament. You don't really need deep knowledge or a conceptual understanding of things to become a good player.

Ok, that blows. You should have to have some kind of knowledge of stuff beyond word associations in order to be a good player. I think good quizbowl should actively reward people's understanding. It may be the case that today one can become good simply by perusing old packets, but this is a problem and needs to be fixed.

You may think that is unfortunate, but from my experience it is the truth. Your experiences may be different. How do you think this should be changed?

There is such a wide array of things that come up in quiz bowl, that one can only learn so much by reading books, and learning stuff in class. Studying old questions and packet archives is the most efficient way to get better in my experience.

Then let's just fold the whole thing up and go home. When you're rewarded more for memorizing lists of Murakami works than for reading Faulkner, we're no longer engaged in anything other than masturbatory regurgitation. That's not an academic game worth playing.

If I were to take two people of equal gender, age, and intelligence who had never played quiz bowl before, know absolutely nothing, and gave them three hours to learn about William Faulkner. One person will read As I Lay Dying during this time period, while the other will memorize character names, and plot summaries of every William Faulkner work. I predict the person who learned about the character names and plot summaries will do better than the one who read As I Lay Dying. The person who read the book will likely be able to nail any tossup on that book, but I would guess the person who learned a little about all the William Faulkner works would do better in general.

Also, some people may argue that individuals who don't look at old packets and packet archives are at a disadvantage, and I agree. I don't feel that quiz bowl should go out of its way to avoid rewarding people who look at question archives.

Actually, I think quizbowl should actively punish people whose knowledge comes from reading packet archives. People whose knowledge of physics comes from associating names with second-order effects have no business getting physics questions before people who study physics. Enjoy ACF Regionals!

Okay. This doesn't make much sense to me. I am not sure why people should be punished for looking over old tournaments, and trying to get better. If you take out the time and work hard to look over questions, I don't feel you should inherently be punished. Looking over old tournaments can only do so much though.

People whose knowledge of physics and science in general is based mostly on word associations can become competent players in my experience, but to be a great science player, you need a deeper conceptual understanding of the subject, and how things relate to each other.

As for the physics at ACF Regionals, you're an experience writer and editor who's a fifth year graduate student in physics, so I trust you know what you're doing.

People who tend to look at archives in their free time are serious players who want to get better at quiz bowl, and I see no problem in rewarding them. There are many free archives and question available to look at on the internet, and one can also create their own archive if they want.

This is just awful circular reasoning. If the only measure of how good one is is one's recall of stuff that has come up before, then this game is stupid and we should all stop doing it. By rewarding this kind of "knowledge" we are incentivizing the wrong kind of behavior that leads to getting better, and I won't have anything to do with it, and neither will any tournament that I'm a part of.

Okay. I will admit that I get most of my questions based on my ability to recall stuff that has come up before. It seems to have worked okay. This is not the only measure of how good one is. Writing questions, doing outside reading and research of subjects relevant to quiz bowl, paying attention in class, and being observant in general can help improve your game.

[Strongside]

Also, there has been some disagreement about the role of question archives in writing question, and it is clear that some people rely more heavily on the archive in quiz bowl.

My opinion is that question writers should focus primarily on writing good questions, and less on if the clues come from question archives, old packets, outside knowldge, outside readings, textbooks, and research.

[grapesmoker]

You don't want to pick on me, so you decide to post on the forums about how my questions are terrible. Whatever. I did not spend a lot of time on those CATO/TACO questions. I didn't have a lot of time to work on them, and I wanted to give Tommy something because the tournament was in need of questions. I will admit that the physics bonus I wrote for Chicago Open wasn't very good.

We've all written shitty questions from time to time. Those questions are, in fact, terrible, and I didn't go seeking them out, you posted them in the forums, which I think makes them fair game for criticism. I think those questions are hugely problematic and writing questions in that mold is bad. Even if your only reference is other people's questions, which it shouldn't be, you can tell that this is so by examining how people write and noting that they're nothing like the things you posted. How much time you had to work on them is beside the point.

And saying that physics bonus you wrote for CO "wasn't very good," is an understatement; it was a really bad question that had parts like "identify this guy who names some waves and some other things too" without a shred of context and also contained downright misleading information about how Alfven waves "deal with" magnetars, which makes no sense.

I will admit I am not a great question writer, but I have definitely gotten better as a question writer though over the past year or two. Also, part of the reason for this is fundamental differences in how I feel questions should be written.

Ok, well, I am a good question writer and I'm telling you that you should not write questions that way. If you seriously have fundamental disagreements about how questions are supposed to be written, it would be interesting to hear them because they must be all kinds of novel.

As for my writing being lists of things in a decreasing order of difficulty of association, isn't that what quiz bowl should do? Maybe I should consider paying more attention to how my clues flow, and decrease the amount of word association present.

No, that is not what quizbowl should do, because mere association means nothing. If I tell you "this guy is associated with event X" you can't reasonably buzz on that; many guys could be associated with event X. "Clues," of that nature are nothing but filler and word salad and in fact, I want to see the word "associated" abolished from question writing unless there's a damn good reason to use it. It's come to mean "I can't figure out how to phrase my question properly, so I'll just plop this connector in here and hope someone buzzes," which is awful, sloppy writing.

[grapesmoker]

Now, let's walk through one of your questions together:

Some lesser-known works written by this man include Language and Power, In the Mirror, and Under Three Flags. His brother Parry is a historian. His most famous work talks about “print-capitalism,” and that work is subtitled, “Reflections on the Origin and Spread of Nationalism.” This man was born in China, and lived in Thailand for a few years, although he is more closely associated with a different Asian country. This professor emeritus at Cornell University was in charge of its Indonesia program, which ironically, was a country that he was banned from in large part due to the fallout and reaction to his most famous work. For 10 points, name this historian, political scientist, and scholar, who is best known for a 1983 work about the titular entities, Imagined Communities.
ANSWER: Benedict Richard O’Gorman Anderson

Ok, so, the first line is just some works by Benedict Anderson. Mildly interesting, but I have no idea what they are about or why I should care since it's just a string of titles. The second sentence is absolutely worthless; why do I care that he has a historian brother named Perry? There's just no context for linking that to anything. The third clue is totally misplaced because if you know anything about Anderson, you know Imagined Communities and probably its subtitle; if you're trying to distinguish between levels of Benedict Anderson knowledge (a dubious proposition to begin with), you probably want to put that towards the end. The next sentence after that is again totally worthless other than telling you that this is someone who studied Asia, which could be done much more compactly, and then sentence after that one is full of more filler, with the only useful information being that this is someone who studied Indonesia (again, if you know anything about Anderson, you know that, so this again should go closer to the end). Finally, there is almost no description of Imagined Communities, which is something that probably should occupy close to half of a Benedict Anderson tossup, given that it's by far his most famous and widely read work.

So yeah, this question is awful! There's obviously some attempt here to order things in a "pyramidal" fashion but there's also a whole mountain of dross and boring nonsense that doesn't help anyone answer the question based on anything but a laundry list of Anderson works. If your philosophy of question writing supports writing these types of questions, then I guess it's no surprise we don't see that many exemplars of your work around; editors probably recoil in horror upon receiving these tossups and rework them into something that actually makes sense.

This may not pertain specifically to question writing, but it shows how even the top players can't answer most of the bonuses at a hard tournament like Chicago Open. I suppose it isn't extremely relevant to this thread.

I don't know why I'm pursuing this, but... yes, many top players cannot answer many of the bonuses at CO. Or, really, any hard tournament. That's why quizbowl is a team sport. Nevertheless, I'm sure that if you had everyone play singles at CO, the top players would in fact separate themselves from the rest of the field by their showing.

You may think that is unfortunate, but from my experience it is the truth. Your experiences may be different. How do you think this should be changed?

Well, I and many others have made recommendations in this very thread about how to do so. The point is to emphasize information obtained in the context of learning something relevant to the topic at hand. I don't want to get metaphysical about quizbowl, but in science, there are definitely "natural kinds" of things that are learned (for lack of a better terminology). You can focus on those things, thereby ensuring that the questions you write reward people who study that thing, instead of people who read Wikipedia's "See Also" section for the Stark effect.

If I were to take two people of equal gender, age, and intelligence who had never played quiz bowl before, know absolutely nothing, and gave them three hours to learn about William Faulkner. One person will read As I Lay Dying during this time period, while the other will memorize character names, and plot summaries of every William Faulkner work. I predict the person who learned about the character names and plot summaries will do better than the one who read As I Lay Dying. The person who read the book will likely be able to nail any tossup on that book, but I would guess the person who learned a little about all the William Faulkner works would do better in general.

The failure of your thought experiment is that it's circular. You have defined "doing better" as "doing better in the kind of quizbowl that would reward you for memorizing Faulkner titles." But this is exactly what I'm arguing against (and even if I were to admit that this person would "do better" I would also argue that this is irrelevant). I want to reward the person who reads As I lay Dying because that person will also very likely go on to read Light in August and The Sound and the Fury and Absalom! Absalom! and that's exactly the kind of growing experience and intellectual curiosity that I think is worth rewarding.

[grapesmoker]

Okay. This doesn't make much sense to me. I am not sure why people should be punished for looking over old tournaments, and trying to get better. If you take out the time and work hard to look over questions, I don't feel you should inherently be punished. Looking over old tournaments can only do so much though.

If you are a really good basketball player and you show up to play soccer, you will not be rewarded for your basketball skills, except perhaps in that they have helped you become generally athletic. My point is: I want to reward people for having primary knowledge of stuff and not for memorizing lists of titles. If you think that memorizing second order analogues of Jahn-Teller effects will help you at Regionals (or, really, at any other good tournament), you are mistaken; what will help you is understanding what things are and why they are that way.

People whose knowledge of physics and science in general is based mostly on word associations can become competent players in my experience, but to be a great science player, you need a deeper conceptual understanding of the subject, and how things relate to each other.

Just this weekend I observed Andy Watkins buzzing in a wholly improbable place on a tossup on Cherenkov radiation. Andy is a good player who knows a bunch of science, but he buzzed on literally the 4th or 5th word of a tossup, on a clue that made absolutely no sense to me and which I'd never heard of. Given that I have studied Cherenkov radiation and have even derived the Frank-Tamm formula in classes, I think this is a problem, because my very real knowledge would not have been rewarded by that question.

As for the physics at ACF Regionals, you're an experience writer and editor who's a fifth year graduate student in physics, so I trust you know what you're doing.

Well, I'm glad we agree on that.

Okay. I will admit that I get most of my questions based on my ability to recall stuff that has come up before. It seems to have worked okay. This is not the only measure of how good one is. Writing questions, doing outside reading and research of subjects relevant to quiz bowl, paying attention in class, and being observant in general can help improve your game.

Sure, of course. Look, I don't want to deny that remembering stuff that comes up is part of the game. I've made some (even very impressive) buzzes on music questions because I've remembered a thing or two from before. But when I happen to beat a knowledgeable music player, to me that's evidence of a question that rewards the wrong thing (like knowing a list of a composer's works without having heard it). No one should be ashamed to buzz because they remembered something that has come up, but questions should be written so as to minimize the rewards one accrues from reading old packets.

[Sen. Estes Kefauver (D-TN)]

My opinion is that question writers should focus primarily on writing good questions, and less on if the clues come from question archives, old packets, outside knowldge, outside readings, textbooks, and research.

The whole point is that it is harder to produce good questions if you only draw clues from certain places (old hard packets) and ignore others (what people learn from in classes).

[The King's Flight to the Scots]

Jerry,

Since you went through Brendan's question as an example of what not to do, could you give us an example of a question that does reward understanding? Real science is fairly distinct from "named-things" bowl, so could you show the community a history or literature question that rewards the real knowledge you want rewarded?

[Strongside]

So I will admit that the Benedict Anderson tossup wasn't great. Those questions were more about giving Tommy and Cameron some questions and helping them out, rather than writing perfect questions.

As for Andrew Watkins getting that Cherenkov Radiation tossup that early, I don't see a problem with it. Andrew is a strong science player, and the Askaryan Effect deals with Cherenkov Radiation. There isn't going to be a tossup on the Frank-Tamm Forumla, or the Smith-Purcell Effect at ACF fall, although I suppose it could have been something else.

Bringing up the Jahn-Teller Effect is interesting, as I got the Jahn-Teller Effect tossup at Lederberg after the Opik and Price clue about one line in. I will admit I don't have a strong conceptual understanding of the Jahn-Teller Effect outside of quiz bowl. If I go up against a really strong science player for an extending series of questions I will almost certainly lose, but I will be able to get the occasional tossup like I did in that situation.

If you are going to critique someone's questions, I suggest you take a nicer tone. It doesn't take much effort, and would probably give you more credibility. Ryan Westbrook gave me detailed feedback about my 2008 Chicago Open questions, and took a positive tone. Maybe you can learn from him.

One problem I have (and probably others) have deals with writing questions for packet submission tournaments. It is hard to get motivated to write great questions for tournaments like these, when you can spend less time writing acceptable questions that are okay or good, especially knowing they might get cut. Now I realize that cutting submitted questions is unavoidable, but it can be frustrating to spend so much time working on questions, and see them cut. Do other people have this issue? How can it be alleviated?

Also, I think it would be more constructive and productive if you make that post you mentioned back in August about how science questions should be written. I know that it would help me.

[Mechanical Beasts]

One problem I have (and probably others) have deals with writing questions for packet submission tournaments. It is hard to get motivated to write great questions for tournaments like these, when you can spend less time writing acceptable questions that are okay or good, especially knowing they might get cut. Now I realize that cutting submitted questions is unavoidable, but it can be frustrating to spend so much time working on questions, and see them cut. Do other people have this issue? How can it be alleviated?

The second part is an understandable sentiment: everyone feels that frustration. "Submit fast, weak questions because they might not make it in anyway" is a totally unacceptable recourse and is what makes editing packet-submission tournaments thankless. A lot of the time questions are cut because they are repeats, not because they're bad. If your response is "well, then, I just won't write good questions in the first place" well, then, your questions will be cut whether or not they're repeats. No one wins!

[Matt Weiner]

I don't think any tournament besides ACF Fall is going to use less than 80% of any well-written packet. ACF Fall, by its limited-answer-pool nature, breeds a tremendous amount of repeats, and also gets a lot of submissions because it is a popular tournament. For Regionals, or for independent invitationals, the situation is different.

[grapesmoker]

Jerry,

Since you went through Brendan's question as an example of what not to do, could you give us an example of a question that does reward understanding? Real science is fairly distinct from "named-things" bowl, so could you show the community a history or literature question that rewards the real knowledge you want rewarded?

Matt, I'll try and do one tomorrow, but one example which I thought was pretty good was the tossup on photons from MO. The second clue, for example, told you that neutral pions mostly decay into two of these particles. If you know something about particle physics (such as that charge has to be conserved and pions are light), you can easily see that the answer should be photons. Now, of course one could conceivably memorize decay branches of various particles but this would be stupid and no one does that. So that's a good example of a clue in a question that rewarded knowledge (the rest of the clues were good too).

[Magister Ludi]

I have a paper to write tonight, but tomorrow I'm going to post a muddled tossup on "The Misfit" from VCU Open 2009 and a very good tossup on "A Good Man is Hard to Find" from MO 2008 to try to offer an example of how literature tossups can reward understanding. Also the VCU Open tossup illustrates a common problem in literature questions when a writer includes lots of quotes in an effort to be "real," but isn't sure which quotes are important and accordingly fills the first half of the question with unhelpful material. I'll also post a couple more examples of tossups that have unhelpful clues early in the tossup, which demonstrate a pernicious trend in writing literature questions in which minor and unimportant incidents/lines/works are used in the leadins making questions ungettable even for players who are very knowledgeable about the subject.

[grapesmoker]

So I will admit that the Benedict Anderson tossup wasn't great. Those questions were more about giving Tommy and Cameron some questions and helping them out, rather than writing perfect questions.

But you can just as easily write a decent Benedict Anderson question. Look, here's a template:

In one of his works this thinker wrote about [idea A]. Another of his works deals with [concept B]. In addition to [Work X] and [Work Y], this thinker criticized [Thinker C] in a work about [thing D] entitled [Work Z]. This thinker's most famous work focuses on [concept E] in [country F]. For ten points, identify this author of [most famou work].

I mean, I just came up with that. All you have to do is to find the things that go in the blanks! I'm not going to do that because I'm not a Benedict Anderson expert, but I don't have to be one to write a tossup on him.

As for Andrew Watkins getting that Cherenkov Radiation tossup that early, I don't see a problem with it. Andrew is a strong science player, and the Askaryan Effect deals with Cherenkov Radiation. There isn't going to be a tossup on the Frank-Tamm Forumla, or the Smith-Purcell Effect at ACF fall, although I suppose it could have been something else.

First of all, I have never advocated, nor would I ever advocate that Smith-Purcell or Frank-Tamm be tossups at this level, or indeed, at any level shy of nationals. Second, from what I can tell from briefly glimpsing the Wikipedia page for the Askaryan effect, it is superficially similar to the Cerenkov effect in that it involves superluminal particle propagation in a medium, but the mechanism looks different; therefore, saying that "the Askaryan effect deals with Cerenkov radiation," is at best misleading and at worst simply false. They appear to be different mechanisms entirely that have one feature in common. This is analogous to the criticism I made of Wesley's Sachs-Wolfe effect tossup from CO, which is that saying that it's "associated with" or "competes with" the Sunyaev-Zeldovich effect betrays a misunderstanding of what those things are and what they do.

Bringing up the Jahn-Teller Effect is interesting, as I got the Jahn-Teller Effect tossup at Lederberg after the Opik and Price clue about one line in. I will admit I don't have a strong conceptual understanding of the Jahn-Teller Effect outside of quiz bowl. If I go up against a really strong science player for an extending series of questions I will almost certainly lose, but I will be able to get the occasional tossup like I did in that situation.

But the point is not to ensure that a science player beats you 9 out of 10 times or whatever. The point is that every tossup should be fair and internally consistent and reward some kind of actual knowledge, and that question was flawed in that it didn't do that.

If you are going to critique someone's questions, I suggest you take a nicer tone. It doesn't take much effort, and would probably give you more credibility.

It is to laugh. I've edited plenty of tournaments and contributed to dozens more; I have all the credibility I could possibly need and I don't have anything left that I need to prove by pretending that questions that suck don't suck. I don't know how many questions you have contributed to packets but I do know that I've never actually seen anything in a Minnesota packet or tournament that even remotely resembles the questions you submitted to CaTo. So even if you're submitting questions, it doesn't look like they're getting used; you should think about why that might be.

Ryan Westbrook gave me detailed feedback about my 2008 Chicago Open questions, and took a positive tone. Maybe you can learn from him.

I didn't edit Chicago Open 2008; perhaps if I end up doing 2010 I will give you some feedback (I've done so already, by the way), though I'll note right now that in recent memory no tournament that I've worked on had submissions that I knew to be from you. But regardless, I don't have anything I need to learn from Ryan Westbrook, certainly not about writing questions or critiquing them.

One problem I have (and probably others) have deals with writing questions for packet submission tournaments. It is hard to get motivated to write great questions for tournaments like these, when you can spend less time writing acceptable questions that are okay or good, especially knowing they might get cut. Now I realize that cutting submitted questions is unavoidable, but it can be frustrating to spend so much time working on questions, and see them cut. Do other people have this issue? How can it be alleviated?

Wow, that's horrible. You should try to write the best questions that you can because they make the lives of editors easier and, presumably, because you take pride in your work. As an editor, I love getting good packets that I can check for repeats and put away as done instead of rewriting every damn thing about them.

Also, I think it would be more constructive and productive if you make that post you mentioned back in August about how science questions should be written. I know that it would help me.

Ok, I'll try to do that soon.

[magin]

I have a bit of a problem with the idea that the important thing is knowledge independent of quizbowl. This seems like a particularly poor substitution for an abstract idea of "importance." A lot of the stuff that I know is based on things I learned reading packets or writing questions, but that doesn't make the understanding any less real, or the subject any less important.

I don't think that every single clue in every tournament has to be based exclusively on non-archive knowledge; as you say, there is a place for rewarding limited understanding (such as understanding from looking at the archives), albeit a much more limited place than currently exists. However, the important thing in choosing an answer is asking "do people playing this tournament know any clues about this answer besides what has come up in the packet archives?", and if the answer is no, or only one or two, then that answer won't be a good answer choice for that tournament. Sure, it's easy to misjudge what other people know about an answer (see my misguided decision to include the tossup on The Princesse of Cleves in the 09 ACF Regionals), but it's imperative that we try to determine what people know (from non-archive sources) to the best of our ability.

[Mike Bentley]

I have a bit of a problem with the idea that the important thing is knowledge independent of quizbowl. This seems like a particularly poor substitution for an abstract idea of "importance." A lot of the stuff that I know is based on things I learned reading packets or writing questions, but that doesn't make the understanding any less real, or the subject any less important.

I don't think that every single clue in every tournament has to be based exclusively on non-archive knowledge; as you say, there is a place for rewarding limited understanding (such as understanding from looking at the archives), albeit a much more limited place than currently exists. However, the important thing in choosing an answer is asking "do people playing this tournament know any clues about this answer besides what has come up in the packet archives?", and if the answer is no, or only one or two, then that answer won't be a good answer choice for that tournament. Sure, it's easy to misjudge what other people know about an answer (see my misguided decision to include the tossup on The Princesse of Cleves in the 09 ACF Regionals), but it's imperative that we try to determine what people know (from non-archive sources) to the best of our ability.

What "people" are we measuring our standard against here? I would argue that the vast majority of "people" playing a regular difficult tournament have very little "real knowledge" outside of their area of study, and thus if this is the primary criteria we go for, the canon is going to have to be extremely small. For instance, how many people on a typical middle-of-the-road team have actually read more than a handful of books on philosophy? How many people have actually played a musical piece being asked about? How many people have indepdently studied or encountered in class enough about King X or Polity Y to be able to answer the question in the first half? I think the number is extremely small.

[No Rules Westbrook]

Nah, I'm with Andrew here. We shouldn't be asking how people learned the info, but what kind of info it is. The fact that a lot of knowledge of it comes from exposure to quizbowl should not mean anything.

Not to mention: I'm very skeptical of the idea that there's a ton of stuff out there that lots of players know independently, but can't be found at least a few times in the packet archives and probably more. Whether a player learned it through a class, or solely through the packet archive, or by writing a related question and researching iffy internet sources - is just beside the point, if the info is important.

Plus yeah, what Bentley said, it's all too easy for a person like Magin to overestimate all the exiting stuff that "real people" must surely know! Real people, even most very intelligent scholarly-type people, tend to know very little.

[grapesmoker]

Some level of knowledge is obviously going to come from quizbowl directly rather than reading. For example, I am unlikely to read Pierre any time soon, and yet I will be rewarded on a Melville tossup if that is mentioned in the tossup. That's fine; there's something to be said for simply knowing that Melville wrote Pierre. The point is that the majority of a question should not be things like lists of titles or extraneous information but rather consist of something like plot descriptions of increasingly more relevant works (in the case of lit questions).

I think part of the problem has been that when people don't know something about a work, they tend to focus on writing tossups on authors, and that combined with the recycling of topics leads to a fast exhaustion of an author who becomes quizbowl-popular. My suggestion is that, at least in literature, people should write more tossups on works rather than authors. It's much harder to exhaust the knowledge of several different and widely read Edith Wharton novels than it is to exhaust knowledge of Edith Wharton titles.

Finally in response to Ryan's point: my bookshelf is full of things that don't come up in quizbowl all that much and which I do know. Maybe that's just me, but I don't think so. Lots of people know lots of different things from contexts having nothing to do with quizbowl and I think we should try and figure out what those things are and maybe write some questions about them. Of course, the set of "stuff people know" and the set of "stuff that gets written about" will have a high degree of overlap anyway because people tend to write about what they know. However, I think there's a lot of stuff we could be writing about that will get answered and enjoyed by players but which we're not writing about because we've been too busy recycling topics.