Past Tournaments as the "Basis" of Quizbowl Knowledge

[Birdofredum Sawin]

I'm going to begin this post with an anecdote. At VCU Open, there was a bonus on the oeuvre of Edwidge Danticat. "Another stupid Danticat question," I said to my teammates. "She comes up at every fucking tournament these days!" Then we sat down to play the CaTO/TaCO, and the very first bonus was on some short story I've never heard of. "Given the state of quizbowl these days," I said, "this is probably some shit by Danticat -- can anyone name a story from Krik-Krak?" And, lo and behold, it WAS a bonus asking for "story from Krik-Krak/Krik-Krak/Danticat."

I have no objection to Danticat, but given her relative importance in the scheme of things, she probably shouldn't be coming up with such metronomic regularity. Why, of all the thousands of authors of equal or greater importance that could be asked about, do people keep asking questions about her? I think the answer to that has to do with the way people relate to previous tournaments. That is, I think that a number of younger players have an uncritical way of absorbing the clues and answers used in previous tournaments (in particular, the clues and answers used in recent, prestigious tournaments). The mindset I'm seeing reflected in people's posts, and which accounts for some of the oddball answers which come up with otherwise surprising frequency of late, seems to be that "if a thing has come up x times [where "x," it seems, can be quite small], then that thing is significant and I should be asking about it too."

My own way of looking at previous tournaments, when my participation in the game was more regular, went something like this. For any subject area, you can roughly divide the answer space into three categories: the core, the periphery, and random outliers. In philosophy, for instance, someone like Kant is obviously in the heart of the heart of the answer space. If I heard questions on Kant and his works at five tournaments in a row, I would think nothing of it: he's so important, and there's so much to ask about in relation to him, that I wouldn't even notice the repetition. By contrast, someone like Fichte is peripheral. He's important, but much less important than a major figure like Kant; if questions on him came up at a few tournaments in a row, I'd think that it was time to lay off him and work some other area of the periphery for a while. And, by further contrast, someone like Salomon Maimon is truly an outlier. If I heard a question on him at all, I'd be like "huh, odd choice"; if I heard questions on him at multiple tournaments in a short span, I'd be like "wtf, Maimon is a truly minor figure -- he shouldn't be coming up this much."

My sense is that some younger players these days have a different take on things. If something comes up, they duly note its existence, and then proceed to learn about it and write on it. It doesn't matter whether the "something" is something important, which should be coming up all the time, or something marginal, which should be coming up rarely, if ever. What matters is that it "came up." This, I take it, is how one Danticat question can spawn another, and then another, until she's coming up all the damn time.

I'd argue that this is a bad way of approaching the game, for a number of reasons. The chief objection which comes to mind right now is that it has a bizarre warping effect on the canon -- people like Danticat loom much larger than they deserve to, simply because some random person happened to write a question on her once, and then someone else who didn't know any better thought she was "important" and wrote his own question on her, and so on, and so on.

Now, I'm aware that the approach I'm suggesting assumes that you can divide subjects up into those which "deserve" to come up with great regularity, those which deserve to come up occasionally, and those which should rarely if ever come up. And obviously, you can only do that if you possess a certain degree of knowledge both of a given subject and of the way it is reflected in quizbowl. I for one couldn't begin to make this kind of assessment with regard to, say, biochemistry. But I think that this is one of the things diligent and knowledgeable editors can and should be doing. So, if you're editing a tournament, and you know that someone minor like Danticat has come up in four of the last five sets you've heard, you might say "ok, I'm going to omit this submitted Danticat question from the set -- she's been overexposed lately." (It occurs to me that this will be impossible to do if you're a harried editor who is throwing together whatever questions he has at his disposal just before the tournament -- so I guess this is yet another reason why that Weinerian approach to set editing is pernicious.)

[Matt Weiner]

Is that burden really on the editor? It seems counterproductive to throw out (what we will assume is) a well-written, difficulty-appropriate question just because the topic has come up at some other recent tournament. Isn't the responsibility more on writers to avoid writing on tired topics?

[Birdofredum Sawin]

In theory, yes, this burden would fall on the writers. But I'm assuming that a lot of the people who do this kind of thing are people who just don't know any better -- they are relative novices to quizbowl, and all they can do is "look at recent tournaments and take what they find there as exemplary" when writing their own questions. This is an approach that we do, and should, encourage in novice writers; I don't think we can also expect them to exercise discretion about which aspects of recent tournaments they emulate. If that assumption is correct, then I think that the control has to be at the level of the editor.

[Sen. Estes Kefauver (D-TN)]

Well, unfortunately, I don't think it's only inexperienced players doing this, which is the problem. Ryan Westbrook and Brendan Byrne basically said they do it yesterday, and I've heard other people who have edited stuff that was decent basically give the same reason when justifying something i thought was odd. When your good writers are doing the same thing, the pickings get slimmer for an editor, because if they are writing the packets that otherwise need no editing, and the editor has to spend all his time on packets by lesser teams, then of course it makes sense to keep those questions. If you get a bunch of good packets, then sure, I think it would be a good thing for editors to try to keep tabs on that stuff. I really think though that Matt is right that we need to try to drive home to players that they should dedicate a greater part of their answer selection to the core.

[Birdofredum Sawin]

Sure, I can believe that it isn't just inexperienced writers who are responsible for this phenomenon. Maybe there are three kinds of people who do this kind of thing. First, the relative novices I referred to in my previous post. They don't know any better, and we can't expect them to be discerning about whether an answer is peripheral, central, or whatever; thus, an editor needs to ride herd on their decisions. Second, Ryan Westbrook. He does whatever the fuck he wants, and clearly nobody is going to persuade him to change his mind about anything. Third, people who ought to know better but don't -- maybe Brendan is one such person. So maybe the goal of this thread should be to get the third group of people to reconsider the error of their ways? That works for me.

[ValenciaQBowl]

Andrew's theory sounds logical to me, and it may help explain something I think I've noticed at the last two COs. I'm pretty certain that no questions (toss-up or bonus) involved Faulkner or Joyce in the CO proper (Faulkner did come up at Trash this year, though) in either of the last two years (unless something came up in a finals packet I didn't play/read or something). I don't mean to suggest that every tournament must involve specifically these two writers, but as two giants whom I particularly love, I just happened to notice the omission of questions involving them. It's not like there aren't lots and lots of works/characters to ask about for these guys proportionate to the difficulty level of any tournament, but I wonder if some writers/artists/scientists are considered passe or too easy simply because of their giant fame and influence. I'm sure if I weren't so lazy I could come up with other examples, and again I want to stress that I'm not making a point specifically about these two figures, just that it may be that it's become more typical to push answer space away from things that many players heard about (too much?) in high school.

[Mechanical Beasts]

I think one of the phenomena that leads to this increased focus on the periphery is the fact that the championships that people care the most about are the ones on hard questions. Since most people and editors include a large number of tossups on hard answers--to avoid the various ongoing music debates, I'll use the example that CO featured a tossup on Musical Offering, not one on Bach--many players, attempting to get better by writing questions, will write more tossups on periphery than on core subjects.

I can't comment on whether this is a smart strategy (since many hard tournaments have bonuses on the lesser-known works of well-known authors, for example) or how this relates to the proportion of easier answer lines that should be coming up at hard tournaments.

I can say that I think I'd very much enjoy a canon that focuses more on the lesser-known works of well-known social scientists, authors, philosophers, and so forth, rather than taking leaps into the periphery that sometimes pay off and sometimes don't, and sometimes (most riskily) cause ten-tournament-long flirtations with minor works and authors.

[Matt Weiner]

As a general point relating to the example used in this thread, I think there are two different kinds of things that suddenly become memetic question topics. One is "a relatively unimportant thing that comes up a lot because of Westbrookian attitudes to the canon." Another is "a recent spate of questions on something that should have been coming up all along." I understand that this thread is not about Edwidge Danticat per se, but let's use her as a convenient example of category two. I think a lot of people read Danticat in classes and once a few people opened the floodgates, everyone else figured this was a good topic to write a question on contemporary academic literature about since they had already heard of her. Ruling out the titans of still-active writers like Thomas Pynchon, I think one could argue that there is no real surplus of Danticat questions as compared to someone else in the second tier. For example: Searching my archive, there have been an astounding 105 questions on Michael Chabon and his works to date. Danticat has come up 14 times (8 of them in the past academic year). It may be jarring for Danticat to suddenly come up at nearly every tournament, but if we're going to ask about present-day literature, perhaps this isn't abnormal compared to similar topics?

Perhaps a flood of questions on Maria Edgeworth or someone else who we can predict has had no recent flood of either publishing or academic interest would be a better example of overdevotion to "the canon" pushing a topic into undeserved frequency.

As for Chris's example, I think it does seem off from a player's perspective when a lengthy quizbowl tournament doesn't have any content about some of the major figures in a major category; at the same time, I'll restate my premise from above, which is that in a submission tournament you don't have the luxury of replacing good questions just to meet smaller subdistributional concerns. If this were VCU Open, NAQT Sectionals, or another tournament where one person/entity writes or selects answers for the whole event, then you could point to oversight if there was a 30-question American literature distribution that didn't include Faulkner.

[No Rules Westbrook]

To be clear, and contrary to Charlie's suggestion - I deliberately try to write on things that have been underexposed (in fact, recall me talking about dredging up the "lost canon" - I'd posit that I, at least as much as any writer, search for underexplored topics instead of grazing on the same ground). If I'm editing, I tend to preferentially choose exciting and underexposed topics (when they're well written and feasible) as opposed to that billionth Baha'i tossup.

Yes, I'm very attached to the notion of working from a canon in many respects, but I ain't the one spitting nonstop Danticat at you. As Andrew says, I suspect it's younger players who see a recent Danticat tu and go "gee, I don't know anything about her, so I'll write a tossup on this to learn stuff" - and as Andrew also agrees, that behavior is something we should generally encourage because it's productive for novices to emulate good writing. I don't think anyone like me or Jerry or Seth is sitting around going "Yeah! Let's do Danticat again! She comes up a lot, so she's awesome!"

A lot of what people are critiquing here is the trend of certain topics in QB mysterious "getting hot" - I satirize that as much as anyone (oh look, Ryan says, another tossup about Machado de Assis - the kids love him!), and I work to prevent it as much as anyone, so I'm confused about people's uses of Westbrookian here.

[Sen. Estes Kefauver (D-TN)]

My whole point though is that you openly adhere to the (ridiculous) notion that the more something comes up, the easier it is.

[No Rules Westbrook]

My whole point though is that you openly adhere to the (ridiculous) notion that the more something comes up, the easier it is.

Yes, I agree with this statement (except I'd note that I only adhere to this when considering non-novice events, generally regs or higher difficulty).

I just wanted to object to the notion that a "Westbrookian approach" creates the Danticat Phenomenon (I'm gonna pretend that's a named effect, of course). It's true that I do support newer writers extensively consulting packets (because it produces a lot of good results), and one suboptimal result of that can be the Danticat Phenomenon. More experienced writers with a broader view of the canon don't usually fall prey to the Danticat Phenomenon.

[Important Bird Area]

I just wanted to object to the notion that a "Westbrookian approach" creates the Danticat Phenomenon (I'm gonna pretend that's a named effect, of course). It's true that I do support newer writers extensively consulting packets (because it produces a lot of good results), and one suboptimal result of that can be the Danticat Phenomenon.

Quick, someone come up with another example, because this effect needs to be doubly-eponymous.

More seriously: I think it's important for newer writers consulting packets from good tournaments to be aware that what they're aiming to replicate is clue structure and (perhaps) difficulty, not the selection of particular answers.

[bmcke]

Do people ever just open the encyclopedia and write about what their finger landed on? I have to assume someone's thought of this before. Why doesn't it happen?

[Cheynem]

What other thing that I believe is problematic about the packet archive is: (which is the greatest thing since sliced bread and indisputably so, I'm not being a luddite here)

The fear among especially newer authors that any clue, once it has been used in a packet, suddenly excludes it from its usage in the near future. I don't want to pick on Auroni, but he made a post in the CO Lit thread (I think) basically expressing this idea, and you occasionally see people, especially in the high school threads, post stuff like "The lead-in to this question was the lead-in to a bonus at HSAPQ Packet #5." Certainly, nobody wants to see five tossups on _Chop Suey_ that all begin the same way, but (and this is especially true at novice and regular difficulty events) this is a dangerous line of thinking because what you are going to produce are obtuse tossups that are coy and vague and obscure which nobody can answer until the final lines. The point of reading old packets is to learn clues that are important and well used for quizbowl. As I said in another thread, this would be a pretty sucky game if people had to learn four or five different facts about a tossup answer every time they wanted to answer it.

[No Rules Westbrook]

Dang it, I cant stop posting lately. I was gonna create yet another topic and title it "Defense of Westbrookian Ethics" but the meat of what I want to say is related to this topic, so thankfully I chose not to.

Here's my theory. Quizbowl is full of a lot of really intelligent people who go to prestigious universities and possess all kinds of knowledge about various fields. This knowledge and level of interest naturally attracts them to a game like quizbowl, but many of those people (even ones that have been around forever and know what qb is about) seem to have trouble coming to grips with exactly the type of game that quizbowl is. They have trouble accepting the necessary framework and parameters within which quizbowl is played.

As Ahmad suggested in his post about "hating to be a game," I think a lot of players way overestimate the value of "real knowledge" in quizbowl (let's, for the sake of this argument, pretend that we can neatly distinguish between "real knowledge" and "qb knowledge," i.e. knowledge that would not have been acquired absent involvement in qb). If there were some neat way of quantifying this - I think a lot of people would be absolutely astounded as to how much success in QB, for the great majority of players (whether it's me or Jonathan Magin), depends upon "qb knowledge." I think a lot of people are unwilling to accept this because they're justifiably very attached to the knowledge they've acquired, their fields of interest, etc. Specifically, they're hesitant to accept that QB is necessarily a game that depends very heavily on binary artificially-acquired chunks of knowledge.

Now, let's look at the argument that topics like Immanuel Kant should appear more than they do in harder events, because Kant was incredibly influential and deserves more space than various obscure people and things that come up. Now, sure, you could write an infinity of tossups on Critique of Pure Experience - I suppose you'd continue to have valid clues at least until you ran out of lines in that text, and lines in all texts written about it. But, at some point, very quickly...most people in qb decide "that's enough about Critique of Pure Experience - I know how influential it is, but it's tired and I want to go write about something else." That's a direct result of the quizbowl format - it's not a game concerned with the in-depth analysis of texts, not concerned with giving each text/thing/person the precise amount of attention that it's arguably entitled to in an academic context. Especially when you get to hard events, qb progresses very quickly from Immanuel Kant to much harder, less influential philosophers. The drive to quickly locate the next important "named thing that can possibly be written about" naturally causes this phenomenon. It's not always true that really famous people stop coming up - Mark Twain has a ton of works and continues to come up a good deal, because people start writing tus on Fennimore Cooper and his Literary Offenses.

I guess what I'm trying to stress is that I tend to see a lot of the trends that people have been bemoaning lately as merely an inevitable result of the nature/format of quizbowl. It's a nature that I think a lot of players struggle with, and their ambivalent feelings rise to the surface as a number of complaints. There are others more inclined to take quizbowl as it is, to pursue it on its own terms and for its own sake.

[Stained Diviner]

Is there some technical solution that could be used to help teach the children? Would it be possible for one of the major quizbowl databases to allow a chosen group of editors to add tags/comments to questions that would allow young writers to see if a question has some issues? Some of the tags could be things like answer too difficult, answer pushing the difficulty edge, FU part, no easy part, difficulty cliff, comes up too often, etc. Somebody reading the questions would see which editors have checked it and whether they had anything to say about it. (So if you saw nobody checked it or only one person checked it, you wouldn't assume anything by the lack of tags/comments.)

Of course, any such system would be useless at first because there aren't going to be a bunch of top players immediately marking up thousands of questions. If successful, however, it eventually would let new players know which questions not to emulate and would also provide lots of feedback for the writers and editors whose work is being marked up.

[ValenciaQBowl]

Boy, that's a good post, Ryan. I don't always agree with your definitions of what's worth knowing for QB, but I wish I had written the third paragraph of the above. It's a succinct summary of my feelings about the game--which is a great thing, worth all of our time and efforts and arguments, but is always first and foremost a game.

Edit: can't count.

[Matt Weiner]

This isn't a particularly new argument, though. Back forever ago there were lots of people whose response to every criticism of, say, that Deep Bench where there was 4/4 trash and packets were required to contain "lies," was to rant about how quizbowl is a pisspoor substitute for actual learning and thus doesn't change its legitimacy at all (since it is always none) by being made as frivolous as possible. Obviously, this attitude is a self-fulfilling prophecy. While Ryan isn't exactly saying the same thing, he is sort of going down that path where we stop considering some ideal goal that will improve quizbowl as we move as close as possible to it, just because we can never completely attain it for whatever reason.

[SnookerUSF]

...how quizbowl is a pisspoor substitute for actual learning and thus doesn't change its legitimacy at all (since it is always none) by being made as frivolous as possible. Obviously, this attitude is a self-fulfilling prophecy. While Ryan isn't exactly saying the same thing, he is sort of going down that path where we stop considering some ideal goal that will improve quizbowl as we move as close as possible to it, just because we can never completely attain it for whatever reason.

First, I think it is unfair to imply that Ryan is uncommitted to improving quizbowl, and purely exists as some kind of superdifficulty leprechaun, always complaining about where his pot of Danticat short-stories have gone.

Is the sort of qb knowledge that Ryan refers to generally destructive to the game of quizbowl and we should seek as writers and editors to eradicate and/or minimize it?

Is it really a choice between this "real knowledge" thing and frivolity or irredeemable obscurantism?

[ValenciaQBowl]

I'm a big believer in starting with commonalities, so let me state that I'm not in any way interested in somehow radically changing our game (as if such a suggestion would matter!) to de-emphasize what I would define as "QB knowledge" (a definition of which, I'm afraid, would take me further afield than I want to go right now) in our questions. I don't want to trade buzzers and 20/20 format for essays and such. I do, however, support a movement toward at least acknowledging that whatever our game determines, it doesn't necessarily determine who is better read, better taught, a harder studier, or, god forbid, a smarter person. However, I do NOT think QB is a "pisspoor substitute" for anything, as it's a game that I've devoted a substantial portion of my time and mental energy to for over 20 years. It's a beautiful game on its own merits, and we don't need to justify it by arguing that it is a substitute for anything else.

I'm all for continuing a quest for an asymptotic approach to some sort of QB ideal fusion of knowledge and gameplay; I just liked Ryan's articulation that our game must always to some extent depend on formulations that compress and summarize and bifurcate information to fit the parameters of the game as we've come to know it. I'm not sure why this is actually that controversial, actually.

[Cheynem]

"Westbrookian" on these boards is starting to remind me of that baseball meme "I'm getting tired of players like ___ and Albert Belle," where Albert Belle's name is appended to a list of various problematic players no matter what the sin just because it's Albert Belle. Despite all of the talk lately about That Darn Westbrook, I'm actually rather confused as to how "powerful" his influence is in the current game (besides obviously on Brendan Byrne). The current trends of way too insane questions coming out of high schoolers, for example, I don't see as being traced to Westbrook, unless I am really naive.

[setht]

I'm a big believer in starting with commonalities, so let me state that I'm not in any way interested in somehow radically changing our game (as if such a suggestion would matter!) to de-emphasize what I would define as "QB knowledge" (a definition of which, I'm afraid, would take me further afield than I want to go right now) in our questions. I don't want to trade buzzers and 20/20 format for essays and such. I do, however, support a movement toward at least acknowledging that whatever our game determines, it doesn't necessarily determine who is better read, better taught, a harder studier, or, god forbid, a smarter person. However, I do NOT think QB is a "pisspoor substitute" for anything, as it's a game that I've devoted a substantial portion of my time and mental energy to for over 20 years. It's a beautiful game on its own merits, and we don't need to justify it by arguing that it is a substitute for anything else.

I'm all for continuing a quest for an asymptotic approach to some sort of QB ideal fusion of knowledge and gameplay; I just liked Ryan's articulation that our game must always to some extent depend on formulations that compress and summarize and bifurcate information to fit the parameters of the game as we've come to know it. I'm not sure why this is actually that controversial, actually.

I think Ryan's claim that quizbowl is a game that can never give a completely accurate reflection of scholarly knowledge in any field is not controversial--I certainly agree with that claim, and I think most people do. I also agree with the claim (which I believe Ryan has made) that "a topic or clue that's come up a bunch in a clump of tournaments has its difficulty artificially decreased (for purposes of high-level events) from its 'natural difficulty level.'" For instance, there have been two recent tossups on the Hecatoncheires, both with very similar clues; I think both had the same lead-in. I would certainly argue that that particular lead-in should not be a lead-in for tossups on the Hecatoncheires for a little while. This is a separate concern from the one Andrew introduced in this thread, namely: should there continue to be lots of tossups on the Hecatoncheires, or should well-informed writers/editors be making an effort to move on to other parts of the Greek myth canon?

I think Ryan is correct that a lot of people are misunderestimating the role of "QB knowledge" in the success of many top players. I think it's fine that many players (including myself) wind up scoring lots of points off of knowledge primarily gained through quizbowl. The point I think various people are trying to make is that while that's fine for what happens during gameplay, that does not mean that the model for question-writing should be "look primarily at old packets as a guide for what topics to write on and which clues to use." Ryan is correct that doing that often gives good results, but the reason for that is that looking at old packets often turns up answers and clues that are good for reasons that have nothing to do with their appearance in quizbowl packets.

-Seth

[cvdwightw]

Mods, feel free to split this stuff elsewhere.

Do people ever just open the encyclopedia and write about what their finger landed on? I have to assume someone's thought of this before. Why doesn't it happen?

Back in the bad old days of quizbowl, this was an accepted if not encouraged practice. Nowadays, not so much. Here's what I mean.

Wikipedia is no excuse for an encyclopedia, but I clicked "Random Article" 6 times before I came across something that might have been academically important: George Whitefield. I then clicked "Random Article" 15 times before I came across a second article on something that might have been academically important: the First Battle of the Isonzo, which I don't think is actually important but the Battles of the Isonzo, grouped together, probably are, and I wasn't going to chance wading through even more random and unentertaining entries to prove my point.

First off, that's twenty-one clicks to find two (2!) entries. Maybe if your encyclopedia is not Wikipedia and omits 99% of the entries on geographic locales, pop culture topics, and living organisms that make up an ungodly proportion of Wikipedia, you will have slightly better luck. In any case, it would have been far easier and far faster to decide, "hey, I think I'll write a tossup on George Whitefield."

Now, let's look at the two entries I've randomly chosen: George Whitefield and (for the purposes of this discussion) the Isonzo River front in World War I. One of these entries concerns a key figure in the Great Awakening, whose name should be familiar to students that have learned about in high school American history. The other of these entries is a series of battles along an often-overlooked theater of World War I, which would likely only show up in a course or book specifically dealing with World War I. Chances are that not only is George Whitefield more important in history than the Battles of the Isonzo, he is far more likely to be answered by the average team. Accordingly, we can then say that George Whitefield should come up more often and at lower levels than the Battles of the Isonzo.

With the encyclopedia strategy, both of these answers would be equally likely to come up, one of which will result in much conversion and one of which won't. But we've already claimed that George Whitefield should come up more often, because he's more important and more accessible (to Americans, at least). Therefore, the encyclopedia strategy severely distorts a sense of what is important and answerable.

With the "past packet" strategy, we find that the Isonzo has come up 12 times in the hsquizbowl.org packet search, mostly as either clues to either Theodoric or Caporetto or the hard part of a bonus, and someone then decides that, "hey, it's come up 12 times before, it must be important!" without having any idea of whether it really is important. The past packet strategy also allows one to make stealth additions to the canon by continually pushing a pet topic as a leadin/hard part and letting unwitting past-packet copiers take the topic and run with it down the difficulty level.

[ValenciaQBowl]

Okay, thanks for redirecting, Seth. So I'll agree with your point (and Andrew's?), too: as a lit person, I particularly care about not writing from old packets, and for what it's worth, I never do. Numerous times in the past I've argued that the lit answer space doesn't necessarily reflect what people read in grad school, though I've always accepted that this is okay, as, in concert with my above post, the game doesn't have to mirror academic canonicity. Danticat and Chabon are reasonable answer choices, but neither may warrant as much attention as they've gotten, and perhaps that comes from using old packets as a basis for answer selection. It's possible, however, that folks who are being discussed more often in middle-brow publications like the New Yorker and NY Review of Books (W.G. Sebald and Bruno Schulz, among others, perhaps, though I know both have come up before) are being missed.

Sorry if I've lost the thread of this discussion, but I'm enjoying it.

[Important Bird Area]

Chances are that not only is George Whitefield more important in history than the Battles of the Isonzo, he is far more likely to be answered by the average team. Accordingly, we can then say that George Whitefield should come up more often and at lower levels than the Battles of the Isonzo.

...

With the "past packet" strategy, we find that the Isonzo has come up 12 times in the hsquizbowl.org packet search, mostly as either clues to either Theodoric or Caporetto or the hard part of a bonus, and someone then decides that, "hey, it's come up 12 times before, it must be important!" without having any idea of whether it really is important. The past packet strategy also allows one to make stealth additions to the canon by continually pushing a pet topic as a leadin/hard part and letting unwitting past-packet copiers take the topic and run with it down the difficulty level.

Hey, I wrote that Isonzo tossup before anyone else had even used it as a bonus part! (Yes, it was for a finals packet, and yes, it was answered correctly before the FTP.)

[Strongside]

So I am a little late to this discussion, but I think this is a good and important thread, and I am glad that Andrew Yaphe started it. I was considering starting a thread with a similar theme myself earlier today, so I am happy to see how many responses this thread has garnered.

I want to respond to and ask questions about a few things that people have said in this thread, and other threads.

1. Charlie Dees said to Ryan Westbrook, "My whole point though is that you openly adhere to the (ridiculous) notion that the more something comes up, the easier it is."

I think this is an interesting statement, and one thatI disagree with Charlie on.

I feel that the more something comes up, the easier it is. While this isn't always true, and there are some outliers, I think the frequency that things come up, is the best way to judge difficulty. I don't know how many quiz bowl questions have been written over time, but I know it is at least a few hundred thousand. To me, searching the contents of a few hundred thousand questions (preferably good questions) to see how often things have come up is by far the best way to judge difficult and accessibility. If anyone has a superior way to judge difficulty, I would be curious to know what it is.

2. Yesterday in the thread about named things in the Science Canon, Chris Ray said, "I'll save my intense rage for what the archive search is doing to quizbowl difficulty for another time, but really that isn't even what I'm saying." Jeff Hoppes mentioned that he wanted to learn more about this, and I would be curious to see what Chris has to say about that. I talked about how I felt that quiz bowl archives and past questions, are the backbone and foundation of quiz bowl. While I agree with what Eric Mukherjee said that they only form part of the backbone, I believe they form a significant part of the backbone.

3. In May, Ryan responded to Ezequiel Berdichevksy's comments about ACF Nationals with this comment, that I thought was interesting,

"For the record, I'm not a huge fan of searching the databases to determine difficulty, and it's not how I do it. It's okay for less-experienced players and editors to get a very rough feel of what has or hasn't come up very recently, but it can also skew your feel for the canon.

I usually just rely on an intuitive impression of how many times I've seen an answer/clue/whatever come up before."

Since Ryan is probably the best writer/editor in quiz bowl, I am curious to see if he has more to say about this. I am not a great question writer in terms of skill or experience, but when I write questions (especially lately), I like to look at my archive to see how much a specific answer or clue has come up. This helps give me a good feel for difficulty, clue placement, and pyramidality.

My question for Ryan is: How do you use question archives/past questions/databases when you write or edit questions? You already stated that you don't use databases to determine difficulty. Do you then use the databases to find clues, or not? If not, where do you find clues?

I hope it doesn't sound like I am calling people out or anything like that, because that is is definitely not my intention. I feel that if we discuss these issues here, it will be good for quiz bowl in general, and will be helpful when writing/editing questions, and determining difficulty.

[Mechanical Beasts]

Brendan, I think that most non-you people get most of their knowledge from classes and from reading, rather than from the archive. Certainly the novice and regular difficulty canon, which have to be directed largely, perhaps exclusively, towards players who don't spend a bunch of time studying for quizbowl, need to be based primarily on inherent measures of difficulty, even if they're not easy to determine.

There aren't many tossups on the ideal gas law these days. There are many on noted cubic (TOTALLY UNIQUE GUYS) equation of state Redlich-Kwong. The ideal gas law is still an "easier" topic. If it comes up zero times for ten years and Redlich-Kwong comes up a dozen times a year for ten years, the ideal gas law will still be an easier topic.

This is hard for you, I understand, because a large part of your knowledge comes from past packets with, perhaps, not as much external reference as some people. This is what's true for the rest of us. That's not an insult, certainly: you're just a different sort of quizbowler, and I think your (voracious and apparently effective) learning style can make some things harder to understand.

[Cheynem]

A superior way to judge difficulty for me would be to consider how many times it has come up in combination with other, equally important factors--is this something that people who study this field would know and find important? Would it appear in academic settings (journals, books, textbooks)? Would the best players of this discipline agree that this is important or gettable? (I'm not saying that they're some aesthetic high poobahs, I'm just saying that if three great physics players have never heard of a physics thing, that's a problem).

Searching old questions is a good thing to do when you write your questions to judge difficulty. But it can't be the only way to judge difficulty because that would just result in all sorts of circular logic, like a mistake being used to justify many more mistakes.

[marnold]

Brendan, I think that most non-you people get most of their knowledge from classes and from reading, rather than from the archive.

I would be surprised if this were true.

[Mechanical Beasts]

Brendan, I think that most non-you people get most of their knowledge from classes and from reading, rather than from the archive.

I would be surprised if this were true.

Okay, I should make myself more precise. At the highest levels, where people are trying to win championships and stuff, people do a crapload of canon-studying. But the canon is the novice's enemy: the notion of a novice canon that has been built up over time ignores the fact that a player's first year playing academic quizbowl is going to start that player off from square one, even though ACF Fall has contained that one tossup on Oe for the last ten years and isn't it time for one on The Day He Himself Shall Wipe My Tears Away at that level? First-year players don't come into the game with more knowledge every year, so it's silly that a novice canon might determine what's "easy" based on what was easy last year plus dc.

[Strongside]

Brendan, I think that most non-you people get most of their knowledge from classes and from reading, rather than from the archive. Certainly the novice and regular difficulty canon, which have to be directed largely, perhaps exclusively, towards players who don't spend a bunch of time studying for quizbowl, need to be based primarily on inherent measures of difficulty, even if they're not easy to determine.

There aren't many tossups on the ideal gas law these days. There are many on noted cubic (TOTALLY UNIQUE GUYS) equation of state Redlich-Kwong. The ideal gas law is still an "easier" topic. If it comes up zero times for ten years and Redlich-Kwong comes up a dozen times a year for ten years, the ideal gas law will still be an easier topic.

This is hard for you, I understand, because a large part of your knowledge comes from past packets with, perhaps, not as much external reference as some people. This is what's true for the rest of us. That's not an insult, certainly: you're just a different sort of quizbowler, and I think your (voracious and apparently effective) learning style can make some things harder to understand.

This is a good post. I agree with you about the Ideal Gas Law and Redlich-Kwong. I learned about the Ideal Gas Law in high school chemistry class, but I didn't learn about Redlich-Kwong until it came up in quiz bowl.

I searched my archive, and noticed three tossups on the Redlich-Kwong tossup in the last year. 2008 VCU Open, 2009 Cardinal Classic, and 2009 Harvard International. There weren't any tossups on the Ideal Gas Law in ACF Style college/open tournaments in the past year, assuming I didn't miss something in my archive. I will admit that if I am playing quiz bowl, I would strongly prefer a tossup on Redlich-Kwong than on the Ideal Gas Law.

I wanted to say something about this.

Brendan, I think that most non-you people get most of their knowledge from classes and from reading, rather than from the archive.

I am assuming you are referring to quiz bowl knowledge when you say knowledge, and are referring to reading in general, and not reading quiz bowl packets. If you are a college/open quiz bowl player who gets most of their quiz bowl knowledge from reading/classes, you probably (though not certainly) won't be an elite player. This is because there is only so much you can learn in a class. I am able to answer some questions based on my own reading, classes, and other stuff, but if you want to be an elite player, most of your knowledge is likely going to come from playing/practicing quiz bowl, and looking over old packets.

[No Rules Westbrook]

Brendan makes a point that I don't think people take into account enough. Good pyramidal quizbowl has been around for awhile now, and especially lately there have been a ton of good tournaments, especially higher-level difficulty tournaments/side events/etc. - so that the canon as it exists now, at all levels, is extraordinarly large and well-developed. I think the canon, as it exists today, does a pretty fair job of reflecting what's important and askable in terms of answers and clues. Is it perfect? Of course not. There are things that shouldn't come up as much as they have, types of clues that are ill-advised, types of clues that are underused and don't show up enough, etc. - and people routinely post here and bring those issues to attention, and they hopefully get corrected. But, I don't think people often give the canon enough credit (by which, I mean, the canon accomplishes a lot of the goals that people want accomplished). Of course, "the canon" doesn't work properly if you just look at a couple of packets written this year and start drawing haphazard conclusions.

To answer Brendan: I was just pointing out with that quote that I don't really depend on a database reporting the shear number of times Answer X has appeared. Rather, when I think of an answer, I try to get a rough sense of how many times I've heard that thing playing qb and in what context (and if I think something has come up too much lately, I pick another answer). I don't usually look at past questions to find clues when I write - sometimes I consult my notes if I'm having trouble unearthing good clues, but usually not - I usually just find a reliable source or two or three on the internet and start copying down good clues, and then I try to arrange them according to my feel for their respective difficulty (looking at considerations like - does Clue X seem like a more basic thing about the answer than Clue Y - obviously, if you're writing about a battle for example - the name of the primary commander and parties fighting and the war and the year are going to be in the class of "basic information ", etc. - and have I heard Clue X before, how many people do I think will buzz on it, is the clue lateral-able so that it leads you to the answer somehow, etc.)

People have criticized me for relying too much on my personal "hunches" about what I've heard come up, what's answerable, etc. - but I think those hunches have proven at least as reliable as other methods people endorse (specifically, I'll take my hunch anyday over someone telling me "no, this is very buzzable, it's really important in field X, people will know it!").

[Mechanical Beasts]

Sure; elite players have a ton of knowledge based on what's come up before. But that's mostly independent of what's hard or easy for 99% of the field of every tournament, and that's what matters, generally speaking.

[Strongside]

I was just pointing out with that quote that I don't really depend on a database reporting the shear number of times Answer X has appeared. Rather, when I think of an answer, I try to get a rough sense of how many times I've heard that thing playing qb and in what context (and if I think something has come up too much lately, I pick another answer). I don't usually look at past questions to find clues when I write - sometimes I consult my notes if I'm having trouble unearthing good clues, but usually not - I usually just find a reliable source or two or three on the internet and start copying down good clues, and then I try to arrange them according to my feel for their respective difficulty (looking at considerations like - does Clue X seem like a more basic thing about the answer than Clue Y - obviously, if you're writing about a battle for example - the name of the primary commander and parties fighting and the war and the year are going to be in the class of "basic information ", etc. - and have I heard Clue X before, how many people do I think will buzz on it, is the clue lateral-able so that it leads you to the answer somehow, etc.)

So I think this is a very interesting paragraph. I am glad that you shared a bit of the process you go through when writing questions.

Even though I have, "the broadest knowledge base in the game" according to one anonymous poster, I don't feel I could be a very effective question writer doing what Ryan does, especially when doing an archive search is relatively quick and painless. At the very least, I would want to check a question archive to make sure I don't have any misplaced clues. This is especially true if I were writing a question, and placing it in a tournament, as opposed to sending it in to an editor or editors for a packet submission tournament. An otherwise 8-9 line tossup could be great, but it could be ruined if it has an egregious out of place clue in the first or second line. A quick search through an archive may or may not prevent something like this, but it couldn't hurt, and could prevent a potentially bad tossup/misplaced clue from finding its way into a tournament.

This must be one of the major reasons that makes Ryan such a great writer.

[at your pleasure]

Here's what seems to be missing from the "past tournaments issue". When using past packets to measure(note the word choice) askability, we should not just look at "how much has it come up but "where has it come up". Even if someone does this for something that is clearly ridiculous at any non-masters level and sees that "Hey, the Watkins-Ryan equation's come up a dozen times as answer since 2005", it should not take much to notice that 10 of those instances were at opens,one was a hard bonus part at ACF nationals, and the 12th was in that tournament that was universally panned for overshooting its target difficulty. While this is imperfect, and many of the people with a skewed view of what's gettable, this will still filter out quite a bit of ridiculous stuff.

[Ethnic history of the Vilnius region]

I find this thread very interesting. Perhaps the reason the perceived youngsters who are writing so many Danticat questions, and perhaps others of us that don't have a good feel for the importance of certain topics in certain areas, are doing so is because consulting the archives is what quizbowlers have been told to do when writing questions. ACF's official writing guidelines refer players to Subash's Ten Tips On Question Writing. Tip 6 states: "When trying to assess the appropriate difficulty level for a tossup answer, consult Regionals packets from 1999 onward. If your answer has not come up AT LEAST twice as a prior tossup or bonus answer, then you should almost certainly not be writing a tossup with said answer." So there is one of the most respected writers and greatest players in quizbowl history specifically telling people to consult past tournaments, and ACF endorses that tip by placing it on its website and referring all writers to it in their writing guidelines.

The ACF guidelines also tell writers to look at How to Write Questions by Jerry Vinokurov, which is also on the ACF website. In the section entitled "Deciding What to Write About", Jerry writes, "For harder tournaments, the field of allowable answers is expanded, but it can still be hard to know whether what you're writing is appropriate. I think the following method provides a decent rule of thumb that one can follow: consult past instances of this tournament." So there again, writers are told to consult the archives by one of the game's greatest player/writers when deciding what to write about.

It seems to me that younger players are getting mixed messages about how to become good players and good writers, so I have a question that I would like to see a venerable player/writer answer that might help out a younger player or someone who should know better:

I want to write a packet of questions for ACF Fall. How should I decide what I should write about?

[Important Bird Area]

ACF official guideline stuff

It seems to me that younger players are getting mixed messages about how to become good players and good writers, so I have a question that I would like to see a venerable player/writer answer that might help out a younger player or someone who should know better:

I want to write a packet of questions for ACF Fall. How should I decide what I should write about?

I think there's a difference here between positive and negative advice from past practice. Certainly Subash's advice, and probably Jerry's as well, is meant as: here's what not to write. The intent is that an archive search will stop you from writing an (unanswerable) regionals tossup on something that hasn't appeared as a nationals bonus part yet.

I don't think any of that is intended that people should be using past tournaments to positively select answers.(of the form: hey, last spring there were three tossups on Danticat! I think I'll write another one because it's the trendy thing to do.)

[Ethnic history of the Vilnius region]

I think there's a difference here between positive and negative advice from past practice. Certainly Subash's advice, and probably Jerry's as well, is meant as: here's what not to write. The intent is that an archive search will stop you from writing an (unanswerable) regionals tossup on something that hasn't appeared as a nationals bonus part yet.

I don't think any of that is intended that people should be using past tournaments to positively select answers.(of the form: hey, last spring there were three tossups on Danticat! I think I'll write another one because it's the trendy thing to do.)

That makes sense. So how does one positively select answers?

[Important Bird Area]

If you are a college/open quiz bowl player who gets most of their quiz bowl knowledge from reading/classes, you probably (though not certainly) won't be an elite player. This is because there is only so much you can learn in a class. I am able to answer some questions based on my own reading, classes, and other stuff, but if you want to be an elite player, most of your knowledge is likely going to come from playing/practicing quiz bowl, and looking over old packets.

I think it's worth separating these three.

To my mind, the first (reading/classes) is the gold standard, what justifies the "Academic" in the titles of the major quizbowl organizations.

The second (knowledge from practicing/playing quizbowl) can be, to my mind, irritatingly meta, but it arises organically from the structure of the game and there's really no way to avoid it. Anyone who's been around long enough will simply have played enough questions to get a feel for giveaway clues and easy bonus parts well outside their fields.

The third (deliberate study of old packets) is something that a lot of people seem to do these days. I know it's possible to become a great player by incorporating this into your program, but I for one have never seen the attraction; I would find this tedious beyond belief.

I think the balance of quizbowl learning needs to tilt toward the first category, accepting the second, and limiting the third as much as we can. (That last may be harsh, but it's more or less the working definition of "real knowledge" as applied to such discussions as reform of science questions.) Otherwise, we run the risk that quizbowl becomes (more of) an isolated, self-perpetuating activity. I don't think we're there yet, but I think we could be if lots of people crammed old packets as a study method; and I think it's worth pushing back against the trend as both writers and editors.

[Mechanical Beasts]

I think there's a difference here between positive and negative advice from past practice. Certainly Subash's advice, and probably Jerry's as well, is meant as: here's what not to write. The intent is that an archive search will stop you from writing an (unanswerable) regionals tossup on something that hasn't appeared as a nationals bonus part yet.

I don't think any of that is intended that people should be using past tournaments to positively select answers.(of the form: hey, last spring there were three tossups on Danticat! I think I'll write another one because it's the trendy thing to do.)

That makes sense. So how does one positively select answers?

Well, perhaps a longer time sample than "last spring." If there were a few tossups on Danticat the spring before last, and the fall before that, and... then perhaps that's a subject of more enduring importance and less a trendy issue.

Also, for a lot of difficulties you can pick totally acceptable tossup answers without having to wonder too much whether you're writing on something not important enough, if that subject's not one you're too familiar with. Don't know enough about history to judge something obscure? Well, I'm sure you can write a regular-difficulty tossup on _Turkey_, and if someone tries to tell you that Turkish history is simply unimportant, uhh... This really gets to be a problem mostly at higher levels, right, where fringe topics and works get explored in more depth.

[at your pleasure]

The second (knowledge from practicing/playing quizbowl) can be, to my mind, irritatingly meta, but it arises organically from the structure of the game and there's really no way to avoid it. Anyone who's been around long enough will simply have played enough questions to get a feel for giveaway clues and easy bonus parts well outside their fields.

The third (deliberate study of old packets) is something that a lot of people seem to do these days. I know it's possible to become a great player by incorporating this into your program, but I for one have never seen the attraction; I would find this tedious beyond belief.

Things in both categories of knowledge, however, can migrate into "real knowledge" since people sometimes do get intriuged by stuff they see in old packets or hear about in practice and decide to read about said stuff. People do not, however, do this just because the stuff has come up, they do it because the stuff intriuges them.

[Skepticism and Animal Feed]

I am on a handheld device in Transylvania so I will keep it short:

I adhere to the Westbrookian view because I reject the Weinerian concept of a natural canon that everybody is taught outside of qb. People and classes are in fact specoalized and subject to instructor and institutional whims and biases. But any idiot, no matter what his background, can open the archive and learn clues. Thus a strict Westbrookian approach is an equalizer that makes quizbowl success open to all.

[Important Bird Area]

I adhere to the Westbrookian view because I reject the Weinerian concept of a natural canon that everybody is taught outside of qb. People and classes are in fact specialized and subject to instructor and institutional whims and biases. But any idiot, no matter what his background, can open the archive and learn clues. Thus a strict Westbrookian approach is an equalizer that makes quizbowl success open to all.

But doesn't the strictly Westbrookian approach leave novice players puzzled as to why they should start playing at all? Whereas a part-Weinerian approach allows players to display knowledge they've acquired outside of quizbowl.

[Mechanical Beasts]

I adhere to the Westbrookian view because I reject the Weinerian concept of a natural canon that everybody is taught outside of qb. People and classes are in fact specoalized and subject to instructor and institutional whims and biases. But any idiot, no matter what his background, can open the archive and learn clues. Thus a strict Westbrookian approach is an equalizer that makes quizbowl success open to all.

A "natural canon" is the only one that can possibly be relevant when you're considering what things novices ought to be expected to know.

Moreover, the reason this thread exists is because we're considering tempering our quizbowl-canon based on what is actually important. (You obviously care about what is actually important, and making sure that it comes up, because you posted as such in the thread about named things.) But if you support a strict Westbrookian approach, and there is something in the archive of no relevance that comes up twenty times, is it okay to write a tossup on it? Is it an important part of quizbowl knowledge? Taken further, in a strict Westbrookian approach (as you construe it), something that is in the archive despite not existing at all--its real world importance is zero because it was just a figment of the first question-writer's imagination--is something that you should learn about to get future questions, and it's okay for future questions to respect this and reward that studying.

i know that Ryan doesn't support that sort of pure model, where you open the archive and learn clues completely independent of how important the things involved are--or even whether or not they exist--and you probably don't either.

[at your pleasure]

something that is in the archive despite not existing at all--its real world importance is zero because it was just a figment of the first question-writer's imagination--is something that you should learn about to get future questions,

Not to derail this discussion, but this would make an execellent Borges pastiche. Anyhow, no matter how specialized the class or biased the instructor, some things will be taught in pretty much every undergrad class on something. For instance, even though one physics professor may spend a lot of time on electricty and another may spend very little, it is reasonable to assume that people who have taking a physics class know about Ohm's law. Likewise, no matter how the professor feels about the importance of the Specie Circular in the grand scheme of things, it's reasonable to assume that people who have taken a US history class have at least basic knowledge of the Second Bank of the US and have probably heard of the Specie Circular.

[setht]

So I am a little late to this discussion, but I think this is a good and important thread, and I am glad that Andrew Yaphe started it. I was considering starting a thread with a similar theme myself earlier today, so I am happy to see how many responses this thread has garnered.

I want to respond to and ask questions about a few things that people have said in this thread, and other threads.

1. Charlie Dees said to Ryan Westbrook, "My whole point though is that you openly adhere to the (ridiculous) notion that the more something comes up, the easier it is."

I think this is an interesting statement, and one thatI disagree with Charlie on.

I feel that the more something comes up, the easier it is. While this isn't always true, and there are some outliers, I think the frequency that things come up, is the best way to judge difficulty. I don't know how many quiz bowl questions have been written over time, but I know it is at least a few hundred thousand. To me, searching the contents of a few hundred thousand questions (preferably good questions) to see how often things have come up is by far the best way to judge difficult and accessibility. If anyone has a superior way to judge difficulty, I would be curious to know what it is.

I disagree. I think the best way to judge difficulty is to collect data on how frequently questions were answered correctly (and at what points, for tossups). The best way to judge difficulty before the questions are played is to use a combination of observations of how questions (especially on the topic at hand, but possibly also on some related topics) have played out in previous tournaments (especially recent tournaments), data on how likely quizbowlers are to encounter the material in a non-quizbowl context (and when they are likely to encounter it--that is, is this material that generally shows up in grad school, or in first year survey courses, or was it all over the news for weeks a month ago), and data on how often and in which tournaments questions on the topic have appeared before. For new writers, I think emphasizing the third part as a negative filter on answer selection is fine, but I think the first two components, when available, are much more useful for positive answer selection criteria. In the specific case of science, I think the second component is readily available to anyone with some time and an internet connection: go find some online course notes and see what things are covered (and in which courses they're covered).

-Seth

[setht]

Brendan makes a point that I don't think people take into account enough. Good pyramidal quizbowl has been around for awhile now, and especially lately there have been a ton of good tournaments, especially higher-level difficulty tournaments/side events/etc. - so that the canon as it exists now, at all levels, is extraordinarly large and well-developed. I think the canon, as it exists today, does a pretty fair job of reflecting what's important and askable in terms of answers and clues. Is it perfect? Of course not. There are things that shouldn't come up as much as they have, types of clues that are ill-advised, types of clues that are underused and don't show up enough, etc. - and people routinely post here and bring those issues to attention, and they hopefully get corrected. But, I don't think people often give the canon enough credit (by which, I mean, the canon accomplishes a lot of the goals that people want accomplished). Of course, "the canon" doesn't work properly if you just look at a couple of packets written this year and start drawing haphazard conclusions.

Okay, to some extent I agree with you: the canon is not in terrible shape. If someone sat down and wrote a tournament straight out of the canon, making no judgment calls based on outside knowledge and using frequency of appearance as the sole rubric for difficulty, the tournament might not be much worse than various other sets of the past year or so (assuming this hypothetical tournament did do things like pay attention to subdistributions, attempt to normalize bonus difficulty [based on frequency of appearances, of course], and use clear, complete sentences). However, I think such a tournament could never be one of the really good tournaments that people look to for years as a shining example of really good quizbowl--in order for that to happen, there has to be more going on upstairs in the writing and editing than picking things out of packet searches. I know (or I assume) that Ryan isn't trying to suggest that such a tournament would actually be good, I just want to point out that while the current canon is not a terrible simulacrum of "real academic knowledge," it's nowhere near good enough to produce good tournaments without some outside input.

People have criticized me for relying too much on my personal "hunches" about what I've heard come up, what's answerable, etc. - but I think those hunches have proven at least as reliable as other methods people endorse (specifically, I'll take my hunch anyday over someone telling me "no, this is very buzzable, it's really important in field X, people will know it!").

I think you generally do a fine job as a writer and editor, but one of your biggest weaknesses is science. I would be interested in seeing how a science set based on your hunches would compare with a science set prepared using the "go look at online course notes" method. I suspect the latter method would produce better results.

-Seth

[grapesmoker]

Look dudes, the canon that we know and love is a mixture of the stuff that comes up because it's important and the stuff that comes up because it's come up in quizbowl before. Some things are really famous and some things are qb-famous; for example, ACF's favorite author William Dean Howells is probably a good deal less known outside of quizbowl to generally well-read people than he is inside quizbowl, where apparently high school juniors are now conversant with the 14 best known Howells novel. That's just going to be a consequence of the fact that some things catch on and get asked about a lot because people keep looking back on past instances of what's come up and going, "ok, this is a thing, let's write about it again." This is particularly prevalent in the literature canon and leads to such absurdities as tossups on Oe being unwritable at this point because everyone and their dog knows more about Oe than Oe himself. In this, I definitely agree with what Andrew is saying; if you find topics that are that done-to-death, the solution is not to find the next most obscure Oe short story (or whatever it is you love by Danticat). Instead, it might be a good idea to move on to something else.

There's nothing wrong with using old packets as a guide to what might come up in the future and to get hints for what to write. The problem develops when everyone decides to write on the same thing. That probably can't be prevented entirely, but I think this is a situation in which packet archives can be used for good. If you're finding 50 hits for something in the last 2 years, that topic might be wearing itself out. For a lot of answer choices, that probably can't be gotten around since a good tossup is a good tossup even if it's come up before. For tournaments like ACF Regionals, I have no qualms about these kinds of repeat topics because they are general-purpose regular difficulty events and if the question is good enough to keep, it'll probably get kept. At that level, there are so many things that are going to come up because they are important that there's necessarily going to be some overlap with trendy answer choices. For tournaments of a higher level, some thought should probably be given to preventing an avalanche of trendiness. For example, as exciting as it is to be able to first-line tossups on Richard Rorty (except when Andrew is in the room), I'm happy to suggest that he be sent to the quizbowl retirement home for a little while, since he's come up in like every other tournament over the last few years.

Also, contrary to Brendan's point above, I would suggest that most people can become reasonably good players just by reading packets; what makes elite players elite is that they go beyond reading packets to the material itself. They have knowledge that you can't get just by looking at a lot of packets. Likewise with the really good writers and editors; they are able to write on interesting and new things because they have mastered the canon but have also managed to go beyond simply repeating what's already in it.

[setht]

I am on a handheld device in Transylvania so I will keep it short:

I adhere to the Westbrookian view because I reject the Weinerian concept of a natural canon that everybody is taught outside of qb. People and classes are in fact specoalized and subject to instructor and institutional whims and biases. But any idiot, no matter what his background, can open the archive and learn clues. Thus a strict Westbrookian approach is an equalizer that makes quizbowl success open to all.

I think there's much less variation in science classes than you might think--there certainly is some, but the variable material is generally a very small percentage of the total course material. In terms of figuring out what topics question writers should write on, then, it seems to me that looking up online course notes is a good option for science questions. If someone is worried that they might be looking at an outlier class, they can always look up an analogous course at one or two other universities--every university is going to have intro courses on quantum mechanics, electromagnetism, etc.; if something shows up in course notes at 3 universities, it seems a safe bet that it's not a random pet topic of the instructor.

To my mind, telling incoming science players "quizbowl success can be yours if you learn the material taught those courses you're going to be taking anyway" is more appealing and novice-friendly than telling them "quizbowl success can be yours if you memorize science questions (including questions on stuff you'll never see in class or care about at all outside of quizbowl) from a bunch of recent tournaments."

-Seth

[AuguryMarch]

This is a really interesting thread on a topic that some of us fogeys have been discussing for a while. I just want to throw my two cents here for some context.

I think what you are seeing now resulting from the focus on old packets is the consequence of a strategy that doesn't scale. The analogy I have in mind is routing around traffic using GPS. There is some recent research suggesting that the overall efficiency of drivers actually decreases once everyone has traffic route information. Everyone trying to optimize using the same data causes a feedback effect that floods the "back roads" with drivers. Similarly if you look at the stock market, you see that most strategies to "beat the market" end up getting subsumed by the market over time (no strong form efficient markets hypothesis for me thank you very much!). Arbitrage opportunities go away.

Similarly, the packet studying strategy in QB is not a strategy that scales. Back in 1999 when I first started playing seriously, the method of pouring over old packets and looking for patterns and regularities was relatively unused among top players. Subash famously memorized 30k+ clues from Benet's to prepare for the famous 2000 showdown with Andrew--he did not memorize old packets. Moreover, the lack of an archive meant that it was difficult to even acquire the tournaments needed to do this systematic data mining (the more I think about it, the more the analogy to the efficient markets hypothesis works). Therefore, the people who possessed the old tournaments (and trust me, we hoarded it) had an advantage over other players in the game.

The unfortunate downside of this, as all of you point out, is that this has caused skews in the "natural" frequency with which certain topics come up. If you are studying packets all the time, relying on the Westbrookian method of relying on armchair intuition, the end result are recency biases (I'm more likely to write about things I've thought about recently) and availability biases (The easier it is to call a given QB topic to mind, the more likely I'm going to think its "gettable"). This is an exact recipe for skewing.

So what's the solution? Obviously the cat is out of the bag--you can't go back to a pre-archive period (nor would you want to necessarily). One possibility is the Hilleman route--go through the entire archive of "good questions" and tag them for frequency. Make these frequency tables widely available and make it such that when you upload new tournaments you get instant data on how the frequencies change. Also track yearly frequency trends. The end result is that we are arguing based on concrete fact instead of intuition (sorry to be so down on intuition) and moreover, it should be easier to spot (and defuse) Danticat phenomenon. You could also write an "answer space sampler" that selects a random Fall - Regionals level answer based on a modified frequency table to help new writers write.

This of course elides the question of clue repetition which in my mind is even more problematic. Does the middle clue stay in the middle over time? If you migrate the middle to the giveaway that is making the game less accessible. If you don't, you are left with the Seth problem-- new players will find that their surest route to success is memorizing old packets and old packet memorizers will have a field day. My feeling is that we have to cede something to the memorizers and so we should come down against clue migration (at least for anything below nationals levels).

edit: utter lack of grammar