High Schoolers and Writing for NAQT

[Ken Jennings]

I'm a longtime NAQT member and editor, though I'm typically too busy to spend much time being a public face of NAQT, on this forum or anywhere else.

But someone pointed me to this thread, where I see that much of the discussion has centered around some imagined core of incompetent, out-of-touch NAQT member/editors who keep the organization hopelessly devoted to "bad quiz bowl" despite the best attempts of new, gifted recruits. A high school player above even used me (well, ) as shorthand for this kind of lousy editor.

I don't know if I really epitomize NAQT's suckiness to the hsquizbowl crowd, but if I do, I want to make myself available for criticism and discussion, so as not to perpetuate NAQT's reputation for unresponsiveness and opacity. That's a reputation that's not wholly undeserved, of course, though I think it's mostly for the usual lame reasons (impossibly busy members, inertia, reluctance to tinker, slowness to achieve internal consensus, etc.) and not anything more malicious. Usual disclaimers apply; I'll be speaking only for myself here, and not for NAQT.

I think many of the specific criticisms about NAQT question quality that I see referenced above are out of date--the "Tolstoy question with the Futurama lead-in," for example. It's true that, editing lit for NAQT in the past, I've seen the occasional trashy clue. I wasn't always a zealot about exterminating these questions, and I seem to recall Eric Hillemann wasn't either, and so they appeared in NAQTsets from time to time. But the general tolerance for that kind of clue faded, and so did mine, and I don't know how many months it's been since I've even seen one of these submitted to NAQT, or how many years since I let one through during my subject-editing pass. (If I like the non-academic clue well enough to leave it in, the question gets recoded as general-knowledge or pop-culture or something similar so as not to clutter the literature set quota.)

It's true that more levels of care go into our high-level sets than to our 'A' sets, but my sense is that NAQT cares deeply about the quality of every question, and feels their faults pretty keenly on those occasions when they're demonstrated. (I've remember times when tossups I've written or edited have been justly criticized for mis-ordered clues and whatnot--hasn't everyone, sometimes?--and I know I've felt bad about it and worked hard not to repeat the same mistakes.) Certainly, behind the scenes, no one at NAQT is winking at the lousy questions that the rubes are falling for while we count our dirty, dirty money. Personally, I know my editing process is identical no matter what difficulty level the question is written for--in fact, as a blind double-check on difficulty-coding, I like to take a pass through each question before I look at how the author coded it. Bad quiz bowl is bad quiz bowl, period, at any level. I think I'm pretty hard on writers who make repeated mistakes with pyramidality, answer choice, smoothness of difficulty gradient, lucidity of prose, specificity of detail, uniqueness of clues, and so forth--and I've gotten much harder in recent years as circuit demands on question quality have risen (a very good thing) and as NAQT has taken on top-tier editors like Subash, Andrew, and Seth to help institutionalize their ideas of good quiz bowl at NAQT (also a good thing).

Anyway, not to protest too much, but I did hope to get across the idea that, in general, NAQT subject editors--even the ones you may not know personally or think of as circuit regulars--do care about current norms of good quiz bowl. I say "in general" because I can't speak for everyone at NAQT, and because I know that lousy questions do get by me sometimes despite best efforts. But I can speak for my own work, which I'm generally happy with. I currently edit all literature, mythology, and film for NAQT, so please do respond with your gripes with questions in those areas--especially on specific questions, as far as question security policies permit. (I also edit NAQT's small quota of "general knowledge" questions, which I'm sure you all love no matter who's editing them.) I don't want to hijack this thread, but feel free to branch off another one called "Ken Jennings Is a Crappy Subject Editor!" and let me have it.

I am hoping to get good feedback when criticism is merited, but, obviously, I'd also like the chance to defend NAQT when I think it's not. If people sneer at NAQT because the questions are letting them down, that's fair enough. Let's work on that; here I am. But if people are going to start sneering at NAQT just because conventional wisdom requires it, we get into a vicious circle that's no good for anybody.

[magin]

Ken,

Thanks for posting. If you're interested in having a discussion, I have a few questions for you: What do you consider the core tenets of a good question? Could you post examples of questions you've written for NAQT that you consider good? Thanks.

[theMoMA]

NAQT was the organization that I first wrote for, and also the first organization for which I edited a complete set of quizbowl questions. There's no need to get hypothetical about whether NAQT's current policies drive away potential editors for their product. It has, and I am one of them. I rarely write questions anymore because I hate using their overly complicated markup language and changing grammar to get from 436 to 425 characters. I rarely edit sets anymore because changing leadins on BrainBusters D TV show sets from pop culture to academic seems like tilting at windmills and my labor is more valuable than that. I rarely involve myself in anything NAQT-related anymore because my future role with the organization has never been clear. Despite my role as one of the most active editors of pyramidal high school questions, I don't know if I would even be considered if subject editorships for high school questions were ever released from their indefinite bondage to old NAQT personalities.

I obviously didn't come into NAQT with the standing of Andrew or Seth, whose comments seem to be colored by the fact that they were handed all sorts of keys to the NAQT kingdom. You guys should remember that none of us are coming to NAQT with the resumes you bring, and that defines our experience as NAQT participants. I never felt like there was a place reserved for me at the top, and I certainly imagine this is one of the reasons that young, promising editors go elsewhere. Maybe some of them will come to NAQT as grad students, as Seth suggests. But I have yet to see a reason for this other than NAQT's history of keeping power with its longterm members until community criticism reaches critical mass, then bringing in big-name editors to placate the community. NAQT has no history of nurturing a pool of young editors; indeed, the past indicates that to make it big in NAQT, you have to earn editor cred somewhere else.

Contrast this to my roles in ACF, PACE, and HSAPQ. When I signed on to ACF Fall 2007, Eric Kwartler basically told me that I could have a prominent role in the future of ACF if I was willing to work to become a better editor. This gave me the community standing to be invited to be the head editor of ACF Fall 2008 and the upcoming PACE NSC, in addition to being sought out for one of the four chief editorships of HSAPQ. These organizations all assured me that the work I was doing would count towards my future role.

More to the point of what we've been talking about...

One of the hardest parts of moving away from NAQT for me is that I know and like R. Hentzel, and as such I suspect a lot of the nefarious things accused of NAQT are not true. But I do get the feeling that this is true: NAQT reads its quizbowl-community critics as only a small portion of the sales demographic pie. This troubles me. I get the feeling that NAQT conflates the community with "elite players," which couldn't be further from the truth. The quizbowl community extends from the best college teams to the high school teams that come to every pyramidal tournament, lose every game, and say they had a great time even when the questions are way over their heads. I also get the impression that NAQT always reads criticisms coming from "the community" as almost parroted from top players. In other words, I think that when Chris Carter criticizes NAQT, NAQT reads it sort of as Matt Weiner criticizing NAQT through Chris. This goes back to the elite player-community member conflation; NAQT seems loathe to believe that individual players of all skill levels align themselves with community standards on their own, choosing instead to believe that they have been almost "contaminated" by interacting with elite players or reading hsquizbowl.org.

This hypothesis has extraordinary explanatory power over NAQT's actions and attitudes. If you view community standards as merely a construction of an elite oligarchy parroted by impressionable, "infected" people, you could easily see your mission as sufficiently placating the community so that it will buy your product, while also producing material contrary to community standards to sell to other areas. If you believe that the community will not grow beyond elite players, it makes sense not to grow it, and instead work with the idea that you're producing questions for those of all standards. I see this view as problematic, and NAQT should at least be worried. If the community is not just an echo chamber for elite players, if the community really is something that can grow, NAQT might be in trouble if it does.

[theMoMA]

Oh, and if the above is not true, I challenge NAQT to act accordingly.

[Stained Diviner]

All of these examples are taken from IS-66. Just to be clear, I think that there are plenty of good questions in IS Sets, and I am picking out a few I didn't like in the field of Literature. I don't want this to turn into an NAQT-bashing thread, since it was cool of Ken to suggest this, and there are other NAQT-bashing threads available. Also, I make no claims to be an expert on question writing and editing--I have been bashed on this board for mistakes I have made.

Packet 10, #10 begins: "This New Orleans native's novel Answered Prayers remained unfinished at his death."

I don't like the first four words of this question, since it allows students to guess the answer without getting the literary clues. Anne Rice and Lillian Hellman were from New Orleans, but that would be 'her death.' I'm not sure about the timing, but I think this question came out about the time that movies were coming out about Truman Capote, so biographical knowledge of him at that time in particular would be unimpressive.

Packet 11, #5 reads:

His short stories include a tale of Aminadab and Aymler attempting to remove a birthmark from Georgiana's face, one where Robin comes to a town searching for his kinsman Major Molineux, and one where Reverend Hopper wears a black veil over his face. Many of his works are set in Puritan Massachusetts, like his novel about the Pyncheon family. For 10 points--name this author of The House of the Seven Gables.

My problem with this is the clue about Puritan Massachusetts. In my mind, a generalized clue like that on a well-known writer such as Hawthorne should only be found in the giveaway, after some specific clues on Hawthorne's well-read works. As is, if the question gets to that point (which it often will given the level of play typical to IS sets), I could see a lot of students asking themselves whether or not to guess Hawthorne based on that clue, which is not as good as hearing a specific clue that a student would recognize as Hawthorne.

Packet 11, #20 begins: "A splintered old black box is brought from the offices of the local coal company, filled with little slips of paper..."

If the answer to the question is The Lottery (which it is), then the beginning of the question should not describe something that sounds to anybody listening like a lottery.

Packet 11, Bonus #16B reads: "This Quaker first mate opposes Ahab's pursuit of Moby-Dick. A coffee company is named for him."

Since this bonus already had an easy part (Ahab), I don't think it is appropriate to give away a second part with a pop culture reference.

Packet 12, #5 begins:

This author's April 2 birthday has become associated with International Children's Book Day. Although he preferred to be known for novels such as To Be or Not to Be and poems like "I Am a Scandinavian," his reputation is based on his collected (*)

The asterisk denotes the power mark, which you'll note comes after it has been established that this is a very well-known Scandinavian children's author. That's Hans Christian Andersen for 15. The end of the question mentions titles of his well-known stories, as it should, but there is not one plot-related clue.

Just to be clear, this packet had questions on Stephen Dedalus and Polonius, so it did have literature questions directed at high school level literature, unlike the two I am criticizing.

Packet 12, #17, which is on The Ransom of Red Chief, states within power, "In the end, instead of the $1500 payoff they hoped for..." The biggest crime here is a power mark being placed too late.

Packet 13, #9 reads:

In May 2006 this man, then under treatment for amyloidosis at the Mayo Clinic, wrote of his sexual awakening at a garden party when he was three. He did this in the course of explaining how his observations of women inform his portrayal of characters like Egwene and Elayne in his twelve-volume series that will end with A Memory of Light. For 10 points--name this author of The Wheel of Time.

This is subjective, but I, without having read any of his works, was under the impression that Robert Jordan was not particularly substantive. Also, the first half of this question has very little to do with his works.

Packet 13 has three literature bonuses. One of them, Feste/Touchstone/Dogberry, seems harder than the others, which are Cerberus/Hercules/Aeneas and Death of a Salesman/Crucible/Marilyn Monroe (with Monroe as the hard part the way it is written). The Shakespeare question isn't impossible, but it doesn't seem to have any easy part at the level of IS.

Packet 15, #1 reads:

He created the character Mathieu, a socialist philosophy teacher, in his trilogy The Roads to Freedom. Bouville historian Antoine Roquentin feels that objects limit his ability to define himself in his novel (*) Nausea. For 10 points--name this French existentialist who declined the 1964 Nobel Prize in Literature he won for works such as the play No Exit.

This actually is a pretty good question. It has several good clues in the right order. I am including it to show how the question length limit hinders NAQT questions. When you get a writer like Sartre who comes up a lot in quizbowl, there have to be a lot of tough clues to avoid buzzer races. It is easy for me to imagine a lot of teams who don't know much about the book Nausea except that it was written by Sartre, so that seems like a likely place for a buzzer race. A buzzer race would be less likely if there were more clues about that novel or clues about the plot of No Exit before giving novel titles, but that would make the question longer. Since NAQT limits the lengths of questions, you are more likely to get buzzer races.

Packet 15, #13 is about the Booker Prize. If the number of literature questions is kept small, then this would be a topic to avoid, replacing it with a question about a work or writer. This is the only Brit Lit tossup in the round.

Packet 15, #15 begins, "The order to destroy his writings upon his death went unheeded..." This is a fairly well-known non-literary clue about Kafka, so it is a bad way to start a tossup. There are no plot-related clues anywhere in the question, though at least several titles are given in the second half of it.

Packet 15, #21 reads:

The novel ends with a discussion in a Madrid taxicab; earlier, a group including Michael Campbell, Bill Gorton, and Robert Cohn arrives in Spain to watch the bullfighting. Lady Brett Ashley is planning to marry Campbell, but she has an affair with Pedro Romero. The narrator, (*) Jake Barnes, is impotent because of a war wound. For 10 points--name this novel about the "Lost Generation" by Ernest Hemingway.

I approve of the first sentence for IS level, but Lady Brett Ashley is named too early--before the power mark and before a lesser-known character. This would be better if it was edited to avoid that problem.

Changed topic title to accurately represent content. -- The Mgmt.

[Stained Diviner]

I am not sure how NAQT can claim to be responsive to the overall circuit. It can point to market measures, such as the large size of HSNCT in recent years and the number of tournaments that use its questions, but there is a difference between being the best option available and being responsive, especially when your competition generally is at capacity, of low quality, poorly marketed, or just getting off the ground.

With the ICT, NAQT made an effort over the past few years to survey participants. Has something similar happened with IS Sets, A Sets, or HSNCT? If somebody wants to dismiss the overall opinions on this board as being biased towards elite teams or the East Coast, they can make that argument, but they shouldn't claim to be meeting the needs of the overall circuit unless they have made an effort to determine the needs of the overall circuit. I have no doubt that NAQT is meeting the needs of some teams, including in many ways mine, but there probably are some things it is doing that are not meeting the needs of the overall circuit, and I don't think anybody has made a substantive effort to determine exactly what those things are.

[Birdofredum Sawin]

I think, whatever historical audience ACF has had, the point is to consider both the "ideal" and "practical" audiences of both organizations. For ACF, our ideal audience is anyone who enjoys questions that are interesting, that reward knowledge before guessing, that reward primarily academic knowledge over general knowledge or pop culture, and that are of an appropriate difficulty. As is so happens, a large part of our practical audience has the same goals as our ideal audience. Furthermore, I think that our attempts to produce tournaments like the most recent ACF Fall demonstrate that we are not interested in being the medium for "a small cadre of self-selected elite members of the quizbowl world, but for everyone who considers themselves a part of our ideal audience. Not only that, but the lack of fragmentation of ACF's practical audience suggests that ACF is responding to criticism and attempting to produce what players want. There is no dedicated faction arguing for much fewer literature questions, or questions in line with the 05 Nationals; ACF Nationals is now much more accessible than it was in 2005 precisely because ACF has been prompt to adapt to public criticism without compromising any of the principles of its "ideal" audience.

I don't think the "lack of fragmentation of ACF's practical audience" suggests anything more than that ACF has happened not to have a fragmented audience. ACF isn't "responding" to criticisms like "it's absurd for 1 out of every 5 tossups to be on lit, while philosophy only comes up once a packet" because (as it happens) nobody is levying that kind of criticism at ACF. There's no obvious reason why ACF's entire audience ("ideal" or "practical") should be so satisfied with the ACF game, especially when you consider how arbitrary the distribution is. Why should mythology and classical music be roughly on par, and each represented about one/fourth as much as lit? They just are, and people seem to be fine with it.

I'm not saying people should be rising up in arms against the ACF distribution, as they have against NAQT's. I'm just saying that it happens to be the case that ACF, at least in the last decade, hasn't really had to "adapt" to substantial, hostile criticisms of basic aspects of its product, because people haven't been raising such criticisms. One could argue that the absence of such criticism indicates that ACF is doing everything right; one might also argue that differences in the "practical" audience of ACF make people less likely to offer such fundamental critiques of the product. For all we know, people who don't like such aspects of the ACF game don't bother criticizing it, but just go play NAQT instead.

I think you're misunderstanding my position. In my opinion, ACF has been very responsible to its ideal audience, which happens to coincide with much of its practical audience. I don't think ACF is just for "the preferences of elite players," I think that's it's for everyone who self-identifies as a member of ACF's ideal audience. NAQT has been responsible to its ideal audience, which happens to conflict with the preferences of many members of its practical audience. Furthermore, I think you're positing an audience ("non-elite players" (which I think is insulting to a large percentage of the circuit), or people who like NAQT the way it is, or NAQT's ideal audience) whose preferences cannot be ascertained, precisely because they have not spoken up publicly on any quizbowl forum. If you believe in the public sphere (and I do), ACF has been responsible to members of the public sphere, while NAQT has not. I don't think that NAQT's ideal audience should include people who do not enjoy academic quizbowl, but I think NAQT's questions have shown that such players are part of their ideal audience. I think that's bad for quizbowl.

Maybe I should be clearer on what I mean by "elite." I don't use the term to mean "the top X% of scorers at a given tournament." Instead, I take it to mean "people who are passionate enough about quizbowl to not only play at tournaments, but to be active members of the community" (which I would loosely define as, e.g., participants on hsquizbowl; conversationalists in the chat room; authors of freelance packets; members of one or another quizbowl organization). That "elite" is obviously "self-identified," since you become part of it by choosing to participate in things like this debate. (It is an "elite" in the sense that it cares about these things in a particular way, not in the sense that it performs at a particular level.) When I further call that elite a "small cadre," I don't mean that it couldn't grow much larger -- indeed, I'd love to see it do so -- but that, as a matter of fact, it really isn't all that large. (How many people are active on hsquizbowl? What is the ratio of that number to the total number of people who play quizbowl more casually in a given year?) I'm not "insulting" the majority of the circuit by labelling them "non-elite"; I'm just pointing out that there are a lot of people who seem to show up to random tournaments without wanting to be "active members of the community" in any substantive way.

I don't know what it would mean to "believe in" (or, for that matter, not to believe in) the "public sphere." My point is that what you call ACF's "practical audience" is basically co-extensive with the set of people who are active members of the community. I agree that ACF has been very responsible to that group; but, again, I don't see any reason to believe that "being responsible to people on hsquizbowl" is equivalent to "being responsible to the public sphere per se." By the same token, I don't see any reason to believe that "speaking up publicly on a quizbowl forum" is the sole way in which the preferences of people who play quizbowl can be ascertained (unless, again, you conflate "being active on hsquizbowl" with "having a preference which is worth heeding").

Andrew

[DumbJaques]

If I was trying to write a terrible question for satire purposes, I doubt I could do much better than including clues about the childhood sexual experiences of Robert Jordan (while discussing, of all things, the precise nature of his present medical problems). Setting aside the trash lit vs. "real" lit thing, I'd hazard that these kind of clues are going to incur the bulk of criticism of NAQT lit tossups. These clues are either useless or fraudable in the extreme (HCA and the children's lit/Scandinavia thing) and in all cases are immaterial to the author's works. NAQT lit questions are amongst the most fraudable of any questions anywhere, despite being far from the worst class of questions anywhere. Even tossups which use clues entirely from the works at hand (see the Sun Also Rises question) often use fraudable clues. I'd hope that the writers and editors (by the way, Ken, if you read this, thanks for taking some time to answer people here) understand that saying characters meet someone in Madrid has the same result as starting the question off with "this work is set in Spain." I think it can be universally agreed upon that such a clue is really bad given that the answer is just really notable for being set in Spain, and many more people probably know that The Sun Also Rises is set in Spain than can identify the personality traits of Lady Bret Ashley.

[magin]

I don't think the "lack of fragmentation of ACF's practical audience" suggests anything more than that ACF has happened not to have a fragmented audience. ACF isn't "responding" to criticisms like "it's absurd for 1 out of every 5 tossups to be on lit, while philosophy only comes up once a packet" because (as it happens) nobody is levying that kind of criticism at ACF. There's no obvious reason why ACF's entire audience ("ideal" or "practical") should be so satisfied with the ACF game, especially when you consider how arbitrary the distribution is. Why should mythology and classical music be roughly on par, and each represented about one/fourth as much as lit? They just are, and people seem to be fine with it.

I don't think the distribution is entirely arbitrary; I think it's a reasonable attempt to balance askable academic categories with answer space (for instance, literature comes up a lot because the answer space for literature tossups is larger than the answer space for mythology). If people have reasonable criticisms of the ACF distribution, I for one would love to hear them, so I can listen to them, think about them, and reasonably respond to them.

I'm not saying people should be rising up in arms against the ACF distribution, as they have against NAQT's. I'm just saying that it happens to be the case that ACF, at least in the last decade, hasn't really had to "adapt" to substantial, hostile criticisms of basic aspects of its product, because people haven't been raising such criticisms. One could argue that the absence of such criticism indicates that ACF is doing everything right; one might also argue that differences in the "practical" audience of ACF make people less likely to offer such fundamental critiques of the product. For all we know, people who don't like such aspects of the ACF game don't bother criticizing it, but just go play NAQT instead.

I don't think that ACF is a flawless organization, but I think that it's been much more receptive to substantive criticism than NAQT (see your own explanations for the difficulty of the 05 Nationals, followed by a significant drop in difficulty of the succeeding ACF Nationals, as well as the 6-line hard cap imposed by the ACF editors on ACF Fall). Also, I think that it is untrue that people with criticisms of the ACF game don't bother to criticize it. Not only that, as an ACF editor, I want to hear criticisms of ACF, so that myself or another editor can talk to the critic and understand why he or she is frustrated and what he or she would like to see changed. If we have an open discussion, we can decide whether the criticism is accurate or not. By not addressing serious criticisms publicly, NAQT is preventing such a discussion from taking place, which frustrates people who've voiced their criticisms.

Additionally, I think there is a very good reason that more people have criticized NAQT than ACF; NAQT has produced more frustration. For many players, math calculation tossups are frustrating. Hearing tossups with vague or misplaced clues and/or cute giveaways are frustrating. Hearing more current events questions than music questions is frustrating for many players, as is hearing more geography than social science, more pop culture than world history (these are all conjectures, since I don't have access to the NAQT distribution, but I wouldn't be surprised if they're all true). List bonuses are frustrating. Tossups whose first sentence does not contain an identifying pronoun are frustrating.

On a level above playing the questions, seeing NAQT consistently respond to thoughtful feedback with "sign up to write questions for NAQT if you have problems" is frustrating. In fact, NAQT's non-presence on this board is frustrating, since this board is the only current place where serious discussion of high school and collegiate quizbowl takes place, to my knowledge. NAQT choosing to produce a plethora of speedcheck sets and TV sets when there are the aforementioned problems with their pyramidal sets is frustrating. NAQT announcing CCCT to be held the same day as ACF Regionals before contacting me, the head editor of ACF Regionals, is frustrating. And so on.

On the other hand, I don't think that ACF has produced a comparable amount of frustration. In the past, people were most frustrated by the difficulty of ACF's questions, which is why we've made a concerted effort to make ACF easier. Then, people were frustrated by the length of ACF tournaments, and we've made them shorter. If people are frustrated with ACF for reasons I haven't mentioned, please bring up those issues, whether publicly or privately (to members of ACF). I think that ACF's questions have been less frustrating for players than NAQT's because they have been, on the whole, consistently better written. I'm sure there have been many fine NAQT questions, but the set of frustrating NAQT questions is much larger than the set of frustrating ACF questions.

Therefore, because NAQT has produced more frustration, they are the target of more criticism than ACF. Not because people who want to criticize ACF don't speak up (perhaps there have been some cases where fear of vitriol on this board has prevented criticisms of ACF; I am personally opposed to vitriol on this board for precisely that reason, in fact, but I can't see how such hypothetical cases make up any significant percentage of criticism).

[Cheynem]

Oh, no, Robert Jordan. I used to get so pissed at that guy in high school because it seemed like an ill-timed Robert Jordan bonus would always come up to hurt our team in a close match.

I'm not a lit expert, so I can't speak too much about the fraudability of lit, but without getting into specifics, at the NAQT set I played this year, I found the history to be eminently fraudable--just way too many "this sounds like stuff that occurred in this country, this is famous from this country, BUZZ" moments or "we'll throw in a pop culture clue halfway through rendering the rest of the question meaningless." I think the shorter length of NAQT questions lend themselves to be frauded more, as do a few issues in pyramidality.

[Sir Thopas]

Yeah NAQT notably doesn't seem to care about linguistic fraudability at all.

[Birdofredum Sawin]

I don't think the distribution is entirely arbitrary; I think it's a reasonable attempt to balance askable academic categories with answer space (for instance, literature comes up a lot because the answer space for literature tossups is larger than the answer space for mythology). If people have reasonable criticisms of the ACF distribution, I for one would love to hear them, so I can listen to them, think about them, and reasonably respond to them.

As it happens, I can speak to this point with a great deal of authority, inasmuch as I'm largely responsible for the current ACF distribution being what it is. The brief story: When the previous incarnation of ACF broke up in the '90s, I helped coordinate a discussion of how ACF could be continued and what changes, if any, should be made in the new version. This unleashed a cacophony of proposals (tweaks in the distribution, adding power tossups, etc.). Out of a more or less Burkean sense that the distribution shouldn't be altered radically from the way it had evolved, I autocratically curtailed the debate and instituted a "new" ACF that maintained the previous distribution.

While I'm very fond of that distribution, I think it's a mistake to rationalize it in the way Jonathan does here. For instance, it makes more sense to say that the nature of the distribution largely defines the "answer space" in a given category, rather than to treat the answer space as a pre-existing given which then dictates the distribution. Another way of putting the same point: If the ratio of music to lit tossups was 4:1, rather than 1:4, we'd have tossups on the fifth-best-known Dvorak opera and Frederic Rzewski instead of on the fifth-best-known Howells novel and Anthony Hecht.

Additionally, I think there is a very good reason that more people have criticized NAQT than ACF; NAQT has produced more frustration. For many players, math calculation tossups are frustrating. Hearing tossups with vague or misplaced clues and/or cute giveaways are frustrating. Hearing more current events questions than music questions is frustrating for many players, as is hearing more geography than social science, more pop culture than world history (these are all conjectures, since I don't have access to the NAQT distribution, but I wouldn't be surprised if they're all true). List bonuses are frustrating. Tossups whose first sentence does not contain an identifying pronoun are frustrating.

On a level above playing the questions, seeing NAQT consistently respond to thoughtful feedback with "sign up to write questions for NAQT if you have problems" is frustrating. In fact, NAQT's non-presence on this board is frustrating, since this board is the only current place where serious discussion of high school and collegiate quizbowl takes place, to my knowledge. NAQT choosing to produce a plethora of speedcheck sets and TV sets when there are the aforementioned problems with their pyramidal sets is frustrating. NAQT announcing CCCT to be held the same day as ACF Regionals before contacting me, the head editor of ACF Regionals, is frustrating. And so on.

On the other hand, I don't think that ACF has produced a comparable amount of frustration. In the past, people were most frustrated by the difficulty of ACF's questions, which is why we've made a concerted effort to make ACF easier. Then, people were frustrated by the length of ACF tournaments, and we've made them shorter. If people are frustrated with ACF for reasons I haven't mentioned, please bring up those issues, whether publicly or privately (to members of ACF). I think that ACF's questions have been less frustrating for players than NAQT's because they have been, on the whole, consistently better written. I'm sure there have been many fine NAQT questions, but the set of frustrating NAQT questions is much larger than the set of frustrating ACF questions.

Therefore, because NAQT has produced more frustration, they are the target of more criticism than ACF.

I'm not denying that lots of the things you point to are frustrating. But again, you seem to treat "producing frustration" as some sort of acontextual absolute, when in fact you are bundling together things which are bad by basically any standard (e.g., the frustration caused by NAQT neglecting to respond to thoughtful feedback) with things which are much more contestable and arbitrary, even if they happen to track the avowed preferences of much of the hsquizbowl community (e.g., the frustration caused by what some people deem an inappropriate ratio of current events questions to music questions).

When ACF was basically the only alternative to CBI, and people who loathed the latter had no choice but to play the former, there was a lot more "frustration" produced by ACF sets. Then, as NAQT became a viable mainstream alternative and ACF evolved into a purer form of itself, people largely stopped expressing frustration with ACF. It's possible that this is because ACF has produced "consistently better written" questions, and has always made a "concerted effort" to improve in accordance with people's expectations. It's also possible that there has been a bifurcation of the quizbowl audience, with what I have described as a "self-selected" group of quizbowl "elites" embracing ACF, while NAQT has tried to appeal both to those people and to a group of people who have no interest in the perfectionist ideology of ACF. (Note that I say this as someone who loves the perfectionist ideology of ACF.) If the latter is the case, as I suspect it is, then it might be a good idea to consider distinguishing criticisms of NAQT which obtain regardless (like "it's really rude of you guys to schedule tournaments on dates which have already been claimed by circuit events") from criticisms which only obtain if you take for granted that "good quizbowl" is identical with "the form of the game preferred by the vocal hsquizbowl majority" (like "it's absurd to think that questions on current events could merit as much room in the distribution as questions on classical music").

Andrew

[Important Bird Area]

Yeah NAQT notably doesn't seem to care about linguistic fraudability at all.

I'd adjust that to argue that "NAQT believes that very few high school players know enough about foreign languages to linguistically fraud anything." If you know of a particularly bad example, that lots of high school teams could be expected to answer without actually knowing about the work of literature (or whatever) in question, either post it here or send me email, depending on packet security.

without getting into specifics, at the NAQT set I played this year, I found the history to be eminently fraudable--just way too many "this sounds like stuff that occurred in this country, this is famous from this country, BUZZ" moments or "we'll throw in a pop culture clue halfway through rendering the rest of the question meaningless." I think the shorter length of NAQT questions lend themselves to be frauded more, as do a few issues in pyramidality.

If it's a history question that's too recent for open discussion on this board, then I'm sure I was responsible for editing it. Please email me with the details and we'll discuss why these questions ended up the way they did. jthoppes [at] berkeley [dot] edu

[Stained Diviner]

I think that two different issues are being confused here.

Andrew states that the distribution is somewhat arbitrary and that, no matter what it is, people can come up with various reasons why it should be changed, which is true.

However, the main complaints with the NAQT distribution have been that it includes too much general knowledge/pop culture/sports. (There have also been complaints about the inclusion of computational math, which I'll ignore for the time being for a variety of reasons.) Furthermore, the people making these arguments aren't arguing that the distribution should go down from 15% to 13% or 14%, they are arguing that it should go down to at most 5%. This is a fundamental change rather than a tinkering.

Adding a classical music question or two to each ACF round would be a change that would alter the game, making certain individuals/teams stronger than they are with the current distribution, but I don't think it would fundamentally alter the nature of the game--it would still be about which team had more academic knowledge. However, changing 10-15% of the NAQT distribution from unacademic material to academic material would alter the game in a more significant way--it would mean that the only way to win a match would be to win on academic questions rather than possibly just hanging close on the academic questions and winning on the trash. It would mean that when I'm picking my fourth starter, trash would not influence my decision, which is different than what happens now.

In other words, I don't think it's equivalent to discuss changes within academic categories and to discuss increasing academic content.

[theMoMA]

I think Andrew is right to say that ACF is only responsible to the quizbowl community (I would advise against labeling the community "elite;" that word produces an unnecessary negative meaning). Speaking for myself as a member of ACF, I have long believed that spreading quizbowl is more than replacing bad questions with good; it's about bringing new voices into community discussions, mentoring new writers and editors, and ensuring the future of the game. Every ACF member will listen to anyone with enough dedication to make themselves heard, and the thing we want most is for new players to join the community.

NAQT's attitudes about its distribution seem to mirror its implicit dogma that the quizbowl community is only one demographic segment in the possible markets for its questions. NAQT is reluctant to engage in discussion about its distribution, and is loathe to change based on community suggestions. NAQT instead chooses to label community discussions "elitist" and implies that adopting community principles would cause it to lose business among those who choose not to discuss. These actions speak to one conclusion: NAQT would rather attempt to please those not in the community than grow the community itself.

[Birdofredum Sawin]

I want to take another shot at elucidating some of the points I've been trying to make in this thread. This time, I'm going to use Andrew Hart's comments about the "community" as a jumping-off point.

I agree that the quizbowl community is a great thing; I agree that it would be nice if it grew even larger; I agree that it extends from the upper crust of collegiate title contenders to high schoolers who "come to every pyramidal tournament, lose every game, and say they had a great time even when the questions are way over their heads." However, I don't think that the quizbowl community (in the sense Andrew and I seem to be using the term) is co-extensive with "everyone who plays quizbowl." It seems clear to me that there are a lot of people who play the game without being invested in a number of the things which matter to many of the people on this board.

Posters on this board often invoke the figure of the not-very-good player who, nonetheless, has "a great time even when the questions are way over [his] head." Yeah, such people obviously exist, and it's great that they are embraced by people on this forum. But I think it's simply untrue to pretend that they are the norm. In my playing career, I've been affiliated with three universities. All of those schools had rather strong teams, but it's just not the case that everyone associated with those teams enjoyed questions that were "way over their heads." I've known lots of people who liked quizbowl, but didn't particularly like questions of ACF length or the ACF distribution. The perpetual struggle at places like Chicago and Stanford (and I find it hard to believe that these schools are exceptional in this regard) has been to accommodate both the people who love the kind of quizbowl favored on this board, and also the people who don't much care for it, but who still come out to practices and would like to play questions more suited to their interests. I don't see why those people should, in effect, be told that they either need to shape up and learn to value the same things we value, or else stop playing altogether.

My position -- as I've argued in the past -- is that there should be a place in the game for people who aren't striving to better themselves, but who just like the game. I'm not saying that we should abdicate all standards. I'm saying that some of the things which are identified on this board as "good quizbowl" simpliciter are, in fact, better seen as criteria best suited to a particular segment of the quizbowl world (call it the "perfectionist" segment, if "elite" strikes you as an invidious term).

I've spent most of my quizbowl career trying to ensure that the perfectionist impulse would have a place in quizbowl: that's why I worked to keep ACF alive, and why I spent so long trying to maintain it as a viable form of quizbowl. I'm delighted that it is doing so well. But I don't think it should be the one and only form of quizbowl, or that people who don't enjoy it should be shamed or shunned. I think people who are interested in perfectionism should have ample opportunity to play a kind of quizbowl suited to their interests, and given the current vibrancy of the community, I think those people are being well-served at the present time. But I also think that people who aren't interested in perfectionism should have a chance to play a kind of quizbowl suited to their interests. Inasmuch as NAQT, in something like its current form, is providing them an opportunity to do so, I think it's a good thing for quizbowl as a whole.

Andrew

[Sir Thopas]

Are you arguing that NAQT being an admittedly inferior product is good for quizbowl?

[Mechanical Beasts]

Are you arguing that NAQT being an admittedly inferior product is good for quizbowl?

i couldn't imagine this is what Andrew means, so I'll rephrase this question, since it's related to one I'd ask.

I think that the goal you elucidated for NAQT is a noble one, certainly if there is something inherent to ACF that puts people off. Do you believe that the goal of making quizbowl accessible to people put off by the ACF model (if I misstate your idea of what NAQT ought to do, please correct me) is mutually exclusive with resolving many of the issues with the NAQT model that have been raised in this thread? Alternatively, what differences between NAQT and ACF are essential in order to accomplish the goal of bringing quizbowl to a wider audience?

[Captain Sinico]

Are you arguing that NAQT being an admittedly inferior product is good for quizbowl?

You're aware that you're talking to Andrew Yaphe here, right Guy? Maybe he deserves better than this glib, loaded question in reply.

MaS

[Sir Thopas]

Are you arguing that NAQT being an admittedly inferior product is good for quizbowl?

You're aware that you're talking to Andrew Yaphe here, right Guy? Maybe he deserves better than this glib, loaded question in reply.

MaS

Yeah, he does. My apologies.

[magin]

Are you arguing that NAQT being an admittedly inferior product is good for quizbowl?

You're aware that you're talking to Andrew Yaphe here, right Guy? Maybe he deserves better than this glib, loaded question in reply.

MaS

First of all, everyone on this board deserves to have thoughtful replies to their posts. On that note, no one from NAQT has seriously responded to Guy's original post (the current discussion with Yaphe seems like it merits a thread to itself); perhaps Guy's original post deserves a thoughtful reply?

[Birdofredum Sawin]

I think that the goal you elucidated for NAQT is a noble one, certainly if there is something inherent to ACF that puts people off. Do you believe that the goal of making quizbowl accessible to people put off by the ACF model (if I misstate your idea of what NAQT ought to do, please correct me) is mutually exclusive with resolving many of the issues with the NAQT model that have been raised in this thread? Alternatively, what differences between NAQT and ACF are essential in order to accomplish the goal of bringing quizbowl to a wider audience?

I'm going to continue my recent policy of responding only to people named Andrew, and say that this very much gets toward what I'm trying to argue. Let me recap what I take myself to be trying to accomplish here. I'd like to explain why I think NAQT's continuing existence is important, in response to a number of people who seem to be starting to think otherwise (as per the original drift of this thread). Basically, my argument runs something like this. As I see it, the locus of interest on hsquizbowl and in the articulate quizbowl community is in a certain perfectionist form of competition. But the reality is that there are a lot of people who are interested in playing some kind of quizbowl, but who have little to no interest in many of the things which people who prefer what I'm calling the "perfectionist" type of quizbowl enjoy. I see these people every time I show up to a Stanford practice: they're the ones who maybe drift in to the room where the hardcore people are playing ACF packets, sit for a round, then leave. I've seen similar people at Chicago and at Virginia and at all sorts of tournaments. I'm not saying that those of us in the community are a priori incapable of understanding these people; I'm saying that, in practice, we tend not to be very interested in them (save for hoping that they will become more like us: i.e., that they will take an interest in "good quizbowl" and want to improve).

I think that there are ways in which NAQT, in its current form, serves such people; and I don't see any reason why such people shouldn't be afforded the opportunity to play. I also think that it's important to distinguish criticisms of NAQT which "obtain regardless" of the kind of quizbowl one prefers, and criticisms which take for granted that everything NAQT does, at every level, should result in the production of a product which would be satisfactory to perfectionists in the community. I'm suggesting that the debate, at the point I entered it, seemed to be proceeding on the latter assumption, inasmuch as top-notch high school players seemed to be complaining that A-sets were beneath their contempt. I was trying to say: No doubt such sets are, but maybe that shouldn't be a concern in the way the discussion was making it out to be.

A general point I'd like to make, then, is that people are failing to differentiate a) universally valid and important complaints about NAQT and b) complaints which take for granted that the perfectionist norms preferred by the community should be ubiquitous. In the former class of complaints I'd put things like NAQT's alleged ethical failings, or complaints about its lack of transparency and responsiveness. There's just no excuse for those, no matter what criteria of quizbowl you're going by. In the latter class of complaints I'd put things like certain objections to NAQT's distribution, and (possibly) some of the disparaging comments about A-sets.

The basis for this argument, again, is my experience of any number of people who show up for quizbowl-related things, but who clearly lack interest in getting better at the game or in playing ACF-type questions. I think it's clear, descriptively, that such people exist. And I would argue, normatively, that they shouldn't be frozen out of opportunities to play.

Andrew

[Sir Thopas]

I'm going to continue my recent policy of responding only to people named Andrew, and say that this very much gets toward what I'm trying to argue. Let me recap what I take myself to be trying to accomplish here. I'd like to explain why I think NAQT's continuing existence is important, in response to a number of people who seem to be starting to think otherwise (as per the original drift of this thread).

That's all well and good, but your post didn't actually address any of the points I brought up originally. The debate at the point you entered it may have shifted, but I don't see why that makes my original post less valid or deserving of a response to allay concerns of mine and others. Furthermore, I feel like all the concerns actual high schoolers have brought up have been more or less unanswered. This debate over NAQT's purpose in the college game can be had, but it's shifted the topic away from what Chris, Sarah, I, and a couple of others, are actually worried about it. Could some plenipotentiary of NAQT please address these instead of all the secondary issues which have arisen?

[Birdofredum Sawin]

That's all well and good, but your post didn't actually address any of the points I brought up originally. The debate at the point you entered it may have shifted, but I don't see why that makes my original post less valid or deserving of a response to allay concerns of mine and others. Furthermore, I feel like all the concerns actual high schoolers have brought up have been more or less unanswered. This debate over NAQT's purpose in the college game can be had, but it's shifted the topic away from what Chris, Sarah, I, and a couple of others, are actually worried about it. Could some plenipotentiary of NAQT please address these instead of all the secondary issues which have arisen?

After looking back at your original post, I'd say that my arguments have, in fact, addressed a number of its points. Take this statement you made: "It is certainly possible to convince players that don’t know any better why their formats are bad; this has happened many times over on HSQB. As such, I am given the impression that NAQT cares foremost with furthering itself, instead of using the furthering of itself to further good quizbowl with it." In my view, this exemplifies the kind of assumption I've been talking about in my recent posts: namely, that the standards of perfectionist quizbowl favored by this board are the only possible standards, and that if NAQT isn't trying to "further" them, it must have some despicable mercenary reason for doing what it's doing. I've also addressed (at some length) your discussion of your bad experience playing an A-set, which seems to be one of the central concerns of your original post. Also, I should note that the argument I outlined in my previous post was not about the "purpose of the college game," but about quizbowl at all levels, including the high school game.

In addition, it might be unrealistic to expect swift responses from an NAQT "plenipotentiary" to a thread which, after all, was initiated a couple of days ago on Christmas Eve. (Though three NAQT members have, in fact, engaged in this thread, and Jeff also spoke to particular issues raised in your initial post.) I know NAQT has a deservedly poor reputation for unresponsiveness, but it seems a bit unfair to demand a prompt reply at this particular time of year. Not every NAQT member is stuck at home writing (or, more precisely, procrastinating on) papers which they failed to get done during the semester.

Andrew

[Sir Thopas]

Yeah, it probably is unreasonable to expect swift responses in December. Many of my points have certainly been answered, but the real point that I wanted to get out of this thread is—Why should all these top high schoolers, all of whom are moving away from NAQT, write for them? I may have been a bit unclear about my motives, but I haven't gotten a response to that yet.

[Ike]

I don't want to take sides with anyone here, but I do want to state my experience with NAQT and how it sort of applies to this thread, because in Ohio, a lot of people fall in that category Andrew points at.

I know when i was first introduced to NAQT I thought it million times better than OAC. No seriously, it owns the living daylights out of that awful format for millions of reasons. I savored every NAQT tournament I attended, whether it be on an A-set or an IS set, it was much better.

Then Weekend of Quizbowl was the first experience I had with the "perfectionist vision of quizbowl," that Andrew points to. It was extremely entertaining, and while the full blown goodness of the ACF vision had not seeped into my consciousness by then, it profoundly influenced it. After a while, I started to dislike NAQT and its gimmicks, and other practices. I felt like it was an epiphany of some sort. And that's what kicked in my desire to learn some stuff.

That said, I really wish NAQT did a better job of giving players the opportunity and the keys to the car for the more perfectionist vision of quizbowl. I had never heard of PACE until the car ride down with Garfield Heights, and except for hearing about a PACE tournament in the state of Ohio at a tournament i could not have gone to. I feel that right now, NAQT is providing a product that appeals a lot to that middle ground, and doing no effort to facilitate the growth of the person who seeks the top ground. Had I not stumbled upon Chris Ray's tournament, I can say I would not even be close to what I am now. Someone (I think Reinstein,) said something about R. Hentzel saying "our format of quizbowl is better than most local formats, and to keep that up." While many may interpret that as a capitalistic viewpoint, I think its a right viewpoint to have no matter what way you cut the cake, but I also wish R. Hentzel would talk to coaches saying that if you're team desires something more out of quizbowl "this is the way to go.."

Also, I think Andrew makes an excellent point on about two types of complaints, and it has merit. For example, if there is a current events distribution that is so high, and makes me want to cringe, let it remain there, but I ask that we move some of the complaints from the perfectionist column to the other column in that:

1.) Nontransparency and pyramidality of clues are used - This will facilitate that process of seeking out challenges above.

2.) Actually not appear to have a capitalist agenda. I didn't very much appreciate R.'s earlier comment about challenging other people to start up another company because "we believe in capitalism" (-its not exactly those words, but something that gets that point across.) Because I think whether we have bad intentions, most people do not want to see NAQT driven out of business.

[magin]

But the reality is that there are a lot of people who are interested in playing some kind of quizbowl, but who have little to no interest in many of the things which people who prefer what I'm calling the "perfectionist" type of quizbowl enjoy. I see these people every time I show up to a Stanford practice: they're the ones who maybe drift in to the room where the hardcore people are playing ACF packets, sit for a round, then leave. I've seen similar people at Chicago and at Virginia and at all sorts of tournaments. I'm not saying that those of us in the community are a priori incapable of understanding these people; I'm saying that, in practice, we tend not to be very interested in them (save for hoping that they will become more like us: i.e., that they will take an interest in "good quizbowl" and want to improve).

I think that there are ways in which NAQT, in its current form, serves such people; and I don't see any reason why such people shouldn't be afforded the opportunity to play.

I agree that such people exist. My question, then, is: are there elements of NAQT that appeal to such people? Can we specifically identify those elements? Are those elements at odds with principles of good writing (such as pyramidality, concrete clues, and rewarding knowledge over guessing)? If such players like non-academic categories, shorter questions, or timed rounds, I don't see why NAQT cannot have those casual-player-preferring elements along with good questions. My issue with NAQT is not that they appeal to casual players instead of me (although I would prefer a different distribution, longer questions, etc), but that they produce many questions at odds with principles of good writing. Therefore, I'd like to see some data, or some statements from people who speak for NAQT, that define these "casual" players and identify what such players enjoy, instead of hypotheses that such players like NAQT for unknown reasons (if, indeed, NAQT does consciously attempt to appeal to such players).

[AlphaQuizBowler]

Andrew Yaphe brought up something that I'd like to address: why do elite teams go to A-set tournaments? I have an experience that I think can help explain. This year, the Walton Varsity tournament was run on an A-set. Walton Varsity has a history of attracting top teams from Georgia (Chattahoochee, Brookwood, &c.), South Carolina (Dorman, James Island, &c.), and other states. My school also attends annually. Those teams, and my team, aren't going to skip an annual tournament or send an introductory team just because it's on an A-set. It's a Varsity competition, so the best teams are going to go. In this case, I think it is the error of NAQT in assigning a clearly top-level tournament a clearly low-level set. Because of this, I don't think that you can make a blanket statement that it's the top players' faults for playing A-sets.

Another issue I see is this. In classifying ACF as "perfectionist" and thus NAQT as "non-perfectionist" (which I'm willing to accept) doesn't that undermine NAQT's authority to award a national championship? In other words, can a competition that is inherently not the best legitimately determine that a certain team is the best?

[Down and out in Quintana Roo]

Andrew Yaphe brought up something that I'd like to address: why do elite teams go to A-set tournaments? I have an experience that I think can help explain. This year, the Walton Varsity tournament was run on an A-set. Walton Varsity has a history of attracting top teams from Georgia (Chattahoochee, Brookwood, &c.), South Carolina (Dorman, James Island, &c.), and other states. My school also attends annually. Those teams, and my team, aren't going to skip an annual tournament or send an introductory team just because it's on an A-set. It's a Varsity competition, so the best teams are going to go. In this case, I think it is the error of NAQT in assigning a clearly top-level tournament a clearly low-level set. Because of this, I don't think that you can make a blanket statement that it's the top players' faults for playing A-sets.

This is a point i just brought up yesterday in a thread that stemmed from this... viewtopic.php?f=6&t=6989#p113107

No one had answered it yet. But this is at least a somewhat valid explanation.

[grapesmoker]

I have a much longer post that I intend to make on this topic, but I do want to note that there is no conflict between the "perfectionist" ideal of quizbowl and the more "casual" players of whom Andrew speaks. I don't think anyone has ever made the argument that we should alienate those people. The vast majority of changes that the "perfectionists" would like to see are actually compatible with serving that community; in addition, there are plenty of structural problems with NAQT that would be problematic regardless of question content, and no one has adequately addressed those issues. I'll try to flesh out those ideas later today.

[Brian Ulrich]

A lot of the concern about NAQT seems to be that, similar to the Alliance of Twelve on Alias, it has wandered from its initial mission and become all about the pursuit of profit at any cost. This view may contain the unexamined view that making money and producing quality sets exist in a zero-sum relationship.

NAQT's first venture outside the circuit-style hs and college market was not TV, but rather some primarily PC and Sports questions it produced for Microsoft around 2000 and 2001. I remember at that time, when I was a new writer, R admitted there was some internal discussion over whether that would take it too far from its mission. The most important argument in favor of going ahead, again, as I recall, was that if NAQT was ever going to be the agent of raising the social profile of quiz bowl in communities, corporate partnerships would be useful.

In addition, NAQT is a business, and as such, is healthy only if it keeps growing. A TV show looking for a question provider will find one regardless of whether it is NAQT. Being that provider not only enhances NAQT's bottom line, but ensures that provider will not be a competitor who could ultimately expand to undermine NAQT, while also ensuring that the NAQT brand is represented in that TV market. If such a show insists on a 150-character limit, then NAQT has to work with that, and will work hard to try and write pyramidally within those confines - our standards mandate at least three buzzable clues in each such question, with the majority ideally having four. If NAQT then decides that as long as it has these old TV questions lying around they might as well package them into "Speed Check" sets, I don't see a problem with that, since they're already written and edited.

NAQT also absorbs money in terms of infrastructure maintenance and hours spent. For example, quiz bowl doesn't just spread into a new area like a disease, it has to be marketed. That takes time for which the person doing the marketing requires compensation, which requires a healthy bottom line. And let's be clear - no one is getting rich off NAQT. If that were the point, R, the fearless leader, would try to do something else with his life.

If a committed new high schooler were to write for NAQT, and that person were to write only tossups for the regular IS sets and above, they would wind up in those IS sets. No one would force them to write anything else. This would not improve the quality of TV questions, but if you don't care about those, the fact their quality doesn't improve shouldn't bother you.

[BuzzerZen]

quiz bowl doesn't just spread into a new area like a disease, it has to be marketed.

NAQT's propensity for marketing quiz bowl to uncharted areas is probably its greatest contribution to the community at large, but it is also gives rise to some of the frustrations that certain people have with NAQT's practices. NAQT seems to me to be the only organization operating in high school quiz bowl with the resources to directly and effectively promote good quiz bowl in areas where bad quiz bowl is prevalent or where quiz bowl doesn't exist at all. (Though I know HSAPQ is beginning to market itself to likely prospects in a number of regions.) I think this is admirable. However, such efforts would be even more admirable if NAQT made more of an effort to guide the teams it brings into the game to the community at large. It certainly doesn't make sense for NAQT to provide marketing support for events run on questions from other providers, but I think even a little more honesty on NAQT's pages about the nature of the community would be appreciated, since the "about quiz bowl" pages currently don't do much, other than linking to this board, to suggest that quiz bowl events run on questions from providers other than NAQT exist, not to mention the other legitimate high school national title. The NSC and the HSNCT have coexisted for a decade now without hindering the growth or legitimacy of either.

[theMoMA]

The things that draw casual players to NAQT aren't necessarily the things that anger more serious players. Short tossups, a timed game, greater emphasis on "awareness of the world around you" questions, and powers seem to be attractive to the casual player. I and others who enjoy variety of competition don't believe these are necessarily bad. NAQT is a different form of quizbowl than circuit quizbowl, and I have no problem with that.

NAQT often holds that it cannot adhere to community standards because its product as it stands is popular with casual players. I don't think this excuses writing out and out bad questions that blatantly skirt community standards. If NAQT finds that these questions actually increase popularity with the casual player, I guess I can regrettably live with the fact that NAQT chooses to believe that invested players are less valuable than the casual demographic. But if NAQT doesn't know why its own product is popular with casual players, and allows bad questions into its sets because it is loathe to change, that's a big problem. It means that NAQT is not responsible to anyone.

Certainly NAQT has shown responsiveness to college voices recently, and everyone appreciates the effort. But the criticism of IS sets seems to follow a different pattern. Guy's complaint about linguistic fraud in high school sets, and Jeff's response that NAQT doesn't believe that enough high schoolers have the knowledge to convert tossups by figuring out linguistics, is indicative of this behavior. Does NAQT really believe that language transparency (which it admits is over the heads of most high schoolers) draws in casual teams? Or is linguistic transparency a tenet of good quizbowl that casual players don't even realize exists and thus cannot color their experience, and NAQT is simply refusing to abide by community standards because it's too much trouble to address the criticisms of a few "elite" players?

A lot of the problems with NAQT have nothing to do with being responsible to casual players as well as "elite" ones. When players bring up bad question-writing practices, it often seems that NAQT says some variant of "Circuit players just hate power tossups, the clock, current events, and/or character limits! Stop meddling and just let us be us!" Most of us are willing (and I personally am more than willing, since I value format variety) to let NAQT be NAQT. We would just like to see many bad question-writing practices that are prevalent in NAQT's questions go the way of the variable value bonus.

That said, I am still concerned that NAQT's organizational attitude towards the quizbowl community seems to be reluctant acceptance at its existence rather than wholehearted embrace, or at least respectful partnership. I would be a lot happier with NAQT if its actions as a whole indicated that it values the quizbowl community as more than one of many potential markets (and, it would seem, as the most inconvenient market) for its questions.

[Important Bird Area]

Many (most?) questions that "blatantly skirt community standards" also skirt NAQT's own standards.

For me, the debate on this board centers around "what current community standards have not yet been adopted by NAQT?" One recent example would be the discussion about cross-subject giveaways.

The question "why has NAQT not yet adopted these standards" brings up the larger questions of accountability, transparency, recruitment of new writers and editors, and so on.

I think we need a new thread to talk about the meaning of "linguistic fraud," so I'll reply to Andrew over there.

[magin]

I think I wasn't clear enough in my explanation of responsibility, so I'll try to be more lucid. In my opinion, responsibility involves ethical behavior and responding to feedback (both with reasonable explanations and procedural changes). I think the question of NAQT's ethics belongs to another thread, so I'll leave that alone. However, I think NAQT has a responsibility to have one or more qualified and able spokesmen to respond to the concerns elucidated in this thread, and to players' concerns in the main. Too often, I think such a spokesman for NAQT has either not responded to public feedback, or responded inadequately.

Secondly, I want to touch on the responsibility involved in producing questions. Most tournaments have two levels of question production: writers write questions and submit them to a centralized team of editors; the team of editors edits the questions and produces some questions of its own. In most instances, I think that the editors are responsible for question quality. New or inexperienced editors are generally given a pass by the circuit in order to encourage them to continue editing (since editors are held in extremely high regard by the circuit, and rightfully so). However, reasonably experienced editors (editors who have edited more than two tournaments, as a rough definition) cannot use inexperience as an excuse, and must therefore be entirely responsible for the questions they produce.

Whenever experienced editors produce a set, I think it's incumbent on those editors to respond to criticisms of the questions they were responsible for. For instance, the social science editor should respond to criticisms of questions s/he edited, and so on. To that end, I think that NAQT has largely not been responsible in this way; the responses to criticisms of SCT on a private forum were a good start, but I think NAQT could do a much better job of responding to criticism. Furthermore, I think that it would be an excellent idea for NAQT to set up such public fora for players who have played a set to critique it (HSAPQ should consider the same for its sets; ACF can continue to use this board, since its sets are almost always played on the same one or two weekends).

Also, I think that arguments that NAQT serves "casual players" as well as more interested players are not satisfactory responses to criticism. First of all, I have not seen any spokesman for NAQT argue that they are producing questions, at least partially, with casual players in mind. Secondly, I think that we can separate "things that casual players like" from "bad question writing;" the overwhelming majority of the complaints I have seen target bad clues/questions, which to me inherently produce some amount of frustration. Not only that, since the more interested/serious players also play NAQT sets, NAQT must be responsible not only to both groups. Thus, appeals to "casual players" handwave criticisms by these more interested players instead of addressing them.

In summary, a growing number of high school players, to my knowledge, feel that NAQT has shirked its responsibilities to them by not responding to reasonable criticisms. Appeals to casual players are not satisfactory responses, since 1) an NAQT spokesman has not defined "casual players" nor asserted NAQT is consciously making any such appeal to them, and 2) I do not see the impossibility of producing sets both casual and more interested players can enjoy, and in fact believe that the production of such sets is extremely important to the growth of good quizbowl.

[Ken Jennings]

Thanks to David Reinstein for taking the time to go through an IS set to review some of his issues with NAQT lit (and "DumbJaques" for following up).

In general, I think most of these are fair criticisms, and even the ones I don't agree with 100% seem well-intended and knowledgeable about nuances of good quiz bowl questions. Starting with the most egregious: yes, the "sexual awakening of Robert Jordan" question reads like a parody of a bad tossup. One of my big knowledge gaps in my subject area is geek lit, epic fantasy in particular, so I feel a lot less confident when NAQT writers produce stuff like this. In this case, I knew about Jordan's illness and assumed (from its prominent placement in the question) that this anecdote was an important one that Jordan had famously published somewhere. I just did a little Googling and it turns out this was just a random entry on his blog! So, yeah, mea culpa. Lousy question.

The other problems that most stand out to me here are the Moby-Dick bonus and the too-easy/fraudable "Lottery" lead-in. The third part of the Ahab/Starbuck bonus was Queequeg...so Starbuck should clearly be the hardest part, and I should have removed the author's coffee clue. I think the "black box from the coal mine" lead-in is okay at this level of play, but the "little pieces of paper" clue comes way too early and could easily have been removed. (If you look at the entire tossup, the subsequent clues are less helpful than pieces or paper, so that clue is not just too guessable, it's also anti-pyramidal.)

NAQT questions get power-marked at the set level, and not by subject editors; I generally agree with most of your power-mark criticisms, but can't take any responsibility for them. I would point out that power-marking is, in some ways, a more demanding task than clue-ordering, since it requires not only gauging difficulty relative to other clues on the same subject, but absolutely, according to some platonic ideal of capital-D Difficulty that should stay pretty consistent over all subject areas. I suspect that every good player could find power mark placements s/he'd quibble over in even the most ably power-marked set...and that it wouldn't always be in the same tossups.

A few of these criticisms don't strike me as all that persuasive, I guess. The very fact that a Booker Prize tossup exists, for example, doesn't make me too unhappy. It's true that the vast majority of lit questions should revolve around literary content, but I don't think that implies that NO tossup should ever be written on more peripheral topics. It looks like this is the only publishing-award question in the set, and it does at least include some author-title clues.

I only saw the successful one of the Capote films, but it was New York and Kansas-set and, I believe, never mentioned New Orleans. My principal criticism of that tossup's lead-in would be that Answered Prayers would be better off before New Orleans, so as to reward players who know the title of the unfinished work alone, and wouldn't have needed any additional information to place it in context.

I can't agree that the "Madrid taxicab" clue makes the Sun Also Rises tossup dangerously fraudable. I associate The Sun Also Rises with Paris and Pamplona myself; I actually think you would have to have substantive knowledge of the book to remember that the action shifts to Madrid at the very end. Or at least, the player who knows the book still has a substantial advantage over the player who's wildly guessing a Hemingway work because Hemingway = Spain. Plenty of novels with famous titles could plausibly end in Spain; that clue doesn't unintentionally narrow down the answer space in the same way that, for example, kiddie lit + Scandinavia = Andersen.

I also don't think I'd tweak the Hawthorne question given another chance. It's true that you don't necessarily have to have read any of his works to know they're set in Puritan Massachusetts, but it's not as if that clue is the lead-in. It follows three specific (and, frankly, fairly easy) clues about three of his best-known stories. If there are Hawthorne readers in the room, this tossup is long over. I think it's legitimate right before the giveaway clue to reward semi-poseurs who at least know generalities about an author's main themes or milieus.

Looking at the weaknesses in these questions is helpful, and definitely prompts me to be more careful when similarly problematic questions are submitted in future. (If you're an experienced writer pondering working for NAQT, I should say that you don't generally have to worry about editors introducing problems like these into your work. Maybe I'm being too candid here, but in my experience, most unpopular NAQT questions are not-entirely-successful attempts by editors to improve even more unacceptable questions submitted by less-experienced writers. Not that that's any excuse.) But in a way I'm actually relieved that these are the most serious lit problems someone could find in a randomly chosen IS set. I don't want to minimize these criticisms, but I don't feel like they convincingly argue that NAQT questions tend to be unacceptably bad, that good writers' efforts would be wasted if they wrote for NAQT, etc. To me, anyway, there's a wide gap between "There are scattered questions in NAQT IS sets that could use small improvements" and the "NAQT is past saving" rhetoric I sometimes see in threads like these. I was bracing myself for pop culture clues, factual errors, "cutie" giveaways, lazy title lists, questions with too few actual buzzable clues, pointless author biography, main plot points or characters' names in the lead-ins, long stretches of clue-free generality, etc.

(I also meant to add before, and forgot, that I'm also happy to discuss more recent questions as well, even if we can't publicly dissect them in this forum. I can always be e-mailed at bugken [at] gmail. Thanks for your participation in trying to improve NAQT's product.)

[Ken Jennings]

Thanks for posting. If you're interested in having a discussion, I have a few questions for you: What do you consider the core tenets of a good question?

That could be an hourlong conversation right there, but here's an abbreviated start at an answer...assuming you mean "tenets of a good tossup," I guess, since that's probably the more demanding form.

At a minimum, tossups need to present important, substantive information on a topic in--without exception--strictly hard-to-easy order. Difficulty should ease up gradually, so as to accurately differentiate teams with as broad a range of skill levels as possible. The referent--the answer--needs to be clearly signaled (either explicitly or, much more rarely, implicitly) as early as possible in the question. Especially with NAQT's stringent length limits, clues need to be dense and specific, and as little time as possible should be spent on vaguer background information or other verbiage that won't help players buzz on knowledge.

Ideally, deep knowledge learned by close personal or academic study of a topic should be privileged, followed (where necessary) by more facile knowledge most likely to have been gained via quiz bowl, followed (in turn) by knowledge accessible just through general cultural literacy. Ideally, some of the clues, particularly the lead-in, should be chosen because they are novel and patently noteworthy or unusual in some way, even to players who aren't experts in the topic. Ideally, all this is done in perfectly crisp, lucid prose that never has to convolute itself to get clues in the proper order or keep referents clear. Ideally, there should be some overarching idea or narrative flow carrying the reader and listeners through the tossup; a series of choppy, discrete, unrelated clues may be just as serviceable, but facts are heard and learned with linkages and context, and it seems appropriate to test their retrieval in the same way.

In practice, sometimes trade-offs are involved in balancing these criteria, and I think reasonable minds can differ as to how successfully a tossup (or set, really) performs that balancing act. There are questions of taste involved in where and how the compromises occur. Even something as simple as clue difficulty can be hotly debated between different players. And I see that I've focused more on construction than answer choice, which comes with its own complicated set of judgments of taste. But hopefully this is enough to kick-start a discussion.

Could you post examples of questions you've written for NAQT that you consider good? Thanks.

Well, I'm not really auditioning for anything here, and I guess it's possible you intend this as some kind of nitpicky trap ("Aha! If you think tossups should reward substantive knowledge, why would you name Dance Fools Dance in the second sentence of your Haruki Murakami tossup instead of using that space to add more plot clues from Hard-Boiled Wonderland?!?") but I'll try to humor it anyway, in the name of improving NAQT transparency. But not this week; I'm on vacation and away from my computer that has most of my NAQT stuff on it. (Well, today I'm actually nursemaiding my six-year-old son who has the flu...some vacation!)

[Tower Monarch]

Looking at the weaknesses in these questions is helpful, and definitely prompts me to be more careful when similarly problematic questions are submitted in future. (If you're an experienced writer pondering working for NAQT, I should say that you don't generally have to worry about editors introducing problems like these into your work. Maybe I'm being too candid here, but in my experience, most unpopular NAQT questions are not-entirely-successful attempts by editors to improve even more unacceptable questions submitted by less-experienced writers. Not that that's any excuse.)

I don't know much about the internal workings of NAQT with respect to editing practices, but this sounds like a testimony for having experienced editors write these questions instead of wasting the apparent time consumed through editing "even more unacceptable" submissions. Would that not also cut the costs of paying some of these less-than-adequate writers??

[Sima Guang Hater]

At a minimum, tossups need to present important, substantive information on a topic in--without exception--strictly hard-to-easy order. Difficulty should ease up gradually, so as to accurately differentiate teams with as broad a range of skill levels as possible. The referent--the answer--needs to be clearly signaled (either explicitly or, much more rarely, implicitly) as early as possible in the question. Especially with NAQT's stringent length limits, clues need to be dense and specific, and as little time as possible should be spent on vaguer background information or other verbiage that won't help players buzz on knowledge.

With you so far.

Ideally, deep knowledge learned by close personal or academic study of a topic should be privileged, followed (where necessary) by more facile knowledge most likely to have been gained via quiz bowl, followed (in turn) by knowledge accessible just through general cultural literacy. Ideally, some of the clues, particularly the lead-in, should be chosen because they are novel and patently noteworthy or unusual in some way, even to players who aren't experts in the topic. Ideally, all this is done in perfectly crisp, lucid prose that never has to convolute itself to get clues in the proper order or keep referents clear. Ideally, there should be some overarching idea or narrative flow carrying the reader and listeners through the tossup; a series of choppy, discrete, unrelated clues may be just as serviceable, but facts are heard and learned with linkages and context, and it seems appropriate to test their retrieval in the same way.

Still with you.

In practice, sometimes trade-offs are involved in balancing these criteria, and I think reasonable minds can differ as to how successfully a tossup (or set, really) performs that balancing act. There are questions of taste involved in where and how the compromises occur. Even something as simple as clue difficulty can be hotly debated between different players. And I see that I've focused more on construction than answer choice, which comes with its own complicated set of judgments of taste. But hopefully this is enough to kick-start a discussion.

If there's a tradeoff between a strict length limit and the quality of a tossup, why not ease up on the length limit a little? I don't think teams are going to count the number of words in every tossup. More often than not, I feel like a strict character limit stifles space for things like connecting words or finishing one extra clue that'd make the tossup sound and play better. I recognize the need for length limits, but quality has to come first.

This rule applies in general. If there's a tradeoff between quality and X, conditionally screw X (this doesn't mean A-sets should have 8 line tossups, but the occasional 4-liner never killed anybody).

Well, I'm not really auditioning for anything here, and I guess it's possible you intend this as some kind of nitpicky trap ("Aha! If you think tossups should reward substantive knowledge, why would you name Dance Fools Dance in the second sentence of your Haruki Murakami tossup instead of using that space to add more plot clues from Hard-Boiled Wonderland?!?") but I'll try to humor it anyway, in the name of improving NAQT transparency.

We're not trying to trap you. We just have the sinking feeling that your image of a good tossup doesn't line up with ours, and we want to iron out why. Also its "Dance Dance Dance".

But not this week; I'm on vacation and away from my computer that has most of my NAQT stuff on it. (Well, today I'm actually nursemaiding my six-year-old son who has the flu...some vacation!)

Sounds like quite a Bataan Death March.

[Birdofredum Sawin]

I have two points to make in response to Jonathan.

I think I wasn't clear enough in my explanation of responsibility, so I'll try to be more lucid. In my opinion, responsibility involves ethical behavior and responding to feedback (both with reasonable explanations and procedural changes). I think the question of NAQT's ethics belongs to another thread, so I'll leave that alone. However, I think NAQT has a responsibility to have one or more qualified and able spokesmen to respond to the concerns elucidated in this thread, and to players' concerns in the main. Too often, I think such a spokesman for NAQT has either not responded to public feedback, or responded inadequately.

Secondly, I want to touch on the responsibility involved in producing questions. Most tournaments have two levels of question production: writers write questions and submit them to a centralized team of editors; the team of editors edits the questions and produces some questions of its own. In most instances, I think that the editors are responsible for question quality. New or inexperienced editors are generally given a pass by the circuit in order to encourage them to continue editing (since editors are held in extremely high regard by the circuit, and rightfully so). However, reasonably experienced editors (editors who have edited more than two tournaments, as a rough definition) cannot use inexperience as an excuse, and must therefore be entirely responsible for the questions they produce.

Whenever experienced editors produce a set, I think it's incumbent on those editors to respond to criticisms of the questions they were responsible for. For instance, the social science editor should respond to criticisms of questions s/he edited, and so on. To that end, I think that NAQT has largely not been responsible in this way; the responses to criticisms of SCT on a private forum were a good start, but I think NAQT could do a much better job of responding to criticism. Furthermore, I think that it would be an excellent idea for NAQT to set up such public fora for players who have played a set to critique it (HSAPQ should consider the same for its sets; ACF can continue to use this board, since its sets are almost always played on the same one or two weekends).

I think this argument about "responsibility" is still not as lucid as it needs to be. What, after all, does it mean to say that it is "incumbent" upon editors to "respond" to criticisms? I don't think anyone is saying that they have peppered an NAQT editor with critiques and been rebuffed. Instead, you seem to be saying "if there is a criticism made on this forum, it is incumbent upon NAQT to reply to it." But when you put the claim that way, I think it's much less intuitively obvious.

I'd suggest that Jonathan is extrapolating a normative claim from a descriptive fact, without (perhaps) realizing that he's doing so. Here's what I mean. As it happens, almost all the people who edit ACF and work on major circuit tournaments are also serious posters on this board. As such, they naturally respond to critiques of their tournaments which are posted here. I think it's great that they do so, but it's not as if they're going out of their way to do it. They already post here all the time; responding to criticism of their tournaments is just part of an average day's Internet activity. As it happens, a number of NAQT subject editors aren't especially active on this board. Thus, they may not take notice of criticism lodged here, or they may not have the time or inclination to engage in a lengthy debate in this forum. (I've been spending huge amounts of time posting here the last few days, but only because a) I'm on break right now, and b) I'm stuck at my computer anyway writing some papers.)

Would it be a good thing if NAQT editors made more time to engage in the give and take of this board? Maybe yes, maybe no; but it is far from clear to me that it is an ethical obligation on them to do so, and that NAQT is currently in dereliction of its duty in that regard. I'm suggesting that your argument here boils down to "it is a responsibility of NAQT subject editors to be active on this board." But I don't think that's true.

Also, I think that arguments that NAQT serves "casual players" as well as more interested players are not satisfactory responses to criticism. First of all, I have not seen any spokesman for NAQT argue that they are producing questions, at least partially, with casual players in mind. Secondly, I think that we can separate "things that casual players like" from "bad question writing;" the overwhelming majority of the complaints I have seen target bad clues/questions, which to me inherently produce some amount of frustration. Not only that, since the more interested/serious players also play NAQT sets, NAQT must be responsible not only to both groups. Thus, appeals to "casual players" handwave criticisms by these more interested players instead of addressing them.

In summary, a growing number of high school players, to my knowledge, feel that NAQT has shirked its responsibilities to them by not responding to reasonable criticisms. Appeals to casual players are not satisfactory responses, since 1) an NAQT spokesman has not defined "casual players" nor asserted NAQT is consciously making any such appeal to them, and 2) I do not see the impossibility of producing sets both casual and more interested players can enjoy, and in fact believe that the production of such sets is extremely important to the growth of good quizbowl.

I take it this is largely addressed to me, but I don't really see how I was dismissively "handwaving" in my discussion of "casual" players. As I understood it, the underlying theme of this discussion was "how can anyone who values quality quizbowl possibly bring himself to work, in good conscience, for an organization which does things like [insert here the standard complaints about NAQT]?" I then presented myself as someone who values quality quizbowl, and explained how it is that I, in good conscience, work for NAQT. For me to give that explanation, I had to present the dichotomy between "perfectionist" and "casual" players, because that's how I understand NAQT and how I justify certain of their practices which seem repugnant to other people in this discussion. You're free to say that I'm wrong about casual players, or wrong about NAQT, or that I'm just engaging in some sort of self-mystification. But in the context of this discussion, I think my invocation of the concept was entirely to the point.

Andrew

[Mechanical Beasts]

I take it this is largely addressed to me, but I don't really see how I was dismissively "handwaving" in my discussion of "casual" players. As I understood it, the underlying theme of this discussion was "how can anyone who values quality quizbowl possibly bring himself to work, in good conscience, for an organization which does things like [insert here the standard complaints about NAQT]?" I then presented myself as someone who values quality quizbowl, and explained how it is that I, in good conscience, work for NAQT. For me to give that explanation, I had to present the dichotomy between "perfectionist" and "casual" players, because that's how I understand NAQT and how I justify certain of their practices which seem repugnant to other people in this discussion. You're free to say that I'm wrong about casual players, or wrong about NAQT, or that I'm just engaging in some sort of self-mystification. But in the context of this discussion, I think my invocation of the concept was entirely to the point.

I think we'd then like to know that you agree with the following statement: that there exist such "repugnant" practices with which NAQT is associated (whether that is because NAQT "just doesn't get it," is a den of thieves, or fully intends to change but hasn't gotten a chance yet is irrelevant) that NAQT does not need to continue in order to satisfy the casual player. That is, when you refer to your justification of "certain of [NAQT's] practices," you do, in fact, mean that you do not use this dichotomy to justify all of NAQT's "repugnant" practices. If so, then we can productively discuss what is necessary for the casual player and what is unrelated to that player's needs.

[Stained Diviner]

Thanks for your response, Ken. My six-year-old got a Wii today, so we're having very different experiences on that front.

I stand by my criticisms, though I respect your point that opinions differ and that there is a difference between finding flaws, which is mostly what I did, and finding completely horrid questions, which I only found one or two examples of out of about six rounds.

Just to add one more bit of constructive criticism to what I said that both of us alluded to but that I want to make very explicit--the key to literature questions is literary clues. The Kafka question above is not the worst question in the world by any measure, but it misses a great, and I think obvious, opportunity to throw in a couple of summaries of Kafka works. The same problem exists with the Hans Christian Andersen tossup. We don't need a ton of HCA tossups, but a few are OK. However, I think any good tossup on him is going to begin with references to plots.

Some of what I'm stating is the result of changing standards. I just looked up a Kafka question I wrote in 2002, and it starts out, "He had a domineering father and very rocky relationships with women, both of which came out in his writing." An Andersen question I wrote in 2003 starts out, "Who completed a collection in 1840 titled A Picture-Book Without Pictures?" Here I am now telling you to do better than I have done, which is of course hypocritical. However, this is one change I have worked on as a writer, and it is something that hopefully both of us will improve at. I am not saying that none of your questions do this--I am saying that more of your questions should do this.

[Birdofredum Sawin]

I think we'd then like to know that you agree with the following statement: that there exist such "repugnant" practices with which NAQT is associated (whether that is because NAQT "just doesn't get it," is a den of thieves, or fully intends to change but hasn't gotten a chance yet is irrelevant) that NAQT does not need to continue in order to satisfy the casual player. That is, when you refer to your justification of "certain of [NAQT's] practices," you do, in fact, mean that you do not use this dichotomy to justify all of NAQT's "repugnant" practices. If so, then we can productively discuss what is necessary for the casual player and what is unrelated to that player's needs.

Um, I'm trying but failing to parse the first sentence here. But I take it you're saying something like this: "Surely you're not invoking this concept to justify all of the practices which people on the board are complaining about? And if not, could you try to distinguish between a) practices you think are defensible on the basis of your perfectionist/casual dichotomy and b) practices which, on your view, need changing regardless?" On the assumption I've got that right, I'll say that yes, I wouldn't want to use this concept to justify everything NAQT does. To my mind, it's important not to assume that every form of quizbowl has to follow the assumptions (about the distribution, about the amount of attention which it is appropriate to lavish on each question, etc.) which are shared by perfectionists. But that doesn't mean that I'm invoking the concept of the casual player to claim that "anything goes." Anyway, here are a few consequences which I would draw from the distinction. (These are just some things which leap to mind out of the recent morass of discussion; I don't mean to suggest that these are the most important things, or the only things I'd change.)

1. Based on this discussion and some threads I was inspired to look up, I think computational math tossups should be eliminated from (at a minimum) the HSNCT (inasmuch as the HSNCT, to my mind, is clearly a "perfectionist" set, and as people have argued ad nauseum, computational math tossups are not consistent with the standards of that kind of quizbowl). I don't know why NAQT continues to insist on these questions -- it could be inertia, it could be that they "don't get it," it could be that they are responding to some sort of significant customer pressure that I don't know about. But I think it's a mistake for HSNCT to include such tossups.

2. I think NAQT should do a much better job about integrating its calendar with that of the circuit, and about making information about the circuit available to anyone who might want it. (E.g., I think there should be a prominent link on the NAQT website to a schedule of circuit tournaments.) Though I have no problem with NAQT providing opportunities for casual players who don't particularly want to improve, I think it should do a better job of making players who do want to improve aware of the venues for doing so.

3. I don't have a problem, a priori, with college tournaments being run on IS-sets, because I don't see how it is unacceptable for casual college players to compete on them. I think it's obviously the case that good college players won't be challenged by these questions, but then good college players shouldn't go to tournaments run on IS-sets. (And since in college there aren't any special incentives for attending IS-set tournaments -- it's not as if you can qualify for SCT on them -- there's no necessary reason why college players should feel compelled to go to these if they don't want to.)

Anyway, those are just some preliminary thoughts on how I'd approach a few of the issues of the day.

Andrew

[naturalistic phallacy]

NAQT should accept your recommendations regarding math comp. I love them.

[Sen. Estes Kefauver (D-TN)]

Since the literature criticisms thread got merged back in here, I want to take on something about the distribution - Ken, why do you seem to think it is acceptable for "geek lit" as you put it to be counted as part of the literature distribution. Even if there are some borderline cases as to what is trashy vs. serious lit, I don't think there's even a remote question that a tossup on Robert Jordan the author is anything but pop culture. Why are you not kicking back these questions, along with the questions on Harry Potter that made it into the HSNCT final and were classified as part of the literature distribution, and undoubtedly even more that I don't have the resources to name because I don't have access to any NAQT database?

[Brian Ulrich]

Even if there are some borderline cases as to what is trashy vs. serious lit, I don't think there's even a remote question that a tossup on Robert Jordan the author is anything but pop culture. Why are you not kicking back these questions, along with the questions on Harry Potter that made it into the HSNCT final and were classified as part of the literature distribution, and undoubtedly even more that I don't have the resources to name because I don't have access to any NAQT database?

I won't speak for Ken, but as someone who writes some of these questions, and deals with similar issues in my own category, I can say such classification guidelines come from above the level of the average editor. It works both ways - CNN and al-Jazeera both count as TV questions because they're TV stations, even though both are dedicated to news programming. A film question is also (in theory) as likely to be Charlie Chaplin as Will Smith, though (I suspect) submissions tend not to run that way.

[Important Bird Area]

Charlie, would it be reasonable to restate your position as "NAQT should have fewer questions on music, tv, and movies, and more questions on academically important novels, plays, and poems"? I'm assuming people don't actually have strong feelings about NAQT's internal labels for questions (compare to the "true RMP distribution" discussion a while bacK).

[Mechanical Beasts]

Charlie, would it be reasonable to restate your position as "NAQT should have fewer questions on music, tv, and movies, and more questions on academically important novels, plays, and poems"? I'm assuming people don't actually have strong feelings about NAQT's internal labels for questions (compare to the "true RMP distribution" discussion a while bacK).

I think we'd both like to hear that HP is being classified as trash, rather than lit. If we just adopt your restatement, it's possible that NAQT vastly decreases its amount of trash produced, but continues to have extra trash by taking away questions from "real" lit. NAQT's internal labels currently allow trash to invade lit and, because I guess "things that are written down" are automatically lit (from Brian's description), there currently exists no limit on how much trash can invade lit.

I'd be more comfortable with a distribution that has as much trash as NAQT does now but which has lit a sacrosanct category, which cannot be accidentally invaded by trash. (And as Dwight's analysis shows, the real victim of the preponderance of trash/GK/etc. here is fine arts, not literature, so we really are talking about two different problems: the tendency of trash to exceed its distribution and invade lit via (what we believe are poor) internal labels, and the initial too-high distribution of trash.)

[rhentzel]

I am disappointed by the low quality of questions, as well as NAQT’s attitudes towards people like me. I may not be NAQT’s target audience, but I am worried by the fact that I seem to be completely ignored in NAQT’s business model.

I own myself surprised to read this; I'll admit that Hunter College High School is not a *typical* member of our target audience, but those of us guiding NAQT certainly consider you and your team to be part--an important part--of our target audience. We want you (and similar teams) to enjoy our product, to enjoy our tournaments, and to esteem our organization. I'm sorry that you have gotten the impression that we don't care; I would like to think that could be ameliorated.

When R. Hentzel said a while ago, for example, that he thought A-level sets were appropriate for all levels of play: (1) I felt this was a complete lie, as the questions are all but devoid of academic content (more on this later);

NAQT A-level sets use the same distribution as our regular Invitational Series and as our national championship set. Even if one takes the extreme position that *all* geography, current events, and mixed-subject questions are nonacademic (in addition to sports, popular culture, and general knowledge), that's still only 27% nonacademic content. Sports and popular culture together are only 7%.

and (2) if it wasn’t, it’s a slightly delusional thought to have, as the questions have been demonstrably incapable of separating between top teams.

I also think this is not true. I'm most familiar with the Twin Cities-area circuit, where I saw the basic set of teams (particularly the same top teams) compete in eight NAQT events (four on regular invitational series, two on A-level questions, one on a modified A-level set, and one (TOMCAT) on what we call SHORTs. I didn't see a significant difference in the overall standings among those eight tournaments.

It does depend, I guess, on the what you mean by "top teams"; NAQT's "introductory invitational series" (i.e., A-level sets) are not specifically targeted at, say, teams that are going to finish in the top ten at the HSNCT; I think that even in matches between those top ten teams the better team is likely to win a significant majority of the time (80%?), but I don't really have clear, numerical evidence for that. I'll have more to say about this in the thread that's specifically about A-level sets.

I’m not saying that every tossup in an IS set should be on My Life in the Bush of Ghosts, but I feel that NAQT should stop marketing A-sets to regions like the DC metro area,

To the best of my knowledge, NAQT doesn't *specifically* market its A-level sets to the DC metro area in any way that's different than our national marketing message.

In the interests of transparency, I think it's unlikely that NAQT will start refusing to sell sets to hosts that are interested in using them.

make a late-season set that is fun for upper-level teams to play on (in preparation for the HSNCT or something),

Did you enjoy the chance to play the DII collegiate Sectionals questions last year? If not, in what way did it not measure up to your expectations of such an event?

How many "upper-level teams" are you counting here? Should the set "be fun" for the country's top 20 teams? Top 100?

This is an idea that we'd be much more likely to adopt; we like writing questions and we like the idea of top teams playing on our questions.

and—perhaps most importantly—have HSNCT be a tournament marketed specifically with the top teams in mind, in distribution, content, and so on.

In my opinion, the HSNCT *is* already marketed with top teams in mind. To focus the discussion, though, how many top teams are you considering? Hypothetically, if we did decide to run the HSNCT purely according to your precepts, how many teams' opinions on what the proper difficulty, distribution, and so should be, would we consider?

Having reread this paragraph, I suspect it could come up as snarky, but I don't mean it that way, I just mean to emphasize that NAQT already works very hard to put on an event that will be enjoyed and admired by the "top teams": we spend a huge amount of time on the questions, we work hard at achieving a uniform difficulty that can distinguish the top teams without being frustrating for the bottom teams, we provide lots of rounds, we spend lots of money to get great moderators from all over the country, and so on. I'm perfectly willingly to believe that the HSNCT is not ideal (or even far from ideal) in your eyes, but it's not because NAQT doesn't want to produce a quality event, and it's not because we don't want it to appeal to top teams. For us, "top teams" is something like the top 150 or so in the country; roughly one-tenth of the number that compete in NAQT tournaments annually.

Computational math, for example, could easily be much reduced in HSNCT, if not in NAQT entirely.

NAQT considers it important to have championships played on questions similar to those on which teams have qualified.

That said, NAQT is not wedded to the idea of having computation questions in its packets, but I think their presence is more commonly enjoyed than a reader of this board might think (after all, nearly 40% of the respondents on our 2006 *ICT* survey) thought there should be "some" or "lots" of computation questions in collegiate quiz bowl).

I suspect that we will work on distributing surveys similar to those we've done at the college level this year at our HSNCT (and possibly more widely, depending on the time we have available). Certainly people's views on the existence, quantity, and quality of computation questions will be covered.

Instead, the message sent out by HSNCT is worrisome. Given all the complaints about it in previous years, and the seeming lack of willingness to address these points, NAQT is sending out the message that they actually don’t care about the top teams at all.

The short response to this is that NAQT cares very deeply about the top teams; we desperately want their participation in our events. We also care about providing quiz bowl that is fun and educational to mediocre teams, poor teams, and even to teams that don't currently play. We want quiz bowl to become a ubiquitous activity in which every school across the country (or even the world) participates.

Our packets and our policies represent our attempt to provide products (at different levels) that satisfy teams with widely different expectations, ideals, and ability levels.

Furthermore, this attitude will only lead to the top teams no longer attending—so NAQT is then sending out the message that they don’t care about prestige, or crowning a true national champion. I am a quizbowl altruist; I would much rather work for a company that cares more about helping out players of all levels, including the uppermost, than lining its coffers with gold doubloons.

I can assure you that NAQT considers "helping out players of all levels" to be of paramount importance. Sadly, I can also assure that our coffers are not lined with gold (and are unlikely to be so in the near future).

Similarly, I feel that the distribution is being skewed away from good quizbowl. I have already aired my gripes about computational math; I’m not going to do so again. However, it seems unreasonable to me that the entire nation has to suffer through one out of every 8 tossups being math because otherwise the players in Belize will keep playing Chip.

1 in 18 tossups are computation.

NAQT is in a great position to spread good quizbowl to regions with poor formats; instead, their modus operandi has been to endear their format to said regions by making it more like the terrible formats that they are giving up for NAQT, rather than trying to explain why the bad parts of the format are bad, as HSAPQ has made a marked effort of doing.

I this this paragraph contains an oversimplification of NAQT's actions and views in this area.

Our basic format and distribution hasn't changed significantly since we introduced our high school invitational series in 1998; whatever changes have occurred have, in general, been toward what I perceive top players would consider "good." I think we have taken every possible opportunity to approach clients with different formats (state organizations, television shows, etc.) with our standard questions and we have always tried to sell them on using our basic format first. Generally speaking, we've been unsuccessful. We would much rather sell cheap tossup/bonus packets in our standard format than expensive custom questions in any other format, but the market just doesn't seem to work that way.

I do honestly wish HSAPQ the best of luck in converting organizations or regions of the country to a more standardized format; perhaps they will succeed where we have failed.

It is certainly possible to convince players that don’t know any better why their formats are bad; this has happened many times over on HSQB.

I think this paragraph understates the difficulties involved in manifesting a major change in the dominant style of quiz bowl in a region. I think that players who seek out this board (or who are convinced to visit it), are going to be much easier to convert--on average--than the "decision-makers" at state organizations, television stations, and even coaches' organizations.

For instance, I've devoted an enormous amount of energy to building up quiz bowl in Minnesota, but the dominant format here remains Knowledge Bowl by a factor of something like 6:1 in terms of number of teams and number of schools. In fact, many of the top quiz bowl programs in the state (who have heard all of the arguments on this board and then some), continue to *prefer* Knowledge Bowl. It may simply be the case that I am an ineffective evangelist, but I don't think that's true.

As such, I am given the impression that NAQT cares foremost with furthering itself, instead of using the furthering of itself to further good quizbowl with it.

What gives you this impression about NAQT?

NAQT cares about furthering good quiz bowl; we would like to see everybody playing quality questions that reward knowledge, produce fair results, teach things, inspire people to learn more in all facets of their lives, meet like-minded people and new friends, and that are fun to attend. That goal is pretty much the entire reason any of us contributes to NAQT; the money isn't very good, the benefits are atrocious, and much of the work is not very exciting. We can reasonably disagree about the proper means to achieve that end, but I think you will acquire a warped view of NAQT if you view its principals as not caring about issues of quality or of furthering quiz bowl as a whole.

As a minor data point, three of our members have been awarded the Carper Award for "meritorious service in sustaining and enriching academic competition" by voters who, by and large, had no association with NAQT.

We don't believe that the only worthwhile quiz bowl is to have regular invitational series questions (or longer/harder versions of such questions) that only includes questions taken an academic high school curriculum. We think that writing the best possible questions--within the constraints of what hosts want to buy or teams want to play--is a reasonable endeavor and one that is completely compatible with furthering good quiz bowl.

Now, to the questions themselves. I have already admitted that I probably should not have been playing an A-level tournament, but I did, and I would like to reiterate: those questions were awful. Absolutely abysmal. I believe I put up a higher bonus conversion on HSAPQ set ACF1, intended to be approximately IS-level in difficulty, than on 81-A. The only explanation for this is pure caprice on the bonuses, and subject matter that is entirely irrelevant to the rest of quizbowl.

I believe that should be IS-80A; not that it really matters for the purposes of discussion, but that's the set for which the following statistics apply.

First of all, your team average 24.54 PPB (82%) at LIFT VIII:

http://www.naqt.com/stats/tournament-te ... nt_id=2695

I'm surprised to hear that you put up a better conversion on ACF1, but I don't have those numbers available, so I'll accept that as true.

That said, I would consider 24.54 to be an extremely good bonus conversion; I'm really dumbfounded as to how you could consider that to be evidence of non-canonical, capricious answers. What am I missing?

As for the answers themselves, on theoretical grounds, I don't believe A-level sets (or IS #80A in particular) are any different from our regular invitational series questions. The questions are written by the same people (in roughly the same proportions), they are edited by the same people, they have the same official guidelines for writing, they have the same distribution, and so on.

IS #80A contained 1,177 answers (roughly 288 tossup answers and just over 3 answers per bonus, on average). 634 of them were *also* answers in some invitational series of 2007-2008. 773 of them were covered by 2006-2008. That seems like a very reasonable overlap to me; one that suggests the answers are being drawn from the same canon.

I know I can’t reveal answers, but when I see a bonus that puts at a huge disadvantage Jews living on the eastern seaboard, I am disgusted.

Stepping back from this particular question, I'll address the more general point:

NAQT realizes that some questions are going to be easier for people who live in certain parts of the country, who are of particular faiths, who attend certain types of schools, who have lived in foreign countries, or who have friends who just won't shut up about a certain novel. We continue to write such questions (as moderately as possible) out of a belief that if we eliminated *all* such influences, it would really carve an enormous chunk out of the things that could be asked in quiz bowl.

The tossups were very short and often devoid of academic content, and with entirely misplaced clues.

A-level questions are shorter--there's no doubt about that--but I disagree strongly that they were devoid of academic content. I don't think there's any significant difference among the academic content of our A-level invitational series, our regular invitational series, and our HSNCT.

That brings me to perhaps the most relevant point. NAQT says that the way to effect change in the overall question quality is to sign up and write good questions.

I think this is one way to effect change (and one that is, perhaps, fairly fun). In addition, writers can become editors or members where they have increasing power (and, in fact, the responsibility) to produce good questions and to define NAQT's policies.

But it's not the only way to change; providing feedback to NAQT (and, in particular, comprehensive, statistical feedback) is valuable. If you take any sort of survey about your tournament, we'd love to know those results and we would take them into account when making decisions. You could also work on non-packet-production projects (for pay!) within NAQT that you think are done poorly or that would free up time for us to work harder on packet sets. Finally, if you are associated with tournaments that use NAQT's custom questions and you would prefer to see them adopt our standard questions, let the tournament organizers know that.

From what I gather, this is simply untrue. NAQT is completely overstretched in its writing capacity. I remember one TD freaking out last year because it was the day before the tournament and he had not received the questions yet. This is not because of the invitational sets NAQT writes. In addition to this, it writes tons of speed sets and TV sets. If I were to sign up, I could not be at all assured that the average quality of questions would go up. What would actually happen, and has been happening, is that my work would bump out some of the worse parts of IS sets. But that bad work would still be used in the tons upon tons of sets in bad formats that are written. Instead of my work replacing bad work, NAQT would just commit itself to writing more and more sets.

I think this is an oversimplification, though I agree that NAQT is stretched pretty thin on question production. The best Christmas present I could have gotten would have been four top-notch writers willing to kick in 500 questions per year, even if they were only interested in writing questions for regular invitational series (or harder) sets.

I won't deny that NAQT looks for opportunities to sell more sets and that we will give any such opportunity due consideration; I hope that we will continue to land new customers and provide them with quality questions. In fact, this very post contained a request for NAQT to produce an additional set of questions.

That said, I'm having difficulty responding to this paragraph since it seems to accept as given a model of our production and marketing efforts that just isn't true. It has been very noticeable to me that our quality had a noticeable increase when (for instance) a single person like Andrew Yaphe or Jeff Hoppes signed up to write. And we took a noticeable hit when big contributers like Subash Maddipoti moved on to run his family business. We didn't start selling more custom sets directly as a result of the first, and we didn't stop selling any as a result of the latter.

For why this is bad, see, for example, two paragraphs ago. I could not, in good conscience, write for NAQT with the express purpose of spreading good quizbowl and making mediocre quizbowl better, if I knew, for a fact, that what would actually happen is that the mediocre quizbowl would get better, and would be replacing perhaps slightly worse quizbowl. The only net benefitter in this situation is NAQT,

Even were I to accept your premises, I don't see that your conclusion ("only net benefitter") follows in this case. It seems like thousands of players across the country would be exposed to better questions (some of which you wrote), our editors would have more time to spend on other questions, more teams would be exposed to circuit-style quiz bowl, you would get paid, and NAQT would get paid. I don't want to say that I think writing for NAQT is the only possible course of action, but viewing the situation as NAQT looking to increase its profits at the expense of cheap, college-student labor is just not accurate.

and there is absolutely no reason for me to write for NAQT so that they can make a bigger profit, instead of writing for HSAPQ, or writing my own set and mirroring it across the country (as I am currently attempting to do). It may be somewhat less lucrative for me personally, but good quizbowl would gain so much more for my work. And that is what really matters to me, and many other people with whom I have come into contact. I just can’t say the same of NAQT.

It's not difficult to "try out" writing for NAQT; maybe it's not a good fit and you wouldn't be happy writing questions for us, but it wouldn't take that much time to send us an evaluation sample and contribute 100 questions after that. You can experience first-hand interactions with our editors, you can see if they make negative changes to your work, you can write only for the types of sets that you want to, you can get statistical feedback on how your questions play out, and you can see what it's like to produce questions within our framework. If you don't like it, you can quit at any time with no negative consequences.

[rhentzel]

I think the big problem, as has been stated before, is the length limit on NAQT's questions. I've been doing a lot of writing for BATE and none of my questions would meet the length limit, yet I feel all do a good job at differentiating between teams of all levels. If NAQT could increase their question length by 100-150 characters, you could add a couple of more clues, perhaps in the beginning of the question, that would greatly help to differentiate between teams. I think what's happening is that writers who ordinarily write good questions write subpar NAQT questions because of NAQT's character limit.

Objectively, if 100 more characters is good, why not another 200 or 500?

I'm willing to concede that more text and more clues should, in nearly every case, result in a question that has better powers of differentiation . . . but I think this is true of any tournament, regardless of the length of the questions; more clues will always provide better differentiation.

But there are drawbacks to longer questions: they take more time to read (so games have fewer questions or a tournament has fewer games), they take more time to write, they take more time to edit, and there is a threshold for the number of clues through which any player can sit without becoming frustrated at not knowing them.

If I had to pick one single characteristic that I thought differentiated the typical reader of this board from the the typical player of NAQT questions (or the typical person who might want to start playing NAQT questions), it is in his or her ability/willingness to listen to clues that he or she doesn't know without getting frustrated.

The interesting question, to my mind, is what criteria does one apply to determine when questions are "long enough"? And this seems like an empirical question: they should be long enough that a packet fairly distinguishes between teams and long enough to include interesting clues. But once those ends have been achieved, I don't understand why they should be made any longer.

For me, the issue question length boils down to judging whether or not a packet sufficiently differentiates between teams in its target audience; if not, the questions need to be lengthened (assuming they don't contain useless text that can be replaced). But if they are long enough to do that, what's gained by making them longer?

I think NAQT's existing packets meet that goal of fairly distinguishing teams, but I would be very interested in seeing any sort of analysis that suggested that wasn't true, either in general or for a specific packet set.