NAQT discussion

[Birdofredum Sawin]

Since the "Underwhelmed" thread seems to have simmered down, I thought I'd start a new discussion prompted by reflections on what people have been saying about NAQT. I know this might sound like a dreary meta-discussion of a thread, but in fact I'm going to try to have something more interesting to say.

What struck me about a number of posts in the thread is the apparent assumption, on the part of numerous posters, that NAQT operates in bad faith or is not to be trusted. Take the tail end of the discussion, which pilloried the survey. I understand that some people might be dissatisfied with the phrasing of some of the questions, though to me "I dislike this because it offends my quiz bowl aesthetics" is a reasonable response. Regardless of its language, the survey a) made public the distribution of ICT, b) allowed people to weigh in about potential changes in that distribution, and c) allowed people to vote on several debatable kinds of questions. I'm not saying that we should all shower praise on NAQT for doing things they perhaps should have done a long time ago, like reveal the distribution. But the negative response to the survey, to be frank, surprised me. The posters seem to assume that NAQT, a month from now, is going to announce something like the following: "All the results are in, and most people want everything to stay exactly as is; so we're sticking to the status quo, and no you can't see the results." If that happens, then this kind of response would be warranted. But to post cynical comments about the survey now is to assume that it's a smokescreen and that nothing is going to come of it. That is, it's to assume that NAQT is acting in bad faith.

I understand that this kind of rhetoric is not uncommon on this, or any, Internet forum. Were a newcomer to this forum to peruse the posts of the last few weeks, he might assume that apologists for CBI, advocates of NAQT, writers who use Wikipedia, and people who eat puppies are all on the same moral plane. Personally, I'm down with the "CBI apologists = puppy eaters" equivalence, but once you get past that self-evident ethical proposition things become much more ambiguous than recent posters seem willing to allow. I want to suggest that people think a little harder about their knee-jerk reactions to NAQT. To start, here are a few statements about NAQT which are, I think, true:

1) NAQT is not CBI. That is, it isn't a morally corrupt organization with no ties whatsoever to the game, and which would like nothing better than to have the circuit disappear.

2) NAQT is not a cash cow. Some people seem to think that the NAQT product should be held to a higher standard because NAQT members are getting rich off the game. Obviously, I'm not saying that NAQT shouldn't be held to a high standard. But I don't think I'm divulging any profound corporate secrets when I say that nobody is getting rich from NAQT. They do bring in a lot of money, but then they also spend a lot of money on things like paying question writers and flying moderators to tournaments. Someone like R., who spends a fantastic amount of time working for NAQT, makes enough money from it to live decently, were he living in Bangladesh or Belize. Again, this isn't an excuse for a mediocre product, but it might reduce some of the animosity about exorbitant fees. (I agree the fees are large; my point is, they aren't being exacted so NAQT members can take their annual month-long vacation to Bali.)

3) NAQT does not conspire against players. This might fall under heading 1, but I thought it was worth making a separate point of it. I myself don't see eye to eye with a number of NAQT people on a number of issues. But none of them are "out to get" anyone. Unlike the people who run CBI -- who have no ties to the circuit, and are frankly contemptuous of it -- the people who work for NAQT are or were active participants in college competition. They believe in the game, even if their views about particular aspects of it might not align with the views of some, or even many, of the people posting on this forum.

I'm posting about this at such length because the posts I'm responding to seem indicative of a growing contempt for NAQT among the elite players (which I define as "people who care enough about the game to read, and post to, this forum"). I worry about this, because I think it would be a disaster for the circuit if that contempt were to lead to a falling out similar to the one which most top programs have had with CBI. At this point, ACF has become the passion of a coterie. Around 15 teams are devoted to the format, but everyone else seems to find it superfluous. However, almost all of the most active writers and editors are partisans of ACF. If there were a real rupture between advocates of the two organizations, it could be fatal for both. ACF would become the province of a handful of elite teams, who would run endless tournaments for the delectation of one another. NAQT would continue to attract a much larger number of teams to its tournaments, but with all the best writers shunning them in favor of ACF theirs would be a Pyrrhic victory; a few years from now, as NAQT writers and editors move on to other things, NAQT would be unable to attract talented writers and editors to replace them, which could lead to a downward spiral of mediocrity.

In short, it's in the best interests of everyone that NAQT not be demonized (as ACF has been unfairly, and CBI fairly, demonized in the past). I think the real problem right now is a certain failure of communication between NAQT and the circuit. I also think that this disconnect is accidental. It just happens to be the case that most NAQT people belong to a previous generation of players. I competed against them as an undergrad, so I know that they're basically well-intentioned human beings. Most of you whippersnappers only know them as "members of NAQT," which makes it easier to assume that they're soulless corporate apparatchiks. I'd like to think that a little more communication could smooth over a lot of the apparent animosity I'm seeing toward NAQT. I'd also like to think that this survey is a promising step toward such improved communication, which is why I hate to see it so immediately dismissed.

Andrew

[grapesmoker]

What struck me about a number of posts in the thread is the apparent assumption, on the part of numerous posters, that NAQT operates in bad faith or is not to be trusted. Take the tail end of the discussion, which pilloried the survey. I understand that some people might be dissatisfied with the phrasing of some of the questions, though to me "I dislike this because it offends my quiz bowl aesthetics" is a reasonable response. Regardless of its language, the survey a) made public the distribution of ICT, b) allowed people to weigh in about potential changes in that distribution, and c) allowed people to vote on several debatable kinds of questions. I'm not saying that we should all shower praise on NAQT for doing things they perhaps should have done a long time ago, like reveal the distribution. But the negative response to the survey, to be frank, surprised me. The posters seem to assume that NAQT, a month from now, is going to announce something like the following: "All the results are in, and most people want everything to stay exactly as is; so we're sticking to the status quo, and no you can't see the results." If that happens, then this kind of response would be warranted. But to post cynical comments about the survey now is to assume that it's a smokescreen and that nothing is going to come of it. That is, it's to assume that NAQT is acting in bad faith.

As one of the people who posted about the survey, I don't think I made any assumptions about what it would lead to. I do think that the way the survey was phrased was overtly tilted to one side of the issue. I don't think the survey gave a reasonable voice to the critics of this year's set; doing so would have entailed laying out the reasons why people have been dissatisfied with it. "I just don't like it" is not an adequate option when so much time and energy has been expended in discussion of this set.

1) NAQT is not CBI. That is, it isn't a morally corrupt organization with no ties whatsoever to the game, and which would like nothing better than to have the circuit disappear.

I don't think anyone believes this. I certainly don't think NAQT is corrupt at all; they're all really nice people, so far as I know. Nevertheless, there seems to me to be a distinct problem with communication and a resistance to change within the NAQT circle, which, if we're talking about "cotries," it's worth noting is itself a closely-knit group of individuals of approximately the same age and coming from approximately the same player background.

2) NAQT is not a cash cow. Some people seem to think that the NAQT product should be held to a higher standard because NAQT members are getting rich off the game. Obviously, I'm not saying that NAQT shouldn't be held to a high standard. But I don't think I'm divulging any profound corporate secrets when I say that nobody is getting rich from NAQT. They do bring in a lot of money, but then they also spend a lot of money on things like paying question writers and flying moderators to tournaments. Someone like R., who spends a fantastic amount of time working for NAQT, makes enough money from it to live decently, were he living in Bangladesh or Belize. Again, this isn't an excuse for a mediocre product, but it might reduce some of the animosity about exorbitant fees. (I agree the fees are large; my point is, they aren't being exacted so NAQT members can take their annual month-long vacation to Bali.)

I'm going to repeat the claim I already made once, which is that this year's ICT set isn't worth the $240 (of my own money) I paid to play in it. I gladly handed over my money to David Rappaport at MLK this year, having received from them a superlative product. I also did not hesitate to pay the $130 for ACF Nationals. While I understand R. and others spend a lot of time on NAQT, I also spend a lot of my time working on tournament sets, and for far, far less financial reward (and unlike R., I don't even have a real job, unless you consider "grad student" a real job). $240 is a lot for me to shell out for a substandard tournament. Hell, $100 is too much to shell out for a substandard tournament, but a tournament that claims to be a premier national event has a responsibility to deliver when it charges a fee that's almost twice the next most expensive event.

3) NAQT does not conspire against players. This might fall under heading 1, but I thought it was worth making a separate point of it. I myself don't see eye to eye with a number of NAQT people on a number of issues. But none of them are "out to get" anyone. Unlike the people who run CBI -- who have no ties to the circuit, and are frankly contemptuous of it -- the people who work for NAQT are or were active participants in college competition. They believe in the game, even if their views about particular aspects of it might not align with the views of some, or even many, of the people posting on this forum.

I personally don't think NAQT "conspires" against players, and I'm not sure anyone really holds such a view. What NAQT has is a resistance to change and an aversion to engagment with criticism. The very fact that you're posting this in defense of NAQT is indicative of this. Why doesn't someone from the organization make a post defending it or explaining some of the things for which it has been criticized?

As you state later on, there's indeed a lack of communication between NAQT and the circuit, but that problem is entirely one-sided. There's been more than enough communication from "our" end (and I use "our" loosely, since I don't presume to speak for anyone else), but none from NAQT. The least that I think I should get for my money, if we're going to be purely consumeristic about this, is an explanation of the issues that have been raised in this forum and a personal reply to my email, which I sent R. about a week ago. The fact that NAQT has repeatedly not done so seems rather contemptuous to me. Although I can't see into the heads of NAQT writers, what I read from such a strategy is that what I have to say doesn't matter and that in any case it doesn't merit a public response. That's fine if NAQT wants to operate that way, but in the future, it will be doing so on someone else's dime.

The survey and revealing the distribution are both good first steps, but even those steps had to be coaxed out of NAQT. What I really want, more than any survey, is for NAQT writers to engage in a public discussion with us, their critics, in a non-facile way. If I could see that I would be far more motivated to grant NAQT the benefit of the doubt in any future misunderstandings.

[Matt Weiner]

It is certainly true that demonization is unproductive in such ultimately mundane matters as quizbowl, and that I and others can sometimes get carried away on the Internet as opposed to face-to-face discussion. However, it is equally true that there are danger signs in the direction that NAQT is heading. Perhaps there was a failure to articulate precisely what the problem with the survey is. The wording of the responses is just an amusing side issue that, if indeed innocent, was a useful way to illuminate how out of touch NAQT can be. The fundamental problem is that the survey seems to be responding to questions no one asked and opening the door for NAQT to go back on some improvements it has made. There were few or no calculation questions or 30 20 10s this year, for example; the best possible outcome of those survey questions is that NAQT sets continue to look just as they are. The other possibility is that a slew of terrible question styles returns. The questions one might expect to see on a survey drawn up in response to this year's tournament dissection-- "should trash continue to be noticeably harder than other subjects," "should there be tossups on gimmicky things like narwhal tusk"--are conspicuous in their absence.

The lack of transparency in general is and certain incidents related to that are how NAQT has brought suspicion of its motives upon itself. For example, take the still-open matter of my team's missing third place trophies. The only communication from NAQT membership on this point was Matt Bruce's not-really-apology which, like seemingly all communication from all NAQT people ever, was disclaimed as not being an official pronouncement of the organization. Take also the writing of intentionally bad questions out of spite, the backroom deal regarding UCLA's Division II eligibility a few years ago, and the tendency to make sweeping and after-the-fact policy pronouncements with the (often explicitly stated) proviso that they are outside the scope of procedures which may be changed by feedback. "NAQT does not consider the proximity of top teams when selecting the site of the ICT" is an example of what I have in mind here.

By refusing to participate in discussion on an informal level and by their explicit policy of taking all praise and criticism collectively (announced as the reason for removing writer identification from questions) NAQT has chosen on its own to be the target of speculation regarding its motives.

I wonder how much they would care, should all the complainers simply quit. Certainly Georgia's $240 is just as good as VCU's. Does NAQT really care about keeping the ICT as competitive as possible? What evidence is there on this point?

And why is no one chastising the pro-NAQT-as-it-is people for their own rancor and unrealistic requests? Is asking for more reasonable answer selection and some more fair game rules really more radical or self-serving than asking for the exclusion of grad students or tossups without clues in them? Is speculating about NAQT's integrity really more paranoid than accusing anyone with the slightest connection to ACF of wanting to destroy NAQT's format and turn it into ACF Part 2? Ultimately one has to realize who is offering the most realistic and compromising way for themselves to be satisfied here.

[Rothlover]

I pretty much agree with whats been added thusfar, with the caveat that the only way I might be elite would be in trash, negging and writing shit for shit.

I don't believe I've ever publically demonized any member of NAQT or publically said anything "bad" about said members. I usually make it clear when posting that I appreciate NAQT's efforts, and the time many people are taking from their work etc to put out sets. That said, I have always argued for more responsiveness, accountability and access to stats on stuff being converted (the last of which I asked for last years ACF nats as well.) The survey seems to be a tolerable step, and I personally am witholding judgement until after survey results are released.

[No Rules Westbrook]

Quite well said, Jerry...

I agree with Andrew only in so far as this: NAQT critics sometimes make arguments that are silly and unlikely, and more importantly, just unproductively distract from the important message. Arguments that fall into this category include that NAQT is a corrupt organization "out to get" certain people, a corporate demon making a fortune, similar to CBI, and composed of people who are intentionally and consciously unlikeable.


Now, I think there are a second class of arguments that are much more reasonable, but still not central to the super-important message. The first of these are comments about NAQT's protest proceudures, moderating and logistical snafus, etc. Sure, these things should be fixed, but they're episodic and transient. The second is the general comment that NAQT is an unresponsive monolithic entity that doesn't recognize criticism of any sort. Now, maybe this is the root or cause of all of the discontent (I'm inclined to think so), and maybe everything would be super otherwise. But, on its face (unless you assume bad faith), the survey reaches out, I agree with Andrew, to respond to this basic criticism.


There remains a final class of arguments that are truly paramount: objective and substantive objections to question quality and gameplay/tournament quality. In other words, the question that an individual player like Jerry asks - why should I pay this amount of money to go to this supposedly national championship tournament to play on these questions (and, for that matter, why should anyone)? This is the question that the survey comes nowhere close to addressing. The only potentially useful part is the revealed distribution. The multiple choice answers contain 4 options that are enough to make someone like me chuckle and one generic "malcontent" option i.e. "I just don't like it" or "it offends my qb aesthetics." The survey's entire tone is stilted enough that I think it just proves to some people that NAQT doesn't "get it" and doesn't care to get it, when it comes to real substantive debates on what types of questions should be expected and demanded written at a tournament like ict. Voting no on computation questions just doesn't feel like progress in that debate.


I'm unmoved by a fairly sweeping apology based on the "state of the game" of qb. If the implication of that position is banishment to a circle jerk on the Isle of the Elites, then that's a shame. Maybe we can pony up for some strippers with the 240 clams we'll save.

[yoda4554]

Actually, one point that's intrigued me. Since I do some freelance writing for NAQT high school sets, I have a small amount of access to some NAQT stuff. And an interesting thing I notice on tournaments is that while many of the questions that get bitched about the most are by people who might be characterized as out-of-touch-with-circuit-and-haven't-played-in-years (particularly on cutesy and stylistically irritating questions), a fair number are written and edited by people who have been quite recently active and are considered to be generally good editor-writers. I wonder if those are just flukes (after all, there are imperfect ACF questions as well), or rare missteps by these writers, or if there's a particular reason that writing for NAQT causes these issues...

[Chris Frankel]

I'm not as pessimistic about the survey as the prior expressed opinions from the thread; the fact that they did put it out and include a long-awaited revelation of the distribution (thank you for doing this!) is saying something significant, and I hope NAQT follows through with its results. I took the survey and also was thinking of starting a discussion thread about the questions it asked (maybe this thread should cover it).

I think people are cynical about it because precedent, unfortunately, seems to give them cause, with the following circumstances illustrating the case:

1. NAQT isn't CBI, and it's unfair to make the comparison. However, the mere existence of a monolithic QB organization with issues of bias, antagonism towards the general circuit, and a refusal to budge when presented with any constructive input from its customers (i.e. CBI) is a very real spectre that almost all players in the circuit know either from experience or from stories of yesteryear. Does this apply to NAQT? Not at the present, but I think NAQT would do well to remember its own roots and do its best not to fall into this image. Building a history of things like the UCLA D2 deal and the Weiner-Eric Smith protest war (and I think it's certainly more likely that such incidents were the result of slightly misguided policy decisions than any sort of bias) hasn't been helpful.

2. That said, NAQT has some non-ICT related business practices, that whether ideological or economic in motivation, do conflict with the existence of the circuit as we know it. The push for running college tournaments on high school IS questions threatens the proliferation of packet submission tournaments and has helped instill in newer players an aversion to question writing and studying more difficult academic material. The CUT format seems like a way to push out graduate students and discourage experienced players from participating. And whether or not it may be written off as half-assed journalism, the Washington Post article on the NAQT family did seem to imply the existence of a desire to replace the packet submission tournament with NAQT-affiliated ones.

3. This has been said, but it bears repeating. NAQT has historically been very resistant to criticism. The problems with questions that people have been discussing date back to at least the era of the first ICT I played in. You can also see from this thread and the earlier one how many people have attempted to issue productive criticism in the past and ended up either being ignored, given a terse lip service type remark, or labelled as malcontents and written off. I also agree that historically some of the most vocal criticism has been harsher than it should have been (I myself have apologized for my most inflammatory tones in the past and have made efforts to reduce it in new posts), but it's still baffling to see a business have so much aversion to taking criticism. Watching NAQT remove the question writer numbers to prevent subpar writers from being held accountable or watching noted current officer of the very historically prominent Michigan program Dave Rappaport have to line his initial (and valid) criticism post with a paragraph of unnecessary platitudes out of the worry that he might otherwise be labelled an asshole and ignored is immensely frustrating.

4. There's never been any transparency in the process with NAQT. Now NAQT has indeed made some positive changes to keep up with the circuit, such as taking out the once-infamous plethora of hockey trivia and vastly reducing the presence of 30-20-10 and list bonuses. I played most of the 2004 ICT packets in practice, and they were certainly better than ICT 2003, as another example. Of course, while some complaints do eventually get attention, a lot of the other ones get ignored, like the insanely low amount of philosophy, social science, and arts in relation to geography and current events. This brings me back to the transparency issue: you can't really predict how NAQT is going to respond to your criticisms because all the discussion goes on behind closed doors, and the best you can get is an announcement saying that changes are being considered and possibly implemented, without any elucidation on what and why. The asymmetric information is annoying because it prevents the team debating on whether to spend $240 for ICT from making an informed decision on whether NAQT's product will likely fit their standards of worth. ACF has done a lot better in this regard, see the transition from Nationals 05->06: People complained about the 05 set's excessive difficulty, Andrew came out and responded by saying how the set came to be so, promised to make toning down the difficulty a top priority for ACF 06, and did so.

5. Despite Letzler's claim (it would be ideal if he could elaborate on that with names and/or example questions, but I imagine that NAQT policy probably won't let him), I still think a lot of the problem is indeed rooted in NAQT's being out of touch with the evolution of the mainstream circuit. As Yaphe mentioned, I don't think it's debatable that ACF seems to have a monopoly on attracting this generation's most proflic and capable question writers/editors (Yaphe and Subash before him being the sole exceptions I can see), and I think that NAQT would do well to examine why this is the case and consider trying to change it. I think NAQT's closed structure and hesistance to engage in any sort of public dialogue is a big factor behind this rift.

For those reasons, I think there is much justification for the current degree of distrust for NAQT. Dave's thread was telling in how many once-silent names came out of the woodwork to express a wide range of concerns about NAQT's direction, many of which were the same ones that that same annoying group of assholes with chips on their shoulder who don't deserve to be listened to have been making for the past 3-4 years.

Now, it is also for those reasons that I actually find some hope in the release of the survey. Putting out that distribution after all this time is a big, big step for NAQT, and I really hope that this thread won't end before some significant discussion of it occurs. I will take the time to do so in the near future.

So in summary, I think that NAQT has made its own bed and made a lot of people unhappy through its past actions and policies. However, I think the survey represents a huge chance for redemption if handled well, and I took it in good faith and hope everyone else with an interest in the circuit does as well, so we can all get a more accurate reflection of how opinions run. The ball is NAQT's court now, and I hope they will follow through and make some positive changes with their survey instead of turning it into another piece of ineffectual lip service as TRASH did.