SCT Discussion

Old college threads.
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

SCT Discussion

Post by Mike Bentley »

Before we start with SCT discussion, can someone from NAQT enlighten us with what the policy for this is? I.e. should we avoid discussing specific questions because some high school tournaments will be using the Div II set later in the year?
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
Aaron Kashtan
Lulu
Posts: 39
Joined: Wed Apr 20, 2005 11:07 pm
Contact:

Post by Aaron Kashtan »

Just one comment: In my opinion, there should not be ANY trash or geography in NAQT packets. No NAQT packet, at least not at D1 level, should ever contain a single trash or geography question. Everyone hates these kinds of questions and there is no valid excuse for continuing to include them.
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Post by Matt Weiner »

Well, some people do like trash. I am still unclear on who wants all this geography, though.
User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3109
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Post by DumbJaques »

I would be very surprised (and upset) if NAQT policy for SCT discussion is changed, since they're being used at at least two high school tournaments. So yeah, as someone running one of those tournaments, I'll ask you please not to discuss specifics.

Also these questions blew.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Post by Important Bird Area »

Anyone who would like to comment on specific history questions from the SCT is encouraged to email me. In the interest of producing ICT questions that are met with something more positive than rage...
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
User avatar
Birdofredum Sawin
Rikku
Posts: 400
Joined: Tue Nov 09, 2004 11:25 pm
Location: Mountain View

Post by Birdofredum Sawin »

Aaron Kashtan wrote:Just one comment: In my opinion, there should not be ANY trash or geography in NAQT packets. No NAQT packet, at least not at D1 level, should ever contain a single trash or geography question. Everyone hates these kinds of questions and there is no valid excuse for continuing to include them.
Hey, good to see an NAQT discussion thread starting off on one of the traditional notes! In response to this: Wasn't there a survey taken at some point? I can't find the link to it just now, but I'm pretty sure that it didn't establish 100% support for the proposition "we must abolish all trash and geography questions." Also, since there is no known form of quizbowl which features absolutely no trash or geography questions, this is a curious complaint.
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Post by Matt Weiner »

That survey wasn't exactly scientific.

Anyway, let's get the usual party started with another familiar post: NAQT wants to have large distros for trash, geography, and current events compared to some other tournaments. Fine. But especially given how large those distributions are, why is there so much fake material presumably counted as the other categories? There were lots of borderline books and so forth in this tournament that I hope were marked trash and not literature. I would like to say more but we need some guidance on how much can be discussed and where.
User avatar
Pilgrim
Tidus
Posts: 647
Joined: Mon Oct 08, 2007 12:20 pm
Location: Edmonton

Post by Pilgrim »

In Division II, I can recall a total of one tossup on non-Grecco-Roman myth, and that tossup was one of the worst in the tournament. There weren't that many bonuses, either. I don't know if NAQT thinks other myth systems are too difficult for some reason, but I'd have to think that at least Norse myth is completely reasonable for this level difficulty and should have gotten more coverage.
Trevor Davis
University of Alberta
CMU '11
MiltonPlayer47
Wakka
Posts: 162
Joined: Wed Jan 14, 2004 7:14 pm
Location: Roswell, Georgia

Post by MiltonPlayer47 »

I also agree that the questions were bad, at least in Div I. There were a lot that were poorly written. I'd like to give examples but it looks like we have to hold off on that.

Regarding distribution, I don't believe that geography, trash, and current events should be abolished from any format. There is no question though that tossups and bonuses from those categories come up too much in NAQT. "General knowledge" clues should disappear entirely, even if they only make up a small part of the regular distribution.
Steven Hanley
FSU (2004-2006)
UGA (2007-2009)
vig180
Wakka
Posts: 190
Joined: Tue Feb 17, 2004 7:59 pm
Location: Athens, GA

Post by vig180 »

With a few notable exceptions, I thought the set was pretty decent. What I particularly noticed was wide variability in difficulty that sometimes led to buzzer races on powers and other times there were incredibly random answers that neither team knew the answers to. I'm neither a big fan of "from random epithets, narrow it down to a common name for a king!" questions nor did I like the random current events stuff (despite liking current events a ton) that were synthesized with academic answers to produce really weird hybrid TUs. I also wish there would be a way to standardize the packet-by-packet distribution, since it seemed there were just too many periods of questions from the same category in short succession. Overall though, I enjoyed the set, but that's probably because I'm an evil, fraudulent player who likes geography and questions shorter than 6 lines.
mcalmvp
Lulu
Posts: 42
Joined: Mon Apr 17, 2006 12:47 am

Post by mcalmvp »

Matt Weiner wrote:Well, some people do like trash. I am still unclear on who wants all this geography, though.
I want the geography....but...anyways, the TRASH distro, esp. on sports, sucked.
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

Aaron Kashtan wrote:Just one comment: In my opinion, there should not be ANY trash or geography in NAQT packets. No NAQT packet, at least not at D1 level, should ever contain a single trash or geography question. Everyone hates these kinds of questions and there is no valid excuse for continuing to include them.
I think this is some sort of clever troll. Even people who are not fans of NAQT typically don't think this.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
ValenciaQBowl
Auron
Posts: 2558
Joined: Thu Feb 05, 2004 2:25 pm
Location: Orlando, Florida

Post by ValenciaQBowl »

Jerry is sensibly assuming that such a strongly stated position might veer toward parody, though surely similarly strong positions, meant in deadly earnest, are taken often on this board.

Anyway, I happen to know Mr. Kashtan is a real human being who plays for UF, and as far as I know, he neither lives under a bridge nor regularly threatens billy goats, so I assume the post is legit.
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

ValenciaQBowl wrote:Jerry is sensibly assuming that such a strongly stated position might veer toward parody, though surely similarly strong positions, meant in deadly earnest, are taken often on this board.
My conclusion is just based on the fact that the post sounds a lot like a parody of a lot of people who complain about NAQT, including myself. I don't have anything against parody, I'm just not sure that post should be taken overly seriously.
Anyway, I happen to know Mr. Kashtan is a real human being who plays for UF, and as far as I know, he neither lives under a bridge nor regularly threatens billy goats, so I assume the post is legit.
I know Aaron a little bit, and I didn't mean to imply that he himself was a troll. I am just trying to point out that it sounds like something that someone might write as a parody of other similar views.

Anyway, I haven't seen any of the questions yet and probably won't for several more days. I have no idea whether they are any good or not, although I was told that Andrew and Jeff were working on the literature and history portions of the set, respectively, so I would assume those parts are probably pretty good. I'm not sure how much mileage we're going to get out of rehashing the NAQT distribution discussion yet again, but surely no one is surprised by the preponderance of geography and trash at this point.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Post by Mike Bentley »

Some general things that NAQT needs to do to improve their questions significantly:

End the puns and figure-it-outs in the first clue of tossups. The 2006 Sectionals, for example, had a tossups that started "This tossup will be easy as this if you answer it" or something along those lines (the answer was ABC). Without getting into details, there were several of these that came up at this year's sectionals where the first clue was worded in such a way that if you figured out the pun or trick (without having any actual knowledge of the subject)

There were a host of bad question writing principles in this set. For example, one question mentioned the author of a novel still in power. Others were fond of placing stock and trivia clues (i.e. "this guy died doing this amusing thing that has nothing to do with what he's important for") before the real clues in tossups.

If you're going to keep the tossups so short, please at least don't waste the space with non-important clues. On the same note, it is a really bad idea to spend 4 lines of a 5 line tossup on, say, one particular work of an author (often with really unhelpful clues beyond the first one). With no room left in the tossup, the questions suddenly fall off a cliff in terms of difficulty and resulted in buzzer races on a ton of clues. It seems to me like these were tossups deemed to be too hard, and were hastily rewritten to include 75% of

The baseball distribution is out of control. I'm not pointlessly argue that the sports or pop culture distribution is too high at this tournament, but NAQT can certainly do a better job balancing the sub-distribution so we don't hear so many baseball tossups and bonuses.

The bonus difficulty was all over the place. One particular problem was bonuses that would ask for three specific things from one work. For example, there would be literature bonuses that all ask for a character from a novel. Instead of doing this (it's okay maybe for really famous works, but this was often not the case), how about asking for the author or the work based on characters or plot information or something. This makes for bonuses with more well defined easy, medium and hard parts, rather than 30-or-nothing bonuses that essentially distinguish whether you've read the book or not.

There were a ton of stupid, cross-discipline tossups in this tournament. These questions seem like they ate into the real distribution (read: non-geography, non-trash, non-current events) in many cases. If you're going to write a shares the name tossup, please try to keep it to one category. Going wildly from history to literature to pop culture and back again is very frustrating. This is fine once in a while, but it seems like it was employed a bunch in this tournament.

If a tossup needs a stupid nacutie at the end of it to be answerable, chances are it's a bad thing to ask about.

The computer science in this tournament was garbage. In fact, I'd characterize pretty much all of it as "computer literacy" rather than computer science. The things that are important and interesting in computer science have very little to do with the ins and outs of specific languages, etc. I'd have much more to say on this subject if I could talk about specific answers.

That's enough for now. Later I'll post something private with specific question critiques that I'll link NAQT writers and people who played in this tournament to.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

I've just looked at the first packet of the set, and I have some general comments that I'll make now before it's ok to discuss the specifics. My overall impression (based on that one packet) was that given the character limit, very inefficient use was made of the provided space. I felt like many of the tossups included information that was not at all useful and often ended up screwing up the pyramidality of the tossups. I also saw way too many questions which essentially boiled down to a "best guess is right 90% of the time" kind of thing, sort of like "this Portuguese author" (not an actual question). Overall, it's probably better than the last really bad SCT set (I can't remember if that was 06 or 07) but there's still a lot of work that needs to be done on these questions.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
Strongside
Rikku
Posts: 475
Joined: Thu Apr 14, 2005 8:03 pm

Post by Strongside »

Overall I liked this question set a lot, and I liked it quite a bit more than I thought I would.

I have more specific critiques of the tournament, but seeing as how the tournament set is being used again, I obviously won't mention them now.
Brendan Byrne

Drake University, 2006-2008
University of Minnesota, 2008-2010
User avatar
Quantum Mushroom Billiard Hat
Rikku
Posts: 435
Joined: Tue Jun 12, 2007 1:16 pm
Location: Midland, MI

Post by Quantum Mushroom Billiard Hat »

I did enjoy playing in the tournament (DII), for the most part, but there were certainly some problems I noticed. There did seem to be a huge number of questions that encouraged guessing off of "hmm, this sounds kind of like ____. " These get annoying to play on pretty quickly. I also noticed the varying bonus difficulty, which made things difficult. I felt like some powers lasted way to far into the question, though I assume this is very hard to fix fairly.
Finally, and perhaps most importantly I had never realized how much I hate bonuses requiring all the answers at once. Since they give no warning at the start of the bonus, it is difficult to keep track of all the parts being asked for. These are also really easy to 0, if no one on the team quite knows what the question is talking about or misunderstands one of the instructions. Bonuses with multiple different clues allow a little more information to be given.

Also, in terms of geography: I really like geography, in part because I can score a lot of points on it. Despite this, I thought there was an excessive amount of it in this tournament. I would be very happy with a 1/1 distribution in a 20/20 game. I think there was at least 2/? in this set of strict geography questions or questions (often history) that I could answer with only my knowledge of it.
MiltonPlayer47
Wakka
Posts: 162
Joined: Wed Jan 14, 2004 7:14 pm
Location: Roswell, Georgia

Post by MiltonPlayer47 »

Bentley Like Beckham wrote:The baseball distribution is out of control. I'm not pointlessly argue that the sports or pop culture distribution is too high at this tournament, but NAQT can certainly do a better job balancing the sub-distribution so we don't hear so many baseball tossups and bonuses.
For Div 1 I remember hearing exactly one baseball question yesterday. And only one hockey. I should have mentioned this before, but one good thing about the set was that there seemed not to be too many different questions from one sport. You might be able to argue about how well-written the sports questions were, but since I know very little sports I can't really comment about the quality.
Steven Hanley
FSU (2004-2006)
UGA (2007-2009)
User avatar
ecks
Wakka
Posts: 248
Joined: Sun Feb 04, 2007 4:03 am
Location: Springfield, IL / Kirksville, MO

Post by ecks »

From the front page of NAQT's website:
SCT Question Set Overlap

Unlike in past years, NAQT's (four-year) Division II question set for 2008 will not share questions with an Invitational Series. The Division I and Division II sets will continue to share questions, and community college SCTs will use Invitational Series #74.
I assume this means it's okay to discuss specific questions, as the questions won't be shared with other packets.
Christopher Stone
Truman State University '09
User avatar
alkrav112
Rikku
Posts: 269
Joined: Sun Dec 26, 2004 8:55 pm
Location: Ann Arbor, MI

Post by alkrav112 »

Au contraire, in Div I, I heard 4 baseball TUs and 2 on NASCAR, both of which are entirely too many. Of course, I loved the skewed golf distribution, but even I acknowledge that any more than 1/1 in a tournament is too much, especially when it is at the expense of my other favorite sport, tennis.
User avatar
ecks
Wakka
Posts: 248
Joined: Sun Feb 04, 2007 4:03 am
Location: Springfield, IL / Kirksville, MO

Post by ecks »

Matt Weiner wrote:Well, some people do like trash. I am still unclear on who wants all this geography, though.
The response to this seems analogous to Weiner's Law #5: I, like other people in this thread, like geography, although I can understand objections to having a 2/? distrubition of it.
Christopher Stone
Truman State University '09
User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Post by Gautam »

My main criticism of the D2 set:

There were some outright hoses. I can remember at least 3 questions where a given series of clues were applicable to more than one things, I buzzed in with X, and the question went on to say "often contrasted with X." 2 of them had factually incorrect info, and I was pretty unhappy about that.

The biology and chemistry in this set was terrible. It seems to me that certain categories were outright excluded in the bio distro, and the chemistry was stupid shit that should not be asked about.

There were a couple of fine questions in the literature and history distro, but overall, too little visual arts, too little non greco-roman myth, some really whacky literature tossups, some very easy bonuses even for this level, an unnecessarily large distribution of general knowledge, a bunch of cuties, and there was this SCT.

Gautam
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Post by cvdwightw »

rhentzel wrote:As Matt Bruce previously announced, unlike in past years, the DII Sectional Championship Set will NOT share questions with one of NAQT's Invitational Series.

However, NAQT will be offering the DII Sectionals set to hosts that want to run end-of-year, regional, high school tournaments for nationals-caliber teams. Phil Blessman's Midwest Championship is already planning to use the set for that purpose.

So, high schoolers who may be interested in competing in such a tournament (if one will be held in their area), may still want to avoid assisting at SCTs where they will be exposed to the questions.

The DI and DII sets will continue to share questions, some in their exact form, others with modifications to make them easier for Division II.
I'm assuming it's not okay to talk about specific D2 questions since it is apparently being used later in the year. Because the two sets overlap, I'm in serious doubt that we can talk about the D1 set either. What seemed like a good idea at the time now in hindsight looks like an absolutely horrible idea, if I'm reading the full post right.

Anyway, I thought the D1 set was certainly better than 2006 and differently bad than 2007. The 2007 set had a lot of questions on things no one cared about. The 2008 set had less "fluff" and more just outright bad questions on canonical stuff.

Personally I think that the NAQT 26-question distribution should include the standard 20/20 ACF model as part of their set. If they want to use the extra 6 questions as 1 geography, 1 general knowledge, 2 current events and 2 trash I guess that's their prerogative and I'm not going to complain so long as the packet doesn't end up with 18 questions heard with a distribution of 2 geography/1 GK/2 CE/3 PC/10 "mainstream academic".
Brian Ulrich
Wakka
Posts: 104
Joined: Mon Apr 16, 2007 1:22 pm
Location: Chambersburg, Pennsylvania
Contact:

Current Events

Post by Brian Ulrich »

As NAQT's current events editor, allow me to copy Jeff's offer and ask for feedback about that category. I'm going to be travelling for the next week, and so probably won't answer, but I will read everything.

It will be helpful if comments about general trends are illustrated with a specific example or two.
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

rhentzel wrote:As Matt Bruce previously announced, unlike in past years, the DII Sectional Championship Set will NOT share questions with one of NAQT's Invitational Series.

However, NAQT will be offering the DII Sectionals set to hosts that want to run end-of-year, regional, high school tournaments for nationals-caliber teams. Phil Blessman's Midwest Championship is already planning to use the set for that purpose.

So, high schoolers who may be interested in competing in such a tournament (if one will be held in their area), may still want to avoid assisting at SCTs where they will be exposed to the questions.

The DI and DII sets will continue to share questions, some in their exact form, others with modifications to make them easier for Division II.
Goddammit, why is it that every time I think NAQT has taken a step forward, it looks like they've actually turned out to take two steps backwards? This is absurd! It locks up discussion of questions until the end of the year, when in fact the set needs to be discussed now, while impressions are still fresh and people remember stuff about the tournament.

I'm going to wait a couple days before I post anything specific to give NAQT a chance to clarify their position on this. This is exactly the kind of communication that needs to take place, and since Matt Bruce has said that he'll be providing this interface, I hope he shows up in this thread and lets us know what the situation is.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
vandyhawk
Tidus
Posts: 555
Joined: Sat Dec 13, 2003 3:42 am
Location: Seattle

Post by vandyhawk »

gkandlikar wrote:My main criticism of the D2 set:

The biology and chemistry in this set was terrible. It seems to me that certain categories were outright excluded in the bio distro, and the chemistry was stupid shit that should not be asked about.

There were a couple of fine questions in the literature and history distro, but overall, too little visual arts, too little non greco-roman myth, some really whacky literature tossups, some very easy bonuses even for this level, an unnecessarily large distribution of general knowledge, a bunch of cuties, and there was this SCT.

Gautam
That pretty much sums up my thoughts of the DI set. So as not to sound completely negative, I will say that there were many fine questions. There were many bad ones as well. In particular, the science was just not good at all. Stock clues or straight definitions of things were used as openers, and in a couple cases of subjects I know very well, they were actually wrong. Science bonuses were the worst offenders of variable difficulty too it seemed. I will say that mean bonus difficulty per round was very consistent (our ppb never strayed outside a range of 4), which is good, but the variance within rounds was quite high. On the subject of "good question writing practices," there were definitely some tossups where one thought to himself, what is the question here? I feel like that was fixed last year but made a return this year, albeit in only a small number of cases.

I'm not going to say much about the trash, geography, etc. parts of the distribution b/c I know that is just part of NAQT. Plus, many people like those portions, and we do just fine in them as well. Like others have said, though, I wish they would stay in their own parts of the distro. The extreme lack of mythology besides greco-Roman was very odd, and not a problem I remember happening before. The lack of visual arts is hardly a new problem and something I wish would get fixed. It seems an easy solution to some of these is a per-round distribution, but we've all been there before...
User avatar
Paul150
Lulu
Posts: 15
Joined: Sun Feb 10, 2008 4:30 pm
Location: Athens, GA

Post by Paul150 »

I have to say that generally, I thought the questions were well-rounded. I actually like a tournament that features a fair amount of trash, geography, and general knowledge questions. In general, I felt that in the D-II packets we used at Alabama this weekend, there was a good balance. Sometimes they had the annoying puns that could make an annoyingly easy power or make it easy for someone to make a lucky guess at the end without the knowledge, but that was generally the exception to the rule.

Some of the academic distribution did irritate me, as there was a match where our UGA C team suffered a 150 point swing on three classical studies questions (A bonus, powered toss-up, and another bonus) towards the end of what was otherwise a close match. But on the other hand, there were a sizable amount of tossup and bonus topics that were favorable to us towards the beginning of the match, so I don't see a huge use in complaining.

I also only recall one or two baseball questions, whereas I felt there was a bias towards a larger amount of hockey questions. This may be (and is probably) due to the fact that readers get through different amounts of questions in a timed round.

And I will add in that the incident with Aaron was a fairly significant disruption that seemed quite unwarranted to me.
User avatar
Sima Guang Hater
Auron
Posts: 1958
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Post by Sima Guang Hater »

wd4gdz wrote:I figured the namesake of Mt. Vinokurov would have this opinion. But the thing is, he was VERY disruptive during matches, even if you weren't in his room. For example, if you beat him on a buzzer race or on a topic he doesn't approve of, he'll curse loudly during your bonus, so that you can't hear the moderator. If you're next door playing your round, you get distracted in the middle of a match if you hear someone scream and kick stuff.
Wow. I get the feeling he's not usually like this, though.

I'm going to agree with Gautam and say the science was very poorly written as well, for the reasons he and Matt K. have already said. Many questions with poor clues, fluff, outright wrong information, and just bad answer selection. This seemed to pervade the other areas as well.

Lest anyone think I'm some kind of hater, I did see a handful of tossups that I found well-written, pyramidal, clue-dense, and interesting, across all subjects. However, in a decent 20/20 ACF packet, maybe 2/2 of the questions will not meet this burden; in an NAQT packet, there are invariably many more. This might be partially due to my bias against geography and sports, but you can also see it in the academic subjects. In general, I'm very glad I was reading; so many people I read for seemed to completely break down at some of the more egregious questions. I believe the people who had the right idea were Jonathan and Charles, who in general treated it all like one big farce and had fun anyway.

As much as I'd like to see NAQT follow a distribution closer to ACF, the silent majority seems to love the high percentage of trash and geography that inevitably make it into the set, so like everyone else except for Aaron, I don't really see any point complaining about it.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
Rothlover
Yuna
Posts: 815
Joined: Wed Feb 25, 2004 8:41 pm

Post by Rothlover »

Hey this was the post I was going to make a couple hours ago before the board went down for a bit for some reason:

To complain about the baseball distribution when there are other serious problems that affect way more than 1% of all tus is idiotic. We already know naqt uses about 1/1 sports per packet, can you really claim to be rational AND bitch when you say 25% of that going to the most accessible, important and deepest sport is excessive?

And, the inability to talk about the questions for months is beyond absurd. Part of what people pay for in an entrance fee is the right to actually pick apart the questions in a community setting. To use bullshit P.C. language, this has a "chilling" effect on actual substantive discourse about a key naqt product, when, in reality, such pointed discussion is needed for the format's legitimacy.
User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4295
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Post by Auks Ran Ova »

If I hear one more obnoxious NAQT "chemistry" tossup for which the answer was selected by throwing a dart at the periodic table, I will kill someone. (Note: as this occurrence is inevitable, I reserve the right to renege on this promise.)

I'm no science player, but the science at this tournament was awful. The vast majority consisted of either questions about scientists (whee, history of science) or organic chemistry questions. This packet set also confirmed for me that NAQT has no idea how to write computer science, the one area of the science distribution where I have some sort of primary experience. As Mike Bentley said, NAQT seems to prefer asking about "computer literacy" to actual computer science.

While, IIRC, our bonus conversion remained rather steady throughout the day, this was accomplished through the rather unorthodox method of splitting the bonuses between instant 0s/10s and instant 20s/30s. A bonus about an author should not ask for two (or three!) of their most famous works, nor should it be [hard work]/"Who wrote [hard work]?", with no other clues/[easy work].

Several packets seemed to have an especially high concentration of antipyramidality, featuring a number of wholly inappropriate clues in power in a variety of tossups. That kind of wackiness causes the game to devolve into buzzer races, which (as we all know) do a terrible job of distinguishing between good (or any) teams.
Rob Carson
University of Minnesota '11, MCTC '??, BHSU forever
Member, ACF
Member emeritus, PACE
Writer and Editor, NAQT
User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Post by Gautam »

ToStrikeInfinitely wrote: I'm going to agree with Gautam and say the science was very poorly written as well, for the reasons he and Matt K. have already said. Many questions with poor clues, fluff, outright wrong information, and just bad answer selection. This seemed to pervade the other areas as well.
Glad to know that this is not just whiny bitch syndrome among first time players....

Gautam
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.
User avatar
ValenciaQBowl
Auron
Posts: 2558
Joined: Thu Feb 05, 2004 2:25 pm
Location: Orlando, Florida

Post by ValenciaQBowl »

Part of what people pay for in an entrance fee is the right to actually pick apart the questions in a community setting.
Dude, serously? It sucks we can't speak publicly of the questions, but what nuttiness is this?
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

Rothlover wrote:To complain about the baseball distribution when there are other serious problems that affect way more than 1% of all tus is idiotic. We already know naqt uses about 1/1 sports per packet, can you really claim to be rational AND bitch when you say 25% of that going to the most accessible, important and deepest sport is excessive?
Yes, clearly a game that's not even the most popular sport in the United States anymore is the most important and deepest sport there is. What the hell are you on?
To use bullshit P.C. language, blah blah
It's not bullshit and has nothing to do with whatever P.C. ghosts you are trying to exorcise. Seriously, your penchant to bring into quizbowl your outside grudges against perceived P.C. Nazis who are all apparently out to get you is retarded.

Getting back to ACTUAL QUIZBOWL:
Dude, serously? It sucks we can't speak publicly of the questions, but what nuttiness is this?
Chris is right. We don't pay for anything other than playing in the tournament itself. However, I stress again the importance of not viewing tournament participation as simply another kind of financial transaction. The circuit thrives on good faith and flags when teams think that the organizations they deal with aren't being honest with them. The practice of discussing questions after tournaments is par for the course in every other situation. To expect players to wait months for the "all clear" is totally unreasonable.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
Rothlover
Yuna
Posts: 815
Joined: Wed Feb 25, 2004 8:41 pm

Post by Rothlover »

ValenciaQBowl wrote:
Part of what people pay for in an entrance fee is the right to actually pick apart the questions in a community setting.
Dude, serously? It sucks we can't speak publicly of the questions, but what nuttiness is this?
Considering discussing the questions within a month of an event is standard for basically every tournament of the last decade, I hardly think its crazy to think that people would want to do that with an event from an organization that wants to be legitimate. However I'm not clear if, in fact, questions can't be discussed publicly for a while. If they can be discussed soon, fine, and someone from NAQT should be saying so. If they can't be discussed, it, as you said, "sucks" and sucking in any aspect seriously hurts an event, just as similar sucky things like not posting stats would. Certainly the bar should be high for NAQT considering the entrance fees and such.
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Post by Mike Bentley »

Rothlover wrote:
ValenciaQBowl wrote:
Part of what people pay for in an entrance fee is the right to actually pick apart the questions in a community setting.
Dude, serously? It sucks we can't speak publicly of the questions, but what nuttiness is this?
Considering discussing the questions within a month of an event is standard for basically every tournament of the last decade, I hardly think its crazy to think that people would want to do that with an event from an organization that wants to be legitimate. However I'm not clear if, in fact, questions can't be discussed publicly for a while. If they can be discussed soon, fine, and someone from NAQT should be saying so. If they can't be discussed, it, as you said, "sucks" and sucking in any aspect seriously hurts an event, just as similar sucky things like not posting stats would. Certainly the bar should be high for NAQT considering the entrance fees and such.
Eh, it's not like the entrance fees of Sectionals are any different than normal tournaments. In fact, they're probably on the cheaper side (around $100 per team) when you factor in the time / effort of writing a packet.

That being said, it still was not a great decision to limit discussion. As I mentioned in my earlier post, I'm going to soon set up a temporary invite-only message board where Sectionals can be discussed openly to get around this problem.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
NoahMinkCHS
Rikku
Posts: 452
Joined: Fri Apr 18, 2003 1:46 pm
Location: Athens, GA / Macon, GA

Post by NoahMinkCHS »

Bentley Like Beckham wrote:That being said, it still was not a great decision to limit discussion. As I mentioned in my earlier post, I'm going to soon set up a temporary invite-only message board where Sectionals can be discussed openly to get around this problem.
This is a great idea, and far more productive use of time than writing another useless "NAQT hates the circuit" post like so many others have done. Thanks, Mike.

Perhaps in the future, NAQT itself will consider doing something similar, that would enable open discussion (more than just a survey) while also permitting re-use of questions at future events (which, in and of itself, would probably be a net positive for the circuit by exposing high school players to harder questions at an earlier age).
Aaron Kashtan
Lulu
Posts: 39
Joined: Wed Apr 20, 2005 11:07 pm
Contact:

Post by Aaron Kashtan »

grapesmoker wrote:
Aaron Kashtan wrote:Just one comment: In my opinion, there should not be ANY trash or geography in NAQT packets. No NAQT packet, at least not at D1 level, should ever contain a single trash or geography question. Everyone hates these kinds of questions and there is no valid excuse for continuing to include them.
I think this is some sort of clever troll. Even people who are not fans of NAQT typically don't think this.
Well, okay, I may have been exaggerating slightly as a result of frustration.
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

The more I read these questions, the more schizophrenic this set appears. There are great questions intermixed with mediocre ones as well as downright awful questions. The tossup answer choices are frequently dubious ("Here are some facts about a person known for only one thing that no one else knows! Here's the thing that person is known for! For ten points, find your ass with both hands and a map!") and much of the text fails to make efficient use of the limited amount of question text. Overall, it looks to me like the set suffers from not having a single good editor (or a group of 2 or 3 editors) who have a lot of regulatory power over what gets in.

To take a historical view of things, it's still better than the earlier SCTs of 03, 04, and 05. But since then, the quality has generally plateaued at "mediocre." The fact that half these questions look like they should be buzzer races between competent teams makes playing on them all that much more frustrating.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
PaladinQB
Wakka
Posts: 168
Joined: Tue Apr 12, 2005 12:13 pm
Location: Cleveland, OH
Contact:

Post by PaladinQB »

The discussion of disruptive behavior and suchlike has been split from this thread and exists here.

bms
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Post by Skepticism and Animal Feed »

Did anyone else think that some of the current events were a bit dated? A lot of it seemed to be from 2005/2006.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

NAQT Discussion Policy

Post by rhentzel »

I apologize for not being able to get online quickly to post this yesterday, but here is the official guidance for discussing the questions at the 2008 NAQT SCT.

The DII set will be used (possibly with minor modifications) at future high school tournaments, and some of those questions are shared with the DI version. Therefore, any discussion of questions may compromise (to a greater or lesser degree) the security of those tournaments.

However, NAQT believes that rapid (and concrete!) discussion is valuable for the the SCTs; players should feel free to include information that "ruins" questions if it is important that it be discussed openly on this forum. NAQT will track such questions and provide a list to hosts using the set in the future so that those questions can be removed. On the other hand, if you don't feel that an issue is big enough (or sufficiently discussion-worthy) to require the use of the forum, please send us your feedback privately at [email protected].

In other words, except for possibly grotesque revelations of content, NAQT is fine with players using their own judgment as to what should be posted.

I (and other NAQT members as is appropriate) will be monitoring this thread to try to answer questions that are raised.

I will also be starting a second thread for "larger-than-a-single-tournament" issues since the last thread I read contained several assertions that there were larger questions that NAQT did not sufficiently address because it focused its responses on specific questions about the SCT or ICT under discussion.
User avatar
Pilgrim
Tidus
Posts: 647
Joined: Mon Oct 08, 2007 12:20 pm
Location: Edmonton

Post by Pilgrim »

Some more general comments on the distribution:

As some others have remarked, the lit was highly frustrating. There were some tossups on works that I found well written and enjoyable, but there were also several questions on works of questionable literary merit, and pretty much all the tossups on authors were poorly structured in terms of work placement, full of biographical clues unrelated to the writer's work, or both.

I don't recall noticing this at the time, but thinking back on the tournament now, it seems like there was a real lack of South American and Middle Eastern history, but this could just be my selective memory.

Also, unsurprisingly, philosophy was underrepresented and most of the questions that were there blew.
Bentley Like Beckham wrote: The baseball distribution is out of control.
Maybe it was a difference between DI and DII, but I was actually pleasantly surprised by what I remember as a low amount of sports in this set compared to other NAQT sets I've played before, and I certainly don't remember baseball being that overrepresented. However, if I recall correctly, there was more than one NASCAR question, and that's just ridiculous.
Trevor Davis
University of Alberta
CMU '11
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: NAQT Discussion Policy

Post by grapesmoker »

rhentzel wrote:However, NAQT believes that rapid (and concrete!) discussion is valuable for the the SCTs; players should feel free to include information that "ruins" questions if it is important that it be discussed openly on this forum. NAQT will track such questions and provide a list to hosts using the set in the future so that those questions can be removed. On the other hand, if you don't feel that an issue is big enough (or sufficiently discussion-worthy) to require the use of the forum, please send us your feedback privately at [email protected].
R., thanks for clarifying this issue. We'll be sure to try and keep the discussion of specifics to a minimum. In particular, anyone who has a copy of the set could cross-check a DI question that they are thinking of commenting about with the corresponding DII round, that would be useful in avoiding compromising the DII questions.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Post by rhentzel »

Pilgrim wrote:In Division II, I can recall a total of one tossup on non-Grecco-Roman myth, and that tossup was one of the worst in the tournament. There weren't that many bonuses, either. I don't know if NAQT thinks other myth systems are too difficult for some reason, but I'd have to think that at least Norse myth is completely reasonable for this level difficulty and should have gotten more coverage.
According to a quick database query, the DII set had 4/4 Greek/Roman mythology questions, 1/1 Egyptian, 1/0 Norse, and 0/1 Mesoamerican.

If you have time, please e-mail us about the issues with the specific tossup that was so bad so we can correct it before the set is used for high school tournaments.

To answer your more general question; NAQT doesn't think non-classical myth is categorically too hard for Division II, but we do think it is harder and should be a comparatively small portion of a set.

What sort of distribution do you think would be appropriate?
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Post by rhentzel »

MiltonPlayer47 wrote:I also agree that the questions were bad, at least in Div I. There were a lot that were poorly written. I'd like to give examples but it looks like we have to hold off on that.
We'd be very interested in looking at examples of questions that players thought were poorly written; if you don't want to retype them, just send us the ID numbers and we can quickly look them up in the database ourselves.
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Post by rhentzel »

Bentley Like Beckham wrote:The baseball distribution is out of control. I'm not pointlessly argue that the sports or pop culture distribution is too high at this tournament, but NAQT can certainly do a better job balancing the sub-distribution so we don't hear so many baseball tossups and bonuses.
According to a database query, this was the sports breakdown for the DI SCT:

Baseball 3/2
Basketball 2/3
Boxing 1/0
Football 2/2
Golf 1/1
Hockey 2/2
Horseracing 1/0
Other minor sports 1/0
Olympics 0/1
Soccer 0/1
Track-and-field 0/1
Volleyball 0/1

As anybody who knows me can tell you, I can't give really provide a personal view on whether this is a reasonable coverage of the world of sports that is informed by a great of depth of subject-area knowledge . . . but the numbers themselves suggest that things are pretty balanced.

What sort of subdistribution in the sports would you like to see?
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Post by rhentzel »

Bentley Like Beckham wrote:End the puns and figure-it-outs in the first clue of tossups. The 2006 Sectionals, for example, had a tossups that started "This tossup will be easy as this if you answer it" or something along those lines (the answer was ABC). Without getting into details, there were several of these that came up at this year's sectionals where the first clue was worded in such a way that if you figured out the pun or trick (without having any actual knowledge of the subject).
NAQT's current policy is that wordplay, analogy, and lateral-thinking has a place in the game of quiz bowl. Perhaps more precisely, we believe that the ability to see the connections implied by such clues is a reasonable skill to test, that such clues do often reward knowledge (though not perhaps knowledge of the superficial subject), and it is something that should be rewarded by a small number of questions.

Such clues shouldn't be the lead-ins of very many questions, as they are not the focus of quiz bowl.
PaladinQB
Wakka
Posts: 168
Joined: Tue Apr 12, 2005 12:13 pm
Location: Cleveland, OH
Contact:

Post by PaladinQB »

One thing I'd like to know with regard to all the distribution questions is if NAQT makes an effort to balance the distribution of the first 20 TU's in each round? We averaged a tad over 23 TU per match and I think our experience was relatively typical.

Leaving 10%+ of each game on the table could really skew the distribution and players perceptions of it. This is particularly true in the case of bonuses, given that even good teams will have several TU's go dead in matches where NAQT's "unique" answer selection takes hold.
rhentzel
Rikku
Posts: 260
Joined: Thu May 15, 2003 4:20 pm
Location: Minneapolis, MN
Contact:

Post by rhentzel »

Bentley Like Beckham wrote:The bonus difficulty was all over the place. One particular problem was bonuses that would ask for three specific things from one work. For example, there would be literature bonuses that all ask for a character from a novel. Instead of doing this (it's okay maybe for really famous works, but this was often not the case), how about asking for the author or the work based on characters or plot information or something. This makes for bonuses with more well defined easy, medium and hard parts, rather than 30-or-nothing bonuses that essentially distinguish whether you've read the book or not.
It is, in fact, NAQT's policy that bonus questions involving three parts about the content of a single work (rather than more superficial questions about, say, its author) should be reserved for works that are extremely well-known, either due to their presence in curricula or because they come up so frequently in quiz bowl that division-I players could reasonably realize that in-depth knowledge might be tested.

It's certainly possible that we asked such a question about an inappropriately difficult work; can you drop us an e-mail at [email protected] with the works that that received dedicated bonuses that you think are not sufficiently well-known to deserve that honor? We can certainly check that against our bonus stats and see if there was a significant difference and adjust of "fame threshold" accordingly for future sets.
Locked