Question-specific discussion: 2016 Division I SCT

Old college threads.
User avatar
gerbilownage
Lulu
Posts: 43
Joined: Tue May 14, 2013 2:03 am

Re: Question-specific discussion: 2016 Division I SCT

Post by gerbilownage » Sun Feb 14, 2016 12:14 am

yup haha, I think I confused "Giuseppe" with "Guell" in my memory somehow
Laurence Li
Westview HS '13
Yale '17
Harvard '20

User avatar
Corry
Rikku
Posts: 330
Joined: Fri Feb 10, 2012 11:54 pm

Re: Question-specific discussion: 2016 Division I SCT

Post by Corry » Tue Jul 05, 2016 9:18 pm

armitage wrote:
Corry wrote: 1793: This might just be a personal preference of mine, but I really don't like tossups on years. When you study history in an academic context, specific years don't really matter. Chronology? Sure, that's important- so write a question on the origins of the Committee of Public Safety. Specific decades? Yeah, sometimes that's important too- so write a question on the 1790s. But why does it matter that the Vendee Revolt specifically happened to start in the year 1793? (For the record, I got this question on the Vendee clue, but I wasn't particularly happy about it.)
I wrote this. In my mind (maybe only in my mind) the names of the various French revolutionary assemblies and factions are less memorable than years in which many important, related events occurred. The answerline "1790s" feels like it would have transparency issues. And as you said, it doesn't seem like this question style barred you from being rewarded for knowing the clues; the D2 version also played fine for the teams I read it to. I'd rethink this type of question if it turns out to have generated lots of negs elsewhere; otherwise I don't think year tossups are invalid in principle.
So this is wayyyy after the fact, but I think the conversion data bears out my concerns with this question.
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor

User avatar
Cheynem
Sin
Posts: 6496
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Question-specific discussion: 2016 Division I SCT

Post by Cheynem » Tue Jul 05, 2016 9:23 pm

What was the conversion data?
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
armitage
Wakka
Posts: 195
Joined: Sun Nov 28, 2010 11:52 pm

Re: Question-specific discussion: 2016 Division I SCT

Post by armitage » Wed Jul 06, 2016 1:11 am

The 1793 tossup went 1/12/12 in 16 rooms. Skimming the other tossup conversion data puts it firmly in the bottom half of Points per Room (along with such diverse topics as the transit of Venus, partially ordered sets, or Durban) and with a really high neg rate (comparable to the tossup on Pavel Chichikov, which I assume literally everyone negged with Raskolnikov). So maybe I misjudged how important the year itself was in French history.

I still don't think it's unreasonable to ask questions on unconventional years, and I think this set should contain a few questions with an experimental flavor.
Richard Yu

User avatar
Corry
Rikku
Posts: 330
Joined: Fri Feb 10, 2012 11:54 pm

Re: Question-specific discussion: 2016 Division I SCT

Post by Corry » Wed Jul 06, 2016 8:55 am

Can we also see the conversion data for Decline and Fall, the San Francisco Bay Area, Devon, and Charles Curtis?
Corry Wang
Arcadia High School 2013
Amherst College 2017
NAQT Writer and Subject Editor

User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5445
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Question-specific discussion: 2016 Division I SCT

Post by Important Bird Area » Wed Jul 06, 2016 12:24 pm

Corry wrote:Can we also see the conversion data for Decline and Fall, the San Francisco Bay Area, Devon, and Charles Curtis?
Decline and Fall: 2/4/6 in fifteen rooms.

SF Bay Area: 0/13/8 in fourteen rooms.

Devon: 2/9/7 in fourteen rooms.

Curtis: 1/6/4 in sixteen rooms.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

Adventure Temple Trail
Auron
Posts: 2613
Joined: Tue Jul 15, 2008 9:52 pm

Re: Question-specific discussion: 2016 Division I SCT

Post by Adventure Temple Trail » Wed Jul 06, 2016 4:49 pm

How many rooms does NAQT believe a question has to be played in for conversion data to be meaningful?
Matt J.
ex-Georgetown Day HS, ex-Yale
member emeritus, ACF

Sailing away on my copper boat

User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5445
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: Question-specific discussion: 2016 Division I SCT

Post by Important Bird Area » Wed Jul 06, 2016 10:26 pm

More than those low-single-digit values... which are wrong anyway because I hit the wrong button on our website and got just the New England site of the DI SCT. Updating now.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

User avatar
theMoMA
Forums Staff: Administrator
Posts: 5652
Joined: Mon Oct 23, 2006 2:00 am

Re: Question-specific discussion: 2016 Division I SCT

Post by theMoMA » Thu Jul 07, 2016 1:23 am

Conversion data is obviously a valuable tool to assess whether particular questions played well, but I don't think a high neg rate, or less-than-perfect conversion by the end, is necessarily indicative of a bad question. For an example of an instance where a high neg rate seems acceptable to me, look no further than the "1793" question. To the extent that those buzzes were people with imperfect knowledge taking the plunge and guessing a year in the French Revolution, I'm ok with that. I suppose it's reasonably foreseeable that an answer that boils down to a year during the French Revolution might lend itself to more aggressive buzzing, and thus more negs, but I don't see that as a reason not to write on the answer. Otherwise, I think you're basically saying that some answers are ruled out because we don't want to invite players to play the game a certain way, and that seems like a dubious decision to me.

As a practical matter, editors and writers don't operate with anything close to perfect foresight when producing questions; instead, we make our best guess as to whether the tossup is gettable and will have an appropriate distribution of buzzes, etc. For an answer like "1793," which has probably never been tossed up before, the writer/editors can only guess, based on experience and feel, whether it will be converted and play well. This typically involves asking whether the answer is something worth knowing, and if so, whether it's something that people actually know. In my opinion, it's worth knowing the chronology of the French Revolution, and in my judgment, I thought it was something people would know (which it appears was true for 75% of rooms).

Tossups generally fall into a range of difficulties. This is practically impossible to avoid, and even if you could, perfect standardization of difficulty and buzz distribution would make tossups much easier to guess, because the answer would never be something more difficult than what 85% of teams know, or whatever the threshold is. I'm ok having a few harder questions in a set to keep teams honest, just as I'm ok having a few questions that might invite more guesses, and thus more negs.
Andrew Hart
Minnesota alum

Locked