Are IS-A sets appropriate for new teams?

Dormant threads from the high school sections are preserved here.
User avatar
Captain Sinico
Auron
Posts: 2857
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: Are IS-A sets appropriate for new teams?

Post by Captain Sinico »

Brian Ulrich wrote:I have no idea whether NAQT is keen to increase IS quantity, though quality is always a concern. As far as the other uses of resources, though, I think you're missing the other end of things. As I noted on the other end, NAQT produces questions for TV shows that air in a few limited markets. These competitions are usually not as prestigious among quiz bowlers, but they are the highest levels of exposure academic competition receives and hopefully create a groundswell of support that could be used to support local hs programs. NAQT introduced the Speed Check sets later as practice material, and I got the distinct impression this was just a case of realizing they got get more mileage out of questions already written. In other words, no one is writing and subject editing just for Speed Check sets, and only minimal set editing is involved.
I find this a highly evasive, disappointing, and incoherent answer. Even if I accept the premise that NAQT is writing these sets to get exposure for quizbowl (to whatever salubrious ends,) I think that quality and production of quality in sufficient quantity ought be a concern above getting the type of exposure for quizbowl offered by television shows that use speedcheck questions and, consequently, that minimal resources should be devoted to the goal of gaining quizbowl exposure in that way until the goal of providing sufficient questions of sufficient quality is met at a significantly higher level of satisfaction than it is now. To my view, NAQT has acted otherwise, which is precisely the type of resource/action conflict that causes me to call NAQT's priorities and intentions into question in my own mind and to conclude that, thought I do not know those priorities and intentions, they are seemingly at odds with my own.
However, I find even that premise highly unsatisfactory. I don't understand how providing speedcheck questions in quantity is meant to (or even ever going to) create "a groundswell of support" for bona fide quizbowl questions when providing the latter isn't evidently a priority (or, at least, not one on a par with providing same speedcheck questions) and NAQT has repeatedly stated that it's strapped for factors of production (vis a vis writers in the main.) In other words, if NAQT thought its production of speedcheck questions was creating demand for normal questions, it should be producing more normal questions to meet that demand, but it's not clear to me that that's what it is doing or wants to do.
Compounding these issues here, I think that NAQT's practices act to exacerbate its scarcity of writers, especially of high-quality questions. For example, I am hardly the only, most ardent, or even most prolific expositor of reservations regarding NAQT's intentions, aims, and practices that have a significant bearing on my assessment of NAQT as a potential outlet for my writing (thought these are perhaps not important in my case, as I outline later) but I don't see anyone responsible clarifying what NAQT's intentions, aims, and practices are to my satisfaction. As a further example, I have long held and continue to hold that NAQT's practice of providing high school questions to college tournaments has served to decrease the number of questions written by college students and poison its own writers' pool.
Thus, I'm not sure what "the other end of things" that I'm ostensibly missing is. To me, NAQT clearly has conflicts for its scarce resources and is choosing to resolve them in a way that either doesn't make sense or is at odds with what I think their priorities ought to be and even with what their stated priorities are, to the extent that the latter exist. I further argue that NAQT's own practices tend to exacerbate these conflicts.
Brian Ulrich wrote:Let's also not forget that several years ago, in 2000-2, at least, NAQT's biggest problem was that it was satisfying the market for upper end high school questions, but receiving numerous complaints that its questions were too hard for most teams.
I'm unconvinced that this was the case. By instructing me “not [to] forget,” you're stating this as a matter of public record. Okay; fair enough: show me the record.
However, even if I uncritically accept this, it seems to have no bearing on the present state, in which NAQT is producing as many nominally introductory questions as and many more speedcheck questions than nominally normal questions by its own reckoning, in which the quality of all these questions is far from unimpeachable, in which there seems to be a great deal of feedback from many quarters decrying these facts, and in which no good justification or even explanation is being offered by a responsible representative.
Brian Ulrich wrote:I'm sure I can't give whatever it is, but what types of assurances did you have in mind?
Well, I'm not sure, because I don't have in mind an attempt to convince me, but others, and I don’t know what assurances others might seek. This question is designed more to ascertain what assurances NAQT is prepared to give, independent of what assurances might be necessary.
However, I will try to answer your question on the premise that you're trying to convince me to write for NAQT (a moot end, since I'm too busy to write much of anything.) I suppose the first, most basic thing I'd ask for is a public statement of principles and direct, substantial answers to my questions by someone accountable at NAQT. There are many things that one might do to back up such statements, but I don't think I can say at this time what would be sufficient, even for me, because the nature of the requirement for backing of such a statement depends sensitively on its contents (mainly on how at odds I consider them to be with observed practice.) For example, I might consider the statement itself sufficient. I'll consequently leave it until such a statement is made to comment on whether I consider such backing sufficient and, if not, what I might prefer to see.

MaS
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

User avatar
Birdofredum Sawin
Rikku
Posts: 400
Joined: Tue Nov 09, 2004 11:25 pm
Location: Mountain View

Re: Are IS-A sets appropriate for new teams?

Post by Birdofredum Sawin »

I thought I'd offer my own take on Mike's requests for answers.
Captain Scipio wrote: Consequently, I wish someone authoritative would post regarding this one way or another, either in this thread or in Guy's thread about motivations for writing. For example, I think authoritative answers are called for to the following questions:
How great a priority is it for NAQT to increase its level of question quality in IS sets and how great to increase the number of such sets?
I'm sure NAQT would like to increase question quality in IS sets. Are you're asking "what trade-offs is NAQT willing to make to increase IS-set quality?" I don't have an answer to that. Also, I'm not sure what the point of the second part of the question is. Are you suggesting that NAQT ought to be increasing the number of IS sets (perhaps at the expense of A-sets)? Or are you suggesting that NAQT ought to be decreasing the number of IS sets (on the assumption that fewer sets would be higher in quality)?
Captain Scipio wrote: Given that level of priority, why are resources devoted to producing things like speedcheck sets and A-level sets?
Again, I don't have an answer to this. Let's say that "a general account of why NAQT produces speedcheck and A sets" is something people still want. (I take it my attempts to justify A-sets, at least in theory, and other people's attempts to justify speedcheck sets don't seem satisfactory.)

Here's another question, though. Let's say that speedcheck sets take some significant, if not huge, amount of resources for NAQT to produce, but that the profit NAQT gains from them allows it to charge less money for a tournament like ICT. Would that make the speedcheck sets more acceptable to you, or would that not matter?
Captain Scipio wrote: Is it definitively acceptable for a writer to write only for a certain level, say for IS sets?
This is an easy one: Yes, that is categorically acceptable. Any NAQT writer can write exclusively for any level of difficulty he pleases. Nobody at NAQT compels writers to produce anything.
Captain Scipio wrote: If NAQT received a large number of good questions from writers wishing to write only for a certain level, would it increase production levels and standards at that level?
Maybe I don't quite get the point of this question, but I don't see how the answer to this could be anything but "yes." If, for example, NAQT suddenly got 500 high-quality HSNCT-level questions from current collegiate writers, the "production levels and standards" at the HSNCT level would by definition "increase." I don't know what else could possibly happen.
Captain Scipio wrote: How can someone entertaining the sincere doubts expressed in this and the "High School Writers" threads credit the answers given? What assurances can NAQT give?
MaS
Again, I'm not really sure what "assurances" would seem satisfactory. I can tell you from personal experience, for instance, that during my roughly five years writing for them, NAQT has never required any writer to produce questions of a particular type. Beyond that, I don't know how to answer this question.

Andrew
Andrew

Ex-Virginia, Ex-Chicago, Ex-Stanford

User avatar
Captain Sinico
Auron
Posts: 2857
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: Are IS-A sets appropriate for new teams?

Post by Captain Sinico »

Birdofredum Sawin wrote:Also, I'm not sure what the point of the second part of the question is. Are you suggesting that NAQT ought to be increasing the number of IS sets (perhaps at the expense of A-sets)? Or are you suggesting that NAQT ought to be decreasing the number of IS sets (on the assumption that fewer sets would be higher in quality)?
My own view is that NAQT ought to increase both the number and quality of IS sets at the expense of IS-A and speedcheck sets. Not to be too pugnacious, but I want to say that the question is not meant to make a point: it's a question that I am earnestly seeking an answer to.
Birdofredum Sawin wrote:Let's say that "a general account of why NAQT produces speedcheck and A sets" is something people still want. (I take it my attempts to justify A-sets, at least in theory, and other people's attempts to justify speedcheck sets don't seem satisfactory.)
Well, all such explanations are satisfactory to various extents according to their explanatory power, but they share the common failing that lack an authoritative backing. My point here is that somebody, clearly, is directing production of these sets. I want to know from that body or from an authorized representative why things are being produced at the levels that they are and what guiding principles are used to determine those levels generally.
Birdofredum Sawin wrote:Here's another question, though. Let's say that speedcheck sets take some significant, if not huge, amount of resources for NAQT to produce, but that the profit NAQT gains from them allows it to charge less money for a tournament like ICT. Would that make the speedcheck sets more acceptable to you, or would that not matter?
That's a good question. Since I think having ICT cheaper is a positive end, such a profit reinvestment scheme would seem to make speedcheck question production more acceptable. However, I'm not certain to what extent NAQT would need to defray ICT costs to make this a net positive and I'm also deeply doubtful that effecting such a scheme is the best solution to this issue.
Let's analyze things a bit further. While the speedcheck questions in se don't seem to me to be a net negative (maybe they suck and displace other quizbowl, maybe they promote good quizbowl by promoting quizbowl generally, but I don't think they have a strong effect either way) the speedcheck questions definitely have an opportunity cost associated with them in the form of resources not spent on better questions. Further, this opportunity cost is, in my view, at least two-fold*: NAQT is losing the revenue from the sale of the better questions and quizbowl generally is losing whatever utility the better questions would provide: I am therefore losing utility as someone who cares about/is part of quizbowl.
So, the problem has become one of compensating my lost utility: what level of profit reinvestment in quizbowl will make me happy enough to condone speedcheck question production? Assuming that the profit from the sets is greater than the lost utility, this should be possible, but I think that it's inevitably a sub-optimal solution as it's essentially a form of Pigouvian quasi-tax: I'm demand that NAQT pay quizbowl back some of its profits and, if they refuse, I penalize them by refusing my services as a question-writer (or whatever) and either way the market seems to experience a deadweight loss. Seen in another way, this is a bargain wherein I have my good will is on the table: I think that a contract point can probably be reached by NAQT’s offering increasing levels of profit reinvestment, but I also think that will be a sub-optimal contract point.
A better solution to the problem as stated from my point of view is for NAQT to produce fewer (or even no) speedcheck questions and instead work on what I view as better questions to make their money that way. In this way, we avoid the deadweight loss associated with the quasi-tax or we reach a better contract point because NAQT has more to offer me.
Actually, I’d like to make a general comment at this point. This situation, I think, illustrates an essential dilemma: NAQT's optimizing its profit in the short-term does not necessarily optimize the overall utility of the NAQT-quizbowl system because it neglects the utility imparted to quizbowl by questions. The resolution to me is simple: in the long run, NAQT is ill-served trading quizbowl's utility+ for profit because NAQT is dependent on the trust of quizbowl for that very profit (for example, quizbowl provides all of NAQT's writers to my knowledge.) Consequently, NAQT ought to pursue ends that serve both.

MaS

*As I earlier posited, I think there’s an additional hidden cost here, namely NAQT’s lost utility in the form of lost production due to alienated or diminished potential writers or customers. I think this is an especially important cost due to its compound nature. However, I don’t include it in the analysis as a cost because it later appears in a different form, namely the “fine” in the Pigouvian quasi-tax I’m levying on NAQT for producing questions that I dislike.

+Obviously, people have different views of what is good for quizbowl, so it's for NAQT to decide what ends it's best to pursue there. However, it seems clear to me that pursuing ends widely at odds with prevailing ideas among potential writers and editors about what's good for the game does not serve NAQT's best interest. For example, at least some potential writers are saying that the ends they see NAQT pursuing are enough at odds with their view of what's good for quizbowl that they're unwilling to work for NAQT for that reason; if that trend is as general as is sometimes asserted, NAQT has made an injudicious choice. To me, NAQT has decidedly made an injudicious choice by not having clear ends one way or another, which is a failing I hope we can resolve here to some extent.
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

Brian Ulrich
Wakka
Posts: 104
Joined: Mon Apr 16, 2007 1:22 pm
Location: Chambersburg, Pennsylvania
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Brian Ulrich »

Captain Scipio wrote:
Brian Ulrich wrote:I have no idea whether NAQT is keen to increase IS quantity, though quality is always a concern. As far as the other uses of resources, though, I think you're missing the other end of things. As I noted on the other end, NAQT produces questions for TV shows that air in a few limited markets. These competitions are usually not as prestigious among quiz bowlers, but they are the highest levels of exposure academic competition receives and hopefully create a groundswell of support that could be used to support local hs programs. NAQT introduced the Speed Check sets later as practice material, and I got the distinct impression this was just a case of realizing they got get more mileage out of questions already written. In other words, no one is writing and subject editing just for Speed Check sets, and only minimal set editing is involved.
I find this a highly evasive, disappointing, and incoherent answer.
If I wanted to be evasive, I wouldn't have taken the time to post. I answered to the best of my ability, primarily by explaining the origins of the situation we have today.
Captain Scipio wrote:Even if I accept the premise that NAQT is writing these sets to get exposure for quizbowl (to whatever salubrious ends,) I think that quality and production of quality in sufficient quantity ought be a concern above getting the type of exposure for quizbowl offered by television shows that use speedcheck questions and, consequently, that minimal resources should be devoted to the goal of gaining quizbowl exposure in that way until the goal of providing sufficient questions of sufficient quality is met at a significantly higher level of satisfaction than it is now. To my view, NAQT has acted otherwise, which is precisely the type of resource/action conflict that causes me to call NAQT's priorities and intentions into question in my own mind and to conclude that, thought I do not know those priorities and intentions, they are seemingly at odds with my own.
Frankly, this is where you need R, or at least someone who can speak authoritatively, to answer, but it seems likely that you and NAQT simply have an honest disagreement over whether trying to produce quality questions for TV shows with short character limits is worthwhile. The way you phrase the situation as "questioning intentions" makes the situation sound far more insidious than it actually is.
Captain Scipio wrote:However, I find even that premise highly unsatisfactory. I don't understand how providing speedcheck questions in quantity is meant to (or even ever going to) create "a groundswell of support" for bona fide quizbowl questions when providing the latter isn't evidently a priority (or, at least, not one on a par with providing same speedcheck questions) and NAQT has repeatedly stated that it's strapped for factors of production (vis a vis writers in the main.) In other words, if NAQT thought its production of speedcheck questions was creating demand for normal questions, it should be producing more normal questions to meet that demand, but it's not clear to me that that's what it is doing or wants to do.
Again, this seems grounded in philosophical differences. I don't know what R would say, but I would say that, while the circumstances of NAQT initially branching out from the type of quiz bowl you like were explained, from the information available to a then-writer, on the thread Guy started, writing for quiz competitions that use other format when NAQT has no power to change that format is perfectly acceptable, and ultimately serves to support the company itself and hence its other endeavors.
Captain Scipio wrote:
Brian Ulrich wrote:Let's also not forget that several years ago, in 2000-2, at least, NAQT's biggest problem was that it was satisfying the market for upper end high school questions, but receiving numerous complaints that its questions were too hard for most teams.
I'm unconvinced that this was the case. By instructing me “not [to] forget,” you're stating this as a matter of public record. Okay; fair enough: show me the record.
Good grief, I don't even remember the main discussion forum from that long ago, let alone know where I can find it archived. This seems like an unreasonable request compounded by the implication that I feel compelled to spend time providing misleading answers for nefarious motives of my own.
Captain Scipio wrote:However, even if I uncritically accept this, it seems to have no bearing on the present state, in which NAQT is producing as many nominally introductory questions as and many more speedcheck questions than nominally normal questions by its own reckoning, in which the quality of all these questions is far from unimpeachable, in which there seems to be a great deal of feedback from many quarters decrying these facts, and in which no good justification or even explanation is being offered by a responsible representative.
As far as the A sets go, the point is that they exist because of criticisms that once existed, criticisms which I gather were before your time. If the present state were to revert to what existed before, those problems would presumably resurface.
Brian Ulrich
NAQT Current Events Editor, 2005-
University of Wisconsin 1999-2003
Quincy University 1995-1999

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Are IS-A sets appropriate for new teams?

Post by Mechanical Beasts »

Brian Ulrich wrote:
Captain Scipio wrote:
Brian Ulrich wrote:Let's also not forget that several years ago, in 2000-2, at least, NAQT's biggest problem was that it was satisfying the market for upper end high school questions, but receiving numerous complaints that its questions were too hard for most teams.
I'm unconvinced that this was the case. By instructing me “not [to] forget,” you're stating this as a matter of public record. Okay; fair enough: show me the record.
Good grief, I don't even remember the main discussion forum from that long ago, let alone know where I can find it archived. This seems like an unreasonable request compounded by the implication that I feel compelled to spend time providing misleading answers for nefarious motives of my own.
You might be misreading here, if unintentionally. By saying "let's... not forget" you imply we should be able to remember (i.e. we should have access to information about) the complaints levied against NAQT circa 2000-2. I think he simply means that he hopes we are provided with some of this record of complaints so we can see for ourselves.
Andrew Watkins

User avatar
Captain Sinico
Auron
Posts: 2857
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: Are IS-A sets appropriate for new teams?

Post by Captain Sinico »

Brian Ulrich wrote:If I wanted to be evasive, I wouldn't have taken the time to post. I answered to the best of my ability, primarily by explaining the origins of the situation we have today.
Okay. I found your answer to the best of your ability to be evasive, disappointing, and incoherent. I'm sorry if that wasn't your intent; I never said it was and didn't believe it was even before you said so. I don't agree with your assessment of the origins of this situation and you seem unwilling or unable to provide evidence supporting it, so there you have it.
Brian Ulrich wrote:...you need R, or at least someone who can speak authoritatively, to answer...
Given that that's what I explicitly asked for, yes, I do!
Brian Ulrich wrote:...but it seems likely that you and NAQT simply have an honest disagreement over whether trying to produce quality questions for TV shows with short character limits is worthwhile. The way you phrase the situation as "questioning intentions" makes the situation sound far more insidious than it actually is.
Whether such a difference exists is precisely what I'm trying to ascertain. Since I have no comprehensive on-the-record statement of what NAQT's philosophy is, I have no good was of doing so. Hence my questions.
When I say I'm questioning NAQT's intentions, I mean precisely this: given what I've seen people say and do in the past on behalf of NAQT, I developed certain notions of what NAQT's intentions and practices are, specifically as relates to how well they align with my own aims for this game. I now find those notions questionable in light of further evidence about NAQT's actions and practices. This is no more or less insidious than the actual reality of the situation and, as I said, I am hardly alone in expressing this sentiment.
Brian Ulrich wrote:...the circumstances of NAQT initially branching out from the type of quiz bowl you like were explained...
They have not been explained, to the best of my knowledge, by someone authorized to speak on the matter by NAQT. Since that's precisely what I asked for, it's hand-waving on your part to try and dismiss this querry in the manner you attempt here.
Brian Ulrich wrote:...writing for quiz competitions that use other format when NAQT has no power to change that format is perfectly acceptable, and ultimately serves to support the company itself and hence its other endeavors.
Okay: I disagree with this assessment whole-heartedly. I don't think this practice serves NAQT's interests in the long run for reasons I outlined above, reasons that you don't address. I also don't accept that this practice is "perfectly acceptable" for reasons I outline above and that you don't address. I'll further add that I don't accept even the premise that NAQT has no power to effect changes in format.
Brian Ulrich wrote:[Your request that I substantiate my statement] seems like an unreasonable request compounded by implication that I feel compelled to spend time providing misleading answers for nefarious motives of my own.
All I'm saying here is, as Andy says, that you claimed that the creation of the A sets was a response to feedback and, further, that that fact is a matter of public record. You go so far as to assert that I knew that it was so at one time: else, how could I be exhorted not to forget it?
I say, on the contrary, that I do not know and I never knew any such thing or, at least, that I do not now know and, if I previously did, I certainly forgot entirely about it. I'm consequently demanding the facts that back your assertion before I accept it, because it's directly contrary to my experience of the matter. If you can't find any facts, that's fine: just don't except to convince me, then. What could possibly be more reasonable than that?
Also, to address your complaint about my "implication," I made no such implication. While I can't prevent you from reading one into what I said, I can assert again that I'm just earnestly seeking explanations and answers for the questions I posed. I further say that I hope my record speaks for itself at this point but, if you choose to attribute my attempts and replies to bad faith, I guess I can't do much about that other than to say YOUR ARE WRONG! and hope we can move on.
Brian Ulrich wrote:As far as the A sets go, the point is that they exist because of criticisms that once existed, criticisms which I gather were before your time. If the present state were to revert to what existed before, those problems would presumably resurface.
Okay. I disagree with everything here. First of all, as I previously said, I don't accept this premise without the facts to back it up and you're unable or unwilling to provide any such facts. Therefore, your founding further arguments on it is a waste of both our time and, frankly, somewhat insulting. Secondly, such criticisms are quite far from "before my time." On the contrary, I was highly active in quizbowl at the times you assert such criticisms were taking place. Finally, even if I did accept your premise, your conclusion doesn't follow. The state of the circuit has changed in a lot of ways in the past 6-8 years, so the assertion that the same questions will meet the same reply is tenuous at best.

MaS
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

User avatar
Stained Diviner
Auron
Posts: 4820
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Stained Diviner »

Complaints that questions in IS Sets are too long or too difficult are not some relic of history--I hear them every year. I'm relatively certain that if you were to take the thousands of coaches across the country that exist, show them an IS set, and ask them to critique it, many of the responses you would get back would be the exact opposite of the critiques made on this board.

NAQT has a limited audience in Illinois. The main complaints are that the questions are too difficult and too long and that NAQT does not follow our distribution (which is another way of saying that NAQT does not have enough math). Because A Sets do not follow our distribution, most tournament hosts are not interested in using them, and I'm pretty sure that A Sets have been used fewer than five times in Illinois ever. NAQT does not have any influence over the Illinois distribution, and it probably never will.

For conferences, NAQT does create a set of questions in IHSA format that are at IS difficulty level. I believe that their market share is in the neighborhood of 10-20%, and the competitors beating them out are Avery and Questions Galore. The conference we're in decided that NAQT's questions are too difficult for Frosh/Soph, so we use NAQT for Varsity and Avery for Frosh/Soph. (Please don't use this point to derail this thread--send me an email if you want to rant about this, and I'll reply to your email. I don't run the conference.)

I'll add that, if I remember correctly, R commented in some thread several months ago that the A Sets are not labor-intensive, since many of the questions are recycled from TV shows that NAQT wants to do and makes money off of. I believe it was in response to suggestions that NAQT should spend less time on A Sets and more time on IS Sets.

I don't think anything I have said yet gets to the heart of Guy's original question and the main point Mike is making, so let me try to get this back on track--talented young people want a way to invest their efforts that will bear fruit, and for people in the quizbowl community the money generally is a secondary concern. In this case there are some specific gaps between what Guy wants NAQT to do and what NAQT does (math, pop culture, question length, quality control, and relationship with the circuit seem to be some of the primary differences), and he probably wants to hear that if he wrote, say, several hundred very good questions for NAQT that he would then get a voice in what the organization does, especially if he can justify his opinions. Furthermore, he probably wants to hear that if several of his cohorts who share many of his views do the same thing, that collectively they will get a very influential voice in what the organization does. If "get a voice" makes it sound too much like an us vs them power struggle, then perhaps it would be more accurate to state that he wants to work for a company that shares his vision and that he wants to know whether the areas of disagreement are fundamental differences in philosophy or are based on some inertial NAQT policies that it is seriously reconsidering.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)

Brian Ulrich
Wakka
Posts: 104
Joined: Mon Apr 16, 2007 1:22 pm
Location: Chambersburg, Pennsylvania
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Brian Ulrich »

For Mike - I apologize for misreading your tone. I thought I you had been active back then, which is why I assumed we'd have the same memories of the feedback NAQT got at that time. As this is not the case, I'm left wishing I knew of an easily searched archive of stuff from that time, which I think was the e-mail listserve used before the yahoogroup.

Beyond that, however, I'm not sure why you're skeptical of the assertion that NAQT began producing a series of easier question sets because of the belief that its standard sets were too hard for many teams. Also, if I remember correctly A-series predated the style of tv question from which they are these days recycled. It's true, however, that NAQT now just adapts its longer style of tv questions into standard qb format to produce the A-sets. This also turns the issue back to whether you believe it is worthwhile for a trivia question company to work with TV shows as well as the circuit, as the Speed Check questions exist only because NAQT felt like doing something extra with its short TV questions, and the A-series, while conceived independently and something that would still exist in some form if NAQT developed anti-TV impulses, has become tied to that field as well as a means of trying to optimally use its resources.
Brian Ulrich
NAQT Current Events Editor, 2005-
University of Wisconsin 1999-2003
Quincy University 1995-1999

User avatar
Captain Sinico
Auron
Posts: 2857
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: Are IS-A sets appropriate for new teams?

Post by Captain Sinico »

Okay, let's get this straight. I'm not objecting to the claim that there is a market (and, at times, a vociferous market) for shorter, easier IS-style questions. I'm not necessarily objecting to the claim that IS-A exists as a result of feedback from that market. I am reserving the right to disagree Brian's assertion that that's why IS-A exists until such time as I see some evidence for that claim (such as someone authorized by NAQT saying so on the record.) I am, however, vehemently objecting and will always vehemently object to the claim that it's a matter of public knowledge that IS-A exists due to such feedback, since that's news to me (and to many others!) and the claim is, therefore, demonstrably false!
Maybe this seemingly isn't an important distinction, but I think it is. If, for example, it's public knowledge at NAQT that that's why IS-A exists, it illustrates the asymmetry of information that NAQT is allowing by not communicating.
So, to return to a middle ground, we all seemingly agree that a market for such questions exists. I'd go even further and say that, even in my quizbowl utopia, there would still be some role for such questions. However, I posit the following reservations:
1. That it in no way follows that such questions should be produced in a 1:N:1 ratio with speedcheck and regular question. In fact, I can't see any justification for that practice and nobody has explained or even defended it yet to my knowledge. My belief is that the ratio should be something more like 1:0:5.
2. That the quality of these questions is unacceptably poor at the present time. In particular, it is widely claimed that they fail even to be reliably easier than normal IS questions. This may be symptomatic of their overproduction but, whatever its cause, it is certainly and categorically unacceptable. I think that, if these questions are to fill their claimed introductory role, their quality is actually vitally important, even if aspects of it are to some extent wasted on the target audience.
3. That these questions shouldn't be widely used for non-introductory events. That they are seems definitively a symptom of their overproduction.

MaS
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Are IS-A sets appropriate for new teams?

Post by Mechanical Beasts »

Captain Scipio wrote:Maybe this seemingly isn't an important distinction, but I think it is. If, for example, it's public knowledge at NAQT that that's why IS-A exists, it illustrates the asymmetry of information that NAQT is allowing by not communicating.
This, I think, is a tremendous part of the problem. While it's understandably uncomfortable for those who can't even say that they themselves speak for NAQT to feel like they're being attributed some sort of perfidy, this community is used to having organizations that are very, very open to the public and to learning of the evil actions of closed organizations. (The obvious example is CBI's copyright-on-tossup/bonus-formats claim; note that I do not mean to suggest that NAQT resembles the evils of CBI in any way save the closed nature of attempting to copyright, essentially, the entire damn game of quizbowl and the way that NAQT is a little more behind-closed-doors than ACF, which doesn't really own doors to close.) (I'd also like to add that NAQT is doing far better at communicating on these boards recently, especially in these recent discussions, than at any time since when I started to post two years ago.)
Andrew Watkins

User avatar
AlphaQuizBowler
Tidus
Posts: 695
Joined: Mon Dec 03, 2007 6:31 pm
Location: Alpharetta, GA

Re: Are IS-A sets appropriate for new teams?

Post by AlphaQuizBowler »

Caesar Rodney HS wrote:
evilmonkey wrote:
Caesar Rodney HS wrote: Anyway, still... point remains: if these are "introduction" packets, how can they qualify as "bona fide" tournaments with full qualification status, etc.?...
I also think that Caesar Rodney Guy makes an excellent point here - if these sets aren't supposed to be played by elite players, then they shouldn't be given the same weight for qualification, since the fields that *should* be playing them don't include the stronger teams.
Thank you for not ignoring the most important point that i tried to make that had, until now, been unmentioned. It makes no sense to me why this is the way it is. It seems totally contrary to what NAQT says its own A-Sets are even for.
This is definitely contingent on NAQT not marketing A-sets to Varsity tournaments anymore. To say that nobody from University of Alabama or Walton Varsity should qualify based on the set quality is ridiculous, if you look at how strong the fields were. In short: if NAQT only has A-sets at JV tournaments, then this is a good idea. If they continue to use A-sets at Varsity tournaments, it is a bad idea to revoke qualification.
William
Alpharetta High School '11
Harvard '15

User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Down and out in Quintana Roo »

AlphaQuizBowler wrote:
Caesar Rodney HS wrote:
evilmonkey wrote:
Caesar Rodney HS wrote: Anyway, still... point remains: if these are "introduction" packets, how can they qualify as "bona fide" tournaments with full qualification status, etc.?...
I also think that Caesar Rodney Guy makes an excellent point here - if these sets aren't supposed to be played by elite players, then they shouldn't be given the same weight for qualification, since the fields that *should* be playing them don't include the stronger teams.
Thank you for not ignoring the most important point that i tried to make that had, until now, been unmentioned. It makes no sense to me why this is the way it is. It seems totally contrary to what NAQT says its own A-Sets are even for.
This is definitely contingent on NAQT not marketing A-sets to Varsity tournaments anymore. To say that nobody from University of Alabama or Walton Varsity should qualify based on the set quality is ridiculous, if you look at how strong the fields were. In short: if NAQT only has A-sets at JV tournaments, then this is a good idea. If they continue to use A-sets at Varsity tournaments, it is a bad idea to revoke qualification.
Well there you go, they should only be used at JV competitions then. Again that's what NAQT says they're supposed to be for fairly explicitly so it would make sense for that to happen.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org

intothenegs
Wakka
Posts: 147
Joined: Sat Dec 29, 2007 5:29 pm
Location: James Monroe

Re: Are IS-A sets appropriate for new teams?

Post by intothenegs »

Caesar Rodney HS wrote:
AlphaQuizBowler wrote:
Caesar Rodney HS wrote:
evilmonkey wrote:
Caesar Rodney HS wrote: Anyway, still... point remains: if these are "introduction" packets, how can they qualify as "bona fide" tournaments with full qualification status, etc.?...
I also think that Caesar Rodney Guy makes an excellent point here - if these sets aren't supposed to be played by elite players, then they shouldn't be given the same weight for qualification, since the fields that *should* be playing them don't include the stronger teams.
Thank you for not ignoring the most important point that i tried to make that had, until now, been unmentioned. It makes no sense to me why this is the way it is. It seems totally contrary to what NAQT says its own A-Sets are even for.
This is definitely contingent on NAQT not marketing A-sets to Varsity tournaments anymore. To say that nobody from University of Alabama or Walton Varsity should qualify based on the set quality is ridiculous, if you look at how strong the fields were. In short: if NAQT only has A-sets at JV tournaments, then this is a good idea. If they continue to use A-sets at Varsity tournaments, it is a bad idea to revoke qualification.
Well there you go, they should only be used at JV competitions then. Again that's what NAQT says they're supposed to be for fairly explicitly so it would make sense for that to happen.
But many tournaments don't get to choose their set- some are arbitrarily assigned A-sets despite being varsity events. In other words, it's not so much that top teams are going and purposely playing at A-set competitions, it's that the ratio of A sets to IS sets is so high that some tournaments which top teams traditionally play don't get the IS sets they should have, and instead get assigned low-level A sets.
Chuhern Hwang
James Monroe High School Class of 2009
University of Virginia Class of 2013

User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Down and out in Quintana Roo »

Well, if that's the case... that's complete crap and pretty lame of NAQT to do that. There are more problems than i initially realized.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org

User avatar
Nine-Tenths Ideas
Auron
Posts: 1554
Joined: Sat Feb 03, 2007 10:14 pm
Location: MD

Re: Are IS-A sets appropriate for new teams?

Post by Nine-Tenths Ideas »

Caesar Rodney HS wrote:Well, if that's the case... that's complete crap and pretty lame of NAQT to do that. There are more problems than i initially realized.
I believe that is what happened with the Delaware Open- initially, an IS Set was booked but was reassigned to an A set.
Isaac Hirsch
University of Maryland '14
Never Gonna Play Again

User avatar
Whiter Hydra
Auron
Posts: 1289
Joined: Tue Dec 04, 2007 8:46 pm
Location: Fairfax, VA
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Whiter Hydra »

Johannes Climacus wrote:
Caesar Rodney HS wrote:Well, if that's the case... that's complete crap and pretty lame of NAQT to do that. There are more problems than i initially realized.
I believe that is what happened with the Delaware Open- initially, an IS Set was booked but was reassigned to an A set.
And LIFT.
Harry White
TJHSST '09, Virginia Tech '13
VP of Technology, PACE
Owner of Tournament Database Search and Quizbowl Schedule Generator
Will run stats for food

User avatar
theMoMA
Forums Staff: Administrator
Posts: 5784
Joined: Mon Oct 23, 2006 2:00 am

Re: Are IS-A sets appropriate for new teams?

Post by theMoMA »

I will resurrect this topic somewhat by saying that the main problems with A-series are not inherent in the concept of A-series. I don't even hold a hard-line stance about question lengths. If these questions are being used for appropriately new-to-quizbowl audiences, I see no problem with pyramidal-style questions that are even shorter than IS questions, since short questions seem to appeal to new players. Leadins are meant to distinguish between top teams, but if the audience is appropriately inexperienced, there aren't going to be many teams benefiting from having an additional clue. For a very novice field, a well-written A series would still be pyramidal and alleviate buzzer races. I don't have a problem with people who are recreational quizbowl players, because their existence provides a starting-point circuit that might nurture committed players in the future.

The real issue seems to be that NAQT does not adhere to the necessary novice difficulty and skirts the principles of good quizbowl to the extreme. A sets seem less like a vehicle for shorter, high-quality questions that recreational players can enjoy and newer players can learn from. They instead seem to be a playground for NAQT writers to produce three-line tossups on too-difficult subjects; rife with pop culture crossover, literature tossups on Harry Potter, and figure-it-out riddles; and containing more geography than any sane person could rationalize.
Andrew Hart
Minnesota alum

User avatar
naturalistic phallacy
Auron
Posts: 1437
Joined: Tue May 01, 2007 12:03 am
Location: Minneapolis, MN
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by naturalistic phallacy »

theMoMA wrote:The real issue seems to be that NAQT does not adhere to the necessary novice difficulty and skirts the principles of good quizbowl to the extreme. A sets seem less like a vehicle for shorter, high-quality questions that recreational players can enjoy and newer players can learn from. They instead seem to be a playground for NAQT writers to produce three-line tossups on too-difficult subjects; rife with pop culture crossover, literature tossups on Harry Potter, and figure-it-out riddles; and containing more geography than any sane person could rationalize.
Yes, this is entirely the case with A sets and is especially the case with the modified A set used for Minnesota league. Many players enjoy the shorter tossups but are utterly surprised by the difficulty of much of the answer space. Perhaps NAQT needs to reevaluate what they consider to make up a "novice set."
Bernadette Spencer
University of Minnesota, MCTC
Member, ACF
Event Manager, PACE
Order Support and Administrative Assistant, NAQT

User avatar
Stained Diviner
Auron
Posts: 4820
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Stained Diviner »

I took my Frosh/Soph Team to a tournament using an A Set a few weeks ago, and I thought it was novice level. The conversion rate was high, and that was despite the fact that many early clues which were pretty straightforward did not elicit buzzes.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Are IS-A sets appropriate for new teams?

Post by Mechanical Beasts »

Shcool wrote:I took my Frosh/Soph Team to a tournament using an A Set a few weeks ago, and I thought it was novice level. The conversion rate was high, and that was despite the fact that many early clues which were pretty straightforward did not elicit buzzes.
Well, it's not that meaningful to say that their conversion rate was "high." How high was it? Do you know that it would necessarily have been lower on an IS set?

Obviously A-sets are appropriate for new teams to some extent; Chicago Open is, too, albeit to an enormously smaller extent. The question isn't just about whether they pass some baseline level of appropriateness: it's also whether A-sets provide unique solvency, i.e. if they provide novice teams with something that IS sets (or housewrites, or HSAPQ, or whatever) cannot. While they're certainly playable by fourteen year olds (and by colleges, apparently), that doesn't necessarily imply that they're the product of choice. (Conversion probably shouldn't be the only criterion here, too.)
Andrew Watkins

User avatar
Stained Diviner
Auron
Posts: 4820
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Stained Diviner »

The stats from that tournament are here. There were a couple of oddities when the stats were first posted, but I think they are pretty accurate now other than the fact that bonus stats are missing. That tournament used powers but not negs in order to encourage the kids to buzz in, had untimed 20/20 rounds, and house-written bonuses.

That tournament had 245 powers and 1146 10's, for a conversion rate of 1391/1800, which is about 77%, which means that in an average match 15-16 tossups got answered. Obviously, I didn't do a controlled experiment and have all the same teams play each other on IS questions, but my honest sense is that the conversion rate would have been noticeably worse.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)

User avatar
setht
Auron
Posts: 1189
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: Are IS-A sets appropriate for new teams?

Post by setht »

Shcool wrote:The stats from that tournament are here. There were a couple of oddities when the stats were first posted, but I think they are pretty accurate now other than the fact that bonus stats are missing. That tournament used powers but not negs in order to encourage the kids to buzz in, had untimed 20/20 rounds, and house-written bonuses.

That tournament had 245 powers and 1146 10's, for a conversion rate of 1391/1800, which is about 77%, which means that in an average match 15-16 tossups got answered. Obviously, I didn't do a controlled experiment and have all the same teams play each other on IS questions, but my honest sense is that the conversion rate would have been noticeably worse.
Regardless of how the same set of teams might have done on an IS set, I think the tossup conversion numbers are somewhat below NAQT's target, suggesting that the set you used should have been a bit easier. I'm too lazy to go through and take each team's PPG and tossup stats to generate PPB; perhaps someone else will do that so we can see what the bonus conversion numbers look like. I wanted to ask: are all of the teams that showed up what you would consider novice or JV teams (that is, the type of team that is the ostensible target of A-set questions)? For instance, would you say that Stevenson is a strong JV team, or are they really not a JV team (by local definitions of JV/varsity, or by "JV equals 9th and 10th graders only" if there's no local definition of JV)?

-Seth
Seth Teitler
Formerly UC Berkeley and U. Chicago
President of NAQT
Emeritus member of ACF

User avatar
Stained Diviner
Auron
Posts: 4820
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Are IS-A sets appropriate for new teams?

Post by Stained Diviner »

For that tournament, the only rule was that players had to be Freshmen or Sophomores. Stevenson has a very good sophomore--you can see Zach's stats at an IS tournament here, with playoff stats here, though it's impossible to draw direct conclusions since he played that tournament with significantly stronger teammates, who were juniors and seniors, and against more varied competition. (I wouldn't say better competition unless you used the playoff stats given that half the teams at the varsity tournament were under 100 PPG.) If you look at the individual stats from the Frosh/Soph tournament, you'll see that nobody came close to him. He is probably the only one there that day who could go to NSC and help his (balanced) team win some matches, and even so his team would have close to zero chance of making the playoffs (under the past system). (There were a small number of other players there who might start for varsity teams who would have a chance of making the NSC playoffs, but their contribution to that varsity team would only be about one tossup per match.)

While it's fair to say that it would have been better if conversion statistics were a bit higher, I don't think it is fair to say that these questions overwhelm younger players to the point of frustration or that a question on The Red and the Black is typical.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Are IS-A sets appropriate for new teams?

Post by Mechanical Beasts »

Shcool wrote:While it's fair to say that it would have been better if conversion statistics were a bit higher, I don't think it is fair to say that these questions overwhelm younger players to the point of frustration or that a question on The Red and the Black is typical.
I don't think any A-set detractors claim either of the above. A-sets are obviously not the exact opposite of a good novice set (one that overwhelms younger players, one for which the mode tossup difficulty is pretty high). But a good novice set would certainly have fewer tossups on The Red and the Black.
Andrew Watkins

User avatar
the return of AHAN
Auron
Posts: 1957
Joined: Thu Feb 01, 2007 10:40 pm

Re: Are IS-A sets appropriate for new teams?

Post by the return of AHAN »

Shcool wrote:I took my Frosh/Soph Team to a tournament using an A Set a few weeks ago, and I thought it was novice level. The conversion rate was high, and that was despite the fact that many early clues which were pretty straightforward did not elicit buzzes.
The games involving my A team had about 85-90% conversion. Our first round match vs. Libertyville didn't see a dead toss-up until #18. Our lowest conversion rate was the New Trier match where only 14 of 20 (70%) were answered.
Jeff Price
Barrington High School Coach
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------

User avatar
Howard
Yuna
Posts: 967
Joined: Fri May 09, 2003 5:42 pm
Location: Ellicott City, MD

Re: Are IS-A sets appropriate for new teams?

Post by Howard »

Johannes Climacus wrote:
Caesar Rodney HS wrote:Well, if that's the case... that's complete crap and pretty lame of NAQT to do that. There are more problems than i initially realized.
I believe that is what happened with the Delaware Open- initially, an IS Set was booked but was reassigned to an A set.
Unless there's been a change, that's not the case. The Delaware Fall Open has traditionally been run on an IS-A set. If Bad Boy Bill sees this, perhaps he'll comment. Obviously, he knows better than I.
John Gilbert
Coach, Howard High School Academic Team
Ellicott City, MD

"John Gilbert is a quiz bowl god" -- leftsaidfred

Locked