Theses on the Transitioning Quizbowl Community

Old college threads.
Locked
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Theses on the Transitioning Quizbowl Community

Post by naan/steak-holding toll »

In private conversations that I've had recently, many people have been remarking at how drastically the collegiate quizbowl landscape is going to change in the next year or two, and indeed how it has dramatically changed over the past few years in general. In particular, a large number of quizbowl's best players will no longer be eligible for titles next year, leaving the field wide open. To paraphrase an ill-timed John McCain comment: the fundamentals of college quizbowl are strong. But we need to recognize weaknesses and address them before they cause issues down the line.

I present the following Theses on the Transitioning Quizbowl Community:

1) The game is becoming more undergraduate-centric and dispersed in talent, despite the (natural) disproportionate representation of grad students among the best players
2) The average quality of questions is at an unprecedented high, but quizbowl is failing to produce enough up-and-coming writers to maintain this standard
3) NAQT must become better integrated with the collegiate quizbowl community


(this is the last part of this post that will remain even remotely cogent, the rest is sort of free-wheeling rambling meant to spark some discussion)

1) is much more a general observation, but I think it's an important one to make. This may be in part due to Nationals becoming easier, but average points per bonus values at this year's ICT and ACF Nationals were extremely healthy across the board, with far fewer teams scoring below 10 points per bonus this year than in, say, 2015. I think this represents an important overall demographic shift - more players are staying on after high school and working to improve, but fewer of these players are actively trying to become "super-players" (I will not attempt to rigorously define this term, but I think most folks know what I mean).

In general, I think the main takeaway of 1) is that Nationals can and should become somewhat easier, but not by too much. Also, there are more people playing and getting good, and this is awesome.

2) As Ike Jose remarked to me, basically everybody has a good amount of "real knowledge" now to the point where it's almost asinine to praise players for having it. This is becoming more and more reflected in the questions people are writing, which are not only doing a good job of rewarding this real knowledge, but also (increasingly) doing a better job of being less repetitive and exploring newer waters. Quizbowl is thus becoming more accessible to a wider audience insofar that you do not need to master an artificially constrained upper-level canon to succeed as much as before.

Unfortunately, fewer people are stepping up to bat as writers for major tournaments, and there were serious concerns about collegiate tournament "slots" in the past year not being filled because teams that have consistently produced tournaments previously did not do so this year (Minnesota on MUT due to heavy commitments, no sets from Chicago/Michigan aside from Auroni's work, etc.) In addition, those who have stepped up often lack the needed guidance to correct easy-to-make writing mistakes and produce sets of adequate quality, as exemplified by this year's ACF Fall, which fell far below the standards of the year's other tournaments despite being one of the year's marquee events.

Some possible takeaways:

- More tournaments should actively seek to mentor new writers and editors, as opposed to just bringing them in and throwing them at a task. This requires that "higher tier" editors be actively committed to both giving feedback and delegating effectively, as well as performing reviews in a timely manner. As always, an effective division of labor makes society as a whole more productive - eager new people gain experience in producing good questions and guidance on how to do so, and the experience of other people is brought to bear without having those people do a lot of grunt work on ten different questions.
- Optional packet-submission tournaments, with relaxed submission requirements (ala WAO) should be more of a thing. These allow people to write on areas that they prefer, without being slapped with having to write a subject they may possess no knowledge of whatsoever (particularly more difficult ones like chemistry or philosophy).

3) I think the urgency of point 3) has been demonstrated by the recent thread about packet distribution. However, working for NAQT has helped give me a limited amount of insight as to the major role NAQT plays in developing quality writers, many of whom are relatively "untapped" by the community producing mACF sets. The fact that NAQT's substantial infrastructure is relatively unused by the collegiate scene except for the production of three sets per year (one of which is only played by novices) reflects a serious missed opportunity. I would argue that this shortage occurs due to two main reasons

- Given NAQT's cost of production and the smaller "pool" of possible revenue from collegiate tournaments, it is not as economical for NAQT to produce collegiate sets
- The representatives of the collegiate community have shown disrespect or even disdain for NAQT's approach to quizbowl in multiple ways, ranging from packet trading to arguing that the format is inferior for asinine reasons (I will cop to falling into falling under these groupings several instances) and aggravating the first issue. This is not just limited to the "elder statesmen" of the collegiate community: some people make arguments implying that geography/current events are a subjects that can't be learned, arguments which hundreds of talented high school students demonstrate to be false every year!

Basically, as a business, NAQT needs reasons to be in the collegiate game and if those reasons aren't apparent, then they may not always be around. I don't think people want NAQT gone, and indeed having more NAQT sets at the college level would probably be a boon.

I don't mean this to be an absolution of NAQT of all sins - I do think there are a some ways in which their distribution is flawed, and the repeated insistence on running tournaments with a clock despite player opposition to the clock and serious logistical issues associated with it. But ever since a larger number of "looped-in" collegiate regulars have started producing a greater portion of NAQT questions (i.e. the people writing for NAQT aren't ignorant of the topics and standards of the rest of quizbowl), NAQT tournaments have consistently been some of the best each year. Just listen to recordings of the 2010 ICT, or even the 2012 ICT (there's a miscellaneous tossup on freaking humpback whales) and compare with the ICTs from the past three years! In addition, NAQT is able to consistently make small inroads into regions like the Rocky Mountains, which is something that a tournament like ACF Regionals really can't boast much about.

Also, a lot of people just prefer NAQT sets! I think there's a not-publicly-expressed tendency to sneer at teams that just show up for SCT, D2 ICT, and never really go above that, but I think trying to find ways to accommodate these teams and bring them into playing more tournaments is better than just giving up on them entirely.

Some takeaways:

- Regardless of whether NAQT changes its policies about collegiate tournaments, people should stop trading NAQT packets. Doing so helps bridge goodwill in a way that can only serve to improve. I will take this as an opportunity to publicly apologize for causing issues by passing the ICT set to the ACF Nationals editors; my intent was to make sure that teams who played D1 ICT did not have an unfair leg up over those that did not by hearing an early clues that "scooped" ACF questions.
- The clock really, really needs to be abolished. Other logistical constraints (such as, perhaps, paper packet requirements) should be minimized if possible, though this could be tricky.
- I think mirror fees for collegiate tournaments can/should be increased slightly for NAQT tournaments, as they have been for ACF tournaments, to reflect the greater resources required to produce an individual college question but also the relatively smaller market for college questions
- Expansion of the number of NAQT collegiate tournaments, if it can be accomplished in an economically profitable manner, would help pave the way for more people to make more money from quizbowl writing (justifying a greater time commitment) and help bring regular collegiate circuits to more regions of the country, and plug home of the "holes" in the collegiate circuit that came up this year. In particular, a fall NAQT set would be awesome to have and help solve the issue of "who is gonna write that second fall regular tournament" that comes up every year.
- Doing all of the above, forging tighter bonds, and building more trust in general, will help give momentum to the sorts of changes that "the community" (if I may) wants to see out of NAQT
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Theses on the Transitioning Quizbowl Community

Post by Cheynem »

Will raises some interesting points. I'd offer two quick thoughts in response:

-who is going to write this new NAQT collegiate set? I'm all for it, but in your earlier points you note that not enough people are writing and that it seems like the "current teams in the game" have become somewhat stagnant in production.

-that brings me to the next point, the more interesting one, of how you can produce more writers. I'll say three possible ideas:

1. Use more packet submission tournaments. I don't subscribe to the Weinerian idea that nearly every tournament should be packet submission nor that this inherently helps create writers, but it does (sort of) lower the burden on writers/editors working on sets and it does create opportunities for those interested in writing to improve their craft in low stakes scenarios.

2. Better mentoring. This is hard, especially when time is of a premium, but experienced editors working with quasi-experienced writers or editors I think can produce good results. I learned so much as a writer from the folks at Minnesota and even from last year's CO, working with Matt Jackson.

3. Better tournament discussions. I think we've evolved from this, but nobody wants to produce a good faith effort and have it received with 10 page bullet points about all a certain category's questions were flawed.

My last idea is just simply calling for better schedule regulation for the college game. We don't want a NCAA of quizbowl juggernaut bureaucracy, but I've long believed a regulatory body that can approve tournaments and balance the schedule might do some good for the issues WIll discusses.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: Theses on the Transitioning Quizbowl Community

Post by naan/steak-holding toll »

Cheynem wrote:Who is going to write this new NAQT collegiate set? I'm all for it, but in your earlier points you note that not enough people are writing and that it seems like the "current teams in the game" have become somewhat stagnant in production.
Let's call this set "NAQT Collegiate Fall" for discussion purposes. If current resources aren't sufficient (they probably aren't), people could be invited on as guest writers/editors for NAQT fall in a similar way to how NAQT already does for SCT almost every year. Leveraging the existing infrastructure, editing experience, and mentoring ability of NAQT is likely to produce a better tournament than a few people striking out on their own (see Missouri Open for instance).

In addition, the number of writers in NAQT has only gone up over time, and includes more and more graduated, experienced collegiate players these days than ever before. Certainly in the short term, it's going to probably get easier for NAQT to get more questions - what's not changing is the cost per question, since I doubt NAQT is gonna cut writer compensation.

In terms of the marketability of this set, I think something around D2 ICT in difficulty level could very easily attract large mirrors among high schools as well. Already high schools are clamoring to play nationals-prep tournaments throughout the year, and high school players regularly use ACF Regionals as study material. I think that, if produced and then mirrored among the wide audience (especially given the NAQT brand) "Collegiate Fall" could easily bring in revenues in the tens of thousands. While perhaps not catering to the cream of the crop of the collegiate game, this set would nonetheless be widely played and enjoyed.
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Aaron's Rod
Sec. of Cursed Images, Chicago SJW Cabal
Posts: 851
Joined: Wed Nov 27, 2013 7:29 pm

Re: Theses on the Transitioning Quizbowl Community

Post by Aaron's Rod »

Cheynem wrote:2. Better mentoring. This is hard, especially when time is of a premium, but experienced editors working with quasi-experienced writers or editors I think can produce good results. I learned so much as a writer from the folks at Minnesota and even from last year's CO, working with Matt Jackson.
One of the problems that I see here is that the path from being a "completely inexperienced" to a "quasi-experienced" writer seems to be really hazy. Sure, you can write questions on your own, but if nobody's telling you whether or not they're good you're not getting better, and I haven't really seen any initiatives to get completely new writers going. Even PADAWAN's requirements stipulated that "You don't have to have previous experience writing collegiate questions, but you should have some experience writing questions at some level." (I myself took a gander at Seth's post at the time but save for Fall didn't meet that requirement, and I didn't end up writing for a full set until almost three years [!] later.) Packet-submission tournaments like ACF Fall are great, but there doesn't seem to be either encouragement for those who submitted good packets to consider writing more, or even the most general feedback for people who submitted bad packets. I know that the latter is really time-consuming, but even a comment or two like like "your literature was too difficult" or "sentences should be written in coherent English" would probably help teams out a lot.

To Will's point on how NAQT does things, I think having an open writer's application as NAQT does is seriously undervalued. It's not wrong to e.g. tap well-known people that you trust to head-edit major tournaments, but I think you're more likely to find the "untapped talent" that you hope is out there with a "y'all come" approach.
Alex D.
ACF
http://tinyurl.com/qbmisconduct

"You operate at a shorter wavelength and higher frequency than most human beings." —Victor Prieto
User avatar
Ciorwrong
Tidus
Posts: 696
Joined: Fri Dec 20, 2013 8:24 pm

Re: Theses on the Transitioning Quizbowl Community

Post by Ciorwrong »

This is a really good post and it's a shame that more people aren't discussing it because Will brings up a lot of interesting points. I'll focus on his proposals:
Periplus of the Erythraean Sea wrote:Some takeaways:

- Regardless of whether NAQT changes its policies about collegiate tournaments, people should stop trading NAQT packets. Doing so helps bridge goodwill in a way that can only serve to improve. I will take this as an opportunity to publicly apologize for causing issues by passing the ICT set to the ACF Nationals editors; my intent was to make sure that teams who played D1 ICT did not have an unfair leg up over those that did not by hearing an early clues that "scooped" ACF questions.
- The clock really, really needs to be abolished. Other logistical constraints (such as, perhaps, paper packet requirements) should be minimized if possible, though this could be tricky.
- I think mirror fees for collegiate tournaments can/should be increased slightly for NAQT tournaments, as they have been for ACF tournaments, to reflect the greater resources required to produce an individual college question but also the relatively smaller market for college questions
- Expansion of the number of NAQT collegiate tournaments, if it can be accomplished in an economically profitable manner, would help pave the way for more people to make more money from quizbowl writing (justifying a greater time commitment) and help bring regular collegiate circuits to more regions of the country, and plug home of the "holes" in the collegiate circuit that came up this year. In particular, a fall NAQT set would be awesome to have and help solve the issue of "who is gonna write that second fall regular tournament" that comes up every year.
- Doing all of the above, forging tighter bonds, and building more trust in general, will help give momentum to the sorts of changes that "the community" (if I may) wants to see out of NAQT
As has been brought up many times, at tournaments with quality staffers, the clock is only an obstacle that prevents teams from hearing all 24 tossups in a packet. It also creates more variability between different readers when you have slower or less competent readers who only get through 14 tossups (this apparently happened to our D2 team at SCT this year) versus the best readers who consistently get through at least 20 tossups. If you have a scorekeeper, you should get to 20 tossups read easily, but why can't NAQT just make packets with like 20 tossups, 20 bonuses and maybe 3 tie breaker tossups and an extra bonus? Wouldn't this lower NAQT's question writing costs? The clock, as mentioned previously, adds a dumb strategy element to quiz bowl when the game should just be about answering questions correctly.

While I won't guess what NAQT's profit margins on SCT and ICT are, does NAQT enough money on those sets to suggest that another set that, if written at say at MUT or EFT level, would generate enough money to turn a profit for NAQT given that NAQT pays the most competitive rates for writers and editors? If it's sustainable financially and the audience is there, I'd say go for it because my team would definitely be intersted. I always feel there is a dearth of tournaments at the regular- and regular+ levels which I feel are noticeably discrete from regular and nationals in difficulty. It's hard to get college freshmen and sophomores to be excited to play a regular difficulty event if their quizbowl experience is pretty limited so I see huge potential in this market. I personally don't really care about the NAQT novice event, but if you played in high school, I'd say ACF Fall is a fine collegiate novice event personally.

I'm not really sure how the mentoring idea is supposed to play out, but one thing I'll say is that people are way overly picky on quizbowl questions. I'm definitely guilty of this, but it's super easy to forget that most people who write questions do it as either a hobby or a side project and their primary focus is on school or work. If people were a little more gentle and constructive in their criticism instead of going pages and pages why a certain econ tossup sucks and isn't "real" or whatever :roll: , maybe more new writers would be willingly to write.

I can only speak to writing for a high school set, and many of the writers from POMMSS never wrote questions before and their questions were fine after editing. It may require a little bit more work to edit a new writer's tossup, but as long as they are willing to improve and practice, anyone can be a good question writer. Again, this is only based on my experience writing and editing a high school set but I think more people should take the plunge and just say "me and 4 other teammates are going to write this tournament". I'd advise getting at least one experienced editor on board, but you can make a playable set very easily. The most important parts of a set are fresh, interesting clues, clean, cogent writing (which is rarely talked about compared to discussion of subject specific jargon and minutiae) and strong difficulty control. I'm not saying that any 5 undergrads can crank out a nationals difficulty set, but I think they can make a collegiate novice or high school set pretty easily if you are familiar with the cannon. After working on these first writing projects, they can work at higher difficulty levels. This seems to be how the system is already in place but few collegiate teams make housewrites. Honestly, High schoolers make housewritten sets (that often aren't very good to be honest), so collegiate players can definitely do so even if the high school market is saturated with mediocre housewrites. People should just start writing because your beginning tossups will probably suck, but you read them in practice or on the IRC, get feedback and then improve them.
Last edited by Ciorwrong on Mon May 15, 2017 10:57 pm, edited 1 time in total.
Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19
UC San Diego Economics 2019 -

at least semi-retired
Ewan MacAulay
Lulu
Posts: 68
Joined: Sun Nov 04, 2012 10:15 am

Re: Theses on the Transitioning Quizbowl Community

Post by Ewan MacAulay »

Periplus of the Erythraean Sea wrote: - Optional packet-submission tournaments, with relaxed submission requirements (ala WAO) should be more of a thing. These allow people to write on areas that they prefer, without being slapped with having to write a subject they may possess no knowledge of whatsoever (particularly more difficult ones like chemistry or philosophy).
This was one of the best things about WAO this year and I agree that flexible submission requirements are a good thing for that kind of tournament, especially if it's only one or two people working on a submission. It worked out much better for me (and probably for the editors) that I got to submit 5/5 science instead of the :capybara: my music/philosophy questions always end up being.

The other thing I might suggest is for more experienced writers to post "guides" on general principles of question-writing/potential pitfalls/useful sources. Every tournament I work on I go back to the notes on question-writing that Seth and Auroni sent out to PADAWAN writers. Similarly, Austin's post on how he wrote the physics for Nationals had a lot of useful stuff to think about in it. I'd be happy to point people to some good chemistry resources if there's interest.
Ewan MacAulay
Oxford 2015
Cambridge 2018
User avatar
Aaron's Rod
Sec. of Cursed Images, Chicago SJW Cabal
Posts: 851
Joined: Wed Nov 27, 2013 7:29 pm

Re: Theses on the Transitioning Quizbowl Community

Post by Aaron's Rod »

Ewan MacAulay wrote:The other thing I might suggest is for more experienced writers to post "guides" on general principles of question-writing/potential pitfalls/useful sources. Every tournament I work on I go back to the notes on question-writing that Seth and Auroni sent out to PADAWAN writers. Similarly, Austin's post on how he wrote the physics for Nationals had a lot of useful stuff to think about in it. I'd be happy to point people to some good chemistry resources if there's interest.
This has been done! Many times. The resources are definitely out there--but of course, none are substitutes for real experience and feedback on questions you've actually written.
Alex D.
ACF
http://tinyurl.com/qbmisconduct

"You operate at a shorter wavelength and higher frequency than most human beings." —Victor Prieto
User avatar
SpanishSpy
Wakka
Posts: 138
Joined: Fri Sep 30, 2016 1:19 am
Location: Arlington, Virginia

Re: Theses on the Transitioning Quizbowl Community

Post by SpanishSpy »

Aaron's Rod wrote:
Ewan MacAulay wrote:The other thing I might suggest is for more experienced writers to post "guides" on general principles of question-writing/potential pitfalls/useful sources. Every tournament I work on I go back to the notes on question-writing that Seth and Auroni sent out to PADAWAN writers. Similarly, Austin's post on how he wrote the physics for Nationals had a lot of useful stuff to think about in it. I'd be happy to point people to some good chemistry resources if there's interest.
This has been done! Many times. The resources are definitely out there--but of course, none are substitutes for real experience and feedback on questions you've actually written.
Just as a clarification, what does Jerry Vinokurov mean by "Marzipan Bonus" in the first article you link?
Alex Wallace
Washington-Lee High School Class of 2015
College of William & Mary Class of 2019
User avatar
Cheynem
Sin
Posts: 7220
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Theses on the Transitioning Quizbowl Community

Post by Cheynem »

Not sure why he picked those two things, but he was referencing bonuses that went like "Is the primary language of these countries English, Spanish, both, or neither?"
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
SpanishSpy
Wakka
Posts: 138
Joined: Fri Sep 30, 2016 1:19 am
Location: Arlington, Virginia

Re: Theses on the Transitioning Quizbowl Community

Post by SpanishSpy »

Cheynem wrote:Not sure why he picked those two things, but he was referencing bonuses that went like "Is the primary language of these countries English, Spanish, both, or neither?"
I see. Thank you for the explanation
Alex Wallace
Washington-Lee High School Class of 2015
College of William & Mary Class of 2019
Locked