The Case for Short Questions

[Victor Prieto]

This post is addressed to the writers and editors of EFT, Penn Bowl, ACF Regionals, Terrapin, and anybody else writing tournaments at regular difficulty or easier for the upcoming year.

Length of a question can have a significant impact on perceived difficulty, both of specific tossups and whole tournaments. In my experience, this phenomenon is particularly enhanced for high schoolers making the jump to collegiate quizbowl (or newcomers altogether). Not only do freshmen with quizbowl experience have to make a big jump in difficulty, but they also have to adjust to a big jump in question length. Most high school players play NAQT the vast majority of the season, which means they spend most of their time playing 291 character tossups or 425 character tossups. As I explain below, most regular difficulty college tournaments are almost twice (!!!) the length of the longest high school regular difficulty tossup. I firmly believe college difficulty should stay the same as it is now. However, I believe that we (meaning, editors of collegiate tournaments) can and should mitigate the second change. By necessity, college tossups need to be longer than high school ones, because the difficulty jump makes it more difficult to avoid speed-check like questions in some categories. However, there is no reason for the current large jump in tossup length that we have right now.

Matt Bollinger alluded to this in the other thread, but at hard college tournaments, the difference between 7-8 line tossups and 9-10 line tossups is pretty substantial when considering perceived difficulty. The extra padding probably doesn’t impact gameplay all that much. At lower levels, meaning, regular difficulty, I’m willing to bet that the perceived difficulty of an 8-line tossup vs. a 6-line tossup is even more drastic.

As I also mentioned in the other thread, editors are creating logistical obstacles for tournaments when they produce sets which are 8 lines in length compared to tournaments which are 6 lines in length, which is why I was very disappointed to see that ACF Regionals chose to raise the line cap from 7 to 8 for next year. Penn Bowl editors could also learn from this and start constraining their tossup lengths, probably saving a ton of time and avoiding last year’s issue. EFT doesn’t have any sort of line cap in its announcement, but it is being stylized as an introduction to new college players, so I hope its editors take a line cap seriously.

There has been discussion about the merits of shorter questions in the past. Andrew Hart, in 2009. Seth, in 2012. Magin, in 2012 (thanks to Brendan Byrne for the tip!). There's some good material in there probably relevant to the discussion.

There was another thread I found some time ago which discussed the importance of length limits, but I can’t find it anymore. I definitely remember that the impact of question length on logistics was discussed, and I think it was made before I started reading the forums in earnest in late 2012, but I don’t remember anything else beyond that. If someone remembers what I’m talking about and would care to post it, I’d appreciate it.

______________________________________________

I read over the first packet of several regular difficulty tournaments from the past year and calculated the average tossup length (units = lines of 10 point Times New Roman) for each packet 1. In addition, I estimated the average tossup length for NAQT questions. Regular high school difficulty questions are maximum 425 characters, while DI SCT questions are limited to 500 characters (perhaps not coincidentally, about 100 characters of 10 point TNR fits on a single line). Finally, I calculated the average tossup length for a couple commonly mirrored high school tournaments this year.

IS set: ~4.25
DI SCT: ~5.00
WHAQ: 5.48
HFT: 6.00
EMT: 7.45
EFT: 7.55
Penn Bowl: 7.80
Terrapin: 7.85

By 7.45, this means that EMT had slightly more 7 line tossups than 8 line tossups. All of the other tournaments here had more 8 liners than 7 liners.

The average number of characters for packet 1 of EMT was 767 (~7 full lines of 10 pt TNR), ranging from 700 to 872 characters. This was the shortest of the four regular tournaments I selected. Yet, it is still a full 153% longer than the longest D1 SCT tossup, and a whopping 180% longer than what is considered regular difficulty in high school. That’s nearly twice as long, and all of the other regular college tournaments are longer!

I honestly advocated for my club to spend a couple hundred dollars on NAQT sets for practices, but not for preparing for NAQT tournaments. It’s because it’s so damn hard to find short, high quality questions that are harder than ACF Fall in the packet archive. Our practice attendance suffers greatly when we’re reading long questions, probably because people don’t want to sit through 8-line physics questions when they’re just there for the history. The faster tossup turnover is a valuable asset for retaining new people in practice. As someone mentioned in the IRC a few weeks ago, NAQT questions create a false sense of "being easier" because they’re shorter and the distribution is quirkier, hard questions go away faster and the difficulty is more fungible.

I think I’ve made my argument on how editors setting longer tossup limits is currently harming the regular college circuit (I don’t think this argument scales up perfectly to higher difficulty). If you disagree with any points I’ve made, I’d like to hear them. If you’re editing a regular difficulty tournament in the future and are intending on setting an 8-line cap, then I’d really like to hear your thoughts.

p.s. I actually prefer longer questions myself. On the other hand, I think the case for short questions is compelling, so I made the argument.

[Victor Prieto]

The second part of this post will be a short discussion to cutting down on question length in mACF tournaments to better lengths. This may come as a surprise to people who have not edited major tournaments before: it is probably much harder to write shorter questions than longer questions, especially at the collegiate level. I don’t think it’s reasonable to ask every regular difficulty tournament to cut down to SCT-length of 500 characters, but I think an upper bound of 650 characters would make a substantial difference in perceived difficulty of tournaments throughout the year.

HELPFUL HINT: Use QEMS2 to write your sets and set character limits very easily! It’s a really great public resource that not enough people take advantage of right now.

______________________________________________

There’s two tactics that you can take to save space:

1. Make clues more compact, by getting rid of excess words.

An example of what I mean:

This leader sent disabled or homeless children to children’s homes in places like Cighic and Siret. As a result of this leader’s Decree 770, women who gave birth to ten or more children were declared “heroine mothers.” In this leader’s final speech, a crowd taunted him by chanting “Timisoara,” in reference to a recent uprising. This man urged for an intensification of socialist ideology in a 1971 speech that was named the July Theses. This man succeeded Gheorghe Gheorghiu-Dej as the head of his country’s Communist Party, and he employed the Securitate secret police. This man and his wife Elena were executed on live TV on Christmas Day 1989 after attempting to flee the country. For 10 points, name this dictator who led Romania from 1967 to 1989.

to this:

This leader settled homeless children in places like Cighic and Siret. This leader’s Decree 770 declared mothers of ten or more children “heroine mothers.” In his final speech, a crowd taunted him by chanting “Timisoara." This man urged an intensification of socialism in a 1971 speech named the July Theses. This man succeeded Gheorghe Gheorghiu-Dej as the head of his country’s Communist Party, which employed the Securitate secret police. He and his wife Elena were executed on live TV on Christmas Day 1989 after attempting to flee the country. For 10 points, name this last Cold War dictator of Romania.

There were no substantial clues removed from this tossup, and it went from 754 characters to 608 characters. I probably took a heavy handed approach, and there might be a wording change that rendered a clue slightly inaccurate, but hopefully you get the idea.

This will only carry you so far, and takes a long time to do. NAQT writers and editors are extremely skilled at accomplishing this - if you’d like to add some tips for people, that would be awesome.

2. Get rid of a lead-in clue or give-away clue.

This one requires a lot of careful thought, but I’m glad that this possibility has been raised in two threads that are going on right now. There are tossups out there with lead-ins that 1) are cool anecdotes but it’s really unlikely that someone will know it (i.e. KILLER WICKED CLUE THAT MUST BE USED EVEN IF SUCH CLUE TAKES UP TWO LINES!!!!) or 2) will do a good job at differentiating Maryland from Stanford from Yale, but is just pointless at regular difficulty. There are also tossups whose last three lines resemble the last three lines of a novice tossup. In this case, the editor did a good job of arranging the very easiest clues about the topic in order, but I posit that this concept is a bad idea at regular difficulty. Those tossups would functionally be interchangeable with tossups which compress those three lines to two or one and a half lines. NAQT questions do this successfully on a regular basis, if you would like to look for examples. A frequent criticism levied against short tossups is that they produce difficulty cliffs, the idea being that there is less room for middle clues. I believe the correct general idea is for regular difficulty tossups to maximize the number of lines devoted to middle clues, by pushing in both directions (cutting into the space of both lead-ins and giveaways).

Example:

An uncommon stereochemical result in reactions following this mechanism is explained by Saul Winstein’s concept of intimate ion pairs. For ambident nucleophiles, this is the reaction mechanism followed by the more [emphasize] electronegative atom, according to Kornblum’s rule. This is the mechanism followed by reactions involving benzyl or allyl triflates. Experimentally, reactions with this mechanism proceed faster at (*) tertiary carbons than at secondary carbons. This mechanism typical works best in polar, protic solvents, such as ethanol or water, since those solutions speed up solvolysis in this mechanism’s rate-determining step. Reactions with this mechanism are best run with weak nucleo·philes, and they produce a mixture of retention and inversion products. For 10 points, name this nucleophilic substitution mechanism in which a carbocation intermediate is produced.

to this:

For ambident nucleophiles, this is the reaction mechanism followed by the more [emphasize] electronegative atom, according to Kornblum’s rule. This is the mechanism followed by reactions involving benzyl or allyl triflates. Experimentally, reactions with this mechanism proceed faster at (*) tertiary carbons than at secondary carbons. This mechanism typically works best in polar, protic solvents that speed up solvolysis in this mechanism’s rate-determining step. Reactions with this mechanism are best run with weak nucleo·philes, and an initial deprotonation step forms a positively charged intermediate. For 10 points, name this nucleophilic substitution mechanism that produces a carbocation intermediate.

I removed the first clue and only edited clues after the power mark, to cut the character count from 884 to 711. I honestly think that this tossup doesn’t differentiate teams any differently at regular difficulty, but makes a huge difference for perceived difficulty (it is still longer than my proposed 650 characters, but that was only an example benchmark).

As I said before, I hope people won't dispute the need for cutting down question length in regular difficulty tournaments. The hard part is how to do so effectively. I hope this helps.

Note: I used EMT questions as my main examples throughout this post. I chose EMT because it had the shortest tossups of the mACF tournaments last year, and the questions were of very high quality.

[Auroni]

This is a very good thread in which I would have posted even had I not been called out by the opening post. I am fully on board with promoting concise writing in quizbowl. I also agree with curbing our impulse to pile on extraneous clues that are cool, but hard. There are two main reasons why I personally tend to produce longer questions (and why I raised the line cap as head editor of ACF Regionals next year).

1) I firmly believe in the value of "contextual phrases," which may not carry clue content by themselves, but which enhance the experience of listening to a question and make it obvious why a particular clue or answer was chosen. For a readily available, but perhaps not perfect illustration of what I mean, the phrase "in reference to a recent uprising," which you deleted from your edited Ceausescu tossup, helps the listener or reader (who is often exactly the kind of new player that you reference in your post), get a decent sense of exactly why the crowds wanted Ceausescu's head, so that they don't take away the impression that they were just chanting a random word. Since this is an educational game, these contextual phrases are valuable so that our questions aren't just isolated statements of fact that have nothing to do with one another.

2) Some categories, such as science and music, naturally require more words to communicate a morsel of information such that it is unique and buzzable. I'm sure that you've been frustrated by incompletely fleshed-out chemistry questions before. If there's the slightest chance that a clue can be misinterpreted or vague, it is always worth shelling out for a bit more space to make it airtight.

I also take the point about the oppressiveness of question length for newer players to heart, having once felt oppressed in such a manner nearly a decade ago, but I am unconvinced that this has to do with the raw character count of the question. I believe that improving the prose of quizbowl questions as a whole through concise writing, avoiding linking unrelated ideas with "and" and "while," and improving word choice will naturally reduce this intimidation factor.

[gyre and gimble]

To build on Auroni's first point, I sometimes like to be "wordy" in order to give players time to process a clue. If a tossup moves from clue A to clue B, I think a player who recognizes clue A immediately should get the tossup over someone who can't recall it right away but knows it. But that second player should be able to get the tossup before someone who only knows B, and not A, but can buzz on B immediately. If you move too quickly between buzzable words, gameplay can actually become a little antipyramidal.

The context that Auroni discusses is very valuable in helping get that second player to the answer.

[heterodyne]

I definitely agree with what both Auroni and Stephen said. In addition to the important gameplay effects of that linguistic "padding", I think there's a less important but still real aesthetic effect. I think it's in fact more tiring to listen to a set of six line questions that are a series of short declarative sentences than a set of eight line questions that have a certain natural-natural-language flow to them.

I'm on mobile, so I can't quote efficiently, but I don't think cutting leadins because they "just distinguish Maryland from Yale from Stanford" is a good idea. Sure, most of your clues shouldn't serve that purpose, but if this is a regular difficulty set your clues should serve that purpose just as they distinguish between Minnesota and Chicago B later in the question. If you have multiple leadin clues then you have a problem, sure, but one clue you think is only buzzable by a few teams isn't the end of the world.

As a larger point, I think that there should be a limit to how much we as a community change our practices to accommodate those who don't like quizbowl all that much - as evidenced by the fact that these hypothetical people are put off by fairly standard college tossup lengths. I find long tossups more fun, as you said you do too. We're certainly not alone in this. Sitting through things you don't know and are not going to know is a pretty important skill for most quizbowlers, and if you can't develop it early then perhaps this is not a game for you.

[jonpin]

Besides any discussion of actual or perceived difficulty, cutting the lengths of questions means games are shorter means tournaments run quicker. By throwing away lead-ins that only serve to distinguish the very best teams in all the land, and which 90% of games will sit through, clueless, you can save a few minutes each game. Alston may have a reasonable point about "we don't need to cater to people who don't like quiz bowl", but at the same time, we as a community shouldn't drive off people who are ambivalent by making their experience worse. Games between bottom-bracket teams, where the entire tossup is getting read, may not be what's foremost on anyone's mind leading up to or during a tournament, but the existence of those lesser-talented teams is vital to the survival of quiz bowl as a game.

[Corry]

I think it's in fact more tiring to listen to a set of six line questions that are a series of short declarative sentences than a set of eight line questions that have a certain natural-natural-language flow to them.

I find long tossups more fun, as you said you do too. We're certainly not alone in this. Sitting through things you don't know and are not going to know is a pretty important skill for most quizbowlers, and if you can't develop it early then perhaps this is not a game for you.

I personally find long tossups to be unbearable. While there are certain benefits to long tossups (i.e. easier to parse clues due to added context, as Auroni noted), for me, those benefits are far outweighed by a deathmarch slog through nine lines of obscurata. And I'd personally consider myself to be, like, a pretty good quiz bowl player.

Something worth thinking about is the psychological benefit of shorter tossups. As Victor alludes to, short questions create the "illusion" that ICT is easier than ACF Nationals, because you feel less piledrived by impossible unconverted tossups if they're short. This is a good thing! Short questions are the ultimate compromise between keeping quiz bowl difficult (since the community probably isn't willing to lower the difficult to college nats) and keeping quiz bowl more accessible and entertaining. This was the first year I attended both ICT and ACF Nats, and I consciously noticed that I had a lot more fun at ICT simply because the rounds felt faster and fresher, as opposed to me being condemned to sit through like fifteen 8-line tossups a round in categories that I know nothing about.

[Benin Rebirth Party]

I think this shortening should especially happen to regs+/nats- stuff as a friendly way for both new players and new staffers to be introduced to hard stuff, and also to get a good number of games in without spending too much extra time (which is inevitable because of worse players and readers).

[Mnemosyne]

I wonder if there's a specialist vs. generalist element to the debate as well. I don't mind long questions on the categories I play, but it's pretty miserable to sit through 12+ really long questions that I have no intention of buzzing on. If you can buzz in almost every subject, then you might not notice the burden of the question length as much as someone who only buzzes in a few subjects.

I know it gets mentioned a lot, but the long questions really hit weak circuits hard when it comes to tournament length. Our This Tournament is a Crime mirror ended prelims after 6:00, and we ended up doing half-packet tiebreakers before finals because the tournament staff didn't want to stay all night. ACF Regionals last year ended at 8:00. Neither tournament had major delays or bad staff - just teams that weren't good enough to buzz early and had to sit through every single question every single round. Maybe that's an argument for bad circuits not playing hard tournaments, but I suspect people find that idea counterproductive.

I think theoretically longer questions are the right way to go, since listing out clues without enough context is fodder for the "quiz bowl = trivia" perception, but in practice questions should be shorter for any tournament that considers high participation across the skill spectrum a goal.

[The King's Flight to the Scots]

I definitely agree with what both Auroni and Stephen said. In addition to the important gameplay effects of that linguistic "padding", I think there's a less important but still real aesthetic effect. I think it's in fact more tiring to listen to a set of six line questions that are a series of short declarative sentences than a set of eight line questions that have a certain natural-natural-language flow to them.

I'm on mobile, so I can't quote efficiently, but I don't think cutting leadins because they "just distinguish Maryland from Yale from Stanford" is a good idea. Sure, most of your clues shouldn't serve that purpose, but if this is a regular difficulty set your clues should serve that purpose just as they distinguish between Minnesota and Chicago B later in the question. If you have multiple leadin clues then you have a problem, sure, but one clue you think is only buzzable by a few teams isn't the end of the world.

As a larger point, I think that there should be a limit to how much we as a community change our practices to accommodate those who don't like quizbowl all that much - as evidenced by the fact that these hypothetical people are put off by fairly standard college tossup lengths. I find long tossups more fun, as you said you do too. We're certainly not alone in this. Sitting through things you don't know and are not going to know is a pretty important skill for most quizbowlers, and if you can't develop it early then perhaps this is not a game for you.

"We can't change line lengths in quizbowl questions because quizbowl questions have long line lengths!" seems like a poor argument to me. Like, you seem to be arguing that quizbowl insiders shouldn't change the game to be casually accessible - but yeah, that's exactly what we should be doing at regular events to extend our reach. Since I think quizbowl is a valuable pursuit, if we can make it more appealing by cutting some of these clues without compromising its competitive integrity, then we should do that, to encourage more of these very non-hypothetical people to stay involved.

Auroni, I'd slightly challenge the idea that additional context always justifies extra text - I've seen cases where the extra context provided by an additional clause is thin, and rewriting the sentence more concisely would achieve the same effect. I do agree with you that better prose would also improve players' experience at many regular difficulty events.

[Ike]

Not sure why this has to be or should be exclusive to regular difficulty events. CO this year will feature some kind of hard/soft cap on 8 line tossups, and I'm striving for bonus text length to not run more than 7 lines. I think unnecessarily prolix bonuses are worse than tossups that take an extra line or two. Having a bonus introduction that is 2 lines long, and then having prompts that are 2.5 to 3 lines each in length is an extra 2.5-4 lines of question text (half a tossup!)

[Victor Prieto]

had I not been called out by the opening post.

Sorry if I was a little blunt there.

I agree that contextual clues are effective in that they can help frame other clues within a larger perspective, but I don't buy the argument that we include them because we're trying to educate our listeners. The point of a quizbowl question is to reward different levels of knowledge. I'm claiming that the current form of collegiate quizbowl doesn't accomplish that as concisely as possible. We agree on that point, that conciseness isn't properly emphasized, but we just diverge a little bit on how to properly achieve that conciseness. I'm okay with that. For example, in the Ceausescu tossup, the contextual clue regarding uprisings is probably fine, I was actually hesitant to delete that bit my first time around (and is why I called my editing heavy handed).

I understand the scenario that Stephen is describing. For example, a tossup about an author may have a middle clue describing characters, then giving their names, and then drop the title of the work. You could either stretch that out by putting words in between and breaking it up into sentences:

"Two characters flip a coin only to find it repeatedly comes up heads at the beginning of a play by this author named for those two characters. That play is Rosencrantz and Guildenstern Are Dead."

Or, compress it down as much as possible.

"Two characters repeatedly flip a coin heads at the start of his play Rosencrantz and Guildenstern Are Dead."

(if this example is not what you were thinking, then please correct me) I still think the latter is favorable, even if you're worried about player A still processing "flipping a coin heads up repeatedly" while player B is reflex buzzing on the word "Rosencrantz," although I totally understand where you're coming from. There's a few ways I can defend the opposite position:

1. A slower moderating pace can help alleviate this issue by creating more time between critical words of clues (e.g. character names and titles). Shorter questions can provide that relief, since it takes pressure off the moderator to rush through questions to finish rounds in a reasonable length of time. That also frees up the moderator to focus more on clarity.

2. The logistical reasons described in other posts provide some justification.

3. Finally, there's a certain point at which the player just has to remember the clue effectively. I'm sure everyone has experienced the pain of recognizing an early clue, trying to remember where the hell you heard it before for a few seconds, and then getting beat to the tossup by the other team. That sucks, but I'd still say that the other team legitimately earned those points.

Auroni is right about science and music tossups often needing more room to properly articulate their clues. However, using that same line of argument, could you not reason that NAQT tossups forgo those context clues to meet their length requirements? I find that 500 character science tossups can be really choppy sometimes, but if you take the same clues from an NAQT tossup and add more context, it probably becomes a lot more smooth while still being able to remain below 7 lines. I think that would be the ideal solution - you'd end up somewhere on the sixth line or maybe just into the seventh, which is right where I want to see college regular difficulty end up.

In my first post, I touched on the "specialist" matter that other people are bringing up now - it's an extremely good point that I also agree with. For veterans, it can be tiresome to sit through very long questions in topics you don't really care about. I bet it's harder for people who don't necessarily have as much patience. Sure, you can say that we don't want those impatient people in quizbowl anyway, but we don't HAVE to make it unbearable to the point where it is severely hurting the recruitment of more casual players. There's a sweet spot that we're missing right now. Look at ACF Fall - over 150 college teams attended, and the top 25 teams don't play. NAQT Sectionals had 79 DI + 166 DII = 245 total teams this year! I'm not saying the only reason teams attend these two tournaments far more than other tournaments is because of the question length, but I bet it's a factor. Speaking personally for my club last year - we had a limited budget, and part of the deciding factor for tournament selection was question length.

[Cheynem]

Yeah, I actually think bonus length tends to be forgotten in line length discussion. Like in most cases your bonus does not need to be that long (why are there lead-ins that go three lines? Why?). Obviously, there are some bulky-ass scientific or philosophical concepts that need a lot of room to explain, and sometimes a rare, well timed long bonus part can be humorous (I enjoy listing all the things that they did to the guy who killed William the Silent and then saying "And then they tortured him!"), but for the most part, bonus parts can be pared down easily.

[Victor Prieto]

I'm on mobile, so I can't quote efficiently, but I don't think cutting leadins because they "just distinguish Maryland from Yale from Stanford" is a good idea. Sure, most of your clues shouldn't serve that purpose, but if this is a regular difficulty set your clues should serve that purpose just as they distinguish between Minnesota and Chicago B later in the question. If you have multiple leadin clues then you have a problem, sure, but one clue you think is only buzzable by a few teams isn't the end of the world.

I want to address this separately - I think this is a very dangerous attitude. There's no WAY this could fly at the high school level. When you're writing a regular difficulty high school tournament, you're not trying to decide games between Stevenson (4) and Barrington (6), nor Naperville North (8) and Auburn (47), even though all of those teams could presumably play each other at regular season events. Sure, the college circuit isn't the same as the high school circuit. But I don't see any reason why it can't be (within reason, there's only so many colleges + universities out there), and this is a helpful step towards achieving that goal. That one lead-in clue could be replaced with something more difficulty appropriate. If you deleted the first sentence of every Terrapin tossup, not even Chris Ray could first-line every one.

Regarding bonuses: I entirely agree that long bonuses are also a drag on tournament length. I specifically excluded them from my post because (1) brevity and (2) I don't think the psychological impact of a long bonus is the same as a long tossup.

[touchpack]

(perhaps not coincidentally, about 100 characters of 10 point TNR fits on a single line). Finally, I calculated the average tossup length for a couple commonly mirrored high school tournaments this year.

IS set: ~4.25
DI SCT: ~5.00
WHAQ: 5.48
HFT: 6.00
EMT: 7.45
EFT: 7.55
Penn Bowl: 7.80
Terrapin: 7.85

This is not quite accurate--an average line of 10 point TNR is slightly more than 100 characters. I took Packet 1 from Terrapin and removed the pronunciation guides and powermarks (which increase line length non-uniformly depending on the length of the individual power/amount of PGs without actually making the question take longer to read, so they should be exempted from this sort of analysis)

The average character count was 825, but the average line length was ~7.25, about a full line difference! In addition, every tossup was under 8 lines, with the exception of an outlier music tossup, which was 8.7, a full line and a half above average. So I'm skeptical about the methodology behind this data.

Also, I think it's important to point out that NAQT's shorter tossups are a deliberate stylistic choice--one which has positives (makes the game feel much more forgiving, allows tournaments to run faster) and negatives (increases the amount of buzzer races). While I agree that length control is a good thing (as I just pointed out, the regular difficulty tournament I edited had an average length of around 7 lines), I think there are differences between the college and high school games, and between mACF and NAQT sets, that shouldn't be tossed out. (as an aside, there should be more mNAQT housewrites! There are a lot of people that enjoy the NAQT format more than ACF!)

That said, stop writing 10 line tossups. They're never necessary, and people don't enjoy playing them. Improve your prose! Training yourself to write under a strict character limit (in my case, NAQT's limit of 500 for SCT/ICT) has made me a much better writer in terms of prose, and I encourage developing writers to try writing tossups with say, a 5 line or 6 line cap as practice. Personally, I think a 6 line cap at fall, 7 line cap at regs, and 8 line cap at nats is a good idea, though I strongly advocate for them to be soft caps since in certain science/music cases (this happens to me like, less than 5% of the time, but it does happen), you really need just a LITTLE bit of extra space to flesh out a description.

[Victor Prieto]

After talking about it with Billy a little bit, we eventually came to the conclusion that using line length is not as effective for determining question length compared to characters. We used different methods for determining line length, and both methods were not exactly consistent. I overestimated, Billy underestimated. However, our character counts are pretty much the same after removing PGs and power marks. Assuming that EMT and Terrapin were the shortest and longest of the two tournaments I surveyed, the character counts for packet 1 ranged from 767 characters (EMT) to 823 characters (Terrapin, std. dev. 54). Anyhow, I hope this doesn’t detract from my larger point that I’m making, which is that I think regular difficulty tossups are longer than they could be and should be.

[CPiGuy]

So, there's been some discussion in this thread about "new players" and how to make (hard) quizbowl more attractive / fun / whatever for them. As one of those "new players" (albeit one who doesn't require much convincing to play quizbowl), I must say I agree that tossups could stand to be shorter. I definitely noticed that D1 ICT questions, despite being equally ungettable to me as other nationals-level questions, were a lot more enjoyable to play than even ACF Regionals, on which I actually stand a chance. I suspect this is because despite being difficult for me to get, because they were shorter, there was less time between questions on which I did have a chance. I understand that for better players, you probably have more than 2/2 of the distribution which you would be hoping to get regularly at higher difficulty levels; having said that, I think that all but the best generalists would have a legitimate chance of buzzing on the first few clues to less than half of tossups. Obviously, for events like national championships, and especially the playoffs of national championships, it makes sense to have lots of hard clues at the beginning in order to differentiate between the very best teams. However, as has been already pointed out, that's not the standard at HS level, and I don't think it should be the standard at college tournaments which are not nationals and opens at which most top players will be in attendance (those being tournaments where first-clue buzzer races would be a real possibility if the first two clues of every tossup were excised). Specifically, I think Regionals and tournaments of comparable difficulty should contain shorter tossups (doesn't have to be NAQT-short, but 6-7 lines rather than 8.) I would advocate that such tossups achieve being shorter, generally (but not always), by just cleaving off the lead-in rather than keeping the lead-in and making the clue gradation faster (the latter, incidentally, is probably the thing I like least about NAQT format -- clues get easy too fast out of necessity, so if you're slow to the buzzer you can get beaten by someone who knows significantly less than you; likewise, you can get questions on things you know less about by being fast). I have two caveats to this, though:
1) as people have pointed out, science and music are categories where this doesn't always work. I'm sympathetic to this, having just written a math tournament which I tried to keep at 7 lines per question. However, science and music writers *already* have to deal with line caps. So, unless science and music are already being given occasional exceptions to line caps, I don't think they should be given exceptions to the shorter ones.
2) None of what I just said applies to tossups-only side events; those are the perfect place to explore interesting material with long, bulky questions (since there are no bonuses).

[Periplus of the Erythraean Sea]

The best college teams answer a large majority of regular questions against other top teams in the first half of the question - see Terrapin for example.

[halle]

I'm probably one of the newest people to quizbowl that's even reading this thread, having started playing less than a year ago, and my experience has been pretty different from Conor's/the perceived average new player's. That is to say that I like long questions. While I can understand the attitude of thinking something like "Ugh, I've got to sit through all these questions even when they're boring?" I think it's just as likely for a new player to be like "Wait, I've got to buzz the very second I hear a clue I think I recognize, even if I'm not sure?" I'm definitely closer to the latter way of thinking. Learning how to keep yourself sharp through a long day of a tournament and learning what level of confidence in an answer you need before buzzing are both skills that new players/teams need to develop, and they both should be considered. In fact, learning when to buzz is probably more specific to new players, so if we're trying to cater to them, leaning towards shorter questions seems counterproductive.

In the Rosencrantz and Guildenstern example, my thought process would probably be something like "oh, that coin toss thing is from R&G, at least I think so, oh they said two title characters, I must have been right, I'm buzzing now." A better and more experienced player wouldn't necessarily need that extra step between the clue and the title drop, but, if we're talking about new players who are still trying to improve, this is certainly a possible way of thinking. Getting rid of those extra words would've turned the tossup from something gettable to a moment of wondering how other players can be so damn fast.

Unless I've been brainwashed by the all-powerful regime of Chicago quizbowl (a possibility I can't entirely rule out), I think that I can serve as an example of a type of new player that's worth considering in these sorts of discussions, as I'd be fairly surprised if no one else feels the same way.

[Cheynem]

Halle kind of gets at this, but I think there are different kinds of long questions from each other. You can have two 8-line or even 9-line questions (my upper bound on college difficulty questions, at least at regular difficulty and beyond). One 9-line question can be long because there are descriptive and interesting clues. The clues chosen are interesting, and the tossup carefully fleshes out all those clues to allow for buzzing. Another 9-line question is vague, babbles, and has lots of uninteresting clues ("A servant of this leader was Doug Brocail. Another servant was Jose Lima. Yet another one was Willie Blair."). Even if you hear all of those questions to the end, I'm 100% sure the first 9-liner is going to be way more interesting and seem like less of a slog than the second.

[Victor Prieto]

The best college teams answer a large majority of regular questions against other top teams in the first half of the question - see Terrapin for example.

This statement got me thinking about how to actually show this empirically, since I thought this may have been a conclusion originating from impressions of playing/moderating/watching top tier games. How often are 10 or more tossups in a game answered in the first few lines? Thankfully, we have a rough estimation for teams buzzing in the first half of the question - the power mark.

For my analysis, I used powers for the top 20 American teams that played Terrapin this year, both overall and against other teams in the top 20. The "top 20" mark was slightly arbitrary (I just picked teams by eyeballing placement, PPB and number of powers). All but two of the teams (Rutgers and the open team "UCSD A") were in the top 25 of the postseason poll, though.

Terrapin power analysis

[explanation of data analysis comes later, skip below if you want to know how I generated the tables and also how to read them]

My major findings:

1. Most of the top 20 teams had fewer powers when playing against other top teams (table 1).
2. Most of the top 20 teams had fewer negs when playing against other top teams (table 1).
3. If every team of the top 20 teams played a hypothetical game against each other, 106 out of 190 games (63%) would see fewer than 10 powers (tables 2 and 3).
4. If you exclude the top 5 teams, the number of hypothetical games with fewer than 10 powers becomes 93 out of 105 (89%).

Important thing to point out again, before I get to my main point: I’m not advocating for a decrease in the difficulty. I’m advocating for a decrease in the number of words read for teams to get to the clues they can actually buzz on. The only data point I have for evaluating the point during a question at which teams buzz is the power mark, hence my detailed analysis of powers between teams.

In games between teams at the very bottom of the top 20, I’ve shown here that at least fourteen out of twenty tossups per game have multiple lines in the powermarked section just whizzing right by. That’s my main point. Obviously, the numbers for teams below 20 should be worse. I don’t care about teams ranked 1 through 14 at regular difficulty, who gives a crap if they get eighteen successive tossups by the third line. With regards to my main argument, I care the most about regular difficulty games for teams between 15 and 60. Teams below 61 can and should play regular difficulty, but ACF Fall, DII SCT, and MUT/EMT should be the difficulty designed for those teams. I think games between teams ranked between 15 to 30 should be getting about 7 or 8 early buzzes per game, and games between teams ranked 31 to 60 should be averaging about 4 to 6 early buzzes per game. Right now, those numbers are instead between 4 to 6 and 1 to 3, respectively.

I would be willing to bet that cutting out two lines out of the powermarked section (by removing clues, but also improving style) would not substantially impact games between middle-tier teams, but have a huge impact on (1) perceived difficulty and (2) tournament logistics. The context clues are important, and I definitely think it’s worth preserving 1 to 2 lines of introductory powermarked clues to help "establish" the tossup on solid ground before getting to serious middle clues (a phenomenon that isn’t present in NAQT tossups that puts me ill at ease). But right now, the slider is set too far right now towards the high end.


_______________


***more in-depth explanation of data***

[Table 1] In the first table, I input overall powers and games played for each team. I then counted powers and games played against other teams in the table. Overall P/G and P/G vs. top teams were then calculated. The change in P/G against top teams (∆P/G) should represent if teams powered more or less when they were playing other top teams. The table is ordered by overall P/G, which I think was the best indicator of team ability that was on the table.

Most of the top teams powered less against other top teams. There are two reasons I can think of that could explain this: one, that teams compete more for powers in certain categories and overall power rates drop as a result, or two, that teams play more cautiously when playing closer games. To try to disprove the second reason, I looked at neg rates vs. top teams - if teams played more cautiously, I thought that neg rates should drop as well (it is also possible that they neg less because questions are powered too quickly before they neg, but I’m not as convinced about that reason). I found that negs did decrease across the board for games played between top teams. The conclusion I drew was that top teams play more cautiously when playing other top teams, and as a result, have lower power rates. My point is that games between top teams see powers on a majority of questions in spite of decreased power rates, so I don’t know if one can confidently assert that top teams play "better" against other top teams.

[Tables 2 and 3] The second table predicts the number of powers in a hypothetical game between any two teams in the set. Essentially, each value in the table is the sum of the overall P/G between both teams. For example, if Yale A played Virginia A in a hypothetical game, the table predicts 10.19 powers per game. Bolded values are games that actually took place (italics means multiple repetitions). Below, there are two counters for number of games above and below 10 powers per game, which shows the majority of hypothetical games in the table would be below 10 powers per game. The last value in the bottom right shows how far off the predictions are, using the predictions from the table and comparing to the actual games that took place. Since the value is -0.88, this means the table predicted the games about 0.88 powers too high, which means the number of hypothetical games below 10 powers should actually be higher.

The third table is identical to the second table, but instead of using the sum of the overall P/G, it uses the sum of the P/G vs. top teams. Using those values came closer to the actual number of powers in the games that took place in this table (the bolded values), only underestimating by 0.13 powers. This table still shows that the majority of games between top teams see less than 10 powers.

Will’s statement seems to be true, since most games between the best seven teams and other top teams do see a majority of the questions powered. However, if you exclude the top seven teams and the hypothetical matchup between teams 8 and 9 (Penn and Northwestern), there are zero hypothetical games which reach an average of 10 powers between both teams. Not only that, this conclusion is based off of table 2, which overestimated the number of powers. If we use table 3, which was closer to the actual values (but underestimated slightly), you only need to exclude the top five plus Stanford (who had an anomalously high power rate vs. Berkeley B).

______

I’m curious to know what could be discovered by applying the same analysis to SCT and ICT stats, or stats at the high school level. If there are any budding quizbowl sabermetricians out there who want to try a similar, go for it, I would really love to see more data (I really would like to see a larger sample size than my own). Also, feel free to copy the data from my sheets and use it for your own nefarious purposes.

[Periplus of the Erythraean Sea]

Most of the top teams powered less against other top teams. There are two reasons I can think of that could explain this: one, that teams compete more for powers in certain categories and overall power rates drop as a result, or two, that teams play more cautiously when playing closer games. To try to disprove the second reason, I looked at neg rates vs. top teams - if teams played more cautiously, I thought that neg rates should drop as well (it is also possible that they neg less because questions are powered too quickly before they neg, but I’m not as convinced about that reason). I found that negs did decrease across the board for games played between top teams. The conclusion I drew was that top teams play more cautiously when playing other top teams, and as a result, have lower power rates. My point is that games between top teams see powers on a majority of questions in spite of decreased power rates, so I don’t know if one can confidently assert that top teams play "better" against other top teams.

I think there's a more elegant explanation which covers both the neg and power issue: when you play against another top team, your total number of chances to buzz in goes down because you are hearing fewer clues playing against them. So you hear fewer clues to buzz on, and therefore your number of buzzes of all types goes down - 15s, 10s, and -5s.

I don’t care about teams ranked 1 through 14 at regular difficulty, who gives a crap if they get eighteen successive tossups by the third line.

The answer is that a lot of people in those top 14 teams do care, and for the most part they're the people who write tournaments.

In games between teams at the very bottom of the top 20, I’ve shown here that at least fourteen out of twenty tossups per game have multiple lines in the powermarked section just whizzing right by.

I think this statement overstates the issue (even if we limit ourselves to looking at just two competing teams) for some reasons:

1) Players may not be correctly buzzing on these clues, but they may still be buzzing on them. Negs in the first half of the tossup matter as well, and I'd be willing to guess that a couple of those happen in most games between these teams. Assuming no vulching, this means the other team's not gonna be powering that tossup.
2) Players aren't always able to remember/interpret a clue immediately - sometimes, people need a line to think through things.
3) Clues can be "added up" to make a buzz later in the question - i.e. if I glean information from clue 2 that narrows things down to five possibilities in my mind, then clue 5 solves that puzzle, I get a clue 5 buzz out of power but clue 2 was nonetheless helpful.

Between these three, I'd estimate that average games between teams in the 15-20 range involve interacting with the power clues in over 50% of tossups. I'm willing to guess that, assuming average tossup length of 7 lines, these teams are most frequently buzzing on lines 4 and 5 - that seems perfectly fine to me. In addition, the importance of (2) and (3) is most acutely realized as tossups grow shorter, because players are deprived of opportunities to think about and contextualize clues, leading to more buzzer races.

I think policing wording is generally a fine way to try to go about things - I plan to have tossups at this year's EFT around be half to a full line shorter, on average. If you can cram more substantive clues into a smaller amount of space, then you generally should, as long as you don't spit out a context-less stream of facts. But in terms of the distribution of where buzzes are happening, I really don't think there's a big issue.

Clean up your tossups, people, and don't write nine lines when you can probably distinguish people with seven.

EDIT: Cleaned up post grammar, etc.

[hftf]

Since there has been a lot of discussion involving Terrapin, I wanted to release some graphs that show where players at the Maryland site actually buzzed.

There were 2063 correct buzzes in 112 games at the Maryland site, which is 22% of the 516 games across all Terrapin sites.¹
Please carefully consider that this data is neither representative of Terrapin nor of collegiate quizbowl.

Distribution of buzzes by team

         

Distribution of buzzes by category

         

Click on the figures to enlarge them. In the first two figures in each section, the dark area represents powers. The last three figures show the cumulative distribution of buzzes.

Let me know if you have any questions, problems, or suggestions. I may post more data later.

____
¹ 112 games = 9 rooms × 12 rounds + 4 rooms × 1 round. 516 games = 60 + 40 + 8 + 41 + 40 + 112 + 31 + 24 + 81 + 49 + 30.

[Rococo A Go Go]

I don’t care about teams ranked 1 through 14 at regular difficulty, who gives a crap if they get eighteen successive tossups by the third line.

The answer is that a lot of people in those top 14 teams do care, and for the most part they're the people who write tournaments.

Even if that were completely true (lots of editing teams appear to rely on retired players like yourself...) I don't think this is a positive way to think of the quizbowl community. Or maybe you just don't need the other teams and want them to go fuck off...but for some reason I suspect that if all the other teams stopped hosting or attending tournaments that are clearly not meant for them, you would find yourself scrambling to change your tune.

[armitage]

I don't think Will is exactly telling the quiz bowl 99% "to go fuck off"...it seemed to me like he was pointing out an actual reason why there might be pushback against Victor's sentiment he quoted, without supporting or opposing it. Also, I felt like people already reached a consensus that regular difficulty should become easier, some time earlier this year. In any case, I'll reiterate that we aim to keep EFT questions more concise this year.

[Periplus of the Erythraean Sea]

I don’t care about teams ranked 1 through 14 at regular difficulty, who gives a crap if they get eighteen successive tossups by the third line.

The answer is that a lot of people in those top 14 teams do care, and for the most part they're the people who write tournaments.

Even if that were completely true (lots of editing teams appear to rely on retired players like yourself...) I don't think this is a positive way to think of the quizbowl community. Or maybe you just don't need the other teams and want them to go fuck off...but for some reason I suspect that if all the other teams stopped hosting or attending tournaments that are clearly not meant for them, you would find yourself scrambling to change your tune.

Considering that I worked to revive a fall tournament explicitly designed to provide an accessible introduction to college quizbowl, it would seem odd indeed if I thought that the 99% of quizbowl should fuck off! What Richard said is correct, especially the bit about making EFT more concise to be more engaging to new players.

If you want good tournaments at each level, somebody's going to want to have to play them, but since the game relies on (paid) volunteers to produce tournaments, there's also going to have to be someone who wants to write them. Until we have a large enough class of professional question producers, which isn't likely to happen in the near future IMO, these paid volunteers are going to have to be interested in producing the tournament they're writing. From personal experience, it's actually much easier to produce a good regular level college (this isn't true for high school) tournament with 7-8 line questions than 5-6 line ones, because you've got more room to fit in middle clues and don't have to police yourself as tightly about leadins, etc. because you know you'll have room for those middle and mid-late clues. In addition, writing more sets like this tends to be a better way of learning, and many (if not most) "volunteer" writers in the college community take on such projects as opportunities to improve, as well as contribute. So, from a supply side standpoint, there's a number of reasons that the kind of people who actually do the writing would prefer to write longer
questions.

There's a balance to be struck, but I think it's basically just Mad Eye Moody's lesson: CONSTANT VIGILANCE. Assuming "regular" stays around the level of this past year's Terrapin (with Regionals being a bit harder), "regular-minus" around EMT/EFT, ACF Fall is returned to 2013/14 levels, then I think difficulty is going to be fine (the big item for difficulty these days IMO is policing hard parts). However, we could do better to curb verbosity and waste less of people's time, or take a step back and ask "is anybody going to buzz on this clue?" before putting a clue in.