A Defense of the Muddy Battlefield Hypothesis

Old college threads.
Locked
User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

A Defense of the Muddy Battlefield Hypothesis

Post by No Rules Westbrook » Mon Jan 26, 2009 3:51 pm

Heh, I started this as a simple response to Jerry in the other thread, but it's really turned into a treatise...so I extracted it and made it its own thread so as not to bog down Penn Bowl discussion.

Okay, after reading through a good portion of this year's Penn Bowl, I think there were some glaring problems - transparency and difficulty cliffs/buzzer races were the big ones, especially in packets written by less experienced authors, which is an indication of some problematic editing. I'll offer a more detailed critique later and cite individual questions in the actual PB thread - many of the questions could undoubtedly have been improved for the benefit of all teams through a few simple insertions and deletions, ones we'd all agree with, regardless of whether or not you agree with my comments below.

Right now, I want to offer a defense of what I've quippingly called my Muddy Battlefield Hypothesis. It goes something like: "Tournaments with the difficulty and length constraints of something like Penn Bowl will always be incredibly hard to write, and grow harder with every passing year, such that it's a delusion when people suggest that these tournaments are usually optimal events for teams of all skill levels to play (NOTE: I said optimal, I didn't say enjoyable - I don't doubt anyone can come to lots of types of qb events and generally have a good time)" I call it "muddy battlefield," of course, because I think that's what these events often resemble for more skilled teams/players.

I want to address what Jerry said in the other thread as a means of demonstrating to people why it's so hard to write events at this difficulty level that please both very skilled and mostly unskilled teams. Jerry says he'd like to think it's possible to write a good tossup on Chandragupta that will please everyone, and there's no doubt that it's possible - even though Chandragupta is one of the most tossed-up subjects around. It's very possible, but here's why it's hard to do: What you'll probably start out doing is you'll try to find a lead-in that's both challenging and useful for the top teams at the tournament - let's say Chicago and Brown - you'll want to find a clue that's both interesting to them, and one that they might potentially be able to buzz off of. The problem is that finding that "just-right" clue is kind of a guessing game - you have to accurately predict what the most knowledgeable team will be able to buzz off, and not any of the lesser-knowledgeable teams - that's hard to do. Add to this difficulty the fact that a restricted canon means you have to be really careful about things like linguistic transparency - you can't give certain clues because it will conclusively restrict the time and context of the question such that only one or two answers are possible at the difficulty level, whereas six or seven might be possible at a Chicago Open difficulty level.

That was an aside, now back to my first point - oftentimes, you're going to guess wrong and pick a first clue that noone can realistically buzz off of - sure, you picked something that looked important and potentially helpful, but it didn't work. Now, you're down to the third or fourth line of the tossup and you're thinking about "medium-level teams" - you're trying to find the same types of clues for them, ones that they'll know and that the teams at the bottom level of knowledge won't know. And you're thinking about the teams realistically, and you only want your tossup to go about 6 lines - so you go ahead and drop a clue like Arthashastra or Kautilya - what you've effectively done is create a sure buzzer race for the most knowledgeable teams, since you guessed wrong with your earlier clue that was supposed to be made for them. And so on.

Let's contrast this with a situation where I, for example, decide to write a tossup on Chandragupta for Chicago Open or some masters event where I have a field composed of mostly very skilled teams. For one thing, I can be way less worried about things like linguistic transparency (although there are still sometimes contextual transparency problems), because the canon is so big that the reward for considering such transparency is almost nothing. But, more importantly is this point - I can start the question off with a whole bunch of hard cool-looking clues which I think might be helpful to very knowledgeable people. If I guess wrong about the first clue being helpful, there's always the second, and then the third, and then the fourth. Hell, I can layer in interesting clues until my tossup is 11 lines long if I want - it's not like dudes like Jerry are going to complain about that. You see, the pressure on me to "guess right" about the difficulty and usefulness of every single clue is much less. If there's ever a point where I feel like there's an unfair difficulty cliff in the tossup, I can just go ahead and layer in another clue in front of it.

Add all of this to one final point - the "very skilled" group of players in quizbowl is getting progressively more skilled by the minute, constantly absorbing more canon and more clues. They're better than they were in 2007, and they'll be better than they are now in 2011 - because, by and large, they're still the same group of people who have been around forever. They haven't necessarily improved their games per se - they've just absorbed more stuff, because they've been around playing and reading packets, writing questions, and so on. Conversely, the "very unskilled" group doesn't change a great deal, they're still starting off from point zero pretty much - so the gap between those groups gets bigger and bigger.

And so, when people pretend like I'm in la-la land when I suggest that tournaments of this nature are really really hard to write with any kind of success, I think they're plum loco. When people suggest that accessibility and length considerations don't tend to foster characteristic problems, they're in fact the ones living in dreamland - I suspect the same dreamland where 25-minute rounds grow on lemonade trees.

I'm not saying Penn Bowl was bad, or that it should do anything that differently - other than, like I say above, some much-needed editing which would have mended the more obvious transparency and buzzer race problems in the set (andsure, going to a 7-line ideal length rather than 6-line might be a step that I'd support). I just want people to have a realistic picture of why certain tournaments tend to turn out the way they do. I don't think there's any way to fix the problems I've noted - nor should we try to fix them, since there are good reasons why sets like Penn Bowl need to exist. But, we should at laest acknowledge the existence of the problems. Sets like these can turn out quite well, but there will always be a limit as to how well - that's just the way it is. Accept it, and stop sounding the clarion call of the tournament that can be all things to all people.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Mechanical Beasts » Mon Jan 26, 2009 4:24 pm

No Rules Westbrook wrote:Accept it, and stop sounding the clarion call of the tournament that can be all things to all people.
I don't think anyone claims that a regular-difficulty event can truly be all things to all people. There's a reason that there are elite players who didn't eagerly sign up to play Penn Bowl, and a reason that there wasn't a whole lot of gossip about why not. Some players don't get their ideal tournament experience from tournaments below a certain difficulty. There's also a reason why there were only a few high school teams at this tournament, and no appearances by West Nowhere C: some people don't get their ideal tournament experience above a certain difficulty.

What a regular difficulty tournament can aspire to do is to be, if not all things, then something to all players: regular difficulty is still playable by, for example, Jerry. It's not, I assume, his favorite difficulty, and that's not surprising. Neither is it the favorite difficulty of a weak college team. That's not surprising. But a regular difficulty tournament aspires to be playable by pretty much everyone. However, that doesn't diminish the relevancy of your point, which is that
No Rules Westbrook wrote:the "very skilled" group of players in quizbowl is getting progressively more skilled by the minute, constantly absorbing more canon and more clues. They're better than they were in 2007, and they'll be better than they are now in 2011 - because, by and large, they're still the same group of people who have been around forever. They haven't necessarily improved their games per se - they've just absorbed more stuff, because they've been around playing and reading packets, writing questions, and so on. Conversely, the "very unskilled" group doesn't change a great deal, they're still starting off from point zero pretty much - so the gap between those groups gets bigger and bigger.
This has to do with the definition of regular difficulty. If a regular difficulty tournament necessarily continues to aspire to be playable by everyone, then sure, you're in a bind with writing. Questions will have to be twice as long in 2020 than they are now, and they'll just be harder to write. If questions don't get longer, then they can't stretch to make the tournament playable by both the best and the worst players. (Or they can, but difficulty cliffs are more of a problem.) And chances are, fifteen line tossups would be almost as bad for the bottom tier of players as questions that are too hard to be accessible, and so in the future regular difficulty loses the bottom tier whatever you do, if this is how you choose to define it.

Alternatively, regular difficulty can have a constant definition, fixed in terms of what regular difficulty is today, and perhaps in 2020, the top five (or fifty!) players in the country will say that regular difficulty just isn't hard enough for them, because they're much better than our five-or-so best players of today, who are currently (let's say) at the fringes of for whom regular difficulty is playable.
Andrew Watkins

User avatar
Skepticism and Animal Feed
Auron
Posts: 3178
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Skepticism and Animal Feed » Mon Jan 26, 2009 4:28 pm

No Rules Westbrook wrote: That was an aside, now back to my first point - oftentimes, you're going to guess wrong and pick a first clue that noone can realistically buzz off of - sure, you picked something that looked important and potentially helpful, but it didn't work.
And whose fault is this? The fault of the team that didn't know that clue and allowed the question to progress to the middle clues.

Unless you're flat out making up clues, or taking them from a source accessible only to yourself, your clue is not by definition "unbuzzable" -- if you learned it somehow, so could Seth Teitler.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

User avatar
theMoMA
Forums Staff: Administrator
Posts: 5686
Joined: Mon Oct 23, 2006 2:00 am

Re: A Defense of the Muddy Battlefield Hypothesis

Post by theMoMA » Mon Jan 26, 2009 5:32 pm

I agree with some of this, but I argue that you, Ryan Westbrook, are living in fantasyland if you think that games between top teams are won on leadin buzzes. Even at length-controlled events, the top teams are usually buzzing on middle clues. I'm not saying that leadin buzzes don't happen, or that they're not important to the outcome of games, but the bulk of points in the matches between top teams come from middle clue buzzes. The reality is that the leadin is more often than not "unbuzzable," even for the best two teams in the tournament. This creates all sorts of problems when the next clue is "this is a treaty that governed 16th-century Spanish explorers," but it's perfectly fine when, like the vast majority of Penn Bowl tossups, the middle clues are properly edited and arranged.

It's true that length-controlled regular-difficulty events are not the best determiner of skill between the top teams, but the results are still legitimate within their own context. One of the most annoying things that I hear with regards to easier questions is how games are won and lost on "buzzer races." Sure, there are times when no one in the room has the foggiest idea where the question is going, and it suddenly announces a really famous clue. More often than not, those are bad questions. But on good questions, the player who has a pretty good idea what the answer is going to be will get the question. Sure, a lot of people might be buzzing on that Coleridge tossup when it says "secret ministry," but the well-versed Coleridge reader who's sitting there thinking "this sounds a lot like Coleridge" will usually get the tossup over the reflex buzzer.
Andrew Hart
Minnesota alum

User avatar
Gautam
Auron
Posts: 1413
Joined: Sun Feb 11, 2007 7:28 pm
Location: Zone of Avoidance
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Gautam » Mon Jan 26, 2009 6:10 pm

Whig's Boson wrote:
No Rules Westbrook wrote: That was an aside, now back to my first point - oftentimes, you're going to guess wrong and pick a first clue that noone can realistically buzz off of - sure, you picked something that looked important and potentially helpful, but it didn't work.
And whose fault is this? The fault of the team that didn't know that clue and allowed the question to progress to the middle clues.

Unless you're flat out making up clues, or taking them from a source accessible only to yourself, your clue is not by definition "unbuzzable" -- if you learned it somehow, so could Seth Teitler.
I will give you some examples of unbuzzable leadin clues from submissions for tournaments I edited:

"8-chloro this azido inhibits growth through p38 MAP kinase and can induce apoptosis in cancer cells."

"The bacteria Xanthonomas uptakes this material from their surroundings to increase their metabolic rate."

"In reverse field experiments, this phenomenon is recorded using waveform digitizers, and is detected by avalanche photodiodes."

There is simply no way to answer a tossup on cAMP from clues about some 8-chloro derivative when millions of halogenated derivatives of different compounds have been studied for anti-cancer properties. I am sure the people who wrote these questions have gotten better at picking useful clues. I've played on questions they wrote or edited recently, and those were much better.

Gautam
Gautam - ACF
Currently tending to the 'quizbowl hobo' persuasion.

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by No Rules Westbrook » Mon Jan 26, 2009 6:21 pm

Yeah, of course, Andrew - I didn't mean to suggest that most events have any kind of high percentage of buzzing on lead-in clues. But, when you're sharply limited as to the length a tossup can be, you have to try pretty quickly to give knowledgeable teams helpful clues that allow their greater knowledge to shine through. You can't be dilly-dallying for very long, or else your tossup will be too long - instead of "lead-in," what I really mean is the handful of clues around the harder part of the question.

And, I don't mean to make this just an issue of tossup length. The limited canon (to meet accessibility goals) is a big part of it too - for one thing, it increases the danger of stuff like linguistic transparency, as I said above. For another thing, it often makes you write on answers like Chandragupta which have been done a billion times before - and finding appropriate clues for those answers which satisfy all types of teams is very hard (i.e. the process of writing which I describe above is even harder when you're writing on something that's been done many times before). I used Chandragupta as an example to illustrate this - I could also pick such super-easy standards as Tale of Genji or Heisenberg's Uncertainty Principle. With the exception of a few things like authors, sometimes there just isn't an infinite set of helpful clues out there just waiting to be plucked from the clue tree. And, my final point was to argue that it's only going to get worse - it's only going to get harder and harder to keep writing tossups on those same old things, because the skilled players keep absorbing more canon and the gap gets bigger.

Now, Andrew seems to be arguing that, even at very easy events, more knowledgeable players usually edge out less knowledgeable ones - on some theory that they often have their buzzer "primed" and usually beat the other player. Sure, I'll buy that - oftentimes, the more knowledgeable player prevails, for a couple of reasons - one of those reasons is that he can often form a faint impression of what the answer probably is, even if the clues aren't terribly helpful, and then he can edge out the reflex buzzer with that built-up knowledge (there are other reasons too - for instance, more knowledgeable players typically have a better mastery of the canon, and can more confidently play mind games about what the answer is likely to be). But, this is all practically an admission of my "muddy battlefield theory" - that's a pretty risky way of trying to get points, edging out reflex buzzers by a split second.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
Haaaaaaaarry Whiiiiiiiiiite
Auron
Posts: 1124
Joined: Tue Dec 04, 2007 8:46 pm
Location: Fairfax, VA
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Haaaaaaaarry Whiiiiiiiiiite » Mon Jan 26, 2009 6:26 pm

theMoMA wrote:The reality is that the leadin is more often than not "unbuzzable," even for the best two teams in the tournament.
Though wouldn't it be better if the leadin is buzzable by a couple of people at the tournament? Otherwise it seems like the line could have served a better purpose.
Harry White
TJHSST '09, Virginia Tech '13, PACE Member
Tournament Database Search
Quizbowl Schedule Generator
Will run stats for food

User avatar
Cheynem
Sin
Posts: 6592
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Cheynem » Mon Jan 26, 2009 6:38 pm

Even if it isn't buzzable per se, the lead-in still has important work--it sets up the looked-for pronoun, usually gives a clue as to time period or context. I'm not saying that meaningless pap as lead-ins ("This writer wrote a lot of books") is acceptable, but, like, an obscure clue for Wordsworth should at least point out that he is a poet and give clues as to his time period and the kind of poetry he is writing. So even if it is not a buzzable lead-in, it still plays a point in narrowing down potential answer topics while still being potentially answerable by someone with very very deep knowledge and some nerve.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by grapesmoker » Mon Jan 26, 2009 7:18 pm

theMoMA wrote:It's true that length-controlled regular-difficulty events are not the best determiner of skill between the top teams, but the results are still legitimate within their own context. One of the most annoying things that I hear with regards to easier questions is how games are won and lost on "buzzer races." Sure, there are times when no one in the room has the foggiest idea where the question is going, and it suddenly announces a really famous clue. More often than not, those are bad questions. But on good questions, the player who has a pretty good idea what the answer is going to be will get the question. Sure, a lot of people might be buzzing on that Coleridge tossup when it says "secret ministry," but the well-versed Coleridge reader who's sitting there thinking "this sounds a lot like Coleridge" will usually get the tossup over the reflex buzzer.
I don't disagree with the gist of this paragraph, but for consideration, please see Brown's first game against Jonathan Magin. That game included several buzzer races, one on the Chandragupta tossup, one on J.S. Sargent, and at least a few other buzzes that could have gone either way because there was very little to buzz on before a clue that helped either team. I don't know if I know more than Jonathan does about Chandragupta, but that question didn't provide any way of deciding that. There were a couple of other questions in that round that were in a similar vein. For example, the Talleyrand question is easy because the plausible number of defrocked 19th century French politicians is very small, so once I put two and two together, I figured my chances of being right were high and went for it. Sure, I was rewarded for knowing something, but I definitely feel like it's the sort of question that wouldn't have done a good job of distinguishing between two decent history players.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
magin
Yuna
Posts: 955
Joined: Fri Oct 27, 2006 5:50 pm
Location: College Park, MD

Re: A Defense of the Muddy Battlefield Hypothesis

Post by magin » Mon Jan 26, 2009 7:41 pm

Ryan is certainly right that events with restricted answer space and tossup length are hard to write well due to difficulty in selecting clues that are both important, not linguistically transparent, and appropriate for the whole field. However, I think having such events is more important to nourishing a healthy circuit than creating many harder (but possibly easier to write/edit) tournaments. Additionally, I think that writing good, non-transparent tossups on accessible answers that a number of players can buzz on before the giveaway is certainly possible. To produce such tossups, I think that editors need a combination of three things:

* Some amount of knowledge about the answer
* Some amount of knowledge about how well known certain clues are (not that you need to know every title, but a rudimentary understanding that The Great Gatsby is more well-known than The Last Tycoon, and so on)
* Finally, and most importantly, careful writing and editing (good research, making sure the question is worded in ways that won't frustrate teams, looking up previous tossups on the same answer to get an idea of which clues should go where, and so on)

I think that it's certainly possible to use descriptions of literary/artistic works, historical events, books/essays, scientific processes, and so on to avoid the problem of dropping notable names too early. Since I've found that most buzzer races occur on titles/names, providing descriptions of, say, battles a historical figure won, novels by an author, and so on as leadins, I think, would help ensure questions don't produce such jarring buzzer races. All of this is relative, but I don't think that even the top players really know a huge amount about every answer that could conceivably come up at a tournament of Penn Bowl's difficulty; with careful editing, I think such tournaments could be very good indeed.
Jonathan Magin
Montgomery Blair HS '04, University of Maryland '08
Editor: ACF

"noted difficulty controller"

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by grapesmoker » Mon Jan 26, 2009 7:53 pm

Coral Gardens and Their Magin wrote:All of this is relative, but I don't think that even the top players really know a huge amount about every answer that could conceivably come up at a tournament of Penn Bowl's difficulty; with careful editing, I think such tournaments could be very good indeed.
The thing is, oftentimes good players don't need to know everything or even very much about a topic to be able to buzz on it. In a tournament with a restricted answer space like Penn Bowl, I'm pretty sure that Bindusara is not going to be an answer, so if I can place a question as asking about an Indian dude circa 4th century B.C.E, it's almost certainly profitable for me to buzz with "Chandragupta." I don't mean to keep harping on this question, it's just a convenient example; this applies very well to a whole range of possible answer choices at easier tournaments because there isn't a very large set of possible answers to choose from.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Mechanical Beasts » Mon Jan 26, 2009 8:05 pm

grapesmoker wrote:
Coral Gardens and Their Magin wrote:All of this is relative, but I don't think that even the top players really know a huge amount about every answer that could conceivably come up at a tournament of Penn Bowl's difficulty; with careful editing, I think such tournaments could be very good indeed.
The thing is, oftentimes good players don't need to know everything or even very much about a topic to be able to buzz on it. In a tournament with a restricted answer space like Penn Bowl, I'm pretty sure that Bindusara is not going to be an answer, so if I can place a question as asking about an Indian dude circa 4th century B.C.E, it's almost certainly profitable for me to buzz with "Chandragupta." I don't mean to keep harping on this question, it's just a convenient example; this applies very well to a whole range of possible answer choices at easier tournaments because there isn't a very large set of possible answers to choose from.
So, then, do you think it's extremely difficult to delay any time-period-fraud late enough in the question to prevent language+time period from becoming a problem? (Or to delay time period and language both, if necessary/possible, whatever.)

Or is it, perhaps, difficult to do that in the length constraints given (that is, that teams that aren't as good as Brown A deserve five lines of question coming after it's clear that we're in the fourth century BC, so we'd need to make the question longer so it's competitive for everyone)?
Andrew Watkins

User avatar
Captain Sinico
Auron
Posts: 2840
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Captain Sinico » Mon Jan 26, 2009 8:26 pm

I don't this this is nearly the issue that Ryan is claiming it is. I think most of the criticisms raised thus far have a lot of validity (regarding how questions are played, mostly) but I have a number of further ones I'd like to raise.
For one, the most experienced players are constantly scraped off the circuit (by the migration beyond college.) Thus the population of dudes who've been around forever isn't so large and really won't ever be. It doesn't seem to me to be larger now than it ever was, for example.
For two, I don't think the model of player development to which Ryan is appealing is anything like right. My experience is that most players saturate at a certain level of play and by and large stay close to that. Perhaps a given player can work very hard to play beyond that over a short period of time, but, by and large, nobody I know has just continued getting better.
Finally, writing interesting, challenging questions has always been a challenge and always will be. It is, however, a challenge that's been well met many times in the past in circumstances not significantly different or more difficult than the current ones are or the future ones seem likely to be. The fact that some tournaments have failed to do this well recently is at least as much an indictment of those tournaments as it is a gauge of how much more difficult circumstances have become (my arguement is that it's much more the latter, but this says a forteori that trying to measure this purely by examining the play of recent questions without further analysis is a fool's errand.)

MaS

PS:
gkandlikar wrote:I will give you some examples of unbuzzable leadin clues from submissions for tournaments I edited:

"8-chloro this azido inhibits growth through p38 MAP kinase and can induce apoptosis in cancer cells."

"The bacteria Xanthonomas uptakes this material from their surroundings to increase their metabolic rate."

"In reverse field experiments, this phenomenon is recorded using waveform digitizers, and is detected by avalanche photodiodes."
Those are just bad lead-ins because they're non-unique. That, likewise, is not a new thing: those clues ought not show up in a finished tournament. I think people had in mind unique leadins that are just too difficult rather than non-unique clues.
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by grapesmoker » Mon Jan 26, 2009 8:31 pm

everyday847 wrote:So, then, do you think it's extremely difficult to delay any time-period-fraud late enough in the question to prevent language+time period from becoming a problem? (Or to delay time period and language both, if necessary/possible, whatever.)

Or is it, perhaps, difficult to do that in the length constraints given (that is, that teams that aren't as good as Brown A deserve five lines of question coming after it's clear that we're in the fourth century BC, so we'd need to make the question longer so it's competitive for everyone)?
Andy, I am not sure I know the answer to your question. I can tell you that my own preference is to err on the side of delaying narrowing down the field of possible answers, but even that's context-dependent. The field of possible important 19th century English politicians is quite large, so figuring that out early is not a problem. On the other hand, the field of askable pre-B.C.E. Indian rulers is small, so having that information early is not good. I think everyone knows, or at least suspects, what tossups on the same answer choices as those featured at Penn Bowl would look like if they were written by me: they'd be about one or two lines longer and would probably focus on being more opaque in the beginning.

This discussion harkens back to the one I had with Andrew Hart after EFT, where Andrew noted that starting a tossup on "law" with a clue about HLA Hart was too early, and I said that given that this was a tournament intended for novices, I was ok with that. I stand by that statement; for a tournament envisioned as an introduction to the collegiate canon, I think that was a fine clue. But for a tournament like Penn Bowl, I would obviously not have written that, since Penn Bowl is a situation where two very good teams might meet, and I would feel that such a tossup would not properly distinguish between them.

In the end, it's all a matter of context and knowing which population you are catering to. I think Penn Bowl took the approach, probably unintentionally, that the best team would be determined over the course of the entire tournament even if some tossups or bonuses would fail in properly distinguishing two teams with non-trivially different levels of knowledge. I would say that overall this succeeded; the teams that were expected to do well did well, so clearly the empirical result was in line with predictions.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Mechanical Beasts » Mon Jan 26, 2009 9:05 pm

grapesmoker wrote:
everyday847 wrote:So, then, do you think it's extremely difficult to delay any time-period-fraud late enough in the question to prevent language+time period from becoming a problem? (Or to delay time period and language both, if necessary/possible, whatever.)

Or is it, perhaps, difficult to do that in the length constraints given (that is, that teams that aren't as good as Brown A deserve five lines of question coming after it's clear that we're in the fourth century BC, so we'd need to make the question longer so it's competitive for everyone)?
Andy, I am not sure I know the answer to your question. I can tell you that my own preference is to err on the side of delaying narrowing down the field of possible answers, but even that's context-dependent. The field of possible important 19th century English politicians is quite large, so figuring that out early is not a problem. On the other hand, the field of askable pre-B.C.E. Indian rulers is small, so having that information early is not good. I think everyone knows, or at least suspects, what tossups on the same answer choices as those featured at Penn Bowl would look like if they were written by me: they'd be about one or two lines longer and would probably focus on being more opaque in the beginning.
Yeah, it wasn't well phrased, but you said, essentially, the important thing. With fewer length constraints, it's possible to be more opaque to experienced players, who have a lot of negative information due to their experience (while lots of players know dude X, fewer know he's the only dude who fits a given set of constraints: less experienced players who are also unwilling to play the freshman method, are pretty likely to err on the side of waiting for better information) while still providing a long-enough tossup to less experienced players.

I guess what I'm really curious about is what we think about how increasing tossup length can solve problems with transparency, at different levels of difficulty (and so what we want out of tossups at different difficulty levels). I think there are several competing interests here.

First, we want to avoid transparency at all levels, and length helps us with that. This is both more a problem at Fall (tiny canon) and less a problem (very few players with a lot of experience play Fall; naturally, then, an even smaller proportion of games feature two experienced teams). This is both less a problem at Nationals (huge canon) and more a problem (high stakes: you don't want to get the result of any game "wrong," and there are a lot of close games between the best teams in the nation). Second, we have few reservations about tossup length at high levels, since experienced quizbowlers are there to play quizbowl, while we have many reservations about length at low levels, since we want to spread quizbowl.

Third, we want as many players in the "intended field" to be satisfied as possible, able to buzz on tossups of the "intended difficulty." This can be interpreted in a number of ways, some of which less favored than others (but I present them all); note that when I say "short" and "long," I obviously don't mean three lines or fifteen.
ACF Nationals is intended for only the elite, so tossups don't have to be long (i.e. you can have a giveaway that might be a Penn Bowl middle clue).
ACF Nationals is intended for only the elite, so tossups don't have to be short (with the side effect that they are good for a Chicago-Brown showdown as well as a Walter Johnson-Hunter).
ACF Nationals is intended to be hard, so tossups should not feature the easiest possible giveaway, so tossups should be short.
ACF Nationals is intended to be hard, so tossups should feature a lot of early, hard clues but still get converted, so tossups should be long.

Perhaps I'm missing something, but i feel like appropriate tossup length is a complicated question.
Andrew Watkins

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by No Rules Westbrook » Mon Jan 26, 2009 9:11 pm

Sorice raises some decent points, but I want to suggest that maybe it isn't just the skill of players that's progressively expanding, it's the canon itself - and the fact that the canon is more available than ever for anyone to see. The type of "good quizbowl" we're playing now hasn't been around all that long, maybe since 2005 - the standards of questions you see today are a relatively new thing. Plus, we have very recently come into an age where tourneys are all available on the internet, so everyone can read them - there's no more hoarding of secret archives of awesome packets for just you to practice on.

The point is that I think the canon of clues/answers is also growing progressively, along with the skill of players. So, it gets tougher and tougher to find appropriate clues for some answers these days - even relatively new players can quickly assimilate information these days.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

Magister Ludi
Tidus
Posts: 677
Joined: Thu Dec 01, 2005 1:57 am
Location: Washington DC
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Magister Ludi » Mon Jan 26, 2009 10:37 pm

Ryan is right. The regular difficulty tournament is entering murky waters where it wants to both be accessible to inexperienced teams yet be challenging for Seth and Jerry. Because the tournament is trying to challenge top teams, the editor it is wildly escalating the difficulty of third parts of bonuses to meet the demands of the ever expanding canon. The difficulty of bonuses cannot stay the same otherwise you'll have a situation where Dallas is 30ing the same bonus on Italian poets as a freshman as he will 3 years later when he plays Penn Bowl as a senior.

Ryan is once again right when he observes that there is a difference between player improvement and canon expansion. We are entering an era of unprecedented canon expansion as more and more quality players emerge. To illustrate this phenomenon I opened one random packet from Penn Bowl 2008. In this packet there was a bonus on American female poets with parts on Plath, Bradstreet, and Millay. In the 2009 edition of Penn Bowl the same exact bonus appeared except Bradstreet was replaced by Louise Gluck as the third part. In this same packet there is a tossup on Atwood with "Rape Fantasies" as the second clue. Now that early clue has become a third part of an Atwood bonus in the 2009 Penn Bowl. I would be willing to bet that "Rape Fantasies" would not have been used as a third part in the 2008 Penn Bowl. I'm sure I could find countless examples of this kind of canon expansion in just one year's time if I compared the bonuses from last year's regular difficulty tournaments and this year's regular difficulty tournaments.
Ted Gioia - Harvard '12
Editor ACF, PACE

User avatar
SepiaOfficinalis
Lulu
Posts: 40
Joined: Thu Oct 09, 2008 3:46 pm

Re: A Defense of the Muddy Battlefield Hypothesis

Post by SepiaOfficinalis » Mon Jan 26, 2009 11:01 pm

Maybe I'll have something substantive to say later, but in the meantime, I think that any tournament that produced such cogent and reflective comments from so many quizbowl luminaries is nothing but an unqualified success (but then, I liked this set too.) My contrarian instincts are defeated, so I'll just reiterate: youz all some nerdz (apparently an insult to some).
Grinnell Quizbowl 05-07
University of Minnesota Quizbowl 08-09.
T.....S...E...O
O.....O...R...L
M......D...H...M

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by grapesmoker » Mon Jan 26, 2009 11:36 pm

everyday847 wrote:I guess what I'm really curious about is what we think about how increasing tossup length can solve problems with transparency, at different levels of difficulty (and so what we want out of tossups at different difficulty levels). I think there are several competing interests here.
Maybe this is worth a separate thread, but it's important to remember that just like difficulty, transparency is a relative measure. In other words, what's transparent to me as someone with a lot of knowledge of what is and is not in the canon, as well as possible answer choices in general, is certainly not transparent to someone who doesn't know how to decide between the competing choices and may not even be aware of what the choices are. Take for example the tossup on Golden Bulls from Penn Bowl: if you have a decent background in European history, you know that there are basically a few commonly named declarations that have been issued at various times throughout history. Pragmatic Sanctions are common as are Golden Bulls, and since at least one of the clues mentioned that it outlined the privileges of the nobility, I was on pretty solid ground buzzing with Golden Bulls. But someone who doesn't know about multiple Pragmatic Sanctions or Golden Bulls wouldn't even know where to begin with that question, so obviously it would not be transparent to that person.

Lengthening tossups may or may not reduce transparency; while I'm a fan of longer questions, I think a better approach would be to find interesting new clues. I think virtually all of the problem questions at Penn Bowl could be rewritten with different clues, some of which may be much less known.
ACF Nationals is intended for only the elite, so tossups don't have to be long (i.e. you can have a giveaway that might be a Penn Bowl middle clue).
ACF Nationals is intended for only the elite, so tossups don't have to be short (with the side effect that they are good for a Chicago-Brown showdown as well as a Walter Johnson-Hunter).
ACF Nationals is intended to be hard, so tossups should not feature the easiest possible giveaway, so tossups should be short.
ACF Nationals is intended to be hard, so tossups should feature a lot of early, hard clues but still get converted, so tossups should be long.
Well, if you ask different people you'll get a different answer of what they think ACF Nationals is about. I think everyone believes that Nationals should fairly determine the best team in the nation, but some people (like myself, for example) also think that it's a fine opportunity to expand the canon into interesting places where it hasn't gone before. I tend to enjoy tournaments which have breadth as well as depth; for me, learning a new clue about Chester Arthur is not as exciting as hearing a tossup on Holderlin, even though I know a lot more about the former than the latter. For the record, I think that where ACF Nationals was in 2008 was about the right level of difficulty for where I think Nationals should be.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: A Defense of the Muddy Battlefield Hypothesis

Post by Mechanical Beasts » Tue Jan 27, 2009 12:09 am

grapesmoker wrote:Lengthening tossups may or may not reduce transparency; while I'm a fan of longer questions, I think a better approach would be to find interesting new clues. I think virtually all of the problem questions at Penn Bowl could be rewritten with different clues, some of which may be much less known.
Sure: when I say that lengthening tossups can combat the problem of transparency, I don't mean that it will make an individual clue any less transparent to me, to Ted, or to you. Rather, it will combat the symptoms so that the tossup gets transparent later, and many of the teams for whom clue n would be transparent have buzzed by n-1, and maybe most of the teams that are left are only good enough that clue n+1 is transparent, et cetera.

(Or you can frame things differently, because what we call "transparency" really denotes "transparency, to an excessive extent." In that case, lengthening tossups can allow those teams for whom clue n would have been excessively transparent to buzz on clue n-1.)

Either way, yeah, choosing different clues would combat the problem head-on: I accept that. I was mostly saying that even something so simple as adding another line or two of clues can help solve the symptoms.
Andrew Watkins

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: A Defense of the Muddy Battlefield Hypothesis

Post by No Rules Westbrook » Tue Jan 27, 2009 12:43 pm

Additionally, I think that writing good, non-transparent tossups on accessible answers that a number of players can buzz on before the giveaway is certainly possible. To produce such tossups, I think that editors need a combination of three things:

* Some amount of knowledge about the answer
* Some amount of knowledge about how well known certain clues are (not that you need to know every title, but a rudimentary understanding that The Great Gatsby is more well-known than The Last Tycoon, and so on)
* Finally, and most importantly, careful writing and editing (good research, making sure the question is worded in ways that won't frustrate teams, looking up previous tossups on the same answer to get an idea of which clues should go where, and so on)
Just to be a little more productive than I usually am with my crabby theory-heavy posts, let me say that Magin's above advice on what editors should do/have is very sound. If you can do these things, you can produce a very playable tournament that anyone can enjoy - you can be very responsible to whatever field you're writing for. Even if someone like me might note the inherent limitations of certain events, there's no doubt that we need editors to produce these events in order to develop a healthy circuit. And, I don't want people in the future to use my theoretical critique of the limitations of some events as justification for attacking well-edited and perfectly decent events which provide a great service.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

Locked