Rewarding Different Levels of Knowledge

[No Rules Westbrook]

Oh dear, another theoretical post that I'm sure will be a rocky ride. Anyway, I want to talk about the broad "levels of knowledge" that players have that should be rewarded in writing questions.

Let's use a very concrete example - a tossup on a book. I'll pick one that's not crazy layperson famous (Moby Dick or Crime/Punishment) but still indisputably famous - let's pick The Blithedale Romance. Very middle-of-the-road difficulty selection, I think most would agree.

Now, let's look at the different "levels" of knowledge people playing the tossup might have. First, and most obviously, you have the group of people who has read the book. You can make more fine distinctions amongst them - people who have read the book the most times, people who have read it most recently (which is maybe more important for knowing shit about it)...or you could posit an even higher level of knowledge like "people who teach classes on that book!" or "people who have written thesis papers on it!" but I'm being realistic here and not accounting for reasonably unlikely scenarios - let's just call this level "people who have read the book."

Now, every single good question writer on the planet tries to reward that level of knowledge over the lower ones, because they believe that such knowledge is more deserving of being rewarded than the lower levels. There's no argument here.

What's the next level of knowledge? If we're being realistic, the next level of knowledge you're likely to encounter in qb is something like "people who have written tossups on the book or studied something like masterplots for it, or done other substantial secondary studying about the book (where "secondary studying" means consulting a source other than those you'd typically consult in academic study of said book). Now, some people might try to hem-and-haw and say that there are levels of knowledge that come before this - that there's some intermediate more deserving kind of knowledge. Really? Okay, maybe there are one or two people out there who have read some literary criticism that mentioned The Blithedale Romance, or have had "academic discussions" about the plot of Blithedale Romance, but have not actually read The Blithedale Romance. But, first off, I don't think there are many people in this category with almost any subject. Secondly, it's not at all clear to me that those people "know more" in any rewardable sense than the people who have done extensive secondary studying (whether that means masterplots, a random internet page, writing questions, reading packets, etc.).

And, of course, you have a third level of knowledge held by people who have cursorily absorbed a few things about the book.

Now, I've begun to see an alarming trend in qb of people who unduly chafe at the prospect of rewarding the people in the second "tier of knowledge" above. I suspect that it's because, as revealed in the previous hsqb thread, a lot of people really hate the idea - yes, despise the very thought - of rewarding the "qb robot." And the robot lives on Tier 2.

Let's think about how small the group of people with Tier 1 Knowledge usually is. Even given an awesome Chicago Open-like field, what percentage of the tourney's participants would you imagine fall into Tier 1? - that is, what percentage have read The Blithedale Romance? I'd guess well under 10 percent. And, Blithedale Romance is very middle-of-the-road....with harder topics, that percentage surely goes down markedly. My point is that distinguishing amongst Tier 2 Knowledge is so often what decides matches - in fact, I'd posit that its influence is dominant in qb. I don't think this is anything to bewail, but rather the nature of the game.

For a long time, it seemed like it was assumed that Tier 1 Knowledge would be protected by admonishing writers to write about things that are truly academically important - that is, we tell writers "make sure that Topic X is studied in the academy to a sufficient degree to merit it being a tossup - especially if you're advancing the canon." This is protection because it ensures that tossups will be on things that could reasonably be acquired through Tier 1 Knowledge - that is, Tier 1 Knowledge is reasonably out there for the taking. I zealously attempt to follow the above admonishment - i.e. I write on Terrence Rattigan because I can make a very good claim that his works are sufficiently studied and well-known to academics. But now, it seems that the "Tier 1 Champions" want some other layer of protection - some additional privilege for Tier 1 Knowledge, some added insurance against the Tier 2 Robots. Some would apparently have me deliberately make sure that Tier 1 buzzes account for a certain percentage of buzzes at a tournament - deliberately attempt to tilt the game in one direction. I think that's a fool's errand.

We should write on legitimately important stuff and structure every tossup so that Tier 1 beats Tier 2 beats Tier 3. If you achieve that, you've written a good question.

If you have Tier 1 Knowledge about an academically important thing, that's great, and you'll almost certainly get the tossup if it's written well. If you don't (and most of the time you won't, frankly, because Tier 1 Knowledge is scarce and time-consuming to obtain), then you fight a battle on Tier 2 Knowledge. I'm tired of the increasingly prevailing view that Tier 2 Knowledge is second-rate "fakeism" to be condemned - quite to the contrary, it's the most influential and characteristic portion of this game.

[grapesmoker]

I think in this discussion it would be useful to look at different categories differently. To use Ryan's example, or indeed the example of any book, at some point you're going to have to mention that some dude named Miles Coverdale is a character in this book. As I was saying in another thread, when you write that tossup on the war of the Spanish Succession, at some point you'll have mentioned Malplaquet or something similar, and people who just remember shit can buzz. I think this is normal and part of the game, but I don't quite understand Ryan's accusations against whoever these "Tier 1 champions" are or what it is that those champions are demanding. I think this is most relevant to the science discussion: there has been an explosion in questions that draw on all sorts of ridiculous clues that aren't even remotely accessible to non-wikipedia-reading scientists, so understandably people like me are upset about that. You'd be upset too if you'd spent years studying something but you still couldn't win buzzer races against people who just memorize some associations.

On the other hand, in literature I think this has been less of a problem. There are some unfortunate tendencies in question writing but I think they are more a problem of answer selection (as in the widely bemoaned "every tournament must have questions on tertiary Oe works," situation). Most literature questions are written pretty well, and I don't think anyone is saying "don't write tossups on The Blithedale Romance because you'll be rewarding the Tier 2 robot" or whatever. It's more like, people are saying, "Don't write tossups on The 40 Days of Musa Dagh because no one has read that and the tossup will basically come down to a question of who has heard of that book."

[No Rules Westbrook]

Yeah, I can certainly sympathize with opposing the "unbuzzable to anyone but wiki readers" sorts of clues that sometimes show up in the top half of tossups. That's not valid rewarding of Tier 1 Knowledge, and shouldn't happen.

[Mechanical Beasts]

Yeah; for example, tossups that start off with The Infant with a Melancholic Face and then cover one work per line until the tragic end are going to be rewarding tier three knowledge all the way. It's reality that people will do more efficiently by reading a plot summary of The Day He Himself Shall Wipe My Tears Away than by reading it (and it's hilarious that we're at the point where I'm saying such things at all!) and if we had the stomach for thirty line tossups, you know, tossups that rewarded tier two knowledge of nine works would be okay. But we don't, so we favor rewarding tier one and two knowledge of a few works.

[No Rules Westbrook]

To be very clear, I also hate descriptions of works that provide nothing buzzable, which is a common recent trend. You may as well just give a list of titles if the descriptions separating those titles aren't buzzable in and of themselves. Replacing useless perfunctory descriptions of stuff with dense buzzable descriptions is something we can all get behind.

[grapesmoker]

Oh yeah, Andy makes a good point. Too often, a lot of questions just expand into basically lists of works by a dude (this goes for philosophy, for literature, for social science, etc.). I think in some situations this is ok, but in many situations it's a symptom of overuse. Like, if you sat down to write a Melville tossup this way, you'd spend the first 4 lines talking about poetry from Battle Pieces and for most tournaments that would suck. You don't always have to go into crazy depth on author titles; you can write a perfectly serviceable question by describing a minor work or two and then moving on to two or three more major works. That's not to say you can never write questions where you do give more minor works, but if you find that happening too often, it probably means that you've reached the limits of the useful knowledge base about that author and maybe it makes more sense to move on to one of the author's works or a different author entirely.

[Cheynem]

Unless you're doing something oddball, like trying to reward knowledge of, say, poetry of a guy better known for novels, I've come around to the belief that good author tossups should only focus on like two works. This allows for more room to offer up deep clues and reward knowledge rather than using perfunctory descriptions of works that turn into title or character bowl.

[Mechanical Beasts]

I agree, Mike, so long as we're talking about finished-product tossups.

From the perspective of "I'm sitting down and about to write a tossup for a packet-sub tournament," an exception might be made for subjects you're not too familiar with. So, for example, what if I (before learning my lesson after Sun 'n' Fun) wrote a tossup on Jamaica Kincaid that spent its second half talking about "Girl" because of my being 100% convinced that that was her best known work? Maybe the first half was split between Annie John and Lucy or something. Well, then even if I'm writing a tossup 150% the length of the desired finished product, my work is 25% too short because half of my tossup might be just plain unusable for the difficulty level.

I think that there's something to be said, in submitting extra-long tossups for editors to cut from, for writing "five-thirds" of a tossup, for example. If you think a good tossup on this author draws on three works, then write five possible thirds of the tossup, order them in the way that makes sense to you, and hope that your high school English teachers didn't have a fetish for fucking with your sense of difficulty. If you've written on two works that are outside the desired difficulty entirely and misordered the other two, the editor still has enough work.

[Captain Sinico]

I think you also need to consider what fraction of your questions will be answered by people with real knowledge. If you're writing questions such that 90% of them are only doing to be answered based on fake knowledge, which is what you suggest to be the case for Chicago Open, I think you're writing poor questions. It is necessary to modulate your questions to try and hit what people in a field like that are likely to really know.

MaS

[grapesmoker]

Unless you're doing something oddball, like trying to reward knowledge of, say, poetry of a guy better known for novels, I've come around to the belief that good author tossups should only focus on like two works. This allows for more room to offer up deep clues and reward knowledge rather than using perfunctory descriptions of works that turn into title or character bowl.

I think you can do it both ways, and a lot of time, it comes down to aesthetic differences. I feel uncomfortable saying to anyone that they should never write on an author using more than a few works; maybe in some cases it's not a bad idea to expand your clue space a little bit. I just think you have to be careful with it; if you find yourself repeatedly writing about some author in a particular style, either change your topic or change your style (or maybe both).

[No Rules Westbrook]

Well, as a quick response to Sorice, I'd say that "what people really know" isn't limited to solely what they know through your preferred methods (which I assume to be academic study). I contend that it's sufficient protection for the preferred methods to simply make sure you write about important things (as I defined above).

Also, the notion of only writing about two works in an author tossup is pretty flawed. If we did that, interesting minor works of authors would never come up - it's pointless to go prattling on for four lines about a fairly minor work. Just give a meaty one/1.5 line description (such that people can buzz if they know the main plot) and then drop the title if it's minor enough.

With the size of the canon today, I can see why younger players may want to look for ways to get some immediate bang out of their buck...rather than ponying up and expending the enormous effort to conquer the burgeoning runaway canon. But, you know me, I'm all about rewarding those who pony up.

[Matt Weiner]

With the size of the canon today, I can see why younger players may want to look for ways to get some immediate bang out of their buck...rather than ponying up and expending the enormous effort to conquer the burgeoning runaway canon. But, you know me, I'm all about rewarding those who pony up.

It takes more effort, and more legitimate effort at that, to learn all the potential clues for Moby Dick than to scribble down two sentences about Pierre. Constantly rewarding surface familiarity with minor works first is a dead end approach to lit writing.

[Mechanical Beasts]

If we did that, interesting minor works of authors would never come up - it's pointless to go prattling on for four lines about a fairly minor work.

What! There's nothing interesting about the coming-up of an interesting minor work if the only capacity in which that happens is trivial. So if we're writing for a level at which there are three lines of acceptable clues about a work, then include it. Otherwise, don't. I don't see why this is an unacceptable algorithm (though I see Jerry's point about trying to motivate going into new cluespace, and I don't have much of a problem with that).

[grapesmoker]

There's a reason minor works are minor. I'm not saying they should never come up, but I think it's much better to reward someone who has read several major Faulkner works than someone who vaguely knows about the existence of Pylon. Even at higher levels of the game, I think we should generally err on the side of including major works from minor authors rather than the other way around. I personally would even prefer to see fewer author questions and more work questions, since it's harder to fraud your way into getting a question early on a book you haven't read.

[setht]

As usual, I'm in agreement with Mike S. and Jerry on this stuff, and in vehement disagreement with several of Ryan's points.

Where do I agree with Ryan? I agree with him that good tossups always try to reward Tier 1 knowledge, then Tier 2, then Tier 3. I agree with him that tossups that stick to Tier 1 clues for many sentences are bad. I agree with him that the number of people who have Tier 1 knowledge of something like The Blithedale Romance is probably pretty low.

I think I disagree with Ryan on pretty much everything else. I'm going to abandon Ryan's "quizbowl robot" because I think it's a useless construct that does not provide an acceptable model for the sort of players we're actually talking about, which I will label "people who only study old packets." I think people who only study old packets are actually limited to Tier 3 knowledge of works like The Blithedale Romance (the number of works at the level of The Blithedale Romance that appear in old packets being so large that no one can know, on average, all the secondary character names or important plot points from such works just from working through old packets). Therefore, most tossups on works like The Blithedale Romance get a small number of Tier 1 buzzes, a larger number of Tier 2 buzzes from people who did some studying on the particular work, and then the people who only study old packets generally get their crack at the tossup slightly ahead of people who don't do any secondary studying outside of class and haven't encountered the work in question in class. This is fine. The same people who only study packets may well have something like Tier 1 or Tier 2 knowledge of authors because it's easier to memorize a couple obscure titles per author than it is to memorize secondary character names and important plot points from all the works at the level of The Blithedale Romance. I say "something like Tier 1 or Tier 2 knowledge" because I think looking through old tossups and latching onto obscure titles does not constitute the sort of "substantial secondary studying" Ryan mentions above; however, on an author tossup the people who only study old packets could well pull a buzz during the portion typically considered Tier 2 clues (or earlier), if the tossup is written to reward title knowledge. In short, I'm saying that when it comes to literature tossups as they are typically written, questions on canonical or semi-canonical works are much less likely to reward people who only study packets than questions on authors.

Science questions in general seem more problematic than literature questions (especially literature questions on works). Part of the problem is lazy question-writing where people lift answer ideas, clue ideas, and clue phrases wholesale from old packets. Unsurprisingly, this is very rewarding for players who only study old packets. We've had plenty of discussion on how to write science questions elsewhere so I'll leave it alone here.

I agree with Mike and Jerry that the real issue here is answer selection. I think Ryan generally does a good job of choosing Tier 1/2/3 clues and putting them in proper order, but I'm less happy with his answer selection predilections. Let's take Ryan's example of Terence Rattigan. Is he "academically important"? Let's say yes, although I was unable to find any mention of Rattigan in the past 4 years of course descriptions (undergraduate and graduate) in Berkeley's English department. Is Rattigan's academic importance enough to make him a good tossup answer selection at a hard tournament? I'm not so sure about that. There are many, many things that are academically important (e.g. large swathes of earth science) that I am confident 0-1 quizbowlers in the country would be capable of answering at all, let alone displaying Tier 1 or Tier 2 knowledge. At the same time, I can think of some academically important things that hardly come up in quizbowl that I do suspect some quizbowlers could answer and some might have good knowledge of--for instance, instead of Rattigan, what about delving deeper into the oeuvre of some important writer? Perhaps we could ask about a Robert Browning poem that doesn't come up often (or at all) like "The Last Ride Together" or "A Toccata of Galuppi's" or "How They Brought the Good News from Ghent to Aix." If we're worried about writing tossups for hard tournaments on things that come up in old packets, Rattigan gets 4 hits on packetsearch vs. 0 for Last Ride, 6 for Galuppi, and 3 for Good News. Now, which of the following sets of quizbowlers is larger: the set of all quizbowlers with Tier 1 or Tier 2 knowledge of Rattigan, or the set of all quizbowlers with Tier 1 or Tier 2 knowlede of "A Toccata of Galuppi's" or "How They Brought the Good News from Ghent to Aix" (I think "The Last Ride Together" is great but I suspect the other two poems are more likely covered in courses)? Given that there are way more courses that cover some amount of Browning's poetry than there are courses that talk about Rattigan, I'm guessing it's the latter group. Would these Galuppi/Good News-type tossups be challenging and innovative and cool? I think so; in fact I think they'd be more exciting than tossups on Rattigan, and I wish people (and in particular Ryan) would look more to this sort of stuff when searching for harder tossup ideas.

I won't argue that Rattigan should never come up, nor that he should never come up as a tossup. I will argue that the number of Rattigan-type tossups should be outweighed by the number of Galuppi/Good News-type tossups, and that both should be outweighed by the number of Blithedale Romance-type tossups--even at ACF Nationals. My main beef with Ryan's writing is that I think he has too many of the first kind of tossups and too few of the second and third.

-Seth

[No Rules Westbrook]

Learning the crap out of Moby Dick is no different than learning a little bit about Pierre and The Moon of the Caribees and so on. Unless you mean actually reading the book, in which case, yes - but realistically, actually reading books will always be moderately inefficient (though it has its benefits, no doubt). That's just kinda how qb is.

I was suggesting that I can see why younger players in particular sometimes drive these discussions the way they do. The notion of conquering the current canon head on is daunting at best.

[Cheynem]

No, learning a lot about Moby Dick is different than learning a little bit about Pierre. Learning a lot about Moby Dick involves careful study, probably even reading the damn thing. Learning a little bit about Pierre is easy--skim through Wikipedia or Masterplots, enough to pick out the clues you'd use in your 1-1.5 description of Pierre. I'd much rather see a question reward deep knowledge of Moby Dick than Wiki-browsing Pierre. Unless of course it is a tossup actually on PIerre.

[grapesmoker]

I am not sure about that, Seth. Maybe there's just a lot more Browning knowledge out there than I'm aware of, but I suspect a tossup on "Good News," or "The Last Ride Together," would see rather poor conversion. I'm no Browning enthusiast, but I've read a few of the more well-known of his works; he's written a lot of poetry and once you get beyond some of the more major things, I'm not sure people know all that much about it.

Learning the crap out of Moby Dick is no different than learning a little bit about Pierre and The Moon of the Caribees and so on. Unless you mean actually reading the book, in which case, yes - but realistically, actually reading books will always be moderately inefficient (though it has its benefits, no doubt). That's just kinda how qb is.

The whole point is not to talk about what qb is like but what it should be like. I'm making the claim that we should reward the person who reads Moby Dick over someone who knows that Pierre is a book and was written by Melville. And of course I don't give a damn about efficiency either way, as it's entirely beside the point.

[No Rules Westbrook]

And anyway, part of the point of this thread is to demonstrate that - you can only read so many books. You've either read a book or you haven't - and if you haven't, what your left with is Tier 2 Knowledge. There's no magical "in between Tier 1 and 2 knowledge" in practice - you've either read shit or your stuck trying to get it other ways. Far more often than not, you're stuck trying to get shit other ways.

[grapesmoker]

And anyway, part of the point of this thread is to demonstrate that - you can only read so many books. You've either read a book or you haven't - and if you haven't, what your left with is Tier 2 Knowledge. There's no magical "in between Tier 1 and 2 knowledge" in practice - you've either read shit or your stuck trying to get it other ways.

But... so what?

[No Rules Westbrook]

So, people need to recognize what a gargantuan role Tier 2 Knowledge plays in this game. It's fun to boast about those few tossups you got cause you "totally read that work! and it was awesome!", but let's not forget about the 11 other tossups that round you got off of very imcomplete secondary kinds of knowledge.

[Matt Weiner]

So, people need to recognize what a gargantuan role Tier 2 Knowledge plays in this game. It's fun to boast about those few tossups you got cause you "totally read that work! and it was awesome!", but let's not forget about the 11 other tossups that round you got off of very imcomplete secondary kinds of knowledge.

I don't disagree with your stated premise, since I think quizbowl is neither a substitute for in-depth classwork, nor an exercise in memorizing binary associations, but something in between where you learn the important facts about important things...or what you call Tier 2.

I disagree with the notion that there's anything Tier 2 about learning to buzz in with Melville when you hear "one of his title characters is a Mr. Glendenning." That's Tier 3. Learning some more advanced things about Moby Dick is Tier 2.

[No Rules Westbrook]

Sure, we can argue about where the boundary is between Tier 3 and Tier 2. Glendinning alone for Pierre is surely 3. Reading a Masterplots-type thing or a description of the plot in a literature encyclopedia or something is Tier 2.

There don't seem to be enough people championing learning in a Tier 2 kind of way today, but rather just Tier 1.

[Matt Weiner]

Sure, we can argue about where the boundary is between Tier 3 and Tier 2. Glendinning alone for Pierre is surely 3. Reading a Masterplots-type thing or a description of the plot in a literature encyclopedia or something is Tier 2.

OK, and so long as the one-sentence clue for Pierre at the start of the Melville tossup is all you need to know, no homo economicus quizbowler is going to learn the full plot of Pierre when he could spend that time more efficiently. Either you keep rewarding people for Tier 3 knowledge of Pierre, or you start taking half your Melville tossup to talk about Pierre so that people need to learn more, or you stop putting clues about Pierre before deep clues from Moby Dick. Options A are B are things you should not do because they're stupid as hell. Do C.

[Mechanical Beasts]

Sure, we can argue about where the boundary is between Tier 3 and Tier 2. Glendinning alone for Pierre is surely 3. Reading a Masterplots-type thing or a description of the plot in a literature encyclopedia or something is Tier 2.

There don't seem to be enough people championing learning in a Tier 2 kind of way today, but rather just Tier 1.

Well, why "champion" that? Isn't everyone's ideal (which we of course can't live up to) achieving first-tier learning? Like, as you said explicitly, most people are stuck with tier two, and strive for tier one. Does second-tier learning need champions?

[No Rules Westbrook]

Yeah, I'm all about championing Tier Two. Maybe that's the whole point of this thread.

Tier Two is the path of the career quizbowl player. People who have a small pool of Tier One Knowledge, and a large cache of medium-depth secondary knowledge on a wide range of subjects. That's the veteran super-generalist - which I'd be happy to defend as the ideal of this game.

[Mechanical Beasts]

Yeah, I'm all about championing Tier Two. Maybe that's the whole point of this thread.

Tier Two is the path of the career quizbowl player. People who have a small pool of Tier One Knowledge, and a large cache of medium-depth secondary knowledge on a wide range of subjects. That's the veteran super-generalist - which I'd be happy to defend as the ideal of this game.

I think it is.

See, I think that Tier Two is inherently a compromise: it's all of our compromise, sure. But we strive for primary knowledge for everything first, and then we settle for mid-range knowledge from reading plot summaries and secondary sources. I don't think the ideal of this game should be "the guy who keeps to whatever primary knowledge he falls into and compromises on everything else to maximize his point-earning-fact-learned-per-minute efficiency." The goddamn game is about learning.

[Birdofredum Sawin]

I started to read this thread, but despaired only a few posts in. Regardless, I'll weigh in and suggest that what I take to be Ryan's central dichotomy -- between "Tier 1" knowledge (e.g. "I read Moby-Dick, now stand back while I buzz on this lead-in describing Chapter 37 of the novel") and "Tier 2" knowledge (e.g. "I know this from having played quizbowl forever, now stand back while I buzz on this middle clue based on my memory of a 2006 ACF Nationals bonus") -- is overly simplistic. To put it simply, there's a lot of knowledge which falls between these two "tiers."

Take literature, both because other people are and because it's an area I can claim to know pretty well. Yeah, I have "Tier 1" knowledge of a lot of books (i.e., "I have actually sat down and read them"). And I have tons of "Tier 2" knowledge. But I would actually say that a significant percentage of my lit answers come not from having read authors/works themselves, but from being generally interested in literature and literary criticism -- which is to say, by reading a diverse body of "stuff about literature" which includes "professional" literary criticism, the New York Review of Books/TLS, popular biographies of authors, etc.

A question which leaps to mind in this context is the tossup on "Augie March" from Magin's first lit singles tournament, whose lead-in was something like "This is Christopher Hitchens's favorite American novel." You wouldn't know that from reading the novel itself, or from quizbowl lore -- unless, of course, that was a recycled clue -- but you could know it from reading Hitchens's literary criticism, or more generally from knowing the great esteem in which Bellow in general and that novel in particular are held by Hitchens and his generation of English writers. That's the kind of thing which, I submit, is worth knowing, but which is omitted from Ryan's perspective. More generally, the whole phenomenon of "taking an interest in a subject (but not in a quizbowl-centric way)" seems to be missing from Ryan's worldview.

[Skepticism and Animal Feed]

Let me make my pet point here: I dispute that quizbowl players, especially top players, gain much of their knowledge from an academic setting. Seth Teitler and Jerry Vinokurov, as science grad students, probably haven't taken a class on literature or history for over half a decade, and yet they have both dramatically improved in both of those subjects over that time.

How? Having been to Jerry's apartment, I suspect I know the answer. His apartment is littered with books on history, philosophy, religion, etc., as well as novels. He reads these books in his spare time and obtains knowledge from them. I don't know how he picked these books, but I bet he wasn't assigned them in a class and probably didn't take them from a syllabus.

If players are in fact gaining most of their Tier 1 knowledge from reading things in their spare time, and not from taking classes, then this focus on "what is academically important" and "what comes up in a Berkeley syllabus" is horribly misguided.

[No Rules Westbrook]

Bruce is, of course, right. The all-too insistent focus on "the academy" is often very discordant with the reality of most knowledge bases.

[Skepticism and Animal Feed]

People are happy to condemn the wrong-headed notion of "quizbowl should test what one learns in school" while arguing against the inclusion of computational math, but somehow as soon as the conversation shifts to this context, it becomes sacred.

[Mechanical Beasts]

People are happy to condemn the wrong-headed notion of "quizbowl should test what one learns in school" while arguing against the inclusion of computational math, but somehow as soon as the conversation shifts to this context, it becomes sacred.

Presumably people are saying that "learned in the academy" is a necessary (or very close to it) condition but not at all a sufficient (or very far from a sufficient) one.

[marnold]

That's the veteran super-generalist - which I'd be happy to defend as the ideal of this game.

Just out of curiosity, what do you then make of the fact that every super-generalist in the game today not named Brendan "Beep-Boop" Byrne objects to this philosophy and the writing styles it produces, in some cases (Sorice and Seth foremost) really strongly. It seems odd for such unanimous disapproval from the group that you supposedly champion.

[Sargon]

I also heartily agree with Bruce. In my experience, the emphasis on "academic importance," leads down one of two very unpleasant roads. First it can and has in recent years lead to questions with several lines of clues that are effectively unbuzzable, a waste of time and effort for everyone involved. The extreme form of this is picking tossup answers that are too hard so that a tossup is basically unanswerable. In that case people with significant, deep knowledge from private study fare the same as people who know nothing at all on the subject. Second, choosing "academically important" answers that few people who are not graduate students in the field could legitimately have encountered perversely rewards "fraud" knowledge and punishes actual learning, since the only way they can be answered is by rote memorization and word associations.

As I have argued before, I think people should write questions aimed at rewarding things people could conceivably know from unguided independent study. With a few exceptions, I am convinced that this excludes secondary literature clues, since even if people read commentaries and whatnot, they will almost certainly not know which ones are considered "important." I don't dispute that a classics PhD might be helped by a clue about a clue about an authoritative commentator on the Aeneid, but I am hard pressed to think that anyone but a classics PhD would know which commentaries are "authoritative." It would be a better use of the space to reward things about the poem that someone might know based on reading any serious commentary.

On that note, I still am not convinced that works tossups reward serious knowledge better than author/creator tossups. A tossup on an author with details from several works maximizes the chance that someone will answer it based on having actually read something mentioned in it. The key is simply to vary the minor works and what is described about them so that it is not easy to fraud with word associations, creating what is in effect a few short works tossups spliced together. Such tossups reward different levels of type 2 knowledge, and also ensure that the widest range of knowledge (from "I have read every word he wrote 50 times" to "I have vaguely heard of this guy") is rewarded.

[cvdwightw]

I think people are making the wrong distinction between "quizbowl" and "the university." The correct distinction is:

"If someone who has studied this subject at a given university level can answer a question or can buzz off a certain clue, then that question/clue is appropriate."

The association is NOT:

"If someone who has studied this subject at a given university level cannot answer a question or cannot buzz off a certain clue, then that question/clue is inappropriate."

Obviously, we make a very wide allowance for "people who might have Tier 1 knowledge from other, non-university sources," e.g., if someone buzzes on a history clue because it's mentioned in a work that person's read, it qualifies as Tier 1 knowledge even if that person has no "real" Tier 1 history knowledge. Every good question writer needs to tacitly acknowledge that university knowledge is a large subset of Tier 1 knowledge.

At some point "shallower" Tier 1 knowledge and "deeper" Tier 2/3 knowledge start to intersect - I enjoy buzzing on the antics of the few characters I still remember from having read Miguel Street a few years ago, but that doesn't mean someone with "deep" Tier 2 knowledge (learned it from old packets, etc.) can't buzz before me off the antics of a character I don't remember.

[No Rules Westbrook]

Well, Marnold, though it's certainly popular to pigeonhole my rather well-known beliefs into a box and claim you flatly oppose them - I think there are several views on this. People like Seth, Sorice, Magin strongly oppose my take (Magin doesn't have a traditional supergeneralist game at all, but close enough). To varying degrees, people like Jerry, Weiner, Lafer have views in the middle - that is, increasingly in the order I mentioned them, I think those players have more respect for Tier 2 kinds of knowledge than the players in the previous group. And then you have Brendan and I, of course. I'm not sure about where others might fall on the spectrum, but I do think there's a continuum amongst generalists - a progression of very different attitudes toward Tier 2 Knowledge and how much we should reward it.

[Matt Weiner]

I fucking love Tier 2 knowledge, I just think you're doing a pretty poor job distinguishing between "Tier 2 knowledge" and "Tier 3 knowledge about hard things." Without some more rigorous exploration of these terms you made up, people are going to fall right back into the "harder = realer than!" fallacy.

[No Rules Westbrook]

Also, I'll note a second argument that I'm making here: that the writing style promoted by the strongest advocates of Tier 1 Knowledge is unfeasible. It simply can't be done well within the parameters of this game. That's my pet argument "some people want quizbowl to be a game it fundamentally is not."

But, that's a more abstract hypothesis and, whatever, fairly difficult to prove either way.

[Charbroil]

...the emphasis on "academic importance," leads...[leads] to questions with several lines of clues that are effectively unbuzzable...[or]...choosing "academically important" answers that few people who are not graduate students in the field could legitimately have encountered.

Now, I'm not knowledgeable at all about question writing at the collegiate level, but try as I might, I don't understand the logic behind this statement--isn't the issue these days that people aren't picking academically important topics in favor of whatever has a name that they can find in Wikipedia (even if that topic is impossibly hard to buzz on)? I don't understand how academically important topics would be more likely to be impossible to answer, especially given that people should be more likely to have primary knowledge of those topics.

Obviously, this assumes that people remain consistent in picking academically important topics of the right difficulty--but if people do that (by, say, using course notes, etc.), aren't they more likely to pick less difficult topics that come up frequently in their sources (course notes, syllabi, etc.)

[Birdofredum Sawin]

So I assume that Bruce's "pet point" in this context was either unrelated to my post, or in fact was lining up behind it? (Inasmuch as reading things like "the New York Review of Books/TLS, popular biographies of authors, etc.," as discussed in my earlier post, has nothing whatsoever to do with "academic settings.") And certainly my erstwhile, quixotic attempts to roll back the takeover of the science category by "stuff science players learn in their classes, and nothing else" coheres with what Bruce is saying.

My point here is that Ryan's simple-minded dichotomy doesn't really capture much about the game, or how it is that many of the top players of my acquaintance know the stuff they know. To offer another example: In our heyday, Zeke was one of the best art players in the game (if not the best), while I was one of the best music players in the game. Neither of us had any interest (per Bruce) in art or music as they are studied in the "academic setting." Zeke just loved art, and knew a lot about it: he took an interest in the subject, frequented museums, looked at non-academic books on various artists, etc. Ditto for me and classical music. I feel that it's that kind of knowledge -- the kind which derives from a serious, if non-professional, interest in a given field -- which is scanted by Ryan's curious "tiers." I also think that that kind of knowledge is one of the great things about the game. That is, playing quizbowl can lead people to become curious about areas which they might never think about otherwise (e.g. art, or mythology, or whatever), and then to develop a passion about those areas. That passion isn't synonymous with "scouring packet archives for questions that have been written on those areas," though Ryan's crude differentiation of knowledge suggests otherwise.

[grapesmoker]

Also, I'll note a second argument that I'm making here: that the writing style promoted by the strongest advocates of Tier 1 Knowledge is unfeasible. It simply can't be done well within the parameters of this game. That's my pet argument "some people want quizbowl to be a game it fundamentally is not."

People actually do write this way though. Just to give an example, look at the science at TIT/IO. A lot of it was really good and did a good job of rewarding people with knowledge. Why can't we do this for other categories as well?

[Skepticism and Animal Feed]

My point here is that Ryan's simple-minded dichotomy doesn't really capture much about the game, or how it is that many of the top players of my acquaintance know the stuff they know. To offer another example: In our heyday, Zeke was one of the best art players in the game (if not the best), while I was one of the best music players in the game. Neither of us had any interest (per Bruce) in art or music as they are studied in the "academic setting." Zeke just loved art, and knew a lot about it: he took an interest in the subject, frequented museums, looked at non-academic books on various artists, etc. Ditto for me and classical music. I feel that it's that kind of knowledge -- the kind which derives from a serious, if non-professional, interest in a given field -- which is scanted by Ryan's curious "tiers." I also think that that kind of knowledge is one of the great things about the game. That is, playing quizbowl can lead people to become curious about areas which they might never think about otherwise (e.g. art, or mythology, or whatever), and then to develop a passion about those areas. That passion isn't synonymous with "scouring packet archives for questions that have been written on those areas," though Ryan's crude differentiation of knowledge suggests otherwise.

My "pet point" is that this process you describe is exactly how most players acquire most of their knowledge, and I think that quizbowl theory should start with this model of knowledge acquisition in mind if it wants to reach non-harmful results.

[Mechanical Beasts]

My "pet point" is that this process you describe is exactly how most players acquire most of their knowledge, and I think that quizbowl theory should start with this model of knowledge acquisition in mind if it wants to reach non-harmful results.

I must be misunderstanding. Why do we, then, ask for academic literature in the distribution? Why don't we just ask for "write on books that you enjoy reading/ you think a typical quizbowler would enjoy reading?"

Alternatively, I guess--in what other was would quizbowl--in theory or in practice--adapt to the model of knowledge acquisition that quizbowlers explore culture sometimes without the aid of a curriculum?

Moreover, I feel like I always have the aid of a curriculum whenever I need or want guidance because I can rely on what I hear come up frequently to be academic by some external standard.

[Sargon]

...the emphasis on "academic importance," leads...[leads] to questions with several lines of clues that are effectively unbuzzable...[or]...choosing "academically important" answers that few people who are not graduate students in the field could legitimately have encountered.

Now, I'm not knowledgeable at all about question writing at the collegiate level, but try as I might, I don't understand the logic behind this statement--isn't the issue these days that people aren't picking academically important topics in favor of whatever has a name that they can find in Wikipedia (even if that topic is impossibly hard to buzz on)? I don't understand how academically important topics would be more likely to be impossible to answer, especially given that people should be more likely to have primary knowledge of those topics.

Obviously, this assumes that people remain consistent in picking academically important topics of the right difficulty--but if people do that (by, say, using course notes, etc.), aren't they more likely to pick less difficult topics that come up frequently in their sources (course notes, syllabi, etc.)

People writing on random things they found on Wikipedia is a problem, but people seem pretty well aware of that. Most people do it, I imagine, because they are having to write questions in an area they don't know well, and have no conception of what is or is not well known. However, my point here is that what people encounter in academic study and what they encounter in independent study often do not overlap and that we should, by and large favor the latter since that is where the vast majority of quizbowlers are in most subjects. Striving for "academic importance" has led to a large influx of abstract theory and secondary literature clues in quizbowl in recent years, which seems on the whole ill-advised. This is particularly true in the social sciences. Questions rarely reward knowing about particular societies or languages, for instance, but often reward knowing about ethnography and theoretical linguistics. There are a few minor ethnic groups particularly Polynesian tribes that come up regularly because they are the subjects of famous ethnographic works, but knowing a thing or two about Hungarians or Siberian tribes, or that Pashtun is an East-Iranian language will never get you points. Certainly with languages I have met far more people with a basic knowledge of interesting features of particular languages and their relationships to other languages than people with a basic knowledge of linguistic theory. It is important that what comes up be academically important, but not all academically important things should come up, even if you would learn them in a lower level course than things which do come up.

[Skepticism and Animal Feed]

I must be misunderstanding. Why do we, then, ask for academic literature in the distribution? Why don't we just ask for "write on books that you enjoy reading/ you think a typical quizbowler would enjoy reading?"

One can divine what is important in a given field without having any academic exposure. Read a few books about US history, and you will figure out that this "Andrew Jackson" guy seems more important than this "Levi P. Woodbury" guy. I'm sure the same is true for composers, for artists, for anthropologists, etc. Multiple people performing the same analysis will come to similar conclusions.

People who are truly knowledgeable in a subject will be able to figure out what is important. These people will serve as editors and will guide the canon in an appropriate way.

[Skepticism and Animal Feed]

but knowing a thing or two about Hungarians ... will never get you points

It's only been three days since RMPFest...

[grapesmoker]

One can divine what is important in a given field without having any academic exposure. Read a few books about US history, and you will figure out that this "Andrew Jackson" guy seems more important than this "Levi P. Woodbury" guy. I'm sure the same is true for composers, for artists, for anthropologists, etc. Multiple people performing the same analysis will come to similar conclusions.

People who are truly knowledgeable in a subject will be able to figure out what is important. These people will serve as editors and will guide the canon in an appropriate way.

There seems to be some notion that "academic importance" is dictated by some esoteric process only accessible to people within academia. But Bruce is correct here; we know that Andrew Jackson is more important than Levi Woodbury, and we know this because we're literate, educated people who read things and have an interest in history. Academic significance is an important metric for assessing what to write about and what kinds of clues to pick because it weeds out a lot of junk, but it's not the exclusive criterion by which it is decided what to write about.

[grapesmoker]

Take literature, both because other people are and because it's an area I can claim to know pretty well. Yeah, I have "Tier 1" knowledge of a lot of books (i.e., "I have actually sat down and read them"). And I have tons of "Tier 2" knowledge. But I would actually say that a significant percentage of my lit answers come not from having read authors/works themselves, but from being generally interested in literature and literary criticism -- which is to say, by reading a diverse body of "stuff about literature" which includes "professional" literary criticism, the New York Review of Books/TLS, popular biographies of authors, etc.

This is a useful and important point. I read a lot of various periodicals and web sites with an academic bent and that's resulted in quite a few buzzes for me on various topics, including literature. I think having a non-professional interest in a topic is an important thing that should be rewarded. For example, I'm often rewarded for reading philosophy on my own, despite the fact that I'm not a philosophy student nor do I have any formal training in the field. But I do spend a lot of time seeking out things that are interesting to me and reading about them (even if I don't always manage to read the primary literature as well) and that gets me a lot of good buzzes.

I think this is in line with what I was trying to say in this and other threads about rewarding people who go out and read two or three major Faulkner works rather than mine Masterplots for additional Faulkner titles. I mean this in terms of incentives. If people see that being genuinely interested in literature and actually reading things gets them points, that shifts the incentives towards reading things. I know I've become interested in a lot of stuff that I've only learned about by way of quizbowl and I think we should reward that kind of intellectual curiosity. I don't think that the way that lots of questions are being written right now actually does that.

[grapesmoker]

Questions rarely reward knowing about particular societies or languages, for instance, but often reward knowing about ethnography and theoretical linguistics.

Paul, I really wish you would not make categorical statements like that which just aren't true. In fact, quizbowl questions have moved quite a bit towards asking about exactly such things, and if you pay attention during tournaments you will find that this is the case. I think it's great that people are doing this, and you seem to agree (that it would be good for this to happen). Well, it actually is happening, so let's please use factual statements to talk about the game instead of something you seem to feel is true but is actually not.

[No Rules Westbrook]

I feel that it's that kind of knowledge -- the kind which derives from a serious, if non-professional, interest in a given field -- which is scanted by Ryan's curious "tiers." I also think that that kind of knowledge is one of the great things about the game. That is, playing quizbowl can lead people to become curious about areas which they might never think about otherwise (e.g. art, or mythology, or whatever), and then to develop a passion about those areas.

My crude tier system aside, I actually very much agree with what Andrew is saying here, and I'm a big fan of cultivating this type of knowledge. By contrast, a number of influential qb thinkers today constantly hammer the importance of "the academy" and that's (1) shortsighted/simplistic and (2) not at all representative of real knowledge bases in this game.



On Weiner's point, I should try to more precisely distinguish between Tier 2 and Tier 3. So, there's no doubt that reading Masterplots or something like that, or an interesting selection from Google Books (an encylcopedia of history, etc.), is valid Tier 2 knowledge. And there's no doubt, I think, that gaining knowledge from writing questions is valid Tier 2 knowledge - because in order to write questions the right way, you have to research the topic pretty thoroughly, learn what's generally important and not important about it, etc.

Accumulating information from old packets is, I guess, the interesting question. I will certainly agree that simply memorizing the most baseline binary associations is Tier 3 Knowledge - knowing "Alexander Chatsky = Woe From Wit", for example. But, I think most good writers already do a decent job of sticking those kinds of clues in "Tier 3 Land."

However, there are so many packets out there today, and you can accumulate a ton of info about most pedestrian qb subjects by becoming a student of those packets. You can in fact acquire info that goes way beyond the more simplistic binary associations. If I were a cynic (and I am), I'd even say - with many subjects, you can acquire a very large proportion of the important, relevant and workable clues for a given subject. That is, you can acquire a substantial portion of the important things about a given topic that can reasonably be clues in quizbowl. I think many people are way too comfortable writing off this often quite substantial body of knowledge as "fakeism," just on their kneejerk reaction that they don't like the idea that someone is studying old packets (and because they are intensely uncomfortable with people "knowing stuff" and not truly "understanding stuff" - which is fine, but this game doesn't and can't do a very good job of testing understanding, with the possible exception of science). On the other hand, I think that kind of knowledge is the lifeblood of the game, and I'm not at all ready to write it off as second-rate fakeism.