2014 ICT: general discussion

Old college threads.
User avatar
ThisIsMyUsername
Auron
Posts: 1005
Joined: Wed Jul 15, 2009 11:36 am
Location: New York, NY

Re: 2014 ICT: general discussion

Post by ThisIsMyUsername »

I was done with this argument, but since Ike has re-opened this, let me briefly respond to add what I hope are my last words on this subject.
Ike wrote: I agree that the Manfred Kuehn clue doesn't test whether someone has actually read Manfred Kuehn or just studied some names of Kant scholars, but there is no need to test such a distinction because on that clue because if you have heard of the name Manfred Kuehn, you must have engaged with Kant in some form. To me this kind of clue is one of those where "the only way you know this clue is by having engaged with the particular scholar or work cited, because they haven't come up in quizbowl at all, or incredibly infrequently, so it doesn't matter if I just name drop it here" clues. (I like to think of them as shibboleth clues.)
I agree with you that if "the only way you know this title/author's name is through engagement", then, indeed, merely name dropping these things suffices, and descriptive clues are unnecessary. The problem is that you have just declared this to be the case, without doing anything to establish it.

The example that you use below to try to justify such a claim is unfortunately quite ill-fated:
As an example of this type of clue working well, I would have buzzed on that Giordano Bruno tossup on the first clue because I have read a bit about Bruno, and the introduction to my book The Expulsion of the Triumphant Beast discusses Frances Yates quite a bit. So while it certainly is true that you aren't testing whether or not I have read Frances Yates, there is no need to do so, because a. - Frances Yates hasn't been populating all those Giordano Bruno tossups all those trendy players have been studying, and b. - that Yates clue can be is most likely learned by intellectually engaging with Bruno - which is what you are trying to reward anyway. So I think for the tossup on Bruno, the lead-in empirically and ideologically worked, and that it would be a mistake to try to argue "well that tossup didn't test whether or not you have read Frances Yates."
Inventing a hypothetical situation in which you beat a set of imaginary people who don't know the first clue to a question you didn't play in real life is not really what the word "empirical" means. Frances Yates (and her connection of Bruno to hermeticism) is name-dropped in the header text (i.e. the three summary paragraphs at the beginning) of the Wikipedia article on Giordano Bruno. I hope you enjoyed reading The Expulsion of the Triumphant Beast, because all that reading is going to do for you is allow you to buzzer-race on the first clue against people who have spent two minutes on the internet, reading exactly the source most likely to be used by people our age who are learning rudimentary facts for the first time.

(I guess I should also re-iterate for the umpteenth time, since I appear to have been misunderstood on one point occasionally: I'd be fine with a lead-in that rewarded people for having read Yates. I'm not trying to claim that reading Yates on Bruno is inherently less valuable than reading Bruno himself. I'm saying that assuming that just name-dropping Yates out of the blue rewards some form of engagement is a suspect assumption.)

Frankly, I don't understand what either you or Yaphe are doing when you tell stories about how you, the people who have read a ton about Author X (primary and secondary material alike), would have buzzed on (and enjoyed) the clues in a tossup. Well, of course you would have! The point of writing well pragmatically is not gearing the questions towards people like you, who have done extensive reading. As a means of determining the first clue of each tossup, fine; but not the questions as a whole. The point is to start with the assumption that most people have limited engagement with any the subjects of hard answer-lines, and to try to tap into that specific limited knowledge for middle clues, so that the question plays well. This is why Yaphe's counterfactual to his Henry Adams question-- in which I hypothetically approve of a tossup on Henry Adams drawn entirely from incidents from Democracy-- is also absurd. I would never write nor approve of question like that, because I have no reason to expect enough people to have read that novel, nor to expect that there are deep layers of knowledge in the field about that novel that need to be distinguished.

The central point of agreement among those criticizing the ICT literature and philosophy seems to be: the answer-lines at ICT were on things people have engaged with intellectually, in real life, and things of which people have knowledge. However, the clues mostly did not succeed in tapping into that engagement or knowledge, because the clues were on things with which the players haven't engaged. Re-connecting the clues to what the field knows will produce better results. I, personally, have an ideological commitment to a more restrictive definition of "engagement" or "knowledge" than that favored by some of the other people posting in this thread, because I don't think "having heard of things" is as valuable as "having read things". This distinction is rejected by many in this conversation. So be it. If only the purely "pragmatic" changes I am suggesting happen in ICT, I'll still be a much more satisfied customer, and I think other people will be too.
Lastly, contra every other lit player who has posted in this thread, I actually find these questions great in the enjoyable sense. They are so much fun! It's the only tournament all year, where you get to play on a somewhat different style of question, and after the tournament is done, you get to go learn an incredible amount of content and figure out if you want to read some of these interesting sounding books.


Yes, I too enjoy going through the packets afterwards and looking up some of the books I hadn't heard of. And I too find Yaphe's choice of works for lead-ins consistently interesting, and am pleased with the results of this exercise, after I perform it. (Although, I have to say: given that we agree that the contents of most of these books are not explained, I'm not sure why ICT would get you more fired up about this than do other hard tournaments that draw on interesting works, unless just hearing titles gets you excited.) However, this dimension should be considered only after all the dimensions of the quizbowl questions functioning as quizbowl questions have been accounted for.
I would hate to see the Mortimer Adler clones of the world ruin one of the last quizbowl bastions of unique experiences.
While it is perhaps tempting for me to respond in kind by calling you a "clone" of some outdated thinker as a means of lazily tarring you with guilt by association (as you are doing to me), I unfortunately can't think of a prominent intellectual who has propounded the value of name-dropping things without addressing their contents; so, I guess you're safe.

EDIT: Grammar
John Lawrence
Yale University '12
King's College London '13
University of Chicago '20

“I am not absentminded. It is the presence of mind that makes me unaware of everything else.” - G.K. Chesterton
User avatar
Birdofredum Sawin
Rikku
Posts: 400
Joined: Tue Nov 09, 2004 11:25 pm
Location: Mountain View

Re: 2014 ICT: general discussion

Post by Birdofredum Sawin »

Having taken my rhetorical flamethrower to Marshall this past week, I want to try to extend an olive branch to John, lest I seem to have a vendetta against the entirety of the current Chicago team.

I think this discussion got off on a bad footing, but has, from a pragmatic standpoint, settled down to something approaching a broad consensus. Rather than attribute any positions to John or anyone else, however, I will simply state my own position and hope that others will agree with it.

If I understand where things are now, people are "fine" with ICT tossups where the leadin material is (for lack of a better shorthand phrase) "typical Yaphe stuff." When I say that people are "fine" with it, I mean something ranging from "outright glee" (Ike) to "grudging acceptance" (others). However, people would prefer a more curtailed version of "typical Yaphe stuff" in future ICT sets, thereby allowing more room for development of middle clues.

Using examples is always dangerous, but I will suggest that this tossup exemplifies the kind of thing that, again, people would be "fine" with seeing going forward:
Myles Burnyeat argued that the opening of this work draws attention to "accomplished mastery" in the person of a geometrician from Cyrene named Theodorus. It includes a critique of Protagoras' maxim "{Man is the measure of all things}," and compares the mind to a {bird cage}. It considers three definitions of the central subject, including (*) perception, a true judgment, and a true judgment with an account. For 10 points--name this {Platonic dialogue} on the nature of knowledge.

answer: _Theatetus_
Those "in the know" (I hope) will agree that Burnyeat's work on the Theatetus is significant and interesting; however, this tossup only devotes a sentence to it (and indeed, mingles substantive clues about the dialogue itself in that sentence) before proceeding directly to clues about intrinsic features of the work itself. Again, and as usual, I am not offering a defense of this particular tossup; I offer it as an exemplar solely for its proportion of "typical Yaphe stuff" to "clues from intrinsic description of a work."

I do not mean for this post to re-open any cans of worms that may have been shut; instead, I genuinely offer it in a spirit of reconciliation and in hopes of arriving at some constructive result from this discussion.
Andrew

Ex-Virginia, Ex-Chicago, Ex-Stanford
User avatar
Adventure Temple Trail
Auron
Posts: 2754
Joined: Tue Jul 15, 2008 9:52 pm

Re: 2014 ICT: general discussion

Post by Adventure Temple Trail »

I'm more or less totally on board with your latest post, Andrew. I didn't know that Theaetetus leadin -- as became apparent to a ballroom's worth of people last Saturday -- but from what I can tell, that leadin was one of the best-written secondary-literature clues in the set (and that tossup one of the best philosophy questions in the set). If more of the secondary-source clues in next year's ICT look like that one, i.e. "mingling" in info about the original text, using enough words in the clue to make clear what the importance is to the primary text rather than feeling like word-association, the 2015 set would assuage most of my concerns from this year.
Matt Jackson
University of Chicago '24
Yale '14, Georgetown Day School '10
member emeritus, ACF
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: 2014 ICT: general discussion

Post by grapesmoker »

I don't think that "someone could learn this from a Wikipedia article" is really a good argument for anything. You can learn lots of things, many of which are even true, from a Wikipedia article. I'm going to guess that's true for almost all answer lines in any quizbowl tournament. But the reality is that people for the most part don't sit around reading Wikipedia articles on Giordano Bruno. I imagine that if you're doing something like that, you already have sufficient interest in Bruno to read either his primary work or some secondary literature about him. Just skimming a Wikipedia article is unlikely to get you the same level of retention as someone who has read more of the source material than you.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
kayli
Auron
Posts: 1525
Joined: Sat Jun 21, 2008 10:15 pm
Location: Illinois

Re: 2014 ICT: general discussion

Post by kayli »

jonah wrote:As long as we're discussing ICT science, I am interested in feedback on the math. I wrote most of it and edited all but one question.
Which ones were these? Could I see the text of them?
Kay, Chicago.
jonah
Auron
Posts: 2383
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: 2014 ICT: general discussion

Post by jonah »

Touko Kettunen wrote:
jonah wrote:As long as we're discussing ICT science, I am interested in feedback on the math. I wrote most of it and edited all but one question.
Which ones were these? Could I see the text of them?
I'm not posting all the math questions, but here are the answer lines. * indicates I wrote it as well as editing it.

Tossups: *complex conjugation, *supremum, *finitude, *Chinese remainder theorem, *discrete [topology, distributions], power series, Charles Hermite, central limit theorem

Bonuses: *integer-valued properties of mathematical objects (polynomial degree/chromatic number/winding number), nonplanar graphs (graphs/Kuratowski's theorem/Jordan curve theorem; not edited by me), George Dantzig (Jerzy Neyman/simple algorithm/John von Neumann), *injective function/Schroder-Bernstein theorem/law of the excluded middle, *logic (first-order/Kurt Godel/arithmetic), *category theory (/commutative diagrams/morphisms), *brachistochrone problem ([least] time/calculus of variations/Beltrami identity), pioneering statisticians (Gauss/Rao/Kolmogorov)
Jonah Greenthal
National Academic Quiz Tournaments
felgon123
Lulu
Posts: 70
Joined: Sun May 10, 2009 4:04 pm

Re: 2014 ICT: general discussion

Post by felgon123 »

Note: this is a shoddy reconstruction of a much more eloquent paragraph that was infuriatingly lost when I hit the "submit" button and was told to log in again.

Final clarifications/thoughts: I agree with several things Eric has said. The tossups on scientists were generally not to my taste. No one has mentioned the tossup on Emile Clapeyron, which was atrocious. The tossup on J. S. Bell seems pretty pointless/opaque to me, but I don't have the requisite knowledge to judge. Is anyone really buzzing on these pre-electronegativity Mullikan clues? This might just be me, but a tossup on Seaborg feels closer to a tossup on, say, Joseph Priestley than I would like to see at a national tournament, i.e. too much of a pure "science history" bent. In any case, the scientist tossups regularly lacked early clues anyone could be expected to buzz on, and others did as well. The example of the "vibration" tossup is a good one; Eric buzzed at the end of the fourth line, and presumably everyone else buzzer-raced on "IR spectroscopy," which were literally the next words after "degrees of freedom." However, I don't want to seem like I'm explicitly gunning for Seth. To pull a problematic question from Selene's set, the tossup on carbon monoxide, which we heard in our game against Penn, exemplifies this same problem. If I remember correctly, Eric (along with everyone else in the room) buzzed as the moderator was beginning to say "(*) 200-fold greater affinity than oxygen for hemoglobin." I might not be a carbon monoxide scholar, but I know some reactions/processes involving it, and I assume Eric knows a lot more than me, so I don't think it's unreasonable to expect some clues we could buzz on before a drop like that into truly common knowledge. To give another example from biology, the tossup on Parkinson's began with two very hard clues, one of which isn't unique, followed by a cliff into "Lewy bodies" in the middle of the third line. It seems to me that, as others have maintained of the literature and philosophy, the most common problem with the science was a lack of buzzable middle clues, often due to a preponderance of overly difficult first, second, and sometimes third clues. With all that said, I want to dispel the notion that I thought the science was "bad" overall; in my first post, I was careful to specify I was taking issue only with "some" questions/tendencies. In addition to pointing out problems with various chemistry tossups, Eric has also pinpointed several of my favorites from that category, and now that I have had a chance to look through the packets, I can say there were a good handful of Seth questions in the last 2-3 tossups of packets that I would have enjoyed had we been able to hear them, in addition to the ones I enjoyed while playing the tournament; there certainly was a decent number of those, and I apologize if I gave the impression there wasn't.
Tommy
UVA '15
Harvard...let's say '23
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5647
Joined: Wed Jul 26, 2006 11:46 pm

Re: 2014 ICT: general discussion

Post by Sen. Estes Kefauver (D-TN) »

felgon123 wrote:Note: this is a shoddy reconstruction of a much more eloquent paragraph that was infuriatingly lost when I hit the "submit" button and was told to log in again.
In my experience, when this happens on hsqb I've always been able to click back to the page where I wrote the post, and it will still have the text sitting there. You just have to copy and repaste it when you log back in.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: 2014 ICT: general discussion

Post by theMoMA »

felgon123 wrote:I’m not one to post extensive breakdowns of individual questions, and I don’t have a copy of the set anyway, so I’ll mostly limit myself to general observations. By and large, I found the science in this set a bit frustrating. I’ll begin by appealing to the useful dichotomy drawn above between empirical and ideological criticism. As I am neither inclined nor qualified to declare Seth’s clues unimportant, unhelpful, or unknowable in any absolute sense, I simply want to call attention to the fact that a considerable number of the science tossups in his categories had 3-4 lines of clues that no one buzzed on, followed by a giveaway or pre-giveaway clue that any mediocre science player, or even any decent generalist, would know. To put it more concretely in terms of my own experience: there were a number of tossups on answers I’m rather familiar with that consisted of four lines of things I didn’t recognize followed by an easy giveaway, often the basic definition of the answer. Even if this is merely indicative of my own ignorance, or the collective ignorance of quizbowlers (since my displeasure seemed to be shared by others), that doesn’t change the fact that a sizeable chunk of the science questions were loaded with clues that didn’t distinguish between the existing levels of knowledge among players, when these differences do exist and very much can be exploited by well-written questions. One need only look at the tournament played a couple hours after ICT, Eric Mukherjee and Co.’s Lederberg 2, which consistently and skillfully distinguished between the knowledge levels of science players on canonical answers, both when written creatively and when written in a more straightforward fashion, as well as on more inspired or difficult answers.

Aside from that, I do have one ideological outcry about a specific question. Our game with Penn was one tossup away from having to be decided by the third tiebreaker question, a truly awful tossup on Hans Bethe, in which the first clue was about how he calculated the Lamb shift, arguably his most important achievement. Players are more likely to know that than they are to know anything at all about Charles Galton Darwin. A buzzer race on an easy lead-in like that would have been a terrible way to lose such an important game, and an embarrassing way to win.

On a more positive note, I did enjoy Selene’s biology more than the rest of the science, and I wish she had handled the chemistry as well. Nevertheless, I very much appreciate the work Seth did for this tournament; regardless of how I felt about some of the questions in his categories, I’m glad they existed and were there for us to play.
felgon123 wrote:Note: this is a shoddy reconstruction of a much more eloquent paragraph that was infuriatingly lost when I hit the "submit" button and was told to log in again.

Final clarifications/thoughts: I agree with several things Eric has said. The tossups on scientists were generally not to my taste. No one has mentioned the tossup on Emile Clapeyron, which was atrocious. The tossup on J. S. Bell seems pretty pointless/opaque to me, but I don't have the requisite knowledge to judge. Is anyone really buzzing on these pre-electronegativity Mullikan clues? This might just be me, but a tossup on Seaborg feels closer to a tossup on, say, Joseph Priestley than I would like to see at a national tournament, i.e. too much of a pure "science history" bent. In any case, the scientist tossups regularly lacked early clues anyone could be expected to buzz on, and others did as well. The example of the "vibration" tossup is a good one; Eric buzzed at the end of the fourth line, and presumably everyone else buzzer-raced on "IR spectroscopy," which were literally the next words after "degrees of freedom." However, I don't want to seem like I'm explicitly gunning for Seth. To pull a problematic question from Selene's set, the tossup on carbon monoxide, which we heard in our game against Penn, exemplifies this same problem. If I remember correctly, Eric (along with everyone else in the room) buzzed as the moderator was beginning to say "(*) 200-fold greater affinity than oxygen for hemoglobin." I might not be a carbon monoxide scholar, but I know some reactions/processes involving it, and I assume Eric knows a lot more than me, so I don't think it's unreasonable to expect some clues we could buzz on before a drop like that into truly common knowledge. To give another example from biology, the tossup on Parkinson's began with two very hard clues, one of which isn't unique, followed by a cliff into "Lewy bodies" in the middle of the third line. It seems to me that, as others have maintained of the literature and philosophy, the most common problem with the science was a lack of buzzable middle clues, often due to a preponderance of overly difficult first, second, and sometimes third clues. With all that said, I want to dispel the notion that I thought the science was "bad" overall; in my first post, I was careful to specify I was taking issue only with "some" questions/tendencies. In addition to pointing out problems with various chemistry tossups, Eric has also pinpointed several of my favorites from that category, and now that I have had a chance to look through the packets, I can say there were a good handful of Seth questions in the last 2-3 tossups of packets that I would have enjoyed had we been able to hear them, in addition to the ones I enjoyed while playing the tournament; there certainly was a decent number of those, and I apologize if I gave the impression there wasn't.
Broadly speaking, there are two different ways to claim that the distribution of buzzes across the tossups in a particular set is not ideal. The first is that, across a given set or area of the distribution, the editors have consistently made choices (emphasizing a particular kind of source, using particular kinds of clues, picking particular kinds of answers, etc.) that have led to a problematic distribution of buzzes. The second is that certain questions in the set seemed to have problems with clue gradation, and thus that the set (or a subdistribution of the set) at large did as well.

I find the second manner of argument problematic to the extent it relies on the inherent assumption that "what I saw when I played the question in a particular game room" is indicative of "how a question played across the spectrum of rooms." I don't think this assumption is a good one. I have no doubt that, for example, you and Eric are among the combinations of players likely to know the most about, say, carbon monoxide. I also have no doubt that a perfectly well-written tossup on carbon monoxide, which is filled with important and eminently knowable early and middle clues, could evade your collective grasp until a relatively late clue for any number of reasons, including the simple fact that even the best players in their categories don't know every important and eminently knowable clue about every possible answer.

Put another way, I think your argument proceeds by stating that "Eric and I didn't know the clues; therefore the clues were too hard; therefore the tossup was bad." And I simply don't think that this logic works very well. It may actually be the case that (a) you and Eric didn't know the clues and (b) the clues were also too hard, but the connection between (a) and (b) isn't a very strong one absent further analysis (such as, to construct an obvious example, "as it turns out, the clues were only available in one PubMed article from 1997"). Unfortunately, such further analysis is almost entirely lacking from your critique. You point to a tossup on Clapeyron, "which was atrocious," but make no additional comments. You point to a tossup on J.S. Bell that you "don't have the requisite knowledge to judge," but nevertheless claim was "pointless" and "opaque." You point to several tossups that, in your observation, didn't elicit buzzes in their earlier clues, but make no effort to connect this to how those questions played in the tournament at large, and provide little to no analysis of the early clues themselves.

I don't think that your argument proves that there was a consistent problem with the early-to-middle science clues. It only shows that players do not always buzz on early-to-middle clues, which is nothing new, helpful, or interesting. (And somewhat ironically, you deride the tossup on Hans Bethe, which apparently did have buzzable early clues, for being "truly awful" because its early clues were too buzzable when, at least from the discussion of Bethe in the thread, the claim about the ease of that clue seems overstated.)

Editors must constantly make predictions about what players will and won't know. It's a broad, bold claim that editors are consistently predicting wrong, one that I think should be supported by evidence much stronger than "everything I know is easy and everything I don't is impossible" writ large.
Andrew Hart
Minnesota alum
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: 2014 ICT: general discussion

Post by Cody »

theMoMA wrote:I don't think that your argument proves that there was a consistent problem with the early-to-middle science clues. It only shows that players do not always buzz on early-to-middle clues, which is nothing new, helpful, or interesting. (And somewhat ironically, you deride the tossup on Hans Bethe, which apparently did have buzzable early clues, for being "truly awful" because its early clues were too buzzable when, at least from the discussion of Bethe in the thread, the claim about the ease of that clue seems overstated.)
I don't want to talk about your whole post, but this in particular is wrong. The lead-in wasn't too buzzable: it was too easy. That's a very important difference that you can't just gloss over by changing the words you use. There's been a pretty large argument over whether "calculating the Lamb shift" is easier than "nucleosynthesis", but that argument is irrelevant since the true question is whether calculating the Lamb shift is too easy for a lead-in. It is, and is used to illustrate the larger problem with scientist tossups.
Cody Voight, VCU ’14.
User avatar
Sima Guang Hater
Auron
Posts: 1958
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: 2014 ICT: general discussion

Post by Sima Guang Hater »

Andrew wrote:I find the second manner of argument problematic to the extent it relies on the inherent assumption that "what I saw when I played the question in a particular game room" is indicative of "how a question played across the spectrum of rooms." I don't think this assumption is a good one. I have no doubt that, for example, you and Eric are among the combinations of players likely to know the most about, say, carbon monoxide. I also have no doubt that a perfectly well-written tossup on carbon monoxide, which is filled with important and eminently knowable early and middle clues, could evade your collective grasp until a relatively late clue for any number of reasons, including the simple fact that even the best players in their categories don't know every important and eminently knowable clue about every possible answer.

Put another way, I think your argument proceeds by stating that "Eric and I didn't know the clues; therefore the clues were too hard; therefore the tossup was bad." And I simply don't think that this logic works very well. It may actually be the case that (a) you and Eric didn't know the clues and (b) the clues were also too hard, but the connection between (a) and (b) isn't a very strong one absent further analysis (such as, to construct an obvious example, "as it turns out, the clues were only available in one PubMed article from 1997").
I think this is an important point, which is why I posted my overall statistics from the tournament (and encouraged/would encourage others to do the same, or at least compile that information behind-the-scenes). I agree that how one tossup played in one game may not be generalizeable, but if the physics was not being powered at the same rate as the other sciences, across several players and several games, I believe that would serve as the kind of evidence you seek.

Tommy made some interesting comments about the biology that are worth addressing. I completely agree with his criticism of the carbon monoxide tossup, which I'll post here:

Image

To shed a little more light on it: the leadin is about CO dehydrogenase and the next clue is about vanadium nitrogenase. These are very difficult, and don't show up in the biochemistry courses I've taken or done a syllabus search on, but the latter seems to have made a pretty big splash in journal articles. The next clue is about a pathway that I haven't covered, but 1. Comes up in quizbowl (occasionally), and 2. Wouldn't surprise me if it were taught in other biochemistry classes not at Brown or Penn. The next clue is about Heme oxygenase, which did come up in both undergrad and grad classes - though the fact that CO is a side product of said reaction isn't something that was drilled into us. But this was small text on one or two slides of literally thousands. The next clue is something taught in AP biology. This adds up to form a very difficult, cliff-y question. All of the clues are excellent and relevant to the field, I just think their arrangement doesn't work that well.

The Parkinson's tossup has a somewhat similar problem. I don't know the leadin (though I don't deny that some classes would cover Braak staging, as far as I can tell the neurology shelf exam and the class I took didn't), but the clue about haloperidol's extrapyramidal symptoms is covered fairly extensively in medical school. However, in quizbowl and pre-graduate classes, there seems to be a few points of access for deeper Parkinsons' knowledge - the lesions in the substantia nigra, the use of L-DOPA, Lewy bodies, and symptoms. Side effects of anti-psychotics doesn't seem to fall into one of those pathways, and since Lewy bodies are a Parkinson's buzzword in quizbowl, that leads to a cliff. This question is perfectly defensible, but were I to write it, I'd probably lead in with the haloperidol thing, then maybe mention alpha-synuclein or talk more about Lewy bodies before dropping the name "Lewy", something like that - i.e. going deeper into the later clues to create a pyramid, rather than layering on independent, harder clues.

I think something that's being lost in this discussion are the fundamental questions of "what level of knowledge should allow an [early/middle/late] buzz on subject x", "how much of the field should be getting [early/middle/late] buzz on subject x", etc, both in terms of how said marks were met, and where said marks should be set. If the intent was for a given player to power ~10-20% of a category that they're above average on and ~50% of a category they purport to know well, I'd say you've nailed it. However, from the conversations I had, it seems the best physics and chemistry players in the field were consistently being foisted with a harder climb than the biology players in the field.

NOTE: Billy Busse, for instance, managed ~6 powers across science (as opposed to 26 at Lederberg), and just from extrapolation I'd guess Tommy managed a similar number (vs. 17 at Lederberg). Ashvin and Sriram, who managed 12 and 10 powers in Lederberg, while playing on the same team, each managed around ~6 powers at ICT while functioning as the science players on their team. I think this shows, to some extent, that the physics was a little top-heavier than you would want.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
setht
Auron
Posts: 1205
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: 2014 ICT: general discussion

Post by setht »

I agree with Andrew H.'s argument (over here) that referencing other people's gut reactions is generally not a great idea for tournament discussion, but I wanted to note that I think the situation for "sifting through commentary in search of useful lessons for future sets" is rather different for ICT than for most post-tournament discussions: I believe we're going to compile detailed statistics from all the scoresheets, so we can go back and check how often a given tossup was answered for 10, how often it was powered, how often it was negged, and how often it went dead. We'll similarly be able to check the distribution of 0s/10s/20s/30s on bonuses. I wish we had yet more data (exactly where tossups were buzzed on; exactly which bonus parts were/were not answered), but with what we get I think it is possible to get real use out of "gut reaction" comments, one person's commentary after looking over the set, etc. It won't be possible for me to carry out every imaginable analysis of the data we do get (it would take too long, and I'm not imaginative enough to come up with all of them on my own); this is why any pointers I can get from you all on things that would be worth checking will be useful to me. In short, I am okay with people engaging in "bad discussion habits" in this case. I should say that I've been happy with the mode of discussion so far, but then again I'm an inveterate discusser of individual questions.

I suppose what is useful (and interesting!) to me may not be that useful or interesting to the other people reading this thread. If you'd like to notify me of some potential broad trend, possible issue with a single question, or really anything, feel free to send me an email ([email protected]) if that seems like a better option than posting here. I do encourage people to keep posting here with anything that might be interesting/useful to the wider audience.

Thanks again for all the commentary, and keep it coming.

-Seth

p.s. I hope no one will feel any hesitation about sending in comments, no matter how vague or unsupported: I will be grateful for any guidance I get. I will not be upset if someone says "the tossup on X felt really hard" and I find upon checking the stats that the tossup on X was converted in 15 out of 16 rooms, with 5 powers. I know there are all kinds of sources of variance in how a particular questions plays for a particular pair of teams. I would much rather spend the time checking a bunch of comments if it nets me one solid idea for an easy fix to improve future sets, then not hear anything.

p.p.s. I invite comments for both the DI and DII sets. I'm not sure I'll be the one mining the DII data (and come to think of it, I may not be the only person analyzing the DI data), but if I'm happy to collect email feedback and pass it on to whoever needs it.
Seth Teitler
Formerly UC Berkeley and U. Chicago
President of NAQT
Emeritus member of ACF
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: 2014 ICT: general discussion

Post by grapesmoker »

The Quest for the Historical Mukherjesus wrote:NOTE: Billy Busse, for instance, managed ~6 powers across science (as opposed to 26 at Lederberg), and just from extrapolation I'd guess Tommy managed a similar number (vs. 17 at Lederberg). Ashvin and Sriram, who managed 12 and 10 powers in Lederberg, while playing on the same team, each managed around ~6 powers at ICT while functioning as the science players on their team. I think this shows, to some extent, that the physics was a little top-heavier than you would want.
I'm confused as to how this comparison makes sense, given that Lederberg is a tournament consisting entirely of science. Surely you need to scale that to a fraction of relevant questions powered.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Sima Guang Hater
Auron
Posts: 1958
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: 2014 ICT: general discussion

Post by Sima Guang Hater »

grapesmoker wrote:
The Quest for the Historical Mukherjesus wrote:NOTE: Billy Busse, for instance, managed ~6 powers across science (as opposed to 26 at Lederberg), and just from extrapolation I'd guess Tommy managed a similar number (vs. 17 at Lederberg). Ashvin and Sriram, who managed 12 and 10 powers in Lederberg, while playing on the same team, each managed around ~6 powers at ICT while functioning as the science players on their team. I think this shows, to some extent, that the physics was a little top-heavier than you would want.
I'm confused as to how this comparison makes sense, given that Lederberg is a tournament consisting entirely of science. Surely you need to scale that to a fraction of relevant questions powered.
I agree, I was only pointing out that spectacular stats in Lederberg didn't necessarily translate to similarly spectacular stats on the science at ICT.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: 2014 ICT: general discussion

Post by Cody »

Yeah, but you have to basically divide by 4 to get the number of science questions to match up, so the power counts aren't really off.

edit: I've also been reminded of a long-standing bug-a-boo I have with the NAQT correctness guidelines. Chemical symbols should absolutely be acceptable for elements [until they are read].
Cody Voight, VCU ’14.
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: 2014 ICT: general discussion

Post by theMoMA »

I'm glad that my post prompted Eric's in-depth criticism, because that was the intent. I'm definitely not saying that people should avoid discussing individual questions, or even that they should avoid extrapolating broader trends from observations across multiple questions. There just needs to be some substance behind the criticism to make it a really useful one, especially when the thesis of a particular post is a broad claim about a trend in a particular subdistribution.
Andrew Hart
Minnesota alum
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: 2014 ICT: general discussion

Post by naan/steak-holding toll »

Nobody's really mentioned this much so far, so I think I'll post it: I thought the history in this set was pretty good and did a good job of being pyramidal, having solid difficulty gradation, and achieving a good balance between difficulty of answerline and rewarding depth of knowledge. The clue selection was also very good, with no over-reliance on quotes or anecdotes, but instead a strong focus on important historical facts and people. Not that helpful, but I think Jeff did a really good job on this front.

The only history question I really have any complaint about was the tossup on the Onin War in the finals, which (if I recall correctly) said "this period" for a long time, making it perfectly reasonable to buzz with "Sengoku-jidai" or "Muromachi period" and be confused when prompted, since it wasn't made clear that the question was looking for a conflict until later on and you don't typically think of the Onin War as a "period" (even though it technically is), especially since Japanese history is conventionally divided into several commonly tossed-up periods.
Will Alston
Dartmouth College '16
Columbia Business School '21
felgon123
Lulu
Posts: 70
Joined: Sun May 10, 2009 4:04 pm

Re: 2014 ICT: general discussion

Post by felgon123 »

I thought I was done posting in this thread, but I had forgotten how much Andrew enjoys misrepresenting everything I say. I understand that my posts, rare as they are, must be a real treat for him to set to work upon, so I won't begrudge him a reply.
theMoMA wrote:Put another way, I think your argument proceeds by stating that "Eric and I didn't know the clues; therefore the clues were too hard; therefore the tossup was bad."
Throughout your post, you attribute a dictatorial disposition to me that, much as you would like it to, does not exist. The attitude I sought to convey throughout my post, perhaps with less success than I hoped for, was, "I could be wrong, but this is how it seemed to me." I don't have the full stats for the tournament at my fingertips. If I did, I assure you, I would check the power numbers on every tossup I mention and bring that in to support my arguments. But instead, I am confined to describing how the questions played out in my rooms, and making educated guesses as to what kinds of buzz distributions the clues produced. I point to questions I think typified certain problems so that a) other science players can concur or expostulate as they see fit, and b) people like Seth with the actual numbers can check to see if they bear out my claims. I apologize for having the audacity to think "the best biology player in the game didn't buzz until the fifth line on a commonly known thing that everyone else in the room was buzzing on" is a useful observation to make. It's possible a couple players powered that tossup, but based on the empirical evidence I have, it sure isn't likely. Your partial retraction due to Eric's agreement with me amuses me immensely. Eric has helpfully broken down the clues for that tossup, showing just how steep a difficulty cliff it suffered from, but I suppose that's all merely subjective and presumptuous, too, isn't it? After all, just because Eric didn't learn these things in his biochemistry courses doesn't mean other people don't. Who is he to say whether those clues are "hard"? Apparently when I say, "Eric didn't know these clues, and I think they're too hard," that's pompous, but when Eric says, "I don't know these clues, and I think they're too hard," that's the "in-depth criticism" you're looking for, and suddenly it's okay to extrapolate trends based on personal observation, so forgive me for thinking you would've challenged me no matter what I said until Eric backed me up. Now, none of that should be taken as a criticism of Eric's fine analysis. My point is that my arguments obviously weren't irrelevant or presumptuous. Quite the contrary, they were useful (dare I say "correct"?), and they invited valuable contributions from a player with more expertise that supported my claims.
theMoMA wrote:You point to a tossup on Clapeyron, "which was atrocious," but make no additional comments. You point to a tossup on J.S. Bell that you "don't have the requisite knowledge to judge," but nevertheless claim was "pointless" and "opaque." You point to several tossups that, in your observation, didn't elicit buzzes in their earlier clues, but make no effort to connect this to how those questions played in the tournament at large, and provide little to no analysis of the early clues themselves.
The Clapeyron tossup was atrocious because a) it's a tossup on Clapeyron, and b) it said "Antoine equation" in the second line, and yes, in that instance, I felt confident that every science player would agree with me. As you may have noticed, no one has leapt up to defend that question (or any of the other questions I singled out for censure), though of course they are welcome to do so. I'm not sure what it is about the "This seemed problematic to me; thoughts?" paradigm I adopted that you find so offensive. I picked out questions I thought had problems so others could share their opinions of them, contextualize them, defend them, check the stats on them, etc. Seth asked for feedback about specific questions and what may have been wrong with them; I gave examples. Is this not productive discourse? You accuse me of making bold and arrogant declarations of failure on the editors' parts founded on nothing more than my lack of knowledge, then fault me for an instance in which I admit my criticism could be wrong by changing my words so that I contradict myself. I said the J. S. Bell tossup "seems pretty pointless/opaque to me," and invited others to discuss/agree/refute by openly declaring my ignorance as to whether the non-Bell's theorem stuff is important.
theMoMA wrote:I don't think that your argument proves that there was a consistent problem with the early-to-middle science clues.
I heartily agree. It suggests, even maintains, that there was a recurring problem, but it doesn't prove that. However, I never claimed to be irrevocably "proving" anything, which would be a bold claim indeed, and I did and do remain open to power stats or the testimony of others "proving" me wrong. So while I approve of your desire to show off your reading comprehension skills, I wish you wouldn't clutter the thread with it.
theMoMA wrote:It's a broad, bold claim that editors are consistently predicting wrong, one that I think should be supported by evidence much stronger than "everything I know is easy and everything I don't is impossible" writ large.
One final generalization: I'm going to presumptuously and subjectively speculate, on the basis of my experience, that few readers of this thread had the propriety not to laugh at this finishing rhetorical flourish, so magnificently out of proportion with anything I actually wrote, even when taken as the fond, loving parody I'm sure it is.
Tommy
UVA '15
Harvard...let's say '23
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: 2014 ICT: general discussion

Post by theMoMA »

As I said above, and consistent with my response to Eric, the issue I take with your post is not that you pointed out things that you thought were problematic, but that you attempted to draw out a systematic problem with the set based on questions you didn't like even though you presented very little evidence about why those questions themselves were problematic.

If this were a post in the specific-question thread with the thesis "I had an issue with these particular tossups," I'd have no problem with it. I guess you've attempted to frame your post that way by characterizing your approach as "This seemed problematic to me. Thoughts?" But that's not how I interpreted your post, which is in the general discussion thread and states its central claim as "a sizeable chunk of the science questions were loaded with clues that didn’t distinguish between the existing levels of knowledge among players." If you meant your posts as a critique of a discrete slice of individual tossups with no larger systematic point, I apologize, but that's not how I understood it.

If you indeed had a larger systematic point to your critique, the entirety of the evidence you marshaled to support your claim is (a) that there were a few questions that were terrible for reasons you originally didn't explain, (b) that you observed several questions on which strong science players didn't buzz until near the end (but again, you didn't initially analyze whether the specific clues are of a kind that are not buzzable, or whether they simply didn't overlap with the knowledge base of the strong science players), and (c) that your "displeasure seemed to be shared by others."

My contention is that your evidence did not support your claim; as I've posted here and elsewhere, I think that bare assertions that questions were bad or that clues were unbuzzable are not very helpful to construct larger points about a set absent more detailed analysis that links these observations together. (And I think that assertions such as "my displeasure seemed to be shared by others" are unfalsifiable appeals to authority.) I appreciate that both you and Eric have taken my post as an opportunity to go into detail examining problems with questions in the set, and I hope you continue to offer feedback about specific questions or the set in general if you've got more things to say. My post was simply a call for you to provide what I see as crucial analysis in this particular context.
Andrew Hart
Minnesota alum
touchpack
Rikku
Posts: 451
Joined: Tue Sep 20, 2011 12:25 am

Re: 2014 ICT: general discussion

Post by touchpack »

So I'm basically going to ignore a lot of what has been said above and present my own personal philosophy of question construction. Feel free to discuss whether or not you think it is good or not.

When constructing my tossup's pyramid, I tend to think about two things when considering my middle clues:

1) Do I have at least one middle clue that is well-known enough to be out of power, but is not well-known enough to be giveaway/near the giveaway?
2) Do I have a clue in power that someone with deep knowledge can reasonably know?

This, in my mind, ensures that the tossup will have a smooth descent from "very difficult leadin clues" to "very easy giveaway clues." In an ACF-length tossup, there will be multiple clues in category 1 and possibly multiple clues in category 2, but in an ICT length tossup, you probably only have room for one clue in each category.

Another thing I like to consider is "if someone else wrote this tossup, and I did none of the research I just did in writing it, where would I buzz on this tossup?" If the tossup is regular difficulty and in a subfield of chem/physics that I know quite a bit about and I'm not powering it/I'm buzzing really late, that's a problem and I need more category 2 clues. If it's Chicago Open level and I'm not buzzing until very late, that might be fine, as long as I'm reasonably confident someone else in the field has a chance of buzzing before me. I find it's a nice mental exercise to determine how difficult your question is.

I would look at examples from DRAGOON/MUT in which I applied this practice, but those sets aren't clear, so instead I'll take some example tossups from this year's ICT that I think exemplified this practice well or poorly.

The tossup on chirality does this well. The cyclodextrin/chiral column chromatography clue is a very solid category 2 clue, and the C1 point group and optical spectroscopy clue are definitely category 1 clues. Someone with reasonably deep chromatography knowledge is powering the question, while people that don't have that knowledge but understand what optical isomerism is can still get a pre-FTP buzz.

The rearrangements tossup is another tossup I would say has a great difficulty curve. A description of the Fries rearrangement is definitely a clue people with deep orgo knowledge have a chance of knowing, as is the name of the Wolff rearrangement. The Hofmann rearrangement is one of the most important rearrangement reactions out there and is definitely widely known enough to be a category 1 clue. (aside: the tossup claims the reaction is named for Roald Hoffmann (two f's in surname)--this is not true, it is named for August Wilhelm von Hofmann (one f in surname))

The vibration tossup is lacking a category 2 clue. The 3N - 6 thing (which is where I buzzed) is a very good category 1 clue, as anyone who has taken physical chemistry should be able to buzz there. The leadin clues seem reasonably important, and the late clues are also good. The only thing this tossup is lacking is a category 2 clue (I would suggest something about the Franck-Condon principle or Raman spectroscopy, were I editing this tossup). Removing one of the 2 leadins and inserting a category 2 clue, in my opinion, would make this tossup perfect.

The tossup on electric potential doesn't seem to have a category 1 clue. Liquid junction potential isn't something I've heard of, but if someone told me it was a good category 2 clue, I might believe them (it seems reasonably important, at least). However, from there, the tossup goes straight to "This quantity's dramatic shift at the endpoint of a titration due to changing reactant concentration is given by the Nernst equation," which not only is going to tempt people to neg with pH before the sentence is over, but the Nernst equation is basically giveaway. This tossup sorely needs a category 1 clue to distinguish people with super basic knowledge from people with intermediate knowledge.

I think people's (Tommy's) frustrations with some of the questions are stemming from the fact that some of the questions were top-heavy, which I definitely sympathize with--I know there were several tossups during the tournament that were frustrating personally for me to play on. As a final note, I will say that I think the tournament overestimated the amount of science history knowledge people have, especially in the physics distribution. The tossups on Bell and diffraction gratings I think exemplify this best--from what I've heard anecdotally, in most rooms these science history clues went over everyone's heads, leading to buzzer races on EPR/description of a diffraction grating. I would suggest that these clues be scaled back, both in frequency and in relative position in tossups, for future NAQT tournaments.
Billy Busse
University of Illinois, B.S. '14
Rosalind Franklin University, M.S. '21, M.D. Candidate '25
Emeritus Member, ACF
Writer/Subject Editor/Set Editor, NAQT
Edmund
Wakka
Posts: 176
Joined: Sun Feb 07, 2010 8:25 pm

Re: 2014 ICT: general discussion

Post by Edmund »

I'd just add that I'm in total agreement with Billy's post above concerning clue difficulty in these specific physical chemistry questions and where they might have done better. I think liquid junction potential is a fine lead-in and I think that "Hessian eigenvalues" followed by "Birge-Sponer" are fine earlier clues, but that they need to be eased into the latter two lines by something genuinely intermediate in difficulty. To my mind the word "anharmonic" achieves this for "vibrations" but there is the risk of being coy or allusive with a one-word clue.

I think we can take as a given from the above that it is a bad idea - at ICT level also - to write tossups on scientists who are only famous for one thing.
Edmund Dickinson
UK Quizbowl
University of Oxford '11
User avatar
Ike
Auron
Posts: 1063
Joined: Sat Jul 26, 2008 5:01 pm

Re: 2014 ICT: general discussion

Post by Ike »

One more thing I would like NAQT to discuss.

Are editors for Division 2 ICT allowed to rearrange questions from packets so that the water downed question of DI appears in a different packet in D2? Specifically, I'm referring to the precession tossup in DI occurring a couple rounds after the D2 tossup on it, and the tossup on the Volta River in D1 appearing a couple rounds after the D2 tossup on it.

A lot of teams, including Illinois talk with their D2 teams about the questions they have played or just played, if the moderator had made it to tossup 23 in the tiebreaker round, the answer line of precession would have been "tainted" by the fact that Illinois knew of the existence of such a tossup. Of course it's entirely possible that they were written independently of each other, since they seem to not share many of the clues - though that's not at all the case for the Volta River tossup. Either way, I think it would be wise so that NAQT editors of D2 can't rearrange tossups that are source from D1, and that NAQT should strive as much as possible to make sure that D2 answer lines don't seep into future D1 rounds or vice versa.
Ike
UIUC 13
User avatar
Steeve Ho You Fat
Auron
Posts: 1138
Joined: Mon Jun 01, 2009 11:48 pm

Re: 2014 ICT: general discussion

Post by Steeve Ho You Fat »

I'm pretty sure the NAQT doesn't allow direct conversions to be in different packets, but I was surprised to see that there were several extremely similar questions (presumably written independently) in different packets. For example, I wrote a bonus on la Quenelle that was in the DII packets several rounds after the DI bonus that had at least one, maybe two parts the same.
Joe Nutter
PACE Emeritus
Michigan State University '14
Walnut Hills High School '11
User avatar
Matt Weiner
Sin
Posts: 8145
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: 2014 ICT: general discussion

Post by Matt Weiner »

I don't see why that's necessarily a problem. If the D1 and D2 questions are independently written, then telling someone "I just heard a D1 question on Morocco, here are some facts about Morocco for you, D2 team" is functionally the same as "I didn't just hear a question on Morocco, here are some facts about Morocco for you, D2 team" -- the fact that it came up later in the D2 set is a coincidence not dependent on it being in the D1 set.
Matt Weiner
Advisor to Quizbowl at Virginia Commonwealth University / Founder of hsquizbowl.org
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 6113
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: 2014 ICT: general discussion

Post by Important Bird Area »

It is indeed the case that:

1) Questions converted from Division I to Division II must appear in the same round in both sets

and

2) There are often independent questions about the same topics that are not such converted questions
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred
Kyle
Auron
Posts: 1127
Joined: Tue Jun 15, 2004 1:16 pm
Location: Seattle

Re: 2014 ICT: general discussion

Post by Kyle »

Madagascar Serpent Eagle wrote:For example, I wrote a bonus on la Quenelle that was in the DII packets several rounds after the DI bonus that had at least one, maybe two parts the same.
I wrote the D1 CE bonus without having seen the MI bonus you wrote for D2.
Kyle Haddad-Fonda
Harvard '09
Oxford '13
Locked