Some math calculation proponents, myself included, have argued that it is possible for a math calculation question to be pyramidal. Unfortunately, both sides in this debate have been working mostly off of gut feelings, since we never really had any solid definition of what makes a pyramidal tossup pyramidal, nor any data on how tossups were answered.http://hsapq.com/policy.html wrote: This article outlines the six reasons why HSAPQ feels that math calculation is not an appropriate topic for tossups in quizbowl.
1. It is not possible to write math calculation tossups in the pyramidal style
I do not seek to reopen the debate over math calculation in quiz bowl; this ground has been covered well enough. However, I would like to discuss pyramidality: how do we define it, and how can we analyze it? In doing so, I will attempt to show that regardless of whether math calculation is pyramidal, the results of playing math calculation questions produce similar buzzing patterns to playing pyramidal tossups.
The first issue is defining pyramidality. The QBWiki states:
How can we analyze if a tossup is pyramidal, though? Imagine if one were to graph the percentage of buzzes occurred on the y axis against the percentage of the question elapsed on the x-axis; I will call this a "cumulative buzz graph". Certainly, (0,0) and (100,100) are points on this curve for all questions; after none of the question has been read, no buzzes occur; after 100% of the question has been read, 100% of the buzzes have occurred (we normalize the y-axis against all "buzzes occurred", so we disregard teams that do not buzz on the tossup). What characteristics will the graph have? If we see sharp, vertical spikes, this is a sign of a difficulty cliff; at a certain point in the question, lots and lots of teams suddenly buzzed.http://doc-ent.com/qbwiki/index.php?title=Pyramidality wrote: Pyramidality is a concept in tossup-writing that states that clues should be arranged in descending order of difficulty, with the hardest information first and the easiest at the end, after the "For 10 points."
The question, then, is what a non-pyramidal question will look like, versus what a pyramidal question will look like on this graph. I claim they will look fundamentally different. When we think of non-pyramidal questions, we think of early buzzer races (because of misordered clues) or late buzzer races (because of misdirection, or too hard clues). If no buzzer races occur in the graph, then I will say that the graph has a "pyramidal buzz distribution."
The Goldfish Tournament
The idea of using cumulative buzz graphs is appealing, but until recently, the data needed to actually create them was unavailable. The Goldfish tournament changed that. We now know exactly where each team buzzed on a given question, for the admittedly small sample size of 50 teams on 30 questions. I have attached the cumulative buzz graph for this year's Goldfish tournament.
Each plot is a tossup; for a given value x on the x-axis, the point on the y-axis is the percentage of buzzes that occurred at or before x percent of the tossup had been read.
What do we notice? First, most of the plots seem to have the "pyramidal buzz distribution" discussed above; there don't seem to be too many buzzer races.
The turquoise plot towards the upper left of center contains the point (45,60), meaning after less than half the question had been read, 60% of the teams that would eventually buzz had already buzzed. That question only had 5 buzzes, though, so the sample size is quite small.
The dark blue plot in the lower right, on the other hand, had no buzzes until over halfway through the question; then, at around 78% of the way through, had a huge leap in the number of buzzes occurred. That corresponds with the end of the question text and the start of the 5-second pause, so we can suppose there was a buzzer race on the giveaway, and that the question was fairly hard.
This brings me full circle back to the original discussion point: is math calculation non-pyramidal? There are two math calculation questions in that graph; they are the ones colored white. Neither one is easily distinguishable from the remaining 28 tossups, which were pyramidal. Hence, I claim that the math calculation questions did have pyramidal buzz distributions.
You can look at the spreadsheet containing my data. As a brief explanation of my method: I read a sample question to myself at the rate I read IS sets, and found I read about 4 words a second. Hence, I considered each "second" after the question ended to be the equivalent of 4 words. I considered the length of a tossup to be the number of words needed to find all buzzes when the tossup was in progress, then let the effective length be the length plus the number of waiting seconds times 4; for math questions, one waits 10 seconds, for non-math, 5.
The x-axis of the graph is the percent of question time elapsed; letting each second after the tossup is dead be 4 words, this is simply the number of words read over the total number of effective words. The y-axis is the percentage of buzzes occurred; this is simply the number of buzzes before time t over the total number of buzzes on the question.
The math calculation questions are question 1 and 26, as observed through the reported answer times (they are the only ones which have buzzes between 6 and 10 seconds after the tossup was finished).
I welcome any comments on the methodology or conclusions I discussed here. I would rather avoid this turning into the usual discussion of math calculation in quiz bowl, though, and stay focused simply on the pyramidality aspect of calculation.
EDIT: Described wrong plot originally in analysis; fixed incorrect description.