## Quizbowl WPA

### Quizbowl WPA

So, I was thinking this morning about whether it would be possible to isolate the effects of specific game actions (say, getting a 10 or -5, or 30ing a bonus) on a team's eventual chances of winning a game. My first thought is that if you were simply to run a regression on a whole bunch of data from tournaments, using team 10's, -5's, bonus conversion to predict team winning percentage (on the tournament level) you should be able to get a pretty good estimate of the all other things being equal effect of, say, taking a -5 on your chances of winning a game.

Has anyone ever tried anything like this? Would it get useful results? Does anyone volunteer to gather all that data?

bms

Has anyone ever tried anything like this? Would it get useful results? Does anyone volunteer to gather all that data?

bms

Brian Saxton, The Ohio State University '12 (I hope)

- Kouign Amann
- Forums Staff: Moderator
**Posts:**1181**Joined:**Sat Dec 06, 2008 12:44 am**Location:**Morristown, NJ

### Re: Quizbowl WPA

Something out of Bill James could probably be adapted to work here. However, I'll leave it to the more mathematically inclined.

Aidan Mehigan

St. Anselm's Abbey School '12

Columbia University '16 | University of Oxford '17 | UPenn GSE '19

St. Anselm's Abbey School '12

Columbia University '16 | University of Oxford '17 | UPenn GSE '19

### Re: Quizbowl WPA

This probably isn't the most useful stat in quizbowl. The reason that WPA tells us something for baseball stats is that there are lots of small, discrete, non-scoring events that add up to relatively rare scores, so the true value of each play is not readily apparent. (And also, because there is a huge, manageable database of results between teams, most of which are within a couple of standards of deviation of each other in terms of performance).

In quizbowl, there are no instances that are recorded in the data that do not lead to a score change. We can probably expect a quizbowl WPA to tell us pretty much the same thing that points per game (or some kind of adjusted points per game that awards appropriate penalties for negs) would tell us. The only real deviants from that would be players who got an inordinate number of first tossups in a game (since quizbowl games are often more mismatched than baseball games, the team that gets the first tossup is almost certainly a better indicator of who would win the game, so a WPA formula would likely reward first-tossup buzzes highly, even relative to their actual value), or who got an unexpected number of tossups during close games, especially towards the end.

Either way, it'd be hard to run the regressions needed for this since there isn't an easily manipulable database of past results out there, and since unlike baseball, the format and difficulty of the games would often differ.

In quizbowl, there are no instances that are recorded in the data that do not lead to a score change. We can probably expect a quizbowl WPA to tell us pretty much the same thing that points per game (or some kind of adjusted points per game that awards appropriate penalties for negs) would tell us. The only real deviants from that would be players who got an inordinate number of first tossups in a game (since quizbowl games are often more mismatched than baseball games, the team that gets the first tossup is almost certainly a better indicator of who would win the game, so a WPA formula would likely reward first-tossup buzzes highly, even relative to their actual value), or who got an unexpected number of tossups during close games, especially towards the end.

Either way, it'd be hard to run the regressions needed for this since there isn't an easily manipulable database of past results out there, and since unlike baseball, the format and difficulty of the games would often differ.

Andrew Hart

Minnesota alum

Minnesota alum

### Re: Quizbowl WPA

Basically, Andrew's 100% right - I just don't see this carrying over too well.

Fred Morlan

University of Kentucky CoP, 2017

International Quiz Bowl Tournaments, co-owner

PACE

former (?) hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator

University of Kentucky CoP, 2017

International Quiz Bowl Tournaments, co-owner

PACE

former (?) hsqbrank manager, former NAQT writer & subject editor, former hsqb Administrator/Chief Administrator

- Mechanical Beasts
- Banned Cheater
**Posts:**5673**Joined:**Thu Jun 08, 2006 10:50 pm

### Re: Quizbowl WPA

Yeah, especially since while negs certainly deserve a value of greater than -5, it's also a very hard to determine value, since if you didn't neg, the other team might have beaten you to the question anyway, or maybe you didn't know it. Maybe the bonus would be on Ann Landers's biography, and maybe it would be "spell these three letter words." You have bunches of numbers, and not much predictive value--as opposed to few numbers and lots of predictive value, as in baseball.

Andrew Watkins

- at your pleasure
- Auron
**Posts:**1716**Joined:**Sun Aug 03, 2008 7:56 pm

### Re: Quizbowl WPA

I think that the best way to account for negs is to subtract the 15 points plus the average bonus conversion, which provides at least a rough estimate of how many points you would probably locked your team out of. However, even this has the issue of not accounting for different conversions in different subjects.

Douglas Graebner, Walt Whitman HS 10, Uchicago 14

"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,

http://avorticistking.wordpress.com/

"... imagination acts upon man as really as does gravitation, and may kill him as certainly as a dose of prussic acid."-Sir James Frazer,

*The Golden Bough*http://avorticistking.wordpress.com/

- AlphaQuizBowler
- Tidus
**Posts:**695**Joined:**Mon Dec 03, 2007 6:31 pm**Location:**Alpharetta, GA

### Re: Quizbowl WPA

I have proposed a similar stat to the new stat-keeping software.Anti-Climacus wrote:I think that the best way to account for negs is to subtract the 15 points plus the average bonus conversion, which provides at least a rough estimate of how many points you would probably locked your team out of. However, even this has the issue of not accounting for different conversions in different subjects.

William

Alpharetta High School '11

Harvard '15

Alpharetta High School '11

Harvard '15

- silverscreentest
- Lulu
**Posts:**76**Joined:**Fri Mar 21, 2008 12:00 pm**Location:**A legend in my own mind.

### Re: Quizbowl WPA

I read the thread title and thought someone was suggesting a public works project involving quizbowl. Instead of painting murals or building roads, it would be constructing buzzer systems or something.

Brick of Silver Screen Test.

### Introducing the Quizbowl Line Score, TU Margin, and Neg pric

Actually, during my senior year, I developed the quizbowl line score to analyze a statistic I call tossup differential: the number of tossups that would have to go to the losing team instead of the winning team in order to produce a tie.

First, to understand this, consider a line score, such as the following (in ACF format):
A brief explanation: the "-" indicates a correctly-answered tossup, "X" is a neg-5, "/" is a no-penalty wrong answer. The numbers are the running totals after each question. For example, on question 1, the left team got the tossup for 10 and 20 on the bonus, earning a total of 30.

Now here are the calculations to compute the TU Margin:
What this means is that, if the losing team had won two buzzer races on tossup the winning team got, the losing team would likely have won.

Now to interpret this statistic in terms of how much a neg hurts is difficult because we don't know whether the team that negged would have gotten the tossup anyways. As an approximation, we will assume that either team could have gotten the negged tossup with probability corresponding to the TU ratio of the remaining tossups. So (1-[#neggingteamsTU/20]) of the time, the price to the negging team is 5 points, (#neggingteamsTU/20) of the time, the negging team paid 5+10+PPB that they would have gotten, with an additional 10+bonuspts of the opponent on that tossup if the other team got it right.

So, if the tossup went dead we compute neg price = 5 + (10+PPB(negging team))*TU(by negging team)/20; which will usually approximate to 1/4 deltaTU + 5. If the other team rebounded the tossup, the neg price is higher: neg price = 5 + (10+PPB(negging team) + (10+opp.bonus))*TU(by negging team)/20 which approximates to 1/2 deltaTU + 5.

As an example, let's compute the cost of the four negs by the losing team in the above game. Of the four negs, two went dead and two were rebounded correctly. The cost of each of the two dead negs is 5 + (10+15.71)*7/20 = 5 + 25.71*0.35 = 14.00. The cost of the two rebounded negs is 5 + (10+15.71+20)*7/20 and 5 + (10+15.71+30)*7/20 respectively, which approximate to 20.00 and 24.50 points. Adding all of these numbers gives a total of 14+14+20+24.5=72.5 points. Thus, if the losing team had not negged at all, the margin of victory would have been just 27.5 points instead of 100.

Let me know what you think. If I've made any egregious errors, let me know.

First, to understand this, consider a line score, such as the following (in ACF format):

Code: Select all

```
+30 -|
/|X -5
+50 -|
+45 X|- 15
|- 45
-----+-----
+85 -|
105 -|
125 -|X 40
155 -|
|- 80
-----+-----
|- 100
/|X 95
/|/
150 X|- 115
180 -|
-----+-----
200 -|
|- 135
|- 165
230 -|X 160
260 -|
=====+=====
*260*| 160
```

Now here are the calculations to compute the TU Margin:

Code: Select all

```
Score 260 160
-TU Pts -100 -70
------- ---- ---
. 160 90
+Negsx5 +10 +20
------- --- ---
BonPts 170 110
10/TU 10 10
+PPB 17.00 15.71
------- ----- ------
27.00 +25.71
deltaTU=27.00+25.71=52.71
. (Margin of Victory) 100
TU Margin=-------------------=-----=1.90
. deltaTU 52.71
```

Now to interpret this statistic in terms of how much a neg hurts is difficult because we don't know whether the team that negged would have gotten the tossup anyways. As an approximation, we will assume that either team could have gotten the negged tossup with probability corresponding to the TU ratio of the remaining tossups. So (1-[#neggingteamsTU/20]) of the time, the price to the negging team is 5 points, (#neggingteamsTU/20) of the time, the negging team paid 5+10+PPB that they would have gotten, with an additional 10+bonuspts of the opponent on that tossup if the other team got it right.

So, if the tossup went dead we compute neg price = 5 + (10+PPB(negging team))*TU(by negging team)/20; which will usually approximate to 1/4 deltaTU + 5. If the other team rebounded the tossup, the neg price is higher: neg price = 5 + (10+PPB(negging team) + (10+opp.bonus))*TU(by negging team)/20 which approximates to 1/2 deltaTU + 5.

As an example, let's compute the cost of the four negs by the losing team in the above game. Of the four negs, two went dead and two were rebounded correctly. The cost of each of the two dead negs is 5 + (10+15.71)*7/20 = 5 + 25.71*0.35 = 14.00. The cost of the two rebounded negs is 5 + (10+15.71+20)*7/20 and 5 + (10+15.71+30)*7/20 respectively, which approximate to 20.00 and 24.50 points. Adding all of these numbers gives a total of 14+14+20+24.5=72.5 points. Thus, if the losing team had not negged at all, the margin of victory would have been just 27.5 points instead of 100.

Let me know what you think. If I've made any egregious errors, let me know.

Greg (Vanderbilt 2012, Wheaton North 2008)

### Re: Quizbowl WPA

Greg, I like where you're going with this, but I'm not sure that your assumption about dead tossups is valid, and I think you may be underestimating the effects of negs on rebounded tossups.

I'm using five major assumptions in my corrections:

1. A team that answers first with the correct answer will always get 10 tossup points and some amount of bonus points, the expected value of which is the overall tournament bonus conversion (not the bonus conversion in a particular game, as this is treated as an acceptable deviation from expectation). The only way tossups can "switch sides" are through negs and buzzer races.

2. Over the long term, any given team will win 50% of its buzzer races. Obviously this does not apply to the short term, e.g. 20 tossups.

3. Bonus conversion, as a rough indicator of what a team knows, can serve as a Bernoulli variable for whether a given team will get a tossup against an empty chair (if they're not stupid and neg). This is the exact same as saying whether that team would have gotten a tossup against any other team, given that the first team negged and the second team failed to pick off the rebound.

4. Furthermore, we can use the ratio of two team's bonus conversions as a Bernoulli variable for whether a given team would have gotten a tossup against any other team, given that the first team negged and the second team successfully picked off the rebound.

5. The effects of negs and buzzer races on a team's performance are roughly independent (there are certain cases where hoses or improper underlining will cause a buzzer race for a neg, but we assume these cases are negligible in well-written tournaments).

Of these, I'm least concerned about the validity of 1 and 2, and most concerned about 3 and 4 (I think teams with bonus conversions around 10 would be able to get about 13-15 out of 20 tossups against empty chairs instead of the 6-7 predicted by the model).

I denote the team that lost by team N (for "Negging") and the opposing team as team O (for "Opponent").

If the tossup went dead, team N loses 5 points plus the marginal utility of hearing all other clues (which is zero points if they didn't know it, and 10+bonus points if they did know it). If the tossup was picked off, then they lose 5 points, plus plus the marginal utility of hearing all other clues (which is zero points if the other team would have beaten them anyway and 10+BCN+10+BCO points if they wouldn't have; we can assume that since buzzer races are long-term distributed with p=1/2, their contributions to "would have" and "wouldn't have" are equal).

In the first case, we have a certainty of 5 points lost, and a certainty that the other team DID NOT KNOW the answer. In this case, I think your assumption is invalid, since you are assuming that the opposing team has a nonzero chance of answering the tossup (which is not true, since the team did not rebound the tossup). I think a better assumption for dead tossups looks at whether the negging team would have known the answer to X out of n un-rebounded tossups, approximated by a binomial distribution with p = percent per bonus (since we equate bonus conversion with the amount that a team actually does know). With a 15.71 bonus conversion (let's assume that's the overall tournament conversion), then we have a 27% chance of converting both unrebounded questions, a 50% chance of converting one question, and a 23% chance of converting neither question. Therefore, we actually get that the team lost 10+.27*(2*(10+BCN))+.5*(10+BCN) = 10+13.88+12.86 = 36.74 points off the two dead negs; this is about 9 points (~1 bonus part) higher than your value.

I'm less certain that it's a bad assumption in the second case, but I'm going to go ahead and try my alternative method on it anyway. Assuming team O had a 17.00 bonus conversion over the long term, then team O is expected to have gotten (17.00/(17.00+15.71)) = 52% of the tossups, and team N is expected to have gotten 48% (therefore, the difference is mainly in negs and random short-term fluctuation in buzzer race wins, as we expected). So, out of the two rebounded negs, we expect that team O would have gotten both questions anyway 27% of the time, they would have split the questions 50% of the time, and team N would have gotten both questions 23% of the time.

So, team N actually loses 10+.27*(0)+.50*(0+10+BCN+10+BCO)+.23*(2*(10+BCN+10+BCO)) = 10+.5*(20+32.71)+.23*(40+65.42) = 60.60 points off the two rebounded negs; this is about 15 points higher than your value.

So, in total, team N actually lost 97.34 points on the four negs.

Computing these values for team O's negs (two negs, both rebounded), we find that team N would have gotten both questions anyway 23% of the time, they would have split the questions 50% of the time, and team O would have gotten both questions 27% of the time. So, team O loses 10+.23*(0)+.50(10+BCO+10+BCN+0)+.27*(2*(10+BCO+10+BCN) = 10+.5*(20+32.71)+.27*(40+65.42) = 54.82 points off the two rebounded negs.

Therefore we can say that the total expected point differential due to negs is (# points lost by losing team due to negs - # points given up by winning team due to negs) = 97.34 - 54.82 = 42.52 points. So, if buzzer races were exactly at 50% for each team, we would expect team O to win by ~40-45 points based solely on effects of negs.

Therefore the actual expected point differential caused by team O winning more than 50% of buzzer races is (point differential - point differential due to negs) = 100 - 42.52 = 57.48. Dividing by deltaTU, I get 1.09 buzzer races that would have had to flip to the losing team in order for buzzer races to have changed the outcome of the game. Therefore, we can assume that the sum total of the negs accounts for a 1.90-1.09 = 0.81 tossup differential difference. If team N had not negged at all, we would have had (100-97.34)/52.71 = 0.05 buzzer races (in other words, had team N not negged at all, and team O negged the same questions, team N would have had a good chance to tie or win); if team O had not negged at all, and team N negged the same questions, we would have needed (100+54.82)/52.71 = 2.94 buzzer races to go differently for team N to win. These values are consistent with the model (team O blew 1.04 buzzer races' worth of breathing room by negging, while team N shot itself in the foot with 1.85 buzzer races' worth of catchup points, so we get a sum total of 0.81 buzzer races that the negs accounted for).

I'll present a second case. Suppose Team A defeats Team B, 255-150. Team A finishes the tournament 8-4 with 215 points per game and a bonus conversion of 18.33. Team B finishes the tournament 6-6 with 158.33 points per game and a bonus conversion of 12.88.

6 tossups were answered by team A without team B buzzing first.

3 tossups was answered by team A after a neg from team B.

3 tossups went dead with neither team earning or losing points.

5 tossups were answered by team B without team A buzzing first.

1 tossups were answered by team B after a neg from team A.

1 tossup went dead after a neg from team B.

Team A's stats: 9 tossups, 1 negs

Team B's stats: 7 tossups, 4 negs

Tossup differential: 105/(28.33+22.88) = 105/(51.21) = 2.05. Note that we do not care about the short-term bonus conversion because if a tossup switches sides, both team's bonus conversions will be different. We can only assume that the average difference is based on overall bonus conversions.

We expect Team A to get 61.1% of tossups against empty chairs. We expect Team B to get 42.9% of tossups against empty chairs. So then, we expect 58.7% of the tossups to go to Team A and 41.3% of the tossups to go to team B.

Team B loses 5+(.429*(10+12.88)) = 14.82 points from the dead neg, and 15+((3*.413*.587*.587)*(10+18.33+10+12.88)+(3*.413*.413*.587)*(2*(10+18.33+10+12.88))+(.413*.413*.413)*(3*(10+18.33+10+12.88)) = 15+(.426*51.21+.3*102.42+0.07*153.63) = 78.30 points from the three rebounded negs, for a total of 93.12 points.

Team A loses 5+(.587*(10+18.33+10+12.88)) = 35.06 points from the one rebounded neg.

So, the five negs resulted in a net loss of 58.06 points for team B. So now, we find that the tossup differential is (105-58.06)/51.21 = 0.92. In other words, if the teams had not negged, team B would have had to win less than 1 additional buzzer race to win, instead of the over two. We find that negs accounted for 1.13 (or 55%) of the tossup differential, while buzzer races accounted for 0.92 (or 45%) of the tossup differential.

I would be interested in seeing if anyone has a better approximation for "expected tossups converted against empty chairs" and "expected tossups converted against a team with bonus conversion X."

I'm using five major assumptions in my corrections:

1. A team that answers first with the correct answer will always get 10 tossup points and some amount of bonus points, the expected value of which is the overall tournament bonus conversion (not the bonus conversion in a particular game, as this is treated as an acceptable deviation from expectation). The only way tossups can "switch sides" are through negs and buzzer races.

2. Over the long term, any given team will win 50% of its buzzer races. Obviously this does not apply to the short term, e.g. 20 tossups.

3. Bonus conversion, as a rough indicator of what a team knows, can serve as a Bernoulli variable for whether a given team will get a tossup against an empty chair (if they're not stupid and neg). This is the exact same as saying whether that team would have gotten a tossup against any other team, given that the first team negged and the second team failed to pick off the rebound.

4. Furthermore, we can use the ratio of two team's bonus conversions as a Bernoulli variable for whether a given team would have gotten a tossup against any other team, given that the first team negged and the second team successfully picked off the rebound.

5. The effects of negs and buzzer races on a team's performance are roughly independent (there are certain cases where hoses or improper underlining will cause a buzzer race for a neg, but we assume these cases are negligible in well-written tournaments).

Of these, I'm least concerned about the validity of 1 and 2, and most concerned about 3 and 4 (I think teams with bonus conversions around 10 would be able to get about 13-15 out of 20 tossups against empty chairs instead of the 6-7 predicted by the model).

I denote the team that lost by team N (for "Negging") and the opposing team as team O (for "Opponent").

If the tossup went dead, team N loses 5 points plus the marginal utility of hearing all other clues (which is zero points if they didn't know it, and 10+bonus points if they did know it). If the tossup was picked off, then they lose 5 points, plus plus the marginal utility of hearing all other clues (which is zero points if the other team would have beaten them anyway and 10+BCN+10+BCO points if they wouldn't have; we can assume that since buzzer races are long-term distributed with p=1/2, their contributions to "would have" and "wouldn't have" are equal).

In the first case, we have a certainty of 5 points lost, and a certainty that the other team DID NOT KNOW the answer. In this case, I think your assumption is invalid, since you are assuming that the opposing team has a nonzero chance of answering the tossup (which is not true, since the team did not rebound the tossup). I think a better assumption for dead tossups looks at whether the negging team would have known the answer to X out of n un-rebounded tossups, approximated by a binomial distribution with p = percent per bonus (since we equate bonus conversion with the amount that a team actually does know). With a 15.71 bonus conversion (let's assume that's the overall tournament conversion), then we have a 27% chance of converting both unrebounded questions, a 50% chance of converting one question, and a 23% chance of converting neither question. Therefore, we actually get that the team lost 10+.27*(2*(10+BCN))+.5*(10+BCN) = 10+13.88+12.86 = 36.74 points off the two dead negs; this is about 9 points (~1 bonus part) higher than your value.

I'm less certain that it's a bad assumption in the second case, but I'm going to go ahead and try my alternative method on it anyway. Assuming team O had a 17.00 bonus conversion over the long term, then team O is expected to have gotten (17.00/(17.00+15.71)) = 52% of the tossups, and team N is expected to have gotten 48% (therefore, the difference is mainly in negs and random short-term fluctuation in buzzer race wins, as we expected). So, out of the two rebounded negs, we expect that team O would have gotten both questions anyway 27% of the time, they would have split the questions 50% of the time, and team N would have gotten both questions 23% of the time.

So, team N actually loses 10+.27*(0)+.50*(0+10+BCN+10+BCO)+.23*(2*(10+BCN+10+BCO)) = 10+.5*(20+32.71)+.23*(40+65.42) = 60.60 points off the two rebounded negs; this is about 15 points higher than your value.

So, in total, team N actually lost 97.34 points on the four negs.

Computing these values for team O's negs (two negs, both rebounded), we find that team N would have gotten both questions anyway 23% of the time, they would have split the questions 50% of the time, and team O would have gotten both questions 27% of the time. So, team O loses 10+.23*(0)+.50(10+BCO+10+BCN+0)+.27*(2*(10+BCO+10+BCN) = 10+.5*(20+32.71)+.27*(40+65.42) = 54.82 points off the two rebounded negs.

Therefore we can say that the total expected point differential due to negs is (# points lost by losing team due to negs - # points given up by winning team due to negs) = 97.34 - 54.82 = 42.52 points. So, if buzzer races were exactly at 50% for each team, we would expect team O to win by ~40-45 points based solely on effects of negs.

Therefore the actual expected point differential caused by team O winning more than 50% of buzzer races is (point differential - point differential due to negs) = 100 - 42.52 = 57.48. Dividing by deltaTU, I get 1.09 buzzer races that would have had to flip to the losing team in order for buzzer races to have changed the outcome of the game. Therefore, we can assume that the sum total of the negs accounts for a 1.90-1.09 = 0.81 tossup differential difference. If team N had not negged at all, we would have had (100-97.34)/52.71 = 0.05 buzzer races (in other words, had team N not negged at all, and team O negged the same questions, team N would have had a good chance to tie or win); if team O had not negged at all, and team N negged the same questions, we would have needed (100+54.82)/52.71 = 2.94 buzzer races to go differently for team N to win. These values are consistent with the model (team O blew 1.04 buzzer races' worth of breathing room by negging, while team N shot itself in the foot with 1.85 buzzer races' worth of catchup points, so we get a sum total of 0.81 buzzer races that the negs accounted for).

I'll present a second case. Suppose Team A defeats Team B, 255-150. Team A finishes the tournament 8-4 with 215 points per game and a bonus conversion of 18.33. Team B finishes the tournament 6-6 with 158.33 points per game and a bonus conversion of 12.88.

6 tossups were answered by team A without team B buzzing first.

3 tossups was answered by team A after a neg from team B.

3 tossups went dead with neither team earning or losing points.

5 tossups were answered by team B without team A buzzing first.

1 tossups were answered by team B after a neg from team A.

1 tossup went dead after a neg from team B.

Team A's stats: 9 tossups, 1 negs

Team B's stats: 7 tossups, 4 negs

Tossup differential: 105/(28.33+22.88) = 105/(51.21) = 2.05. Note that we do not care about the short-term bonus conversion because if a tossup switches sides, both team's bonus conversions will be different. We can only assume that the average difference is based on overall bonus conversions.

We expect Team A to get 61.1% of tossups against empty chairs. We expect Team B to get 42.9% of tossups against empty chairs. So then, we expect 58.7% of the tossups to go to Team A and 41.3% of the tossups to go to team B.

Team B loses 5+(.429*(10+12.88)) = 14.82 points from the dead neg, and 15+((3*.413*.587*.587)*(10+18.33+10+12.88)+(3*.413*.413*.587)*(2*(10+18.33+10+12.88))+(.413*.413*.413)*(3*(10+18.33+10+12.88)) = 15+(.426*51.21+.3*102.42+0.07*153.63) = 78.30 points from the three rebounded negs, for a total of 93.12 points.

Team A loses 5+(.587*(10+18.33+10+12.88)) = 35.06 points from the one rebounded neg.

So, the five negs resulted in a net loss of 58.06 points for team B. So now, we find that the tossup differential is (105-58.06)/51.21 = 0.92. In other words, if the teams had not negged, team B would have had to win less than 1 additional buzzer race to win, instead of the over two. We find that negs accounted for 1.13 (or 55%) of the tossup differential, while buzzer races accounted for 0.92 (or 45%) of the tossup differential.

I would be interested in seeing if anyone has a better approximation for "expected tossups converted against empty chairs" and "expected tossups converted against a team with bonus conversion X."

Dwight Wynne

socalquizbowl.org

UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

socalquizbowl.org

UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

- Mechanical Beasts
- Banned Cheater
**Posts:**5673**Joined:**Thu Jun 08, 2006 10:50 pm

### Re: Quizbowl WPA

Yeah, the issue with your model (the reason you're dissatisfied with 3 and 4 giving you 6-7 tossups for a ppb of 10) is that a tossup is easier than the mean difficulty of a bonus part. (Tossups are what, usually the difficulty of an easy part, or a little harder?) So realistically, you want to derive how a team does on easy parts from how it does on bonuses in general (since teams that get 10ppb don't get 10 10 10 10 10 10 10 10 10 10; they get 10 0 20 0 0 30 10 10 0 20 or whatever).*cvdwightw wrote:I would be interested in seeing if anyone has a better approximation for "expected tossups converted against empty chairs" and "expected tossups converted against a team with bonus conversion X."

If we assume that bonuses are written to the 85-50-15 standard, which is mostly untrue, but whatever, then a team that earned 10ppb actually answered 65 other parts for every 85 easy parts it got. So the proportion of bonuses-not-bageled should be 85/150 = 17/30 = 11.333/20 tossups.

The above analysis could indeed be bogus and perhaps this doesn't reward teams that do, in fact, know some deeper things enough. It also relies on a model that's probably not true.

* Of course, if you miss the easy part and get the middle, it comes out as a 10. This isn't really a problem, because we're not interested in how well you do against some set canon of easy-part-or-tossup answers. What we care about is looking at some conversion figure that doesn't equate 10ppb to only answering 6 tossups. Using the model of "do you know SOMETHING about the topic we picked as a bonus, and we don't care if you know two or three things" works well enough.

Andrew Watkins

### Re: Quizbowl WPA

I don't get your math, but everything else looks solid. Suppose instead we use this model to extrapolate tossup performance against empty chairs from bonus performance: there are some subjects I know a lot about, some that I know nothing about, and many that I know something about. When averaging these all up, I get a certain number X as my bonus conversion. Dividing by 30, I get that I know X/30 percent of all bonus parts. I can roughly approximate this as a (X/30)^3 chance of getting a 30, a 3*((X/30)^2)*((30-X)/30) chance of getting a 20, a 3*(X/30)*(((30-X)/30)^2) chance of getting a 10, and a ((30-X)/30)^3 chance of getting a 0. All I care about is whether or not I get a 0 (if I know something, then I'll probably get the tossup at some point), which is ((30-X)/30)^3. Subtract that from 1 and multiply by 20 to get the expected number of tossups.everyday847 wrote:Yeah, the issue with your model (the reason you're dissatisfied with 3 and 4 giving you 6-7 tossups for a ppb of 10) is that a tossup is easier than the mean difficulty of a bonus part...

If we assume that bonuses are written to the 85-50-15 standard, which is mostly untrue, but whatever, then a team that earned 10ppb actually answered 65 other parts for every 85 easy parts it got. So the proportion of bonuses-not-bageled should be 85/150 = 17/30 = 11.333/20 tossups...

Using the model of "do you know SOMETHING about the topic we picked as a bonus, and we don't care if you know two or three things" works well enough.

As some benchmarks:

5 ppb: 8.43 tossups answered

10 ppb: 14.07 tossups answered

12 ppb: 15.68 tossups answered

15 ppb: 17.5 tossups answered (this actually corresponds to an 87.5% conversion rate for an average team, in the center of the 85-90% tossup conversion rate target)

17 ppb: 18.37 tossups answered

20 ppb: 19.26 tossups answered

22 ppb: 19.62 tossups answered

I'm still not sure that this works for two teams that

*both*know the answer and determining the chances of either of them buzzing first, but this solves the empty chair problem.

Dwight Wynne

socalquizbowl.org

UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

socalquizbowl.org

UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

- Mechanical Beasts
- Banned Cheater
**Posts:**5673**Joined:**Thu Jun 08, 2006 10:50 pm

### Re: Quizbowl WPA

Yeah, I like your numbers better, and I was asking myself what I was doing to numbers as I was doing my math anyway.cvdwightw wrote:I don't get your math, but everything else looks solid. Suppose instead we use this model to extrapolate tossup performance against empty chairs from bonus performance: there are some subjects I know a lot about, some that I know nothing about, and many that I know something about. When averaging these all up, I get a certain number X as my bonus conversion. Dividing by 30, I get that I know X/30 percent of all bonus parts. I can roughly approximate this as a (X/30)^3 chance of getting a 30, a 3*((X/30)^2)*((30-X)/30) chance of getting a 20, a 3*(X/30)*(((30-X)/30)^2) chance of getting a 10, and a ((30-X)/30)^3 chance of getting a 0. All I care about is whether or not I get a 0 (if I know something, then I'll probably get the tossup at some point), which is ((30-X)/30)^3. Subtract that from 1 and multiply by 20 to get the expected number of tossups.everyday847 wrote:Yeah, the issue with your model (the reason you're dissatisfied with 3 and 4 giving you 6-7 tossups for a ppb of 10) is that a tossup is easier than the mean difficulty of a bonus part...

If we assume that bonuses are written to the 85-50-15 standard, which is mostly untrue, but whatever, then a team that earned 10ppb actually answered 65 other parts for every 85 easy parts it got. So the proportion of bonuses-not-bageled should be 85/150 = 17/30 = 11.333/20 tossups...

Using the model of "do you know SOMETHING about the topic we picked as a bonus, and we don't care if you know two or three things" works well enough.

As some benchmarks:

5 ppb: 8.43 tossups answered

10 ppb: 14.07 tossups answered

12 ppb: 15.68 tossups answered

15 ppb: 17.5 tossups answered (this actually corresponds to an 87.5% conversion rate for an average team, in the center of the 85-90% tossup conversion rate target)

17 ppb: 18.37 tossups answered

20 ppb: 19.26 tossups answered

22 ppb: 19.62 tossups answered

I'm still not sure that this works for two teams thatbothknow the answer and determining the chances of either of them buzzing first, but this solves the empty chair problem.

Andrew Watkins

### Re: Quizbowl WPA

And here are the statistical results (based on the Goldfish tournament) for predicting whether a negging team would have gotten the tossup right given that the opponents got it right at the giveaway.

%chance(N gets it)=19.487% + 2.989% (PPBN) - 1.880% (PPBO)

The R^2 value of this regression is 0.587. This is good in comparison to Dwight's approximation, which is more inaccurate than the simple mean.

%chance(N gets it)=19.487% + 2.989% (PPBN) - 1.880% (PPBO)

The R^2 value of this regression is 0.587. This is good in comparison to Dwight's approximation, which is more inaccurate than the simple mean.

Greg (Vanderbilt 2012, Wheaton North 2008)

### Re: Quizbowl WPA

That does look like a much better approximation to use. Are you on board with my approximation of the "would have gotten against empty chairs if hadn't negged" statistic and use of (long-run) overall bonus conversion instead of the single-game bonus conversion?

If so, then it seems that there is a reasonable quantification of the effects of negs and buzzer races on a game.

Step 1: Compute the tossup margin as given in Greg's first post (essentially, point differential/deltaTU, where deltaTU is the combined expected score given that a team answers a tossup).

Step 2: Compute the effects of negs on each team's performance:

2A. A number of negs will not be rebounded by the other team. Compute the expected points lost as follows:

- find the parameter

- use the parameter

-add this to 5*

2B. A number of negs will be rebounded by the other team. Compute the expected points lost as follows:

- find the parameter

-use the parameter

-add this to 5*

2C. Add the expected points lost due to dead negs and the expected points lost due to rebounded negs.

2D. Repeat 2A-2C for the other team.

Step 3: Take the point differential, subtract the result of Step 2 for the losing team (points lost by the losing team due to negging), and add the result of Step 2 for the winning team (points the winning team lost by negging). Divide by deltaTU to find the new TU margin. This represents the expected number of buzzer races that would have had to go the other way in order for the losing team to have won, if no negs had occurred. Note that this value may be negative, in which case it means we would have expected the other team to win if no negs had occurred, and it may be more than the original TU margin, in which case it means that the winning team could have won by more if it hadn't negged.

Step 4: Subtract this value from the original TU margin to find the expected TU margin due to negs.

This may help teams quantitatively answer the question "Are we negging too much?" Certainly, if the "points lost to buzzer races" is negative, then the team is negging too much.

If so, then it seems that there is a reasonable quantification of the effects of negs and buzzer races on a game.

Step 1: Compute the tossup margin as given in Greg's first post (essentially, point differential/deltaTU, where deltaTU is the combined expected score given that a team answers a tossup).

Step 2: Compute the effects of negs on each team's performance:

2A. A number of negs will not be rebounded by the other team. Compute the expected points lost as follows:

- find the parameter

*p*, given by*p*= 1-(((30-*X*)/30)^3), where*X*is the negging team's bonus conversion. This gives the chance that the team would have gotten the tossup had it not negged.- use the parameter

*p*in a binomial distribution; sum from*n*= 0 to*d*(=number of dead negs) of (*d*C*n*)*(*p*^*n*)*((1-*p*)^(*d*-*n*))*(*n**(10+*X*))-add this to 5*

*d*2B. A number of negs will be rebounded by the other team. Compute the expected points lost as follows:

- find the parameter

*q*, given by*q*= .19487+.02989**X*-.01880**Y*, where*Y*is the opposing team's bonus conversion.-use the parameter

*q*in a binomial distribution; sum from*n*= 0 to*r*(=number of rebounded negs) of (*r*C*n*)*(*q*^*n*)*((1-*q*)^(*r*-*n*))*(*n**(10+*X*+10+*Y*))-add this to 5*

*r*2C. Add the expected points lost due to dead negs and the expected points lost due to rebounded negs.

2D. Repeat 2A-2C for the other team.

Step 3: Take the point differential, subtract the result of Step 2 for the losing team (points lost by the losing team due to negging), and add the result of Step 2 for the winning team (points the winning team lost by negging). Divide by deltaTU to find the new TU margin. This represents the expected number of buzzer races that would have had to go the other way in order for the losing team to have won, if no negs had occurred. Note that this value may be negative, in which case it means we would have expected the other team to win if no negs had occurred, and it may be more than the original TU margin, in which case it means that the winning team could have won by more if it hadn't negged.

Step 4: Subtract this value from the original TU margin to find the expected TU margin due to negs.

This may help teams quantitatively answer the question "Are we negging too much?" Certainly, if the "points lost to buzzer races" is negative, then the team is negging too much.

Dwight Wynne

socalquizbowl.org

UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

socalquizbowl.org

UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry