a new stat

[theMoMA]

Barring some revolution in which Mike Bentley invents a program that records average words before buzz, the stats that we put in to SQBS are pretty much the only things to work with for a new quizbowl stat.

The struggle with quizbowl stats is to find a way to relate individual performance to team performance; finding just how Player X's PPG relates to their team's win-loss record. So taking a page from baseball sabermetrics, I looked at the Win Share. In baseball, win shares are more complicated than I care to explain. But at its very core, it's a number that quantifies the amount of wins that each player adds to his team.

The win share is a backwards-looking stat. In other words, it doesn't attempt to be predictive, only to explain what happened. A common problem with stats is the attempt to blend explanatory power with predictive power, which I will not do. The win share is also backwards-looking in that it starts with team wins first, then assigns values to the players who contributed to those wins. In other words, it doesn't assume that Team X should have 10 wins based on its lineup; it looks at Team X's 7-3 record, then calculates each player's contribution to those seven wins.

So first...what inputs are needed? Just very basic ones, all of which are commonly spit out by SQBS. For each player, this data is necessary: player 10s, player -5s, player tossups heard, team bonus points, team wins, team losses, team 10s, team -5s. From this can be derived other necessary stats like bonuses heard, PPB, etc.

A major assumption that I made in the win share is that player contribution to bonus points is contingent on answering tossups. It's not necessarily an assumption that a player contributes the same percentage of tossup and bonus points to their team; it is simply a recognition that a team cannot score bonus points without the player first buzzing in to correctly answer a tossup. Hence, in the win share system, a tossup is worth 10 + PPB.

So essentially what the win share does is calculate each player's percentage contribution of their team's points, then multiplies that percentage by the team winning percentage.

Here's how it calculates each player's percentage of team contribution:

It starts off with the player's positive contributions: This is simply the number of 10s a player has multiplied by their team's points per bonus:

Plus = (player 10s * 10) + (player 10s * team PPB)

Then it calculates the player's negative contributions. This is slightly more complex. In addition to subtracting the number of points lost via -5s, this also attempts to account for crowding out of teammates' buzzes. It accomplishes this by calculating the percent chance that the player's teammates would get the tossup had the player not buzzed (obviously this is a very rough estimate) and multiplying it by the number of negs, then multiplying that number by the team's points per bonus plus ten. Though this is the rough number of points that a player costs his team via negging, it is expressed as a positive number.

The exact formula for the negative contributions is:

Minus = ((((team 10s * (player TUH / team TUH)) - player 10s) / (player tossups heard - player 10s)) * player negs * (team PPB + 10)) + (player negs * 5)

Breaking this down...the (player TUH / team TUH) is a necessary adjustment for players who didn't play for the entire tournament. The rest of that first part is basically just (team 10s that weren't earned by player X / team tossups heard that player X didn't receive 10 points on), a reasonable estimate of the chance that, had a player not buzzed, his team would have gotten 10 points. The second part is just the raw negative values of the player's negs.

So to get the TPC, or Total Player Contribution, simply take Plus - Minus.

TPC = Plus - Minus = ((player 10s * 10) + (player 10s * team PPB)) - ((((team 10s * (player TUH / team TUH)) - player 10s) / (player tossups heard - player 10s)) * player negs * (team PPB + 10)) + (player negs * 5)

At this point, all you have to do is find the total TPC for each team by summing the individual TPCs. Then take player TPC / total team TPC to find the player's contribution to the team, or %TPC. Then take the player's %TPC and multiply it by team win % to get each player's number of win shares. You can also multiply %TPC by total wins to get player wins, though this should be used with caution in tournaments in which the participants do not all play the same number of games.

So the formula is just:

(TPC / total team TPC) * team win % = win shares.

I worked out these numbers on Excel for the Illinois Open, and the results are available here. Though I also added player wins as the last column, the sheet is sorted by the second-to-last column, win shares, because the teams didn't play the same number of games.

I think the negative contribution stat works really well. You can see that negs on teams with higher PPB and better supporting players cost more points than on teams with lower PPB and fewer supporting tossup-scorers, as evidenced by Jerry's and Seth's neg totals being relatively similar, but Jerry's account for nearly 200 more penalty points.

I should stress that the win shares aren't meant to order players by skill; the stat only attempts to describe how valuable each player was to their team in that particular context.

[Captain Sinico]

This is pretty cool. I wonder if it isn't necessary to also compensate positive contributions by somehow accounting for crowding-out of teammate's buzzes.

MaS

[theMoMA]

This is pretty cool. I wonder if it isn't necessary to also compensate positive contributions by somehow accounting for crowding-out of teammate's buzzes.

MaS

My initial reaction is no, because no matter who gets the 10, as long as it's one of your teammates, it's always a good thing, and no adjustment is necessary. But I could be underthinking this...maybe if you elaborate on what you mean it will give me a better understanding of why this might be necessary.

Also, I should add...that "adjustment" of multiplying team 10s by (player TUH / team TUH) is only a quick-and-dirty approximation (what it's trying to get at is number of team tens while the player was in the game). Obviously if this stat were directly available in SQBS, it would be able to calculate team tens while in the game from the scoresheets without the extensive manual labor that makes the approximation useful.

[ValenciaQBowl]

I assume Mike is referring to dead ducks: when the other team has negged, and your team gets to take the "rebound." And even if he wasn't, this is a potential problem for this type of stat. All of us have played with (or are) the person who will vulch (to use an old term for interrupting even after the other team has negged) or at least snap the buzzer damn quick the millisecond the reader has finished the dead TU in order to get the points (and, I guess, the attendant "glory").

It's not a fatal problem for this measure, but it's something that could maybe skew it sometimes. I know when I've been fortunate to play with better players at CO and such, I'm always happy to let them take dead ducks, and I coach my players never to vulch and not to sweat their personal ppg. Value to a team can be discerned in other ways.

But your idea is pretty cool!

[Sima Guang Hater]

Andrew, Do you have some kind of program or macro that will do the computations for you? If so, I'd love to get a copy and run some recent tournaments through it (Fall, Parfait, Deep Bench, Bob next weekend, what have you).

This seems like a pretty interesting statistic, and in particular, the negative contribution column is interesting to note when you analyze your game.

EDIT: woo post 100.

[theMoMA]

Andrew, Do you have some kind of program or macro that will do the computations for you? If so, I'd love to get a copy and run some recent tournaments through it (Fall, Parfait, Deep Bench, Bob next weekend, what have you).

This seems like a pretty interesting statistic, and in particular, the negative contribution column is interesting to note when you analyze your game.

I'm pretty much a macrotard, so I have no idea how to set that up. It's pretty easy to set up a spreadsheet to do the calculations for you, but summing the team's TPC is kind of annoying (because there aren't an even number of players on each team, and after you resort the spreadsheet the numbers can get jumbled, so what I do is calculate them by sorting by team then using the sum feature, then just entering the sums manually. there's probably an easier way)

[theMoMA]

Also, with the vulturing thing...this is a limitation of current statkeeping methods. Until we get a device or program that lets us figure out average words before buzz, we can't figure out whose points are on early buzzes and whose are vultured tens.

However, this isn't a huge problem. Whoever gets the tossup gets the points, which is always a good thing, even if those points aren't as well-earned as those gained on early buzzes. Perhaps there is a small segment of terrible people who vulch their teammates out of points by zeroing tossups that others would have converted, but there's really no way of knowing this or measuring it (even with our hypothetical words-before-buzz device).

To run some kind of normalizing calculation on negs-against (such as take the number of negs that opponents have against your team, then run a similar calculation to the "minus" calculation above) is impossible because you don't know which 10s correspond with which negs without going too far in depth. Even if you could easily match up vultured 10s with negs, it's still an exercise in folly to normalize them out of a player's stats. The fact is that the player pressed the buzzer and got ten points, and his team then got, on average, their PPB to boot. Those points are real and must be accounted for, and tacitly assuming that the other teammates knew the answer by somehow redistributing those points doesn't make sense.

[cvdwightw]

This looks like a good stat. There is a tradeoff in mathematical modeling of anything between simplicity and accuracy, and I think this fits a nice balance.

The great thing about this stat is that it is more-or-less irrelevant of personal contributions to bonuses. As long as someone gets the tossup, you get roughly your expected bonus points. A more complex model would be to take game-by-game totals, although I haven't done anything to see how far of a deviation from the original stat a game-by-game model would have.

For this to be used predictively, the only major adjustment would be to compensate for losses as well. This does a good job at quantifying players' contributions to wins, but says nothing about how they performed in losses. As an explanatory stat this is fine. There are players who have an excellent command of giveaways so score like 100 ppg against novices (presumably in a winning cause) but drop to 20-40 against good teams (presumably in a losing cause). Thus, if a specialist was paired with one of these players, the specialist's win share would be significantly lower than their actual predictive score. Also, a good player who cannot carry an otherwise weak team to victory will have a low win share (fine) but this says nothing about his/her contributions with a better supporting cast.

Again, this is an excellent explanatory stat, and I think there are ways to modify it into a predictive stat. I look forward to future discussions on this topic.

[Gautam]

So, when dealing with powers, are these the changes to be made?

a. Player + = (Player Powers*15 + Player 10s*10) + ((Player 10s+Player 15s)*PPB)

I'm sure no one disagrees with that.

b. Let (((Player Powers*15 + Player 10s*10)/(Player 10s+Player 15s)) = X.

Then, Player - = ((((Team 10s+15s * (player TUH/Team TUH) - Player 10s+15s)/ (Player tossups heard - player 10s+15s))*player -5s * (team PPB + X)) + (player -5s *5)

The X is supposed to be like average points received per tossup answered by player. It won't raise the number by too much, (maybe like 1 or 2 points). This probably penalizes players who power a lot and neg a lot, but then again, their negative contrib. has to be lower, right?. There probably are better ways to do this. Someone help?

The rest of the figures are basically the same.[/list]

[Captain Sinico]

I'm not talking about vultures necessarily; I'm talking about the value of a buzz. A buzz where you're right is effectively worth less to your team if the other three members knew the answer and nobody on the other team did, for example. This is exactly analogous to the fact that a neg costs more than 5 points. Therefore, I wonder if a right answer's value shouldn't be corrected down to account for the fact that a player's right buzzes are potentially beating their teammate's right buzzes, in the same way that a wrong answer's value is corrected down to account for the fact that they are potentially beating their teammate's right buzzes. I'm not sure how one might do this or if it's necessary; it's just a thought.

Later,
MaS

[cvdwightw]

I've done an analysis (with a corrected BC for Irvine, since that 7.33 in round 2 is not right) on WIT.

Corrected formula: I subtracted from the original Player+ the value of (Team 10s*(Player TUH/TUH) - Player 10s)*(10+PPB)*Player10s, as, again, a rough estimate of what would have been converted had the player not buzzed.

Original Formula vs. Corrected Formula:
The top 11 remain the same rank-wise. No other participant is more than 1 away from the original ranking with the corrected ranking. In terms of win shares, this tends to skew win shares even more favorably toward higher scoring players. In the extreme case, Seth's win shares increase by more than 1 win using this corrected formula (his teammates' win shares decrease accordingly).

I know one tournament's worth of data doesn't make a case, but this correction doesn't appear to significantly mess up tournament "win share" rankings, although they do significantly affect the numerical value of those win shares. Here is where I think the dichotomy is: when you beat your teammates' potentially right buzzes with a correct buzz, you will still earn the expected bonus points, so you're only cheating your teammates out of 10 individual points. When you neg, you not only beat your teammates' potentially right buzzes, but you lose the PPB you could have gotten. Indeed, when altering the formula such that

Player+ = 10*Player10s - (Team 10s*(Player TUH/TUH) - Player 10s)*10*Player10s

(thus, the bonus points otherwise earned are not lost), we do see a small numerical correction but practically no change in the rankings. I think this makes logically the most sense as a correcting factor.

[theMoMA]

I'm not talking about vultures necessarily; I'm talking about the value of a buzz. A buzz where you're right is effectively worth less to your team if the other three members knew the answer and nobody on the other team did, for example. This is exactly analogous to the fact that a neg costs more than 5 points. Therefore, I wonder if a right answer's value shouldn't be corrected down to account for the fact that a player's right buzzes are potentially beating their teammate's right buzzes, in the same way that a wrong answer's value is corrected down to account for the fact that they are potentially beating their teammate's right buzzes. I'm not sure how one might do this or if it's necessary; it's just a thought.

Later,
MaS

I'm still not sure that this is entirely necessary, and I think it begins to blur the line between explaining what happened and trying to predict what might happen in the future (or attempts to measure something other than points contributed to the team, translated into wins). The correction for negs is necessary because negs have real detrimental value to a team and there is a real chance that a team could earn 10 + PPB. However, getting a ten before a teammate has never has a detrimental value to the team.

In the end, the best-case scenario if a player didn't buzz on a neg is that his team gets 10 points and receives a bonus, whereas the best-case scenario if a player didn't buzz on a ten is just the status quo. I don't believe any adjustment is necessary for this stat.

But that's not to say that making this adjustment could have useful explanatory power in some other light (such as measuring average worth of a buzz, or something similar to Samer's PATH), but in the framework of this stat, it doesn't make any sense to make this adjustment.

[cvdwightw]

I understand where both Mike and Andrew are coming from, and I think given my results it does not make sense to needlessly increase the complexity of the model to take this into account - the results will appear similar anyways (a correcting factor of <0.01 win shares) without additional calculations.

For a predictive model, I think the following things, for certain, need to be added:
1. A correcting term similar to what Mike describes, to determine the "value" of a player's buzz to his/her team.

2. A correcting term for winless teams, who obviously contribute something predictively but contribute 0 win shares (the idea being that a player who scores 20 ppg on a winless team is probably better than a player who scores 0 ppg on a 2-win team, but this is not currently implemented in the model).
3. A correcting term for games played, to estimate a player's value over the entire tournament.
4. A "strength-of-schedule" term for comparison across tournaments.

I would propose the following:
1. A correcting term to Player+ of -A*(Team10s - Player10s)/(TU Heard - Player10s)*10*Player10s, where A is an as-yet-undetermined constant. I'm not sure what exactly to make A, or whether to vary by team/player. An interesting side effect to this correction is that a team TPC is no longer roughly proportional to team total points, and can be highly skewed by the presence one player doing most of the scoring.

2. Add a "+1" term - a forfeit win over an imaginary team. This does not significantly affect the overall win shares for teams at the top of the pack, but will drastically improve the "position" of decent players stuck on bad teams. I'm not sure if this is a good thing, but it's the best I've come up with so far.

3. Dividing "Total Wins" by games played instead of total team games yields a score here. Alternatively, with some extra SQBS work we can find out a team's record in the games played, although with half-games and stuff this is probably too difficulty.

4. I'm not exactly sure how to do this one. Obviously people shouldn't get more credit for beating up on bad teams (although the win shares will rightly reflect this happening). Perhaps we use some kind of PAPG statistic to determine a team's "Strength of Schedule", I don't know.

[millionwaves]

I've made an excel spreadsheet to calculate each player's win share - you have to do it by individual team, but the process goes pretty quickly, now.

I tested it a couple of times, and I think I have everything correct, but if I don't, let me know and I'll try to fix it.

You can find it here: Excel Win Share Calculator.

I'll adjust it to include powers if we can come to a consensus for how to do them in the stat.

EDIT: The webspace is no longer down for maintainance.

EDIT 2: The sheet now has shiny if-then statements, so it now works for teams of less than four and people who don't get negs.

[Ray]

remember when paul came up with some sort of slope coefficient heh

[AuguryMarch]

remember when paul came up with some sort of slope coefficient heh

Ray, do YOU remember pistachios? They have a slope coefficient of 0.57, even higher than Seth Teitler's at ICT '07! Outstanding!

[Ray]

if they come up with the technology so that 2 men can have a baby together will you do it with me

[cdcarter]

Crunched the numbers for TIT:
http://www.quag.geek.nz/paste/9237ec5bb183cc99.txt

EDIT:
And the UGA mirror: http://www.quag.geek.nz/paste/089f3fc4231bd152.txt (I added PPTH because those numbers weren't in the posted stats. This is based on (PTS / GP*20).

MLK might take me some more time because they used brackets and the Scobo TC hosting, so my script gets all confused.

EDIT 2:
Not as long as I thought http://www.quag.geek.nz/paste/2cbf8cc567ca1236.txt