What is the Best Way to Tell How Good a Player Actually Is?

Dormant threads from the high school sections are preserved here.
Locked
jesposito
Lulu
Posts: 18
Joined: Sun Dec 17, 2017 9:38 pm

What is the Best Way to Tell How Good a Player Actually Is?

Post by jesposito »

Looking at a lot of tournaments recently, I became interested in player rankings. I noticed that, much like points per game in basketball, points per game isn't always the best indicator of the best players. It is hard to know when a player is just the best player on a bad team, instead of just the best player. Also, it is pretty susceptible to getting messed up by small sample sizes, a problem shared by power percentage. I was wondering if anyone had a better way to rank players based on statistics, whether it is another existing statistic, a team measure, or (best of all) a formula. Thanks!
Joe Esposito
Aurora, Ohio, 2019
Ohio State University, 2023
User avatar
tksaleija
Wakka
Posts: 240
Joined: Fri Jun 30, 2017 8:27 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by tksaleija »

As odd as it may sound, PPB is a good indicator for how strong a player might be. By definition, bonuses are not indicative of one player, but they can be used to observe possible trends. For example, a player may be putting up 50-60 PPG on average, but his team is only getting around 10-12 PPB. Again, this is not a definite answer, but it shows that he or she may not have the deeper knowledge that is needed for bonuses. On the other hand (and I'll use a real-world example here), Sarod Nori of Okemos is a phenomenal player who puts up insane PPG averages, sometimes numbering in the hundreds. In addition to this, Okemos also has a superb PPB average, usually around high teens to low twenties. Now, if you observe that the other players are also very strong, then this is a moot point, but, in the case of Okemos, Sarod is far and away the strongest player on their team when it comes to tossups, so it can be reasonably (though not concretely) inferred that he makes the most contributions to his teams PPB. Again, this is all very specific to certain circumstances, but observing the stats of other players and a team's overall PPB can sometimes serve as an indicator to the "true strength" of a player.

Edited to change "decent PPB" to "superb PPB" so as to not seem as though I'm demeaning a notable accomplishment like 19+ PPB
Aleija Rodriguez
University of Michigan 202x
MCMC/MCCC 2019
High Dependency Unit
Yuna
Posts: 864
Joined: Thu Nov 15, 2012 6:45 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by High Dependency Unit »

I'm a big fan of powers per game as a base statistic.
Michael Borecki
Middlesex Middle '13,
Darien (co-captain) '17,
Bowdoin College (club president) '21
The Sawing-Off of Manhattan Island
Rikku
Posts: 433
Joined: Fri Feb 24, 2017 4:41 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by The Sawing-Off of Manhattan Island »

2017 in amusement parks wrote:I'm a big fan of powers per game as a base statistic.
Agreed; I find power percentage when used in context is also helpful to distinguish how good specialists are (A player with something like 50/10 probably has really really deep lockdown knowledge in one field, which you can't necessarily say for a generalist that goes 50/50, who may just have random pockets across many disciplines). Power percentage can be skewed by low players with low buzz numbers though, so you definitely need to keep an eye on ppg as well.
User avatar
SirMrGuy
Lulu
Posts: 39
Joined: Tue Sep 26, 2017 10:50 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by SirMrGuy »

In my opinion, player performance is too dependent on teammates and level of specialization to really be able to get a full picture from stats alone. This is especially true if multiple players on a team have a lot of overlap (e.g. two literature players), or if players' strength in one subject is particularly strong in a particular circuit, creating overlap with other teams as well.

That aside, I agree with Michael and Vishwa that powers per game and power ratio are reasonably good indicators of strength, albeit with several large grains of salt.
Steven Liu

Cedar Drive MS '15
High Tech '19
MIT '23
I do mathy stuff for Groger Ranks.

"I'm going insane because I miss Steven so much" -Michael Li
Sarod Nori
Lulu
Posts: 25
Joined: Sun Jun 18, 2017 7:51 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by Sarod Nori »

I am a big believer in using win shares to determine the strength of a player (https://www.qbwiki.com/wiki/Hart's_Win_Share_Statistic). This is similar to PPG but attempts to show how negs prevent other people on a team from answering and can therefore cause a team to lose up to 95 points (neg for team A + power for team B + 30 on the bonus for team B + possible power for team A + possible 30 on the bonus for team A). An added benefit to this is that each person on the team receives a number that is (roughly) equivalent to their involvement in a team's success throughout a tournament, allowing for calculations such as team balance (how well a team is able to survive if it loses its top scorer).

It was stated earlier in this thread that PPB is a good indicator of a player's strength. However, I don't believe that this is necessarily true. Perhaps this is due to the difficulty of defining what exactly it means to be "good" but, assuming for the sake of simplicity that PPG correlates directly to a player's strength, PPB can't supplement PPG because tossups and bonuses are fundamentally different. For one thing, a team does not receive more points if it answers a bonus before the moderator is done reading it (other than at timed tournaments) and (excluding PACE) a team does not have to compete with the other team to see who can answer the bonus first. In tossups, it is in one's best interest to buzz in early (even after power) because failure to do so will result in the other team buzzing in and earning points.

This makes it difficult to quantify how much of a team's PPB is due to a single person; most bonuses are one or two clue questions that almost always have giveaways, letting teams identify what the answer is without having "true" knowledge (everybody can answer a bonus part that asks for the author of Romeo and Juliet, while not everyone can answer a tossup on Shakespeare based on some obscure tidbit of Romeo and Juliet); giveaways in tossups are mostly after the FTP and someone may buzz in correctly before this point in the question.

On the other hand, I strongly believe in using PPB in order to determine the strength of a team (with a few factors that adjust for difficulty differences between packets), but that's a different story altogether.
Sarod Nori
Okemos '18
Illinois '22
Cornell '25
browen
Lulu
Posts: 39
Joined: Tue Aug 02, 2016 1:33 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by browen »

Like Michael, I use powers to rate players. When I draft my preseason rankings each year, I look at how many powers each team is returning and then go from there (checking out B-team players, seeing if they were at full strength at nats, etc.). Sometime you see fourth scorers on top teams score something like 13-10-5. In those cases I increase the number of powers for those players as they likely have one section of a subject fully locked down and will be spending their summer learning clues from their now-graduated former teammates. One player I allocated "bonus powers" last year was Rohan Cherukuri of Harker, who as a ninth grader went 11-12-2 as the third scorer on a team that finished tied for twenty-fifth at HSNCT. I move teams down the list if a student has more negs than powers in a tournament.
Brian Owen
Dorman High School Assistant Coach (2014-present)
Dorman '13
5/31/2009: Never Forget
User avatar
El Salvadoreno
Rikku
Posts: 346
Joined: Wed Nov 30, 2016 7:45 am
Location: Northwest Suburbs, IL

Re: What is the Best Way to Tell How Good a Player Actually

Post by El Salvadoreno »

I actually disagree that powers is the best way to judge a player. Sure powers are important, but 10s are just as important. If a player has a low power percentage, but gets nearly every tossup, that person is arguably better than a player with more powers, but that is more specialized (especially in an extreme manner). I agree in general that stats are imperfect, but PPG is probably best (RE: Weiner's Law #8)
Ricky "Slick Rick/Rico Suave" Rivera
Carmel Catholic- Class of '17
Notre Dame (Mendoza)- Class of '22

My Namesake
User avatar
c*t
Wakka
Posts: 238
Joined: Sat Dec 03, 2016 8:34 pm
Location: Richmond, VA

Re: What is the Best Way to Tell How Good a Player Actually

Post by c*t »

It comes down to the player in the context of their team. The best player on a weak team, assuming they're decent, probably has a high PPG and a respectably, but low power ratio. What Ricky's saying could be true, but it's also possible the high power percentage player you're comparing them to is on a really, really good team, and just doesn't get 10s because their team either powers everything or there's another player that picks up the 10s from negs. I think raw number of powers is *usually* a safe way to go, though that's obviously also distorted when you have a team of four really strong players.
Catherine Qian (she/her/hers)
Maggie L. Walker Governor's School '18
University of Pennsylvania '22
Secretary, PACE
User avatar
db0wman
Wakka
Posts: 142
Joined: Fri Jul 21, 2017 3:37 pm
Location: Champaign, IL

Re: What is the Best Way to Tell How Good a Player Actually

Post by db0wman »

https://docs.google.com/document/d/1LeD ... sp=sharing

I created a very crude, simple method of ranking players. It is definitely flawed, so feel free to post any criticism you have.
Dylan Bowman
Uni '20; Illinois '23
User avatar
El Salvadoreno
Rikku
Posts: 346
Joined: Wed Nov 30, 2016 7:45 am
Location: Northwest Suburbs, IL

Re: What is the Best Way to Tell How Good a Player Actually

Post by El Salvadoreno »

cat wrote:It comes down to the player in the context of their team. The best player on a weak team, assuming they're decent, probably has a high PPG and a respectably, but low power ratio. What Ricky's saying could be true, but it's also possible the high power percentage player you're comparing them to is on a really, really good team, and just doesn't get 10s because their team either powers everything or there's another player that picks up the 10s from negs. I think raw number of powers is *usually* a safe way to go, though that's obviously also distorted when you have a team of four really strong players.
But this holds true for more than best players on weak teams. Because powers happen less than 10s in general, a player who is able to get a lot of 10s can win games, where as a player with mostly powers but less overall tossups is not as powerful in that since unless they are in a group. What I am saying is that a player with wide breadth will beat a player with depth but not bredth, hence why I think powers alone are flawed. Obviously, players with both will always win and have high power counts, but they will also have high PPGs. In other words, PPG is better able to consolidate the import of both depth and breadth.
Ricky "Slick Rick/Rico Suave" Rivera
Carmel Catholic- Class of '17
Notre Dame (Mendoza)- Class of '22

My Namesake
Wartortullian
Rikku
Posts: 376
Joined: Mon Dec 19, 2016 10:02 pm
Location: New Haven, CT
Contact:

Re: What is the Best Way to Tell How Good a Player Actually

Post by Wartortullian »

Even with perfect information (i.e. we could run infinitely long games in which every possible combination of players is pitted against every other possible combination), any ranking of players is doomed to be imperfect. This is because skill is non-transitive. Consider the following three hypothetical players in a hypothetical tournament:

Alice can get every science question on the lead-in, every literature question outside of power, and nothing else.

Bob can get every history question on the lead-in, every science question outside of power, and nothing else.

Charlie can get every literature question on the lead-in, every history question outside of power, and nothing else.

Assume the tournament contains equal amounts of all three categories. In head-to-head games, Alice will beat Bob, Bob will beat Charlie, and Charlie will beat Alice. It is therefore impossible to say that one player is better than another, because under any reasonable definition, "betterness" must be transitive.

This is the main reason why no one uses PATH anymore, and why PPG, despite its flaws, is still the main method of ranking individuals. Advanced stats (as amazing as they are) probably won't affect individual rankings, because unless every player is a perfect generalist, it's impossible to definitively answer the question.
Last edited by Wartortullian on Thu Mar 01, 2018 6:15 pm, edited 1 time in total.
Matt
browen
Lulu
Posts: 39
Joined: Tue Aug 02, 2016 1:33 pm

Re: What is the Best Way to Tell How Good a Player Actually

Post by browen »

El Salvadoreno wrote:
cat wrote:It comes down to the player in the context of their team. The best player on a weak team, assuming they're decent, probably has a high PPG and a respectably, but low power ratio. What Ricky's saying could be true, but it's also possible the high power percentage player you're comparing them to is on a really, really good team, and just doesn't get 10s because their team either powers everything or there's another player that picks up the 10s from negs. I think raw number of powers is *usually* a safe way to go, though that's obviously also distorted when you have a team of four really strong players.
But this holds true for more than best players on weak teams. Because powers happen less than 10s in general, a player who is able to get a lot of 10s can win games, where as a player with mostly powers but less overall tossups is not as powerful in that since unless they are in a group. What I am saying is that a player with wide breadth will beat a player with depth but not bredth, hence why I think powers alone are flawed. Obviously, players with both will always win and have high power counts, but they will also have high PPGs. In other words, PPG is better able to consolidate the import of both depth and breadth.
Breadth won't get you far late in tournaments. If there is a one-up match between someone who went 9-47-3 in the prelims versus someone who went 22-15-8 I'm picking the latter to win every time. Low negs is another stat I look at regarding when rating players. If a top player isn't negging at least once a game they are playing far too conservative on the buzzer.
Brian Owen
Dorman High School Assistant Coach (2014-present)
Dorman '13
5/31/2009: Never Forget
User avatar
34 + P.J. Dozier
Wakka
Posts: 197
Joined: Sun Oct 15, 2017 10:01 pm
Location: Chicago, IL

Re: What is the Best Way to Tell How Good a Player Actually

Post by 34 + P.J. Dozier »

browen wrote: If a top player isn't negging at least once a game they are playing far too conservative on the buzzer.
I don't think this claim necessarily holds water in all cases. There are a few incredibly skilled players that manage to put up respectable power numbers and not end a tournament with more than ten negs. I think looking at the number of negs in context with the number of powers is, on the whole, more of a reliable metric.

I'd also like to echo the sentiments of others in this thread that there is no one best way to tell how good a player actually is. Sure, a specialist may put up twenty powers in a tournament by powering a majority of their categories' tossups, but if said specialist were to be pitted against a team with a similar player (or attempted to play solo against a generalist à la IPNCT), their strength would be hampered. How good a player is varies from case to case and requires context.
Wonyoung Jang
Belmont '18 // UChicago '22
ACF; NAQT; PACE
User avatar
El Salvadoreno
Rikku
Posts: 346
Joined: Wed Nov 30, 2016 7:45 am
Location: Northwest Suburbs, IL

Re: What is the Best Way to Tell How Good a Player Actually

Post by El Salvadoreno »

Thiccasso's Guernthicca wrote:
browen wrote: If a top player isn't negging at least once a game they are playing far too conservative on the buzzer.
I don't think this claim necessarily holds water in all cases. There are a few incredibly skilled players that manage to put up respectable power numbers and not end a tournament with more than ten negs. I think looking at the number of negs in context with the number of powers is, on the whole, more of a reliable metric.

I'd also like to echo the sentiments of others in this thread that there is no one best way to tell how good a player actually is. Sure, a specialist may put up twenty powers in a tournament by powering a majority of their categories' tossups, but if said specialist were to be pitted against a team with a similar player (or attempted to play solo against a generalist à la IPNCT), their strength would be hampered. How good a player is varies from case to case and requires context.
You make my point better than I can.
Ricky "Slick Rick/Rico Suave" Rivera
Carmel Catholic- Class of '17
Notre Dame (Mendoza)- Class of '22

My Namesake
User avatar
c*t
Wakka
Posts: 238
Joined: Sat Dec 03, 2016 8:34 pm
Location: Richmond, VA

Re: What is the Best Way to Tell How Good a Player Actually

Post by c*t »

Thiccasso's Guernthicca wrote:(or attempted to play solo against a generalist à la IPNCT)
It's funny this is brought up because I was just thinking about it the other day. Generalists that don't have enough deep knowledge or don't also have a specialized category to get them past the elimination heats and into the top 16 are screwed. Seems to me like only in the 1v1 single elimination heats are generalists going to have the upper hand.
Catherine Qian (she/her/hers)
Maggie L. Walker Governor's School '18
University of Pennsylvania '22
Secretary, PACE
Locked