On-going season rankings

Dormant threads from the high school sections are preserved here.
Tanay
Rikku
Posts: 427
Joined: Wed Jun 03, 2009 5:05 pm

Re: On-going season rankings

Post by Tanay » Sun Nov 28, 2010 2:59 pm

Edward Powers wrote: Team A gets 75% TU conversion against team B and averages 20 PPB. Team B gets 25 % conversion against Team A but gets 30 PPB.
This scenario is incredibly rare, almost to the point of being unfathomable. In HFT's elite field, only five teams averaged 75% tossup conversion over the course of prelims, and just over a third hit 60% conversion, which is in itself a sign of a strong team. Additionally, if one team is getting fifteen tossups in a game, and the other is getting five, the final standings generally do a pretty good job of indicating which team is better.

Take this more realistic possibility:
Team A gets 11 tossups and averages about 18ppb
Team B gets 9 tossups and averages about 24ppb
Final score: 310-320

This would be a game between two teams that are probably similar in caliber. This is the type of game for which someone running a rankings system would have to look at head-to-head, and not the blowout situation (15 tossups?!) posed above. Team A got more tossups. Team B wins. Bonus conversion matters.
Tanay
ex-Berkeley, ex-Bellarmine

User avatar
Huang
Rikku
Posts: 442
Joined: Wed Dec 17, 2008 9:29 pm

Re: On-going season rankings

Post by Huang » Sun Nov 28, 2010 3:03 pm

Edward Powers wrote: Back to the real point: those of you who want to use PPB instead of overall results at tournaments, head to head, in playoffs, of course, explain to me why Cistercian should fall beneath a team it had just defeated by over 300 points in a tournament it finished in a tie for 1st? You only can do this by ignoring Cistercian's significant superiority in TU conversion, which led to a comfortable win. Why would you ignore such a fundamental stat, the independent variable in tournaments, while PPB is always a dependent variable?
I'm not sure how long ago TQBA Kickoff was, but St. Marks did have 21.33 PPB versus Cistercian's 17.84 PPB (maybe Cistercian was missing a player?). Fred probably took that into account. As for TU conversion and PPB, I like to think of TU conversion as "defense" and PPB as "offense" (someone, maybe Bruce, posted this opinion elsewhere on the board) since you deny points by getting TUs against your opponent and score a majority of your points off of bonuses. I don't know if you can use one tournament (Quad States?) as a definite indicator of how well Cistercian plays "defense" (and given their bonus conversion at TQBA Kickoff, Fred probably thinks their "offense" is also a bit inconsistent).

Edit: Switched up ideas in my head
Sandy

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Sun Nov 28, 2010 3:58 pm

Fred, I too have other work to do, so I will try to be brief.

Your Chattohoochie-Alpharetta question is excellent, and in a variety of ways. Let me have a little fun and change the question slightly to put you in the difficult spot you just put me in. Suppose the match in question was not just for the Championship of the Georgia HFT mirror, but was, instead, the final in a national Championship, with all the statistical numbers in Alphretta's favor, but the WIN in Chattahoochie's. Would you give the Championship trophy to Alpharetta? A rhetorical question, for sure, for you would know that you could not deny Chattahoochie it's rightful Championship because: IT WON. (I KNOW I KNOW---you would probably point out that one loss would not eliminate Alpharetta, etc. etc.---I understand). But surely you see why I emphsize the simplest stat of all, winning and losing? It subsumes all others and renders debate, ultimately, pointless. You are what your record says you are, and stats are for...well, you know.

But you do ask a fair and a tough question, and it is tougher because we are early in the season, so Alpharetta has time to come back and prove the loss was an anomaly, as the overall stats for the tournamant suggest it was. My analysis above admits that when all is said and done, an excellent but still inferior team can sometimes defeat a superior team because all averages are just that---averages, so, TU conversion rates will be variable too. For a truly great team, they will probably be 55-60% or higher, on average, even against quality opposition. But since they are AVERAGES, a poor match of 40-45% conversion, let's say, might occur, in which case if the opponent was an excellent one, then the upset can occur.

The stats you cite, even if I did not know Alpharetta's team up-close and personal ( I do--we played them at PACE last year and they were excellent and my understanding is its whole team is back) sugggest that this was probably a minor upset, since Chattahoochies stats overall are quite outstanding as well. So I, because of non-tournament factors like knowing Alpharetta's performance at PACE, knowing its whole team is back, and knowing this was only one tournament, I would not punsih Alpharetta by lowering its rankings. But I would reward Chattahoochie by giving it a nice boost in the rankings. The outside factors I mentioned are the discretionary criteira we all bring to bear when making such tough evaluations. As long as they are not arbitrary, they make sense.

But to close, your example is nothing like the one's I've discussed. Alpharetta lost by 20 and finished in 2nd behind Chattahoochie. St. Mark' lost by 300 plus points to Cistercian, Cistercan finshed AHEAD of St. Mark's, and then you dropped Cistercian behind St. Mark's in your rankings. You also know the details of SJHS at YALE's FaCT vis-a-vis Cheshire , another example where winning by a large margin and finishing a full 2 games ahead in the standings meant less than Cheshire's 24 to 22 PPB lead over SJHS---they were ranked higher and have kept that ranking lead since FaCT, which was their only tournament so far, while we have finished near the top of the 8 tournaments we have attended and for the most part even in losses have performed reasonably well against Top 10 teams. Further, at HFT there were 15 nationally ranked teams from your top 100, I beleive, and we played 6 of the top 60 of them head to head, winning 4 and also winning our 2nd flight outright with a 4-1 record against the 5 other championship calibre teams in our playoff bracket. What happened in the rankings? We dropped, and one of the teams we defeated head to head AND in flight placement climbed ahead of us in your rankings. Of course your task is not easy---so many teams, so many tournaments, so many close matches---I understand and woUld never complain about a ranking, for they are just for fun. But your criteria needed to be questioned, so I did. Hope this response helps in some way. And keep doing what you're doing---it generates interest and fun, which are reasons enough to keep doing it. Now I must get some work done too!!!!
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Sun Nov 28, 2010 4:28 pm

TANAY,

You are correct---75 % TU versus 25% TU would be extremely rare. But this was a model designed to illustrate the deeper point about a stat largely ignored compared to PPB, and that is TU conversion stats, especially in playoffs. Power stats might suffice where they exist, but when tourneys do not use powers, TU conversions are decisive stats to know. Finally, the rare 75%-25% stat was brought up precisely because after Cistercian won Quad States and defeated St. Mark's by over 300 points, Cistercian's rank fell below St. Mark's, with St. Mark's 1 or 2 point superiority in PPB apparently being the reason. To do this to Cistercian required ignoring the fact that Cistercian had to have had an overwhelming advantage in TU conversion (I do not have the exact stat-line, but logic requires such a conclusion in a 300 point plus victory). This, I thought, was a flaw in his ranking method, so I addressed it with the extremely rare hypothetical you discuss. Ironically, you then suggested what I suggested to Fred---in the case of very close teams, one should go head to head---and Fred has consistently suggested that head to head is, essentially, irrelevant because there are so few samples. So I challenged Fred for both of his stands---making PPB too important vis-a-vis TU conversion,and dismissing head to head as irrelevant.
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

sacagawea
Lulu
Posts: 21
Joined: Tue Aug 24, 2010 9:18 pm

Re: On-going season rankings

Post by sacagawea » Sun Nov 28, 2010 5:42 pm

Huang wrote: I'm not sure how long ago TQBA Kickoff was, but St. Marks did have 21.33 PPB versus Cistercian's 17.84 PPB (maybe Cistercian was missing a player?). Fred probably took that into account. As for TU conversion and PPB, I like to think of TU conversion as "defense" and PPB as "offense" (someone, maybe Bruce, posted this opinion elsewhere on the board) since you deny points by getting TUs against your opponent and score a majority of your points off of bonuses. I don't know if you can use one tournament (Quad States?) as a definite indicator of how well Cistercian plays "defense" (and given their bonus conversion at TQBA Kickoff, Fred probably thinks their "offense" is also a bit inconsistent).

Edit: Switched up ideas in my head
To briefly clarify, the Cistercian "A" team at TQBA Kickoff consisted of the two lowest scorers from last years HSNCT Cistercian A team and a couple of kids from last year's HSNCT Cistercian B team. This "A" team was missing two players who have already posted 100+ ppg in other tournaments earlier this year (including Vimal who posted 104.5 ppg in this very tournament as a player on the B team.) The closest thing to the A team that Cistercian has shown thus far this year was in a couple of games (namely against St. Marks and Seven Lakes B) at TQBA Quad States in which 3 of the 4 HSNCT Cistercian A team players had the opportunity to play together. Sadly, these two games are far too small a sample size to say anything, but the team did score 24.61 ppb and 7.5 15s per game on average in these two vs. the 21.18 and 6.9 15s per game in the rest of the tournament on HSAPQ 15.
James Yoder
Cistercian Preparatory School Class of 2011
Harvard Class of 2015

User avatar
Mewto55555
Tidus
Posts: 708
Joined: Sat Mar 13, 2010 9:27 pm
Contact:

Re: On-going season rankings

Post by Mewto55555 » Sun Nov 28, 2010 5:44 pm

I think another consideration might want to be made here, which is that the results of one or two head-to-head games don't really mean much. When both teams are putting up 22 ppb, and say, 5 powers per game, they are combining for 10 powers per game, meaning each tossup is 12.5 points (on average), so each tossup bonus cycle is about 35 points, for a 70 pt swing total. Make 3 or 4 buzzer races go the other way, and all of a sudden your 350 point blowout is a much closer match. While multiple wins over a long season is a pretty good indicator of skill, one or two wins early on is not necessarily decisive.

In addition (and more importantly?), we should also consider the fact that even if Team 1 consistently beats Team 2, that shouldn't guarantee Team 1 a higher spot in the rankings than Team 2. I assume the purpose of optimal rankings are to simulate what the results would be if we could somehow host a perfectly fair, upset-free tournament with all these teams involved. In such a case, we would invariably see circles of death, due to simple considerations (Team A is significantly better than Team B who is significantly better than Team C at history, B>C>A at science, and C>A>B at lit, and they are approximately equal everywhere else). When that happens, B is going to beat C every time, C is going to beat A, and A is going to beat B, no matter how many times we run the matches (barring one of the teams getting a good packet for them). However, one of these has to be ranked above the others, and PPB over the rest of the tournament is the most logical way to make that ranking. Furthermore, its entirely possible that due to, say, its complete lit dominance, team C also has a good shot of beating teams ranked 5-10 places higher than them, whereas A and B have a much lower chance. In that case, it seems perfectly reasonable to rank C above B, even if team Bs extra 4 or 5 tossups per game make all matchups between the two blowouts. Any thoughts on this?
Max
formerly of Ladue, Chicago

Blahhunter
Lulu
Posts: 19
Joined: Sun Apr 05, 2009 2:10 pm

Re: On-going season rankings

Post by Blahhunter » Sun Nov 28, 2010 5:57 pm

I think it could also be noted that higher PPB, higher PPG, and higher positions on Fred's rankings can be linked? It seems that those teams who have higher PPBs also do better overall. I suppose the logic goes that those with high ppb convert the harder parts of bonuses, and would therefore have deeper knowledge. Those teams with deeper knowledge would then benefit from that knowledge when playing in a pyramidal format and would thereby convert more tossups, which allow for them to get even more points on bonuses. Since we are providing dramatic examples; would anyone rightfully rank a team with say 10.00 ppb on a normal high school set in the top 20, and is said team did beat some team out of the top20, would it not be considered a fluke? When two approximately equal teams play though, I agree with Mewtwo's comment above. (In fact such an instance did occur last year between SJHS A, Kellenberg A, and Hunter A at FaCT).
Zihan Zheng
Hunter College High School '13

Flash Bomba
Lulu
Posts: 69
Joined: Sun Feb 28, 2010 12:42 am

Re: On-going season rankings

Post by Flash Bomba » Sun Nov 28, 2010 6:04 pm

Another issue with Win-Loss is the packet itself. Team A could easily have a better grasp on any given subject, but if the favorite things from that subject for someone on team B comes up, B will get more of the questions than normal and win the round despite being arguably worse. I know that for teams like mine with a fairly weak grasp of the quizbowl canon this happens a lot.
Thomas Hsu
University High School '11
Brown '15

Windmill Tump
Wakka
Posts: 138
Joined: Sun Mar 28, 2010 9:27 am

Re: On-going season rankings

Post by Windmill Tump » Sun Nov 28, 2010 7:10 pm

In my opinion, an important purpose of these rankings is to compare teams across the country, and so some quantitative statistics are more suitable. Like Tanay said, points per bonus, and to a lesser extent, powers per game are the best to compare with, as for teams with approximately similar skill levels, those values should stay fairly consistent, no matter the opponent or one particular bad packet. Of course there will be problems, like when a team answers only a small number of tossups, and the bonuses it receives are either favorable or not. However, I don't think this is too big of a problem - if that happens to a team in one particular game because of a bad packet, uncharacteristic negs, whatever, then it should still somewhat balance out with other games, and if it happens repeatedly, well then the team probably won't do too well on the bonuses in any case, and eventually shouldn't be ranked that high. About Coach Power's question about the results for a national championship, I'm not sure if the rankings are relevant to that, or really to any one particular tournament, as they have different purposes: again, these rankings compare teams across the country based on a variety of results and so points per bonus become more important, while a tournament declares a winner and standings based on every game, whether a "fair" one or one with unfortunate negs and packets, and so clearly win-loss is looked at.

Head to head records are obviously important, but they are much more impacted by a particular packet, conditions on that day, etc., and so if we're going to compare teams nationally, then, at least this early in the season, I'd prefer points per bonus. If one team consistently beats another team over the course of several tournaments, not just one, that is significant if we're comparing the relative strengths of those two teams to another, but I still don't know how important that would be to a general ranking of all teams.
Sid Dogra
Detroit Country Day School '13
Michigan '17
NYU '21

User avatar
i never see pigeons in wheeling
Rikku
Posts: 391
Joined: Mon May 31, 2010 3:57 am

Re: On-going season rankings

Post by i never see pigeons in wheeling » Mon Nov 29, 2010 12:20 am

Yeah, if Fred were to prioritize TU's when doing rankings, the sample size required would be enormous. The only reliable evaluation for this would be head-to-head matchups, but acquiring those among the diaspora of quiz bowl high schools is a great challenge. Even if those results were acquired, we would need many, many matches in order for a statistical average to be acquired. Ed brought up the Cistercian-St. Marks example several times during this thread. The problem with using that standard is that Cistercian beating St. Marks by that margin could be a fluke. You would need many, many games for that to statistically average out to a reliable figure. In pyramidal quiz bowl, the most reliable indicator of knowledge is PPB precisely because it is the variable least susceptible to wild swings depending on head-to-head tossup figures, and bonus conversion again is intertwined with TUs/powers per game because knowledge depth is critical to both.
And it's not like the pure figure is taken without any consideration of extenuating circumstances. If teams only got a couple TU's but 30'd both of the resulting bonuses, it would be taken into account versus the team that acquired the majority of TU's and (obviously) had a lower bonus conversion. But PPB is still the most reliable indicator.
Ankit
Cal '16
Bellarmine College Preparatory '12

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Mon Nov 29, 2010 2:54 am

Mewto 55555, Blahunter, xpmath and drno: You all make excellent points, so I concede to much of what you say. PPB is an extremely important stat---I never denied that. And Mewto, you make a very interesting argument concerning why team A>B>C>A...but the raw statistics Fred is using could never account for such subtleties between teams. Further, when dealing with the Top 100 in the Country, which is what THIS thread is about, the odds on such consistent imbalances in category strength become less likely in the aggregate, but you have made a very nice and nuanced point. Blahunter, your 10 PPB team is not the type of team this thread is about---surely such a team is almost always going to be overwhelmed by the Top 100 teams on this list, although there might be an odd upset here or there. And xpmath, you are correct to assert that a blowout win might be a fluke, and therefore the use of Cistercian's match is not necessarily conclusive in any analysis. But it might also be true that it was not a fluke. Either way, it misses the deeper point I was making: Since it may or may not have been a fluke, which only time could correct if it were, why not at least give credit for it until we learn otherwise? To say it might have been a fluke is one thing; to turn around and rate the losing team higher than the winning team, especially if the winning team also won the whole tournament, seems arbitrary to me---it suggests that because it might have been a fluke, we then conclude, in the absence of any evidence whatsoever, that it WAS a fluke because, get this, the losing team's PPB was 22 and the winning team's was 20, or something like that, since we all KNOW that PPB is a more reliable stat. But is it? A 22PPB with 5 or 6 bonus opportunites in a match because of low TU conversion versus 20PPB with 13 or 14 bonus opportunities in a match, which apparently happened in the Cistercian match, misses the overwhelming fact that on bonus opportunites Cistercian scored 260-280 points, while St. Mark's scored 110-130 bonus points. Now I am not the world's greatest mathematician, but it seems to me that the 260-110 edge to Cistercian was discounted because an easy stat---PPB---was available, while the more important stats---bonus opportunties and total bonus points per match or per tournament were missing, or as Ankit suggests, would be too difficult to compute. But when ranking the BEST teams, why not put in that effort, especially when all other indicators between the teams are close? Further, one of you, I believe it was xpmath, suggested that rankings have little to do with national tournaments. Not true: they can effect seedings, and, as everyone knows, seedings can be crucial to advancing. Finally, I trust you all understand I am not a luddite. I know that PPB is a great indicator. I never argued otherwise. But when a team outplays its opponents in a tournament and then is dropped beneath all of them when the rankings come out, this is perplexing, especially if the tournament was geographically diverse, as, for example, HFT was. Teams who are rated highly but never play each other actually play for the first and perhaps the only time all year due to geograpahic necessities. Then, one team wins, but its victory against a deep and geographically diverse field is discounted because of a single stat that is also a dependent variable: PPB. This makes paper stats more important than actual tournament play, not just against one team, which could be a fluke, but against a field of teams, which is not a fluke. If it is, then let's just stop playing pyramidal tournments all together and just have paper bonus tournaments. Of course, no one would come. My point? PPB is a great indicator. But when it indicates that the team that won the playtoff bracket should be rated below every team in the bracket that it just finished ahead of in a fair and challenging tournament, after the tournament is over and reported, it is a flawed indicator needing the adjustment of intelligence. My hopefully intelligent contribution to such a corrective analyisis is this: TU conversion is the stat that is ignored in such cases. But it need not be, since total bonus opportunities are reported in every tournament, for this is how PPB are computed---the total points are divided by the total opportunities, and these opportunites are dependent upon and exactly equal to TU conversions. So, why not create a complimetary stat to go with PPB? Why not include Bonus Opportunity Percentages (BOP's)? And contrary to Ankit's argument, I think it would be quite easy to do because, as just stated, total bonus opportunities are already reported in the overall stat line. Let's say a team played 12 matches in a tournament and had 175 bonus opportunities. Assuming the traditional 20 TUH per match, then this team hit 175/240 TU's giving it a bonus opportunity stat of 72.9%---clearly an outstanding team. And so on and so forth. I'd bet such a stat could be easily added to the overall tournaments stats in the SQBS format. Then Fred and the rest of us would have 2 or 3 great and more refined indicators available to us---PPB, Powers, and BOP's, which could be generated automatically via the SQBS program or others comparable to it. And, an even more refined BOP stat would be Bonus Opportunities in the Playoffs, since the Prelim stats might be skewed because of very imbalanced Preliminary brackets. And since most tournaments report their results in two phases---Prelims and Playoffs--- a stat could be created for greater refinement: Bonus Opportunity Percentages in the Playoffs (BOPP's!!! ).

So, what do you think?
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

Windmill Tump
Wakka
Posts: 138
Joined: Sun Mar 28, 2010 9:27 am

Re: On-going season rankings

Post by Windmill Tump » Mon Nov 29, 2010 4:06 am

With regards to the whole fluke discussion, I think we actually agree for the most part - if we're comparing the strengths of only two teams, that play each other fairly often, then those head to head games suddenly become a lot more important. However, we're talking about a ranking of the top 100 teams across the country; even ignoring flukes, comparing two teams at a time to come up with these rankings doesn't work well, obviously because of the numbers first, and second because of something similar to the situation that Mewto said. Suppose you have two teams where, with regards to tossups, one team is average at literature and history, while the other knows almost no literature and is strong at history. The first will probably win most of their games against the second, as they might get all the literature tossups and maybe one or two history. However, against a different field that is generally above average at both literature and history, the second team may still lose the literature, but win history, while the first loses in both, and we don't know what happens. This example is poorly constructed, but it's the best I can come up with right now, and it hopefully shows why I don't think we can base a ranking for teams across the country, whose purpose is to presumably give a vague prediction as to what would happen if these teams all played at one tournament, on head to head results. I do think that people should look more carefully at teams that do well against those that supposedly have better points per bonus, such as yours, but ultimately, I think points per bonus should still be the most important factor in these national rankings.
In the case of Cistercian and St. Mark's, I think Fred probably was basing part of his ranking on the TQBA Kickoff stats, which were explain above, and where St. Mark's had 21.33 points per bonus as opposed to 17.84 for Cistercian "A", as he can't be expected to look at the individuals playing everytime to realize that it wasn't the usual team.

As for the use of these rankings for seeding, I don't think that was really what I was saying, and I'm sorry if that's what I implied. Clearly the rankings can affect seedings, which I agree with; as I said, I think they're a general prediction of how teams would hypothetically do at a tournament, and so should be used for seedings. I just wanted to say that these aren't supposed to tell you EXACTLY what will happen at a tournament, as that's impossible to know. Instead, they're used for comparing teams nationally, and so your question, clearly in jest, about awarding championships to the team with highest points per bonus isn't really the point here, since that's not what Fred's rankings are doing. I think that Fred feels that points per bonus give the best idea of how teams compare to each other nationally, which is why he values it for these hypothetical rankings. It would make no sense to order teams by points per bonus in a real tournament, because all the teams are there - there's no point in trying to guess how they'd do against each other at that tournament if it's already happened.

For your idea of Bonus Opportunity Percentages, seeing as it's 3 AM, I don't really want to type up a huge reply with examples, but I think some of the previous posts address something like that, such as Joe's post. It's an interesting idea, but I'm not sure how it'd work out to give an accurate depiction that included opponent strength as well. The prelims/playoffs idea is a good start, but again, there could be three tiers of teams, which I think is what happened at our last tournament, where it went DCC A, DCC B, then everyone else, and the game scores show it.

In any case, this is an interesting discussion, if ultimately all hypothetical. A lot of what I said above probably seems incoherent, and I'll apologize in advance for that; hopefully, you'll get a general idea of what I mean.

EDITED because I apparently missed a point when I read Coach Power's posted.
Sid Dogra
Detroit Country Day School '13
Michigan '17
NYU '21

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Mon Nov 29, 2010 4:46 am

xpmath, I agree, it is after 3AM, so an extended discussion can wait. But, I do respect your thoughtfulness here, and I do agree that if Fred was using the TQBA PPB stats rather than the Quad States PPB stats, his ranking makes more sense. And, in general, I understand the use of PPB when comparing teams across regions---probably 95 % of the time PPB correctly identifies the superior team. My real issue with Fred was when actual tournament results contradicted the expectations that such PPB suggested, but Fred seemed to ignore the tournament results and stick with PPB anyway. To me this makes tournaments and their results irrelevant, and this can never be considered a wise thing to do.

And I do know that sometimes upsets occur, but to automatically dismiss results that might be upsets dismisses the effort of the winning club in favor of mere paper stats which is unfair to the winning team. Time of course might reveal that the results of any single tourament were in fact upsets, but until that time comes, why should not the winning team earn the recognition its victory seems to have warranted? Everyone knows Fred's ranking are ongoing and subject to revision as more data comes in throughout the year, but until data comes in showing that Team A's victory over Team B was an upset or a fluke, why not use the actual results in tournament competition as a whole, and not just the PPB stat line alone, to rank teams, especially if the tournament in question was geographically diverse, which helps remove the speculative nature of stats like PPB alone across regions? There is actual competition to assess, and the if the team with the lower PPB prior to the tournament wins the tournament, and still has a lower PPB after the tournament, does that not reveal that we use PPB too glibly while ignoring the most esssential skill in pyramidal play---the ability to earn bonus opportunities in the first place via superior TU play?

So, when one uses a stat too glibly, should we not make an intelligent correction to such use? That is all I have been suggesting throughout all of my posts in this thread.

And before closing, I thank you for your thoughtful responses, for which you have no need to apologize, for they have made this discussion more interesting to me a well. Now we should probably both get some rest!
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

Chandragupta Maurya
Lulu
Posts: 69
Joined: Tue Oct 05, 2010 7:29 pm

Re: On-going season rankings

Post by Chandragupta Maurya » Mon Nov 29, 2010 9:37 am

Huang wrote:I'm not sure how long ago TQBA Kickoff was, but St. Marks did have 21.33 PPB versus Cistercian's 17.84 PPB (maybe Cistercian was missing a player?).
Cistercian had a 3-man team and was missing a few players at that tournament
Vimal Konduri
Cistercian Preparatory School '13
Harvard '17

Windmill Tump
Wakka
Posts: 138
Joined: Sun Mar 28, 2010 9:27 am

Re: On-going season rankings

Post by Windmill Tump » Mon Nov 29, 2010 3:38 pm

But Fred isn't simply ignoring tournament results; even in his newest set of rankings, in the second sentence he says that he almost moved LASA above State College, but didn't, due to State College missing a player and having an impressive performance at ACF Fall, which I think are acceptable reasons. I think tournament results are definitely looked at, but once again, the majority of tournaments don't have a field as geographically diverse as HFT did, and in that case, you can only compare two or three teams to each other. I just think that when a tournament only lets you compare two or three teams, and you want to compare teams across the country, the head to head results shouldn't be given as much importance, although clearly they should still be given some. I think I explained why I think that way in my last post. The exceptions are tournaments like HFT, which give you a great idea of how the top teams do against each other. However, I think the results at HFT were actually taken into account, at least for the top teams there, which are around what I'd expect. Some teams are indeed ranked lower than what I personally think they should be, including yours, but Fred himself said that he'll try to look at those kind of teams better in the future.

I wish there was a good way to keep track of how good teams are at getting tossups while taking into account the strength of their opponents, but I feel there are just too many problems with using tossup conversion, many of which have been said before in this thread. No kind of stats are completely perfectly, and ultimately it comes down to which one you think is most useful for your purpose, and for this purpose, I think bonus conversion is the best to use, whereas you think differently, and that's fine. If we can come up with a stat that perfectly tells you the relative abilities of teams, well that would be amazing, but I don't think that exists.
Sid Dogra
Detroit Country Day School '13
Michigan '17
NYU '21

User avatar
The King's Flight to the Scots
Auron
Posts: 1451
Joined: Mon Jan 26, 2009 11:11 pm

Re: On-going season rankings

Post by The King's Flight to the Scots » Mon Nov 29, 2010 4:15 pm

These are temporary rankings...if you learn things and do better at the next tournament, you'll move up. Prove that you were underrated by beating some of the teams above you.
Matt Bollinger
UVA '14, UVA '15
Communications Officer, ACF

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Mon Nov 29, 2010 4:17 pm

xpmath---You've written another thoughtful response and analysis which I cannot really improve upon. And you are doubtless correct---TU conversion stats are inextricably linked with the quality of one's opponents in ways that PPB are not, so PPB is doubtless the more reliable stat. Still, if one is allowed the assumption that if only playoff stats should be seriously used, the variability should stabilize somewhat, making it a more reliable statistic to aid in evaluations. But for it to be truly useful, we probably need to have it become part of the normal stat line at tourneys. Until this is done, most will find it too cumbersome a stat. And, because PPB cannot be directly impacted by your opponent the way TU conversion percentages can, most will rely on PPB rather than BOPP's (I trust you enjoy my new acronym :grin: ).

And of course you are correct--there is no fail safe set of stats that can make predictions certain. But this is probably a good thing---it keeps some wonderful mystery in the process and allows us to be pleasantly surprised, sometimes even astonished, by the great uspets that come from out of the blue, like Adair County or Bellarmine at last year's HSNCT. Both gave all underdogs some hope, and this, I think, is a good thing.

Nice chatting with you. Good luck with your season.
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Mon Nov 29, 2010 4:29 pm

Matt,

If defeating those above you guaranteed moving up, this part of the thread would have been unnecessary. The whole point was that teams were defeating teams and then FALLING in the rankings behind the very teams that had just been defeated.

Still, what you say is, in the final analysis, absolutely true: These rankings are transitory, and excellent performances in the future should change them, and whether or not they do, if genuine improvement has taken place relative to one's competitors, then one's National performances at year''s end will reveal this, at which time a team's performance will earn it its final high National ranking if it has in fact improved significantly, and no opinion poll can change this.

So, as usual, with respect to the essential issue being discussed, you are concise, to the point and, in my judgment, correct.
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: On-going season rankings

Post by cvdwightw » Mon Nov 29, 2010 5:37 pm

Coach Powers,

I think you're missing a few things. First off, when two teams are of roughly equal ability, tossup conversion becomes the most important factor in the margin of a game between them. When both teams have high (>20 ppb) bonus conversions, every buzzer race or improbably-right-up-one-team's-alley tossup becomes magnified that much more. It's certainly not unheard of for a team to win a game by 300 points and lose to the same team in the same tournament, because the packet and buzzer races go a different direction the second time around. (For an extreme example, see the Minnesota-Chicago series of matches at 2009 Illinois Open).

Second, as I think multiple other posters mentioned, these rankings are supposed to be somewhat predictive. That is, they're not intended to explain past results, but rather predict what would happen if two teams played each other right now. Obviously, these predictions are going to be wrong in the absolute, but if predictions are things like "team #25 and team #30 are really close in ability, so each has about a 50% probability of winning a game against the other one," then team #30 beating team #25 doesn't really say a whole lot about what will happen the next time team #30 plays team #25 (I just skimmed the rankings, so I have no idea who these teams are). On the other hand, team #25 putting up 4 ppb more than team #30 probably gives a slight edge to team #25 (probably not significant, but we don't have a whole lot to work with; for instance - a 20-16 difference puts the higher ppb team over the top, assuming no negs, when scoring 9 tossups to opponent's 10, but not when scoring 8 tossups to opponent's 9).

Third, your BOP (as with all tossup conversion stats) does not adequately take into account negs (by both one's team and one's opponents). If one team had a BOP of 64.9 but negged on 15% of the questions, and one team had a BOP of 52.7 but negged on only 5% of the questions, which team is better? I'd argue that it's impossible to tell - the first team is merely the more aggressive team, and will accordingly be the first team to buzz on more questions. Crudely modeling this, we might expect the two teams to each score the same number of tossups, as the extra 2 or so open-play tossups gained by the first team due to aggression will be given back through the extra 2 or so rebounds given to the second team by the first team's negs.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: On-going season rankings

Post by Down and out in Quintana Roo » Mon Nov 29, 2010 6:00 pm

Man, i miss Byko's rankings.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org

User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4042
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: On-going season rankings

Post by Auks Ran Ova » Mon Nov 29, 2010 7:25 pm

Carangoides ciliarius wrote:Man, i miss Byko's rankings.
Yeah, no one acted irrationally about those!
Rob Carson
University of Minnesota '11, MCTC '??
Member, ACF
Member, PACE
Writer and Editor, NAQT

User avatar
Auroni
Auron
Posts: 2998
Joined: Thu Nov 15, 2007 6:23 pm
Location: Brooklyn

Re: On-going season rankings

Post by Auroni » Mon Nov 29, 2010 7:47 pm

Ukonvasara wrote:
Carangoides ciliarius wrote:Man, i miss Byko's rankings.
Yeah, no one acted irrationally about those!
Yeah, man, I was speechless and awe-struck when one team that went undefeated at one tournament in New Hampshire topped the rankings.
Auroni Gupta
UIUC
ACF

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Tue Nov 30, 2010 3:01 am

Coach Chrz!

You mean you do not appreciate the Morlanesque beauty of Fred's rankings, nor the wonderful discussion his methods have engendered, including my very own contribution of the BOPP? I am shocked :shock: and disappointed :cry: , for these are truly things of great wonder, are they not? And is not Auroni correct about Byko's rankings? Coach Chrz, you're not one of those people who always think the grass is greener on the other side, are you?

Further, are you missing the comedy here? After writing virtually a novel on the topic across 6 or 7 posts, Dwight tells me I am missing some things!!!! Read his post! It is a good one one, and I can concur with almost all of what he says. For example, my BOPP is discussed, a miraculous idea if I must say so myself :razz: , but it is also not yet refined---that wasn't its purpose when I raised it---so adding the nuances of negs and how they might affect BOPP( :lol: ) is an excellent contribution, don't cha think?

But the notion that I missed the idea that these rankings are meant to be predictive and not to explain past results is not quite fair, for are not rankings based upon an understanding and appreciation of those very same past results? For the longest time this Fall, State College and LASA had not played a match, while other teams had played dozens, yet SC and LASA were both ranked 1 & 2, precisely because of past performances and the awareness that both had most or all of their players from last year's great teams. My point? Their rankings were based upon previously recorded and publicly known performances. Those were so good that they allowed us to place them at 1 & 2 and predict great performances in the future despite not having played any matches this year---and when they finally did come out to play, those predictions were essentially vindicated---SC's at ACF Fall at UVA, and LASA's at HFT, with SC's fine 2nd there with a player missing and apparently others somewht under the weather. Does Dwight really believe I do not understand this? Or is Dwight himself missing something?

I believe he is. After writing my short novel on this topic, I think he might have missed the essence of my argument, which really agrees with his---so, do you see the comedy yet? What he missed is the fact that Fred was ignoring some performances and was actually ranking teams who had lost to other teams ahead of them anyway in the absence of any other evidence other than one dependent variable: PPB. So I challenged Fred's methods here because they could lead to a misunderstanding of past results and thus undermine their predictive power in the future, while also denying teams the recognition they have earned up to this point in the season. Now Dwight's citation of the 2009 Illinois tournament is wonderful, and it does show enormous point swings between teams in the very same tournament. But Fred did not use such information to make his case, so the Illini tournament is not really on point given the issue with which I was wrestling. It may be true that the next time Cistercian plays St. Mark's, St. Mark's will win by 300 points. BUT IT ALSO MAY NOT. And until that evidence comes in, why deny Cistercian the recognition it has earned? To replace actual tournament results with one's own's theories is fundamentally anti-scientiic, is is not? And I know all of the arguments about one match meaning little. But it was not simply one match---it was an entire tournmament which Cistercian WON. The result? Cistercian fell beneath the team it had defeated both head to head and in placement in the tourney. You really want to tell me I am missing something? Now I have admitted that it is possible this result was an upset---but to turn this possibilty into a fact not yet in evidence really misses something---the very purpose of tournaments in the first place. If Fred is allowed to do this, or anyone else because they have substituted one aspect of excellence---PPB--for the entirety of excellence--TU conversion, wins head to head, tournament excellence, etc., etc., then we should just close shop. To dismiss tournament results is incomprehensible to me. Yet apparently I am missing something. And this to me is comic. We broke away from the NAC and all it represents to create PYRAMIDAL TU'S and to reward people with a monoply on the answering of bonuses precisely because these TU's were converted...and yet we now tend to ignore ignore them and love PPB. Yet one is not allowed to answer any bonus without hitting TU's. So I point this out and basically I am told that TU's are too variable. PPB is more reliable. No kidding!!!!!!! But just because TU conversions are more difficult to assess, it does not mean we cannot learn how to do it, for in the final analysis, it is the decisive skill in tournaments. PPB is wonderful---I have said that over and over. But when it becomes a substitute for the entirety of skill in quzbowl, so much so that other crucial factors are ignored, I think a corrective is required, and I think that is what has been missed in all that I have written. The BOPP was my contribution. Is it a crude idea right now? Yes. Might it ultimately be unworkable? Yes. BUT, maybe it won't. If it does become a useful tool, then a contribution has been made. If not, I tried but followed an unfruitful path. I can live with that. And I also do recognize that all of this is for fun. It is not life and death, and all that I have written is amicable.

So Coach Chrz---now do you still miss the Byko rankings? Isn't this really much more fun?
Last edited by Edward Powers on Tue Nov 30, 2010 10:15 am, edited 1 time in total.
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

Romero
Wakka
Posts: 157
Joined: Thu Jun 05, 2003 11:05 pm
Contact:

Re: On-going season rankings

Post by Romero » Tue Nov 30, 2010 8:29 am

Edward Powers wrote: But it was not simply one match---it was an entire tournmament which Cistercian WON. The result? Cistercian fell beneath the team it had defeated both head to head and in placement in the tourney. You really want to tell me I am missing something?
You are missing something.

Bonus Conversion

Yes, Cistercian had a lower BC at Quad than St. Marks on HSAPQ 15 (22.14 to 21.78). St. Marks also had a higher BC at Kickoff on IS-96 (21.33 to 18.86).

Game Results

Vimal's Cistercian Team (the A-team in spite of any team name) has played St. Mark's 3 times this season. St. Marks record is 2-1. The one loss happened in the game where St. Mark's best player had a suspected appendicitis attack and had to leave in the middle of the match. Did someone forget to mention those two losses or the medical emergency in previous discussion?

Tournament Placements

St. Marks finished one spot ahead of Cistercian at Kickoff and Cistercian finished one spot ahead of St. Marks at Quad States.

Tournament Victories

Cistercian chose to leave Quad States early to attend their homecoming festivities and left that tournament tied without a final. Cistercian was tied with Seven Lakes and had just a one game advantage on St. Marks. Cistercian has outright won two tournaments this year (one in Oklahoma and one in Houston, both on A-sets) but neither included any other schools in Fred's top 100.

Cistercian may be a better team than St. Marks but the data we have does not support this contention at this point. In the future Cistercian will have a chance to prove it is worthy of a higher ranking.
Chris Romero
Texas Quiz Bowl Alliance & Louisiana Quiz Bowl Alliance
Quiz Bowl Coach
Former Player, Texas A&M University ('96-'07)
Former Player, Catholic High School-New Iberia, LA ('92-'96)

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Tue Nov 30, 2010 10:09 am

Thanks Coach Romero. Now the comedy reverts back to me, for I was missing something, something for which I have been arguing---reliance on actual results from tournaments to aid in the formulation of rankings as the year progresses. And as it turns out,I was apparently relying on reports from Quad States alone, and I should not have. And, if the evidence you have brought forth about St. Mark's and Cisterican concerning all of their matchups this year is accurate, and I have no reason to doubt you, then I stand corrected, and apologize to St. Mark's for not knowing its full record thus far this year. I suppose the only piece of evidence in my possible defense might be the fact that Fred had Cisterican ranked ahead of St. Mark's prior to Quad States, and St. Mark's finished behind Cistercian there and yet ended up ranked ahead of Cistercian after the tournament. But even if this might be the case---and I am no longer sure that it was the case---I was still arguing without the very evidence I had insisted Fred use, so guess who has egg on his face and needed the correction Coach Romero has rightly rendered?

So thanks for correcting my ignorance, Coach. And, if Fred knew all these things when I did not, then I owe Fred an apology as well.

I told Coach Chrz that things could be quite interesting and funny even without the Byko rankings. At least I was correct about that! I just did not expect that the laughter would rightfully be at my expense. :oops: :oops: :oops:
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

More Discussion Fodder

Post by cvdwightw » Tue Nov 30, 2010 9:02 pm

So I went ahead and performed that wacky stat I was talking about over in this thread. I used as my guinea pig the results from IS-96 that are currently complete and usable (15s/10s/-5s for each team in every game, BC for each team, no lightning rounds or other stuff like that). Well, okay, I didn't use all the results, just results from teams that looked strong. I used IS-96 because it's a tournament that's been played in a lot of places nationwide and because naqt.com presents the data in a way very conducive to doing mental arithmetic on opponent data (note: take these rankings with a grain of salt, since my arithmetic may be off).

Anyway, I used a slightly modified version of the model in which expected values were converted to whole numbers (e.g. a value of 0.200 for a1,T was converted into 4 ten-point tossups) instead of doing everything in decimals and then rounding the final answer to the nearest 5 points. I did it the original way, too, with all the decimals. Anyway, you don't care about that. You care about this:

Top 10 performances on IS-96, that I can find, so far
1. Richard Montgomery (9-0, 375.56 ppg, 236.67 ppga)*
2. Bellarmine A (8-1, 323.33 ppg, 269.44 ppga)
3. St. Anselm's (7-2, 330.56 ppg, 258.88 ppga)**
4. Hunter A (6-3, 318.89 ppg, 266.67 ppga)
5. TJ A (5-4, 307.78 ppg, 290.56 ppga)
6. Georgetown Day (4-5, 278.88 ppg, 303.33 ppga)
7. MSJ A (3-6, 279.44 ppg, 315.56 ppga)
8. Walter Johnson (2-7, 269.44 ppg, 328.33 ppga)
9. La Jolla A (1-8, 250 ppg, 327.22 ppga)
10. St. Mark's (0-9, 230 ppg, 367.22 ppga)

*Note that all ten of these teams "defeated" all other teams I entered statistics for, so these are only the results of the games against each other. St. Mark's did tie Kellenberg when I tried it the original not-rounding-to-whole-numbers way.
**No, overall ppg didn't exactly correspond to finish order. Bellarmine ended up winning a lot of close games (many of the games in this hypothetical tournament were won by 5 or 10 points) - their absurd power percentage and bonus conversion appeared to help them more than their habit of negging against bad teams hurt them.

Things this does:
-It takes into account, perhaps even to a fault, strength of schedule - witness, for example, Georgetown Day finishing ahead of Walter Johnson, La Jolla, and St. Mark's, all of which had better numbers on paper.
-It penalizes teams for negging against poor teams (since a negged tossup that isn't converted just ends up as a dead tossup).
-It measures a team's tossup performance as well as can be measured given variable schedule strength. It doesn't take into account the actual results of games, however.
-For some reason, it appears to constrain games between the best teams to a range of about 550-650 total points (the max. combined score I can find anywhere on my spreadsheet in 640). I have a feeling this is due to bonus conversions being around 20. Then again, this is what we'd expect to see if these two teams played an infinite number of times, not just once (although we're counting it as one win/loss/tie).

At some point I will write a MATLAB script to efficiently compute a-values (and the resultant giant round robin) for each pair of teams, at which point I will compile the data and release full results instead of just this teaser.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

mkanu
Lulu
Posts: 42
Joined: Thu Oct 01, 2009 10:16 pm

Re: On-going season rankings

Post by mkanu » Tue Nov 30, 2010 11:57 pm

At UofL Arts and Sciences Challenge there are a few teams that would rank in that top 10
http://www.naqt.com/stats/tournament-te ... nt_id=3344
Meghamsh Kanuparthy
duPont Manual A Team Captain
duPont Manual High School Class of 2012
Harvard University Class of 2016

Angry Babies in Love
Yuna
Posts: 819
Joined: Wed Dec 19, 2007 10:09 pm
Location: MD//DC

Re: On-going season rankings

Post by Angry Babies in Love » Tue Nov 30, 2010 11:58 pm

cvdwightw wrote: 1. Richard Montgomery
I like this new statistic. Can this govern, like, everything?
Raynell Cooper
Arcadia ES '04
Richard Montgomery HS '11
George Washington University '15
University of Maryland, College Park '17
Hella things, National History Bee and Bowl

User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: On-going season rankings

Post by Down and out in Quintana Roo » Wed Dec 01, 2010 12:00 am

mkanu wrote:At UofL Arts and Sciences Challenge there are a few teams that would rank in that top 10
http://www.naqt.com/stats/tournament-te ... nt_id=3344
I really don't think you read Dwight's post at all, did you?
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org

mkanu
Lulu
Posts: 42
Joined: Thu Oct 01, 2009 10:16 pm

Re: On-going season rankings

Post by mkanu » Wed Dec 01, 2010 12:02 am

Not really... Now I feel stupid. :oops:
Meghamsh Kanuparthy
duPont Manual A Team Captain
duPont Manual High School Class of 2012
Harvard University Class of 2016

User avatar
Remember-the-Alamo-Remember-Goliad
Wakka
Posts: 150
Joined: Thu May 27, 2010 10:28 am

Re: On-going season rankings

Post by Remember-the-Alamo-Remember-Goliad » Wed Dec 01, 2010 3:08 pm

I see that Cistercian Prep has become something of an "objet du scandale" on these venerable pages in the course of only a few days. Being both old and Catholic, I have good reason not to enter into the fray of arguments and counter-arguments about the justice found in certain rankings. The worth (and ranking) of my teams will continue to be clarified as we all move toward Atlanta in 2011.

Exemplum Gratiae: This weekend's Houston Hoedown should bring together a repertoire of schools whose teams are well-respected and whose track records are impressive. Perhaps these results will provide a higher level of clarity before winter's pallor causes all to flee toward warm hearths, warm drinks, and warm greetings.

If any intemperance was shown on the part of my 'Children of Light' I shall certainly correct them on that score -- Catholic education is a big believer in that kind of thing. I'm hopeful that reciprocity will be shown by others.

And, for the titans -- Chris Romero, Ed Powers, et alii -- I'll let them work out the details.

. . . . And to all, a good night!
Gregory Schweers
County of Dallas
City of Irving
Cistercian Preparatory School Team
The only Monk on this whole board

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: More Discussion Fodder

Post by cvdwightw » Wed Dec 01, 2010 10:46 pm

cvdwightw wrote:At some point I will write a MATLAB script to efficiently compute a-values (and the resultant giant round robin) for each pair of teams, at which point I will compile the data and release full results instead of just this teaser.
So I now have processed data from all 172 teams that played IS-96 and have fully reported statistics. MATLAB is awesome and cranked out the game results in like a second. Results are attached. A few things:

1. IS96_ProcessedStats sheet contains an ordered list of team rankings - you can sort by average ppg as well. IS96_ProcessedGames contains the expected score of each team against each other team. For an easy way of figuring out how your team would be expected to do against all other teams on IS-96, copy the row with team names to a blank sheet, copy the row representing your team to the row beneath it, and paste the column representing your team to the row beneath that (highlight your column - Copy - Paste Special - check Transpose on).

2. This produces some non-intuitive results in terms of finish at individual tournaments. For instance, the inevitably-discussed Cistercian "B" team finished behind four teams at its own tournament by this metric, despite finishing third at the tournament. I'm not sure whether these non-intuitive results are caused by input error (when you're inputting stats for 172 teams and their opponents, you're bound to make a mistake somewhere) or say something about which teams ended up in the wrong prelim bracket or playing other teams on the wrong packet.

3. On the other hand, this provides some interesting comparisons: it projects that, for instance, TJ B and Chattahoochee; Irvine and Richmond Senior; and Mission San Jose D and Paramus Catholic B would all end up with identical records if they played this giant round-robin tournament at a consistently "expected" level of play.

4. I'd like to do this on HSAPQ-15 or some other set, but collecting the data is a giant pain because there's no easy way to read-and-add opponent performances on those sets (SQBS format is a pain to compute this with, plus that means I have to go search for all the tournaments using the set). The script is entirely ready, however, so all it needs is the relevant data (Team Name, TUH, Powers, Opp. Powers, Tens, Opp. Tens, Negs, Opp. Negs, BC - in that specific order for the script to work).
Attachments
IS96_ProcessedStatsAndGames.xlsx
Results for a simulated 171-game round robin among teams playing on IS-96.
(132.91 KiB) Downloaded 252 times
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

User avatar
Mewto55555
Tidus
Posts: 708
Joined: Sat Mar 13, 2010 9:27 pm
Contact:

Re: On-going season rankings

Post by Mewto55555 » Wed Dec 01, 2010 11:30 pm

Am I crazy, or is that missing the teams who participated in MFAT?
Max
formerly of Ladue, Chicago

User avatar
Nine-Tenths Ideas
Auron
Posts: 1549
Joined: Sat Feb 03, 2007 10:14 pm
Location: MD

Re: On-going season rankings

Post by Nine-Tenths Ideas » Wed Dec 01, 2010 11:50 pm

Mewto55555 wrote:Am I crazy, or is that missing the teams who participated in MFAT?
This tournament used bounceback bonuses, hence it not being featured.
Isaac Hirsch
University of Maryland '14
Never Gonna Play Again

User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: On-going season rankings

Post by Down and out in Quintana Roo » Thu Dec 02, 2010 12:55 am

41-127-3. Ouch. Well, that was the A Team minus our only good player, so, yeah. Still, ouch.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org

User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: On-going season rankings

Post by Sen. Estes Kefauver (D-TN) » Thu Dec 02, 2010 2:31 am

Carangoides ciliarius wrote:Well, that was the A Team minus our only good player, so, yeah.
Tactful.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: On-going season rankings

Post by cvdwightw » Thu Dec 02, 2010 3:03 pm

Isaacbh wrote:
Mewto55555 wrote:Am I crazy, or is that missing the teams who participated in MFAT?
This tournament used bounceback bonuses, hence it not being featured.
I was able to hunt down "regular" bonus conversion from the stats linked to in the tournament thread on this board. Results now for 190 teams.
IS96_ProcessedStatsAndGames_v2.xlsx
Version 2 - now with MFAT stats
(166.05 KiB) Downloaded 187 times
Remember that "win" means in the long run, we would expect the team to outscore the opponent (thus theoretically winning more games), while "tie" means in the long run, we cannot expect either team to score more points (thus theoretically the teams would split their games). Right now I'm not going to try Monte Carlo simulating the tournament - that would require trying to figure out a reasonable standard for bonus variability.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

Edward Powers
Auron
Posts: 1107
Joined: Thu Sep 25, 2008 2:52 pm

Re: On-going season rankings

Post by Edward Powers » Thu Dec 02, 2010 3:48 pm

Dwight,

This is quite interesting. Could you do it for the recent HFT and all of its mirrors around the country, assuming enough teams played in all of these tournments to make the effort worth the time you might need to invest?
Ed Powers
Coach
SJHS Academic Team
Metuchen, NJ

User avatar
dxdtdemon
Rikku
Posts: 387
Joined: Sat Mar 03, 2007 4:46 pm
Location: Beavercreek, OH

Re: On-going season rankings

Post by dxdtdemon » Thu Dec 02, 2010 3:59 pm

I agree that this is quite interesting. Were Awty International and Youth Initiative real teams, or just hypothetical statistical barriers?
Jonathan Graham
Beavercreek HS 1999-2003, Ohio State 2003-2007, Wright State (possibly playing)2012-2015
moderator/scorekeeper at some tournaments in Ohio, and sometimes elsewhere
"Ohio has a somewhat fractured quizbowl circuit, with a few small pockets of intense competition (like in Mahoning County) and with the rest scattered around the state."-Chris Chiego

User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5492
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: On-going season rankings

Post by Important Bird Area » Thu Dec 02, 2010 4:10 pm

Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: On-going season rankings

Post by cvdwightw » Thu Dec 02, 2010 4:52 pm

Edward Powers wrote:Dwight,

This is quite interesting. Could you do it for the recent HFT and all of its mirrors around the country, assuming enough teams played in all of these tournments to make the effort worth the time you might need to invest?
As I've said before, the biggest obstacle with this is the time it takes to collect the necessary data.

In order to get the input opponent results, you need to add up all the powers, tens, and neg-5s scored by the opponent in each match. NAQT-formatted stats make this easy because all the relevant game information is on the same page. SQBS-formatted stats are harder than NAQT-formatted stats because you have to manually search the team detail page for each team. Gimranov-formatted stats are harder in their own way - they don't require a manual search, but rather lots of clicking back and forth between games. In short, while it probably takes me 30-60 seconds per team to mine data from NAQT's website, it will take me a lot longer to do it with other stats formats.

Another problem is that a significant amount of reported statistics is split into prelims and playoffs, without a combined stats page. Tournaments like this (including, I think, 3 HFT mirrors) make it even harder for me to collect the data, because I have to do everything in two chunks and compute the bonus conversion.

My next endeavor will be inputting the GINVIT stats into the IS-96 matrix. I'd prefer to run HSAPQ Tourn-15 before HFT, for two reasons. First, to see how my rankings match up with Fred's subjective rankings of Great HSAPQ-15 Performances. Second, because seven tournaments on that set (FRIGHT, CBCT, Loyburn, VCU Fall, BELLEROPHON, Tar Heel Cup and Quad States) all reported combined stats in usable SQBS-format, plus I trust that the Missouri folks will do the same thing this Saturday to make eight usable tournaments (as opposed to like 3 HFT tournaments that reported combined stats and don't have weird errors). Trevor's Trivia (UTC) has only prelim stats, but they're still usable. Danville Fall has full stats, but apparently did not track negs, which is going to be hard to get around.

I mean, the short answer is that if people want to help me with the relevant data collection, I can do this for pretty much any set being played this year (as long as hosts provide useful and reasonably accurate team statistics). All I need is an Excel sheet with seven columns: Team Name, Tossups Heard, 15s, Opponent 15s (just input zero in these columns if the tournament doesn't have powers), 10s, Opponents 10s, -5s, Opponent -5s, Bonus Conversion.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

User avatar
AlphaQuizBowler
Tidus
Posts: 695
Joined: Mon Dec 03, 2007 6:31 pm
Location: Alpharetta, GA

Re: On-going season rankings

Post by AlphaQuizBowler » Thu Dec 02, 2010 5:45 pm

With Fred's rankings, Dwight's computer models, and the hsquizbowl poll, it seems like it won't be long before we have the QBCS: Quiz Bowl Championship Series. Wouldn't that be great?

All kidding aside, though, do you think, Dwight, that you could run this analysis on past HSNCT statistics as a test of its modeling ability?
William
Alpharetta High School '11
Harvard '15

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: On-going season rankings

Post by cvdwightw » Thu Dec 02, 2010 8:11 pm

AlphaQuizBowler wrote:With Fred's rankings, Dwight's computer models, and the hsquizbowl poll, it seems like it won't be long before we have the QBCS: Quiz Bowl Championship Series. Wouldn't that be great?

All kidding aside, though, do you think, Dwight, that you could run this analysis on past HSNCT statistics as a test of its modeling ability?
First off, I'm not sure that this is a relevant test of its modeling ability - this just computes what would happen if every team played exactly at the level expected given the entire tournament. Obviously, within any single game, some teams will play at a higher level, and some at a lower level. Sometimes, both teams will play at a higher level or lower level than expected. Anyway, the short answer is that I can do it.

In fact, if we get data in the next two hours or so, I can get results tonight. However, in order to meet that deadline, I will have to count on the resources of hsquizbowlers who are aimlessly browsing the forums with nothing better to do. If you fit that profile, here is how to enter the HSNCT data:

Step 1: Open this spreadsheet and go to the "HSNCT_RawStats" sheet.

Step 2: Go to this website and start clicking on teams. Add up the opponent powers (P), tens (TU), and neg-5s (I) for the team you clicked on, and put them in the correct column in the spreadsheet.

Step 3: Hit the back button, click on another team that hasn't been entered yet, and do the same thing.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

User avatar
Excelsior (smack)
Rikku
Posts: 379
Joined: Sun Jan 25, 2009 12:20 am
Location: Madison, WI

Re: On-going season rankings

Post by Excelsior (smack) » Thu Dec 02, 2010 8:50 pm

Below, I've linked a Greasemonkey script that'll save you some arithmetic when entering this data (I would've attached it, but apparently you can't attach .js files). It adds an extra row to the bottom of the "Games" table on team-performance.jsp containing the sum of the opponents' powers, tens, and negs. Seeing as how this was written in roughly the past three minutes, though, I can't guarantee it won't break.

This is the script: http://userscripts.org/scripts/show/91795. It may be wise to avoid looking at the source code, lest your eyes burn from the sheer terribleness of it.
Ashvin Srivatsa
Corporate drone '?? | Yale University '14 | Sycamore High School (OH) '10

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: On-going season rankings

Post by cvdwightw » Thu Dec 02, 2010 9:51 pm

Less than two hours later, here are results!

Note that everything in the "ProcessedGames" sheet assumes a 20-tossup match, and that any match between two good teams is going to overperform expectations because almost all the negs will get picked up (an a,T value of 1 for both teams instead of what it ends up being averaged across the tournament). I don't actually think that a Maggie Walker-State College final would have ever ended 285-270 after 20 tossups, but that's what's predicted based on expected performance.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: On-going season rankings

Post by cvdwightw » Fri Dec 03, 2010 5:50 pm

No, my ultimate goal is to not hijack this thread with scores of Excel spreadsheets, it just seems that way.

Attached are the results of a Monte Carlo "round robin" simulation of HSNCT. First off, there are 1000 simulations of the giant round-robin tournament (I would have done 10,000 except that it would have taken like 2 hours to do and I only wanted to wait about 10 minutes). Second, I modified the code so that teams had randomness as best I could estimate (assume bonuses distributed normally, assume a,P/T/N values were probabilities of power/ten/neg), and did a question-by-question simulation, including using the NAQT tiebreaker procedure. Average scores and stuff like that are not listed, just how many wins out of 1000 each team got against each opponent.

What you can do with this, assuming you think that 1000 hypothetical 20-question games between two opponents is enough to be reasonably accurate, is use it with the double-elimination brackets attached somewhere in that 2010 HSNCT thread to figure out the chances of any given team advancing to any given place, all other matches held constant. I don't know, I think that's cool. For instance, Adair County's "fluke" run was partially caused by meeting two teams in the later playoff rounds that, according to the model and the Monte Carlo simulations, they had a little less than a 50% chance of beating (for what it's worth, I already calculated their chance of T-7th, all other matches constant, at 2.48%).

Enjoy.
Attachments
HSNCT2010_MonteCarlo1000Results.xlsx
(213.79 KiB) Downloaded 176 times
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

User avatar
ryandillon
Wakka
Posts: 151
Joined: Sun Jul 12, 2009 10:47 am
Location: Detroit

Re: On-going season rankings

Post by ryandillon » Fri Dec 03, 2010 6:52 pm

I'm not really an expert on all this statistical stuff, but I just wanted to see what everyone thought of this.

At HSNCT, we ended up tied for 7th. After looking at the stats though, I saw that our bonus conversion would have put us at 22nd in the tournament. After seeing that, I kind of thought maybe our placement was flukey, especially after seeing the discussions about Adair County's tied 7th finish and low ppb compared to those finishing around them. However, when we went to PACE the following weekend, we followed up with a 10th place finish. Once again, our ppb put us far lower than we finished, 21st overall at PACE. I really don't feel like finishing in the top 10 at two national tournaments would be a fluke, but at PACE we only beat two teams (Dunbar and Walter Johnson) with ppb higher than us. At HSNCT we beat four teams (Torrey Pines, Stevenson, Seven Lakes, and Eden Prairie) with higher ppb than us. Just wondering what to think of it.
Ryan Dillon

Detroit Catholic Central Class of 2011
University of Michigan Class of 2015

User avatar
dtaylor4
Auron
Posts: 3733
Joined: Tue Nov 16, 2004 11:43 am

Re: On-going season rankings

Post by dtaylor4 » Fri Dec 03, 2010 7:16 pm

ryandillon wrote:I'm not really an expert on all this statistical stuff, but I just wanted to see what everyone thought of this.

At HSNCT, we ended up tied for 7th. After looking at the stats though, I saw that our bonus conversion would have put us at 22nd in the tournament. After seeing that, I kind of thought maybe our placement was flukey, especially after seeing the discussions about Adair County's tied 7th finish and low ppb compared to those finishing around them. However, when we went to PACE the following weekend, we followed up with a 10th place finish. Once again, our ppb put us far lower than we finished, 21st overall at PACE. I really don't feel like finishing in the top 10 at two national tournaments would be a fluke, but at PACE we only beat two teams (Dunbar and Walter Johnson) with ppb higher than us. At HSNCT we beat four teams (Torrey Pines, Stevenson, Seven Lakes, and Eden Prairie) with higher ppb than us. Just wondering what to think of it.
PPB is not the be-all and end-all of determining who is good at quizbowl. It is useful for comparing teams on the same set across different tournaments.

You beat teams with higher PPB by getting more tossups. It would be difficult to simply use PPB to determine the ability of a team to get tossups.

User avatar
Down and out in Quintana Roo
Auron
Posts: 2907
Joined: Wed Apr 09, 2008 7:25 am
Location: Camden, DE
Contact:

Re: On-going season rankings

Post by Down and out in Quintana Roo » Fri Dec 03, 2010 10:03 pm

Getting bonus questions correct takes nothing but knowledge, well and talking to teammates i guess.

But getting tossups right, i've always thought, takes both knowledge and skill... the skill to know when you really are confident in that answer, when you feel like you can take a guess, and when you feel like you can come up with the answer in the 2-5 seconds after you buzzed sometimes. So, buzzing skill is something that takes a lot of time to get good at, and with a school like DCC that's been doing "good quizbowl" about as long as any other school in the country, you've got a lot of experience building that.
Mr. Andrew Chrzanowski
Caesar Rodney High School
Camden, Delaware
CRHS '97-'01
University of Delaware '01-'05
CRHS quizbowl coach '06-'12
http://crquizbowl.edublogs.org

Adventure Temple Trail
Auron
Posts: 2613
Joined: Tue Jul 15, 2008 9:52 pm

Re: On-going season rankings

Post by Adventure Temple Trail » Sat Dec 04, 2010 2:13 am

Dwight: with people such as Coach C and myself looking back to they days of an objective (if flawed, as Byko's was) ranking model for teams, is there any way one could engineer (and run) a new nationwide set of team rankings from the statistical stuff you've been working on? Or would that require a completely different approach, since this model only works for an individual question set?

Also out of curiosity, does anyone know/remember what the Byko formula was, and why ti might go wrong, doing something such as putting a one-tournament winner from New Hampshire on top? I remember hearing it was weighted somehow based on the previous rankings of teams, meaning beating the number 2 team moved you much further up than beating the number 200 team.
Matt J.
ex-Georgetown Day HS, ex-Yale
member emeritus, ACF

Sailing away on my copper boat

Locked