Byko's Ratings

Byko · Post by **Byko** » Fri Jul 20, 2007 12:26 pm

Aw heck, I'm up for causing a little more trouble:

http://ratings.aiquizbowl.com

I'll recommend that thorough discussion of the algorithm and things like that go into the Theory section of this forum.

I'm sure there are things I've left out in the FAQ that ought to be explained (and I know that a better explanation in the Theory section on the website is probably necessary), so feel free to bring them up.

Post by **DumbJaques** » Fri Jul 20, 2007 12:38 pm

That looks really cool (also, you frighten me). I would suggest that teams that have played less than some number of games (at the very least like 20) probably shouldn't be eligible. Also, were playoff/championship wins counted differently at all? I'm curious.

Byko · Post by **Byko** » Fri Jul 20, 2007 12:52 pm

DumbJaques wrote:That looks really cool (also, you frighten me). I would suggest that teams that have played less than some number of games (at the very least like 20) probably shouldn't be eligible. Also, were playoff/championship wins counted differently at all? I'm curious.

Good points. I took some of your comments into account (so, for example, notice how Decatur Lutheran and Columbia (IL) weren't on my ballot for the top 25 poll) when doing my human vote.

I didn't weigh playoff wins any higher than any other win. When I've brought this up to some people, one or two people thought I should. I'm interested in more input on this, and if there's interest, I might consider playing with it next season.

And trust me, entering close to 10,000 games for the entire season is pretty painful. I actually still have a couple tournaments that I haven't entered, but they're fairly disconnected nationally. If I have time, I might do that.

gonzagaeagleahy · Post by **gonzagaeagleahy** » Fri Jul 20, 2007 1:28 pm

Mr. Bykowski, I don't know if you're missing any tournaments for us or something, but you have Gonzaga ending up 113-32 when I know we finished around 151-34.

Tegan · Post by **Tegan** » Fri Jul 20, 2007 2:06 pm

I hate to pull rank or something here, but given that this is an Illinois thing:

Does Illinois even have a "Columbia High School"? In 13 years of being to every big tournament in the state (including state), I'm not sure I've ever even heard of them ......

This is likely a typo ..... they're probably from SC or something like that.

(adding via edit) .... actually, just examining the Illinois records ..... I know there are many incomplete ..... MS went 85-22 (just as an example).

The Time Keeper · Post by **The Time Keeper** » Fri Jul 20, 2007 2:09 pm

Tegan wrote:I hate to pull rank or something here, but given that this is an Illinois thing:

Does Illinois even have a "Columbia High School"? In 13 years of being to every big tournament in the state (including state), I'm not sure I've ever even heard of them ......

This is likely a typo ..... they're probably from SC or something like that.

Clicking on them shows that they seem to be an IHSA only team who are ranked that high because of their winning percentage (9-1) and the ranking list doesn't disregard teams with a very low number of games played.

Tegan · Post by **Tegan** » Fri Jul 20, 2007 2:21 pm

..... having now looked at who they played ..... strength of schedule didn't figure in either ......

Reminds of the time a coach showed up at sectionals demanding the #1 seed because they were undefeated ...... TWO AND OH!

Sad thing: they got the #2 seed. Some people can just be swayed by the jedi mind tricks of a Sith lord

The Time Keeper · Post by **The Time Keeper** » Fri Jul 20, 2007 2:22 pm

Tegan wrote: Reminds of the time a coach showed up at sectionals demanding the #1 seed because they were undefeated ...... TWO AND OH!

That's just awesome.

Stained Diviner · Post by **Stained Diviner** » Fri Jul 20, 2007 2:45 pm

Columbia took 2nd in Class A.

Byko · Post by **Byko** » Fri Jul 20, 2007 3:15 pm

gonzagaeagleahy wrote:Mr. Bykowski, I don't know if you're missing any tournaments for us or something, but you have Gonzaga ending up 113-32 when I know we finished around 151-34.

Tegan wrote:(adding via edit) .... actually, just examining the Illinois records ..... I know there are many incomplete ..... MS went 85-22 (just as an example).

I know that there are other results that you guys have that I haven't incorporated. If you happen to know what I'm missing, feel free to send me your results (remember, for it to be useful, I have to know what teams it was that you defeated, not just a statement that "we went 9-1 at ____ tournament") and I'll be happy to incorporate it. In fact, I'll try to do that as much as possible for anybody who's interested. As we get into next season, I'll try to find a way to streamline the process a little more so that I'm not entering all the results by hand, but we'll worry about that a little later.

Tegan wrote:..... having now looked at who they played ..... strength of schedule didn't figure in either ......

Sadly, it did. Here's the one flaw in the system--teams that are "disconnected" from the circuit can often end up well out of balance from the rest of the field. The end result is that overall record can end up being far more of a determining factor than the strength of schedule. It's a major anomaly, and as a result, it requires human oversight to see which teams really don't belong as highly rated as a computer thinks they are.

From my data, had I not included the one loss I know for certain that Decatur Lutheran had taken, the entirety of the teams competing in IHSA Class A doesn't even touch the rest of the national circuit. As a result, it's going to pretty much be out on its own island apart from everyone else.

The way that the rating system works (as described much better than I could in one of my links in the References section on the Theory page of the website) is that it requires there to be traceability among teams (I could get into the theory of directed graphs and spanning to explain this better, but I'd probably lose most of my audience with it, so this is the best I can do). In other words, you have to be able to create a chain of losses or ties from any one team to any other team. Now, clearly that's never going to be possible nationally (though regionally, this could work quite nicely in many circumstances). So we do the best we can using some methods described in the Theory section of the website (yes, I'm "leaving this as an exercise to the reader").

The main point is that large single-elimination tournaments, when not combined with enough data to supplement them, tend to cause anomalistic results like you're seeing here with Decatur Lutheran, Columbia, Gladstone, and other such teams. It's somewhat to be expected--Decatur Lutheran's win over a then 9-0 team that had beaten teams that had gone 8-1, 7-2, etc., looks that good to a computer, but the reality is that there isn't enough connected data at present to say that it's not that big of a win (or that the winning team just isn't that strong to begin with). So, again, the more data that goes into the system, the more accurate the results will be.

I'm happy to answer other questions or respond to other thoughts anyone has.

vcuEvan · Post by **vcuEvan** » Fri Jul 20, 2007 5:14 pm

Dachspmg wrote:
iambusyeating wrote:Although you may have a point, MW B did beat State College A, TJ A, and Gonzaga at least once over the year.
That was our TJ A that incorporated elements of TJ C, and IIRC that was William and Mary. ;)

Which the algorithm doesn't care about. I guess thats an impossible weakness to fix though.

Sen. Estes Kefauver (D-TN) · Fri Jul 20, 2007 7:00 pm

I like how some tournament in Michigan makes it possible for a winning team to score 835 and the loser to score 600. WTF? Was Batman playing?

Quantum Mushroom Billiard Hat · Fri Jul 20, 2007 7:16 pm

charlieDfromNKC wrote:I like how some tournament in Michigan makes it possible for a winning team to score 835 and the loser to score 600. WTF? Was Batman playing?

That tournament was not NAQT format. The questions were all short, and worth 15 points each, with one bonus question worth 10 points. The games were also longer than most.

quizbowllee · Post by **quizbowllee** » Fri Jul 20, 2007 8:57 pm

charlieDfromNKC wrote:I like how some tournament in Michigan makes it possible for a winning team to score 835 and the loser to score 600. WTF? Was Batman playing?

No. I wasn't.

AKKOLADE · Post by **AKKOLADE** » Fri Jul 20, 2007 9:03 pm

squareroot165 wrote:That tournament was not NAQT format. The questions were all short, and worth 15 points each, with one bonus question worth 10 points. The games were also longer than most.

So, Arena League Quiz Bowl.

I'm splitting this discussion so Dave stops stealing my thunder!

Byko · Post by **Byko** » Fri Jul 20, 2007 10:09 pm

squareroot165 wrote:
charlieDfromNKC wrote:I like how some tournament in Michigan makes it possible for a winning team to score 835 and the loser to score 600. WTF? Was Batman playing?
That tournament was not NAQT format. The questions were all short, and worth 15 points each, with one bonus question worth 10 points. The games were also longer than most.

I was disappointed to see that nobody broke 1000 in a game there this year, though. Back in 2001 (or 2002--not sure which), Eisenhower defeated someone something like 1310-105. I'll have to look back in my archives.

One other note: a couple people have sent me detailed results listing who defeated whom (and even scores, in many cases). Thank you, thank you, thank you! That's exactly the kind of information I need. Others of you have sent me information such as, "We had this record at this tournament and lost to this team in the finals." Unfortunately, all I can record from that into the database is the loss to that team in the finals. Since strength of schedule is a key component, it's necessary to know exactly which teams were defeated and which teams were lost to. I understand that, at this point, it may be difficult (or even impossible) to reobtain that data. That's okay--again, this was my beta test year. But at least it's something to think about for next season.

Stephen Colbert · Post by **Stephen Colbert** » Fri Jul 20, 2007 11:16 pm

Tegan wrote:Does Illinois even have a "Columbia High School"? In 13 years of being to every big tournament in the state (including state), I'm not sure I've ever even heard of them ......

Columbia is about 30 minutes south of St. Louis. I'm not sure they play in any tournaments at all, maybe their conference? Their 9-1 record would pretty much account for the state series, so I'm guessing they play a few more matches. The IHSA lists their '05-'06 record as 15-4, so evidently it's not much more than that. Columbia ICS won the IESA Class A state tournament in 2006 and were back at state 2007 (they're 8th graders had confirmation, so they fielded a considerably weaker team the night of state). So, the high school maybe getting some pretty good talent.

Alejandro · Post by **Alejandro** » Fri Jul 20, 2007 11:25 pm

Would you include conference games in the ranking? Our school did pretty bad at IHSA and NAQT, but we were also missing people at those tournaments (although I doubt it would've made much of a difference for NAQT).
-Alejandro

Matthew D · Post by **Matthew D** » Fri Jul 20, 2007 11:47 pm

Did you have to attend an national tournament to be included in the ranking?

quizbowllee · Post by **quizbowllee** » Fri Jul 20, 2007 11:55 pm

Matthew D wrote:Did you have to attend an national tournament to be included in the ranking?

Obviously not, since so many teams on there didn't.

I think the biggest flaw with this ranking - and it's no fault of Byko's - is a lack of access to results.

Brindlee Mountain, for example, played well over 100 games this past year - likely close to 200. But, this had us at 29-17. Ironically, it did seem to account for most of our losses

Anyway, I think the point here is that Byko needs us to be vigilant in posting results.

That being said, most tournaments in Alabama don't keep stats on a computer, but just have the old-fashioned poster board with Ws and Ls on them.

Byko · Post by **Byko** » Sat Jul 21, 2007 12:14 am

NCRedhawks wrote:Would you include conference games in the ranking? Our school did pretty bad at IHSA and NAQT, but we were also missing people at those tournaments (although I doubt it would've made much of a difference for NAQT).
-Alejandro

Yes. Since most conferences don't have results posted online (a few in Indiana do, as do one or two in Ohio), I haven't included them in large part. But I'll work to include them if you pass data my way.

quizbowllee wrote:I think the biggest flaw with this ranking - and it's no fault of Byko's - is a lack of access to results.

Brindlee Mountain, for example, played well over 100 games this past year - likely close to 200. But, this had us at 29-17. Ironically, it did seem to account for most of our losses

Anyway, I think the point here is that Byko needs us to be vigilant in posting results.

That being said, most tournaments in Alabama don't keep stats on a computer, but just have the old-fashioned poster board with Ws and Ls on them.

Ding, ding, ding! Johnny, tell Lee what he's won!

Seriously, though, you're exactly right. The nice thing about this, though, is that all it takes is just knowing who defeated whom. In some cases, what you're talking about is all I have to work with, and it may be good enough. I don't know, for example, during which rounds teams picked up wins and losses in prelims at Dorman, but when I see a prelim round robin in which teams went 5-0, 4-1, 3-2, 2-3, 1-4, and 0-5, I have all the information that I actually need. Sure, scores and stuff would be nice for aesthetic purposes, but it wouldn't affect the actual calculations.

Again, I just did this as a beta test this year and wanted to give people something to argue about over the summer so that it wouldn't be so quiet. At least now there may be more of an expectation of what I'd need to be able to incorporate tournament results into next year's ratings.

quizbowllee · Post by **quizbowllee** » Sat Jul 21, 2007 12:19 am

If nothing else, I will try to take a digital picture of the poster boards from every tournament we attend and e-mail them to you.

-Lee

Matthew D · Post by **Matthew D** » Sat Jul 21, 2007 1:57 am

Well I know that three of the tournaments we played in where kept on computer... I was just wondering how we faired this year...

rchschem · Post by **rchschem** » Sun Jul 22, 2007 1:03 pm

This is a mighty, mighty effort. Truly impressive.

Hopefully your job search will turn out successfully, and soon. :)

Do the scores of the matches won and lost matter? If so, I'll send you some.

Eric

Byko · Post by **Byko** » Sun Jul 22, 2007 6:55 pm

rchschem wrote:This is a mighty, mighty effort. Truly impressive.

Hopefully your job search will turn out successfully, and soon. :)

Do the scores of the matches won and lost matter? If so, I'll send you some.

Eric

Wait a second--should I be looking for a new job? Is there something that you know that I don't?

While it's nice to have the exact scores (for aesthetic purposes), they don't affect the actual ratings--just wins and losses. I'm curious what people think about whether margin of victory should or should not matter. It would certainly be more difficult to implement (especially with the various formats out there), but I'm interested in people's opinions.

Post by **theMoMA** » Sun Jul 22, 2007 7:03 pm

I'd say that margin shouldn't be incorporated. Margin is a forward-looking stat (predicting future wins), actual wins are a backward-looking stat that tells you how teams actually did. Since this is trying to rank how teams actually did at the end of a long season, and there's no reason to try to predict how they'll do in the future (graduations making the teams completely different in future games and whatnot) I think straight up W-L (obviously with consideration for strength of schedule) is the way to go.

rchschem · Post by **rchschem** » Sun Jul 22, 2007 7:59 pm

theMoMA wrote:I'd say that margin shouldn't be incorporated. Margin is a forward-looking stat (predicting future wins), actual wins are a backward-looking stat that tells you how teams actually did. Since this is trying to rank how teams actually did at the end of a long season, and there's no reason to try to predict how they'll do in the future (graduations making the teams completely different in future games and whatnot) I think straight up W-L (obviously with consideration for strength of schedule) is the way to go.

Well said. The unholy ass-whipping MW A put on my A team at the HSNCT by no means represents what would happen at our next meeting.

My team will be even weaker. :)

Matt Weiner · Post by **Matt Weiner** » Sun Jul 22, 2007 10:32 pm

Accounting for roster variation would be more helpful than accounting for margin, though I don't know how easy it would be to automate that even if the SQBS data necessary was always present, which of course it is not.

Frater Taciturnus · Post by **Frater Taciturnus** » Mon Jul 23, 2007 12:51 am

I'm not entirely convinced we were at any point near the rank of 101st, though we did occasionally have a very good match and got an upset of a better team (Cave Spring, Walter johnson, and that other one[the one that was a total fluke]) I think a strength of schedule factor's impact would be something interesting to measure.

Edit: Dave, the Battle of the brains tournament included several teams from cetral VA that are on your list. The results are at
Southeast region
Central

Edit the II:

squareroot165 wrote: That tournament was not NAQT format. The questions were all short, and worth 15 points each, with one bonus question worth 10 points. The games were also longer than most.

I apologise if my craptastic math skills are wrong, but at 25 potential points per question, and 1,435 aggregate points scored, there would have to have been ~60 TU minimum in the round. Was this quizbowl endurance testing?

evilmonkey · Post by **evilmonkey** » Tue Jul 24, 2007 11:04 am

I'm pretty sure I know the answer to this, but is there any differentiation between Chip's questions and NAQT?

I think you also missed a few conferences in Indiana. Several of the conferences have full results on the inqblots website

Stat Boy · Post by **Stat Boy** » Tue Jul 24, 2007 11:26 am

I'm pretty sure I know the answer to this, but is there any differentiation between Chip's questions and NAQT?

Yes.

Mike Bentley · Post by **Mike Bentley** » Tue Jul 24, 2007 11:38 am

I could probably help you out with a program that would be able to extract stats from an SQBS page. You'd just have to match up the school in your central database of schools with the school name that appears on the SQBS generated website.

If you have some examples of other common stat posting formats I could probably help you out with that, too.

Shoot me an e-mail at [email protected] if you're interested.

Frater Taciturnus · Post by **Frater Taciturnus** » Thu Jul 26, 2007 3:17 pm

Has anyone else thought about comparing the statistical results of Byko's rankings and the human poll Fred recently conducted? ( I might have missed it as I skimmed through the discussion.)

TheCzarMan · Post by **TheCzarMan** » Thu Aug 09, 2007 11:14 am

Bloomfield High School (NJ), 4-1 at Half Hollow (Beat Kellenberg C, Port Jefferson NY A, Colonia B, and Chatam A, lost to ?

4-2 at Governor Livingston (Beat Livingston B, Pingry C, Bergen C, Leonia A, Lost to Seton Hall Prep B, Delbarton A)

2-2 at Millburn (Beat Colonia A, Monsignor A, Lost to Delbarton A and Glen Rock A)

2-4 at State NAQT (Beat MAST A and St. Peter's A, Lost to Bergen A, Prep B, Pingry B, and Hi Tech A

BroNi · Post by **BroNi** » Sun Aug 12, 2007 8:58 pm

Bloomfield...

At Half Hollow, you lost to Commack A in the first round of the playoffs.

TheCzarMan · Post by **TheCzarMan** » Sun Aug 12, 2007 9:31 pm

BroNi wrote:Bloomfield...

At Half Hollow, you lost to Commack A in the first round of the playoffs.

Thanks, the name of the school slipped me, all I remember is this one kid absolutely dominating the question about White Castle to win it.

dyetman89 · Post by **dyetman89** » Sun Aug 12, 2007 9:42 pm

TheCzarMan wrote: all I remember is this one kid absolutely dominating the question about White Castle to win it.

You've got to be kidding me. If you want just one reason why so many of us detest trash, here's a fine one; what a question to lose a game on.

Sen. Estes Kefauver (D-TN) · Sun Aug 12, 2007 9:45 pm

WHAT THE HELL IS WRONG WITH EVERY TEAM IN NEW YORK THAT'S NOT STUYVESANT OR HUNTER

TheCzarMan · Post by **TheCzarMan** » Sun Aug 12, 2007 10:22 pm

dyetman89 wrote:
TheCzarMan wrote: all I remember is this one kid absolutely dominating the question about White Castle to win it.
You've got to be kidding me. If you want just one reason why so many of us detest trash, here's a fine one; what a question to lose a game on.

It was run in an NAC style, although the questions did have a lot more trash than you'd see in NAQT.

Sen. Estes Kefauver (D-TN) · Sun Aug 12, 2007 10:32 pm

TheCzarMan wrote:It was run in an NAC style, although the questions did have a lot more trash than you'd see in NAQT.

lol isn't that redundant? I mean, NAQT has more than its share of trash, but NAC is the trashiest (literally and figuratively) tournament ever.

The Infanta · Post by **The Infanta** » Wed Aug 15, 2007 4:44 pm

charlieDfromNKC wrote:WHAT THE HELL IS WRONG WITH EVERY TEAM IN NEW YORK THAT'S NOT STUYVESANT OR HUNTER

What do you mean?

BuzzerZen · Post by **BuzzerZen** » Wed Aug 15, 2007 5:03 pm

TheInfanta wrote:
charlieDfromNKC wrote:WHAT THE HELL IS WRONG WITH EVERY TEAM IN NEW YORK THAT'S NOT STUYVESANT OR HUNTER
What do you mean?

Charlie means, "Why don't teams in New York like good questions?"

Sen. Estes Kefauver (D-TN) · Wed Aug 15, 2007 6:08 pm

Yeah thats it.

The Infanta · Post by **The Infanta** » Wed Aug 15, 2007 8:51 pm

Unfortunately, while it is true that the majority of New York teams only frequent

tourneys, you do have us (Kellenberg). Half of all the tourneys we attended this year used NAQT questions (not including HSNCT). We also host an annual one of our own in the fall using this format. Our moderator likes to expose us to different ones so we can be more well rounded. So, please count us with Stuy and Hunter as the few that are familiar with the good kind. Like most of you, I also prefer it to

.

Sen. Estes Kefauver (D-TN) · Wed Aug 15, 2007 8:56 pm

OK dude, don't take so much offense to this crap. I'm just using Stuy and Hunter as the examples in a post that's not really ultra serious. I don't really care which exact teams in New York like NAQT, since the vast majority still dont play it.

The Infanta · Post by **The Infanta** » Wed Aug 15, 2007 10:35 pm

No offense taken. I was just letting you know.

BroNi · Post by **BroNi** » Sun Oct 14, 2007 12:46 pm

I would like to open up an issue for discussion, which is in a way directly related to Byko's ratings. I know there have been previous discussions about this, but Dave's system has changed things.

At Rutgers yesterday, another coach and I were discussing some points concerning the upcoming tournament at Kellenberg, LIFT. He said he might only be bringing his B, C and possibly D teams.

So the question is - what is the best way to report the results to Dave, or even just to post them online. Should I, as TD, simply and automatically identify the teams that any school brings as A, B, C, etc.... Should we let the coach determine what teams he or she is bringing? I guess I am asking when does an A team cease to be an A team and become a B team. Are A teams just when you have ALL 4 members or is an A team an A team if ANY one from the A team is on it?

I know that I am opening a hornet's nest here, but I am curious as to what most think. I know, as a coach, that in the coming weeks, our own A team will be missing some of its members, and will obviously not do as well as the full squad. And if the tourney's results are reported to Dave, as I think they should be, our rating will suffer. Through the years, I have always considered that whatever team I bring to a tournament is my A team, regardless as to who was on it - and I simply added their results to my A team's overall stats.

In a related vein, there was at least one time last year at a tournament where once we got the brackets, it was obvious to me that my B team was in a bracket of death, while my C team was in a relativly easy bracket. Without hesitation, I switched my 2 teams. And my B team in the easier bracket did very well, while my C team got slaughtered. When the results got to Byko, our C (which was really our B) team's ratings went way up, and our B team....well, you know the rest of the story.

Should I have switched them?

My own opinion is first of all, there is no easy answer to the above situations. I suspect that as a coach, I would do the same in the future, and even with my A team. I want them to make the playoffs. And if I can do that by calling them my "B" team because the brackets are not balanced, then so be it.

As a TD, just to make it easier for me, I wouldn't like to have schools with no labeled "A" team. I can just see it as a major source of confusion. Carried to the extreme, a coach could say that I don't have my A, B or C teams here today, but I do have my D, F, and Q teams. Imagine trying to set up even brackets with team letters like those? Who's to say? So you don't have your A team here, or you switched your B and C teams. In the end, so what? Are Byko's ratings really that important?

To me, what you do at Nationals is what matters. Opinions? Let them fly!

BuzzerZen · Post by **BuzzerZen** » Sun Oct 14, 2007 1:01 pm

I think that it is unacceptable to switch around the composition of teams after seeing the brackets. Tournament directors construct brackets with an eye to balancing them out, and need to have a general expectation that a school's A team is better than its B team and its B team is better than its C team, and so on. Switching your B and C teams is unfair to the tournament director and your teams' opponents. The fact that there was a "bracket of death" was obviously unfortunate, but it smacks of gamesmanship to move teams around in an attempt to secure victory.

Byko · Post by **Byko** » Sun Oct 14, 2007 1:03 pm

These are good thoughts and comments, Brother Nigel. I'm still not sure whether it was really Detroit Catholic Central A or Detroit Catholic Central B that was in the final four at the Danville tournament earlier this season.

My short thought on this is that I think it depends on your team and how much strength is or is not available. For example, there was one time where Charter went to a tournament last year and was missing its two top players. I opted that they were probably closer to Charter B than Charter A, so that's how I included it.

I'd say this is, unfortunately, almost an intractable problem, but it's good to discuss and see what people are willing to accept. After all, my rating system is just "for entertainment purposes only." It's an estimate, and nationally, it won't be a good estimate until there's a lot more crossover among regions.

cvdwightw · Post by **cvdwightw** » Sun Oct 14, 2007 1:16 pm

I have a further theoretical question: some San Diego teams have a penchant for dividing their best four players among two different teams and then randomly calling one team A and one team B. This frequently happens with Torrey Pines, and for example Rancho Bernardo's four players from last year's nationals team were split between the A and B teams at TWAIN (although I believe this may have been an attempt to qualify two teams for HSNCT and if that was successful they may put their four best players together for future tournaments). This results in teams that are often very similar in record, but not indicative of team strength, or in the extreme case of one player being way better than the rest of the team the B team having a 5 or more game better record than the A team. How would such team-splitting affect the rankings?