Head-to-head as a tiebreaker (split from UCU announcement)

Old college threads.
Locked
User avatar
Matt Weiner
Sin
Posts: 8411
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Head-to-head as a tiebreaker (split from UCU announcement)

Post by Matt Weiner » Wed Mar 06, 2013 4:11 pm

Tees-Exe Line wrote:Any tiebreaker to enter the final will first regard teams’ records in their head-to-head matches, and if that is insufficient (or the tournament is bracketed), points per bonus.
Is this a joke?

[Split from this thread--mgmt]
Matt Weiner
Founder of hsquizbowl.org

User avatar
Tees-Exe Line
Tidus
Posts: 622
Joined: Mon Apr 12, 2010 5:02 pm

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Tees-Exe Line » Wed Mar 06, 2013 4:12 pm

No. Is there some reason it's ridiculous?
Marshall I. Steinbaum

Oxford University (2002-2005)
University of Chicago (2008-2014)

Get in the elevator.

User avatar
Matt Weiner
Sin
Posts: 8411
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Matt Weiner » Wed Mar 06, 2013 4:13 pm

Head-to-head is a stupid tiebreaker for reasons that have long been established. Why would you ever not use PPG in this situation? And are you seriously proposing using PPB instead of PPG even in an intrabracket tie? What?
Matt Weiner
Founder of hsquizbowl.org

User avatar
Tees-Exe Line
Tidus
Posts: 622
Joined: Mon Apr 12, 2010 5:02 pm

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Tees-Exe Line » Fri Mar 08, 2013 4:34 pm

grapesmoker wrote:Matt is right: h2h is a terrible tiebreaker (double-values a single game)
I'm not wedded to the format I posted here, but I must say I haven't heard a convincing argument against head-to-head tiebreakers. In particular, the proposition that it double-values a single game is not correct. Normalizing the weight on every other game to 1, the weight on the head-to-head game is in the range (1,2). If that game were in fact double-counted, there would be no tie. So the question is whether the head-to-head match should have a weight epsilon greater than the weight on every other match, for epsilon < 1, and I think there's a strong argument that it should. The tiebreaker is supposed to answer the question "which is the better of these two teams?" and the data from the match in which those teams met is more informative than data from other matches; hence a higher weight is justified.

More philosophically and perhaps quizbowl-relevant, matches between good teams feature more aggressive buzzing since one cannot afford to wait for additional clues, so I would argue doing well in that situation is demonstrative of better quizbowling than running up the score against crappy teams. That argues further against a PPG tiebreaker, or at least calculating PPG against all the teams rather than some selected subset.

I'm not going to divert attention from this tournament to a stupid argument about proper tiebreaking, but since there's such a consensus against the head-to-head tiebreaker I favor, I'd like to hear more than just assertions that up-weighting the head to head match is somehow obviously bad.
Marshall I. Steinbaum

Oxford University (2002-2005)
University of Chicago (2008-2014)

Get in the elevator.

User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5518
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Important Bird Area » Fri Mar 08, 2013 4:47 pm

Tees-Exe Line wrote: I'm not wedded to the format I posted here, but I must say I haven't heard a convincing argument against head-to-head tiebreakers.
Read this thread from 2008 and then get back to us.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Sen. Estes Kefauver (D-TN) » Mon Mar 11, 2013 12:02 am

It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White

User avatar
theMoMA
Forums Staff: Administrator
Posts: 5689
Joined: Mon Oct 23, 2006 2:00 am

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by theMoMA » Mon Mar 11, 2013 1:09 am

Horned Screamer wrote:It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
Not to mention the team that won the head-to-head matchup is also likely the team that took the worse loss. (In a 10-team round robin, consider two teams tied at 8-1. If the #1B team beat the #1A team, then the #1B team necessarily lost to a team at #3 or below, which the #1A team necessarily beat. Yet the #1B team gets the tiebreaker. This seems perverse to me.)
Andrew Hart
Minnesota alum

User avatar
Tees-Exe Line
Tidus
Posts: 622
Joined: Mon Apr 12, 2010 5:02 pm

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Tees-Exe Line » Mon Mar 11, 2013 11:16 am

Horned Screamer wrote:It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
Thanks for re-stating the very point I already addressed in my previous post. But when it comes to totally unnecessary, pointlessly hostile forum posts, we know whom to turn to. At least this time you didn't out and out lie.

As for people whose arguments actually deserve discussion, if the thread Jeff linked above is what established the consensus in favor of a PPG tiebreaker and against head-to-head, that consensus ought to be revisited. As I understand the analysis, the sample was taken from tournaments featuring repeated match-ups. Each of the candidate tiebreakers was calculated before the second (or third...) match between heretofore-tied teams, and the "prediction" about the outcome of that match was tested against the actual outcome, yielding statements like "PPG predicted the outcome of the repeat match 70% of the time." If I've misunderstood the procedure, please correct me.

That procedure has a number of flaws, but probably the most important one to address and resolve is one of interpretation about the nature of a tiebreaker. First, let me propose that the best way to break a tie is to play a complete packet between the two tied teams and award the tiebreaker to the winner of that match. This debate is about which second-best tiebreaker to adopt given time/packet/whatever constraints. The notion that the proper second-best tiebreaker should "predict" the outcome of that unplayed match is crucially different from actually playing it, however. I submit that the correct objective for the proper tiebreaker is to produce a binary ranking between two teams based solely on those teams' results in the tournament that produced the tie. The binary ranking is different from a ranking of all the teams in the tournament in which you would break in favor of which of two teams had the higher ranking in that overall list.

Aside from the mistaken notion that prediction is the goal of tiebreaking, there are (I believe) a number of problems with the 2008 thread. First, at the time Andrew Hart made a rather undefined but I think essentially correct critique of the sample. A random set of tournaments that happened to feature tied teams before the repeat match-ups is just not drawn from the relevant population, which is tournaments that featured actual tiebreakers. Unfortunately, since it seems like PPG is hegemonic we won't have enough variation in the sample to analyze the alternatives, so perhaps something like the 2008 sample is the best we can do. Second, the outcome in this case is a binary prediction: was the tiebreaker in question was "right" or "wrong" about the outcome of the repeat match? If, as I suspect, a linear probability model was used, than in some sense that is required to be incorrect by construction. At the very least we'd need corrected standard errors around the percentage-correct estimates. Since the inputs have a lot of variance empirically, it would be far better would to use probit or logit. (Were the PPG and PPB tiebreakers specified as binary or continuous variables in that procedure, ie "Team A has higher PPG" or "Team A's PPG - Team B's PPG" or some variation thereof?)

Finally, I should say that it's quite possible to use a function of several tiebreakers. That seems to be the philosophy behind the NAQT "D-value," and I don't see why it can't be done here. I imagine it wouldn't be hard to program such a function into the tournament statistics software, and we can discuss what the function would be.

Let me conclude by saying that IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament, you should not be satisfied with the status quo. To my mind, by rewarding teams for doing well in unimportant matches, it overvalues a lot of uninformative data.
Marshall I. Steinbaum

Oxford University (2002-2005)
University of Chicago (2008-2014)

Get in the elevator.

User avatar
Cheynem
Sin
Posts: 6613
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Cheynem » Mon Mar 11, 2013 11:32 am

But if you use head to head, does that not reward the team who did not do well in "unimportant" matches (i.e., losing to a team below them)?

Edit: However, as the World Baseball Classic demonstrates, breaking ties by PPG does lead to intense fistfights.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Tees-Exe Line
Tidus
Posts: 622
Joined: Mon Apr 12, 2010 5:02 pm

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Tees-Exe Line » Mon Mar 11, 2013 11:37 am

Cheynem wrote:But if you use head to head, does that not reward the team who did not do well in "unimportant" matches (i.e., losing to a team below them)?
I'm not exactly sure what you're saying, but I guess relative to other tiebreakers head-to-head DOES reward the team that did NOT do well in a less important match (which is downweighted by definition)--and that's a plus!

My point about unimportant matches is really about clubbing baby seals, though. The match in which the head-to-head winner lost is almost certainly against a reasonably good team albeit not one part of the relevant comparison for the tiebreaker, whereas the PPG differential is usually dominated by lopsided matches whose data I'm proposing is just not as important for the bilateral comparison at issue.
Marshall I. Steinbaum

Oxford University (2002-2005)
University of Chicago (2008-2014)

Get in the elevator.

User avatar
Matt Weiner
Sin
Posts: 8411
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by Matt Weiner » Mon Mar 11, 2013 11:44 am

Marshall, speaking as a board staffer, you can't post about the tone of other people's posts or tell them to stop posting; don't do so.

Speaking as a quizbowl contributor, let me say that, like all other attempts to make up for a paucity of the actual data being analyzed by performing additional arithmetic wizardry, the infamous thread about tiebreaker correlations is not what caused or changed anybody's opinions on this issue. The reason everyone involved in good quizbowl does it this way (which is, in fact, a great reason to do it that way) is because of years and years of constant observation and discussion of the fairness of all the methods that used to be tried arriving in an evolutionary fashion at this very solid consensus. If the (completely correct, and presented over and over again to you for that reason and no other) point about arbitrary double-counting and rewarding losing to a worse team doesn't communicate this to you, just remember to look at the premise of what you are trying to evaluate: a bracket/round-robin is evaluating a team's strength against THE FIELD (the entirety of teams within the bracket), not "any one arbitrarily chosen opponent." That's why teams' overall record is the first ranking. PPG is superior to any head-to-head result or to PPB as a tiebreaker because it reflects the teams' performance on all questions played, against the entire field, and thus does not change the very frame of reference as to what is being ranked between the tie-creating and tie-breaking stat.
Matt Weiner
Founder of hsquizbowl.org

User avatar
Cheynem
Sin
Posts: 6613
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Cheynem » Mon Mar 11, 2013 11:50 am

I guess I don't get the "clubbing baby seals" argument--especially when each team is playing the same schedule, which makes PPG that much better. The point of using PPG is to see how well this team is playing over a series of games, easily comparable since both teams are playing the same other teams. It factors in the scoring (the only way to determine who is playing well) from the various games--I don't think the point about "clubbing baby seals" is valid (more on that in a sec), but either way, it's fair to look at it because both teams had equal opportunity to "club seals," so to speak.

I don't think the "clubbing seals" analogy is valid for a few reasons:

1. If this tournament is like last year's, there are no baby seals. Even the teams at the bottom of the field last year pulled off some upsets and were competitive. I think Rob and D. David Seal were near the bottom last year; they aren't baby seals (they beat us), and on the right packet, putting up a lot of points on them is just as impressive as putting up a lot of points about Kurtis and Libo.

2. This isn't like a sport where when you get a big lead, you can empty your bench or just hold on to the ball. In quizbowl, you still try to get the question to the best of your ability and thus improve your PPG. The only way to "ease up" in a blowout would be to just not buzz, which is very patronizing to the other team.

I don't know if I'm going to change your mind on this and I'm obviously not super invested in a tiebreaking procedure for a side event, but I just wanted to share my thoughts.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Tees-Exe Line
Tidus
Posts: 622
Joined: Mon Apr 12, 2010 5:02 pm

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Tees-Exe Line » Mon Mar 11, 2013 12:01 pm

I used the clubbing seals analogy because it's relevant to the larger question of tiebreaking in most tournaments, obviously not really a good analogy for this one. Addressing your and Matt's points that the frame is to see how teams played against the field, the whole point is there's a tie so two or more teams did equally well. Now we have to decide which, in fact, did best, and it's appropriate to change that frame to one of bilateral comparison.

As for "double-counting," that just isn't what's happening, yet everyone keeps repeating it as though it decides the issue. For that reason I've concluded that the consensus is not correct and just because everyone agrees on something doesn't mean they're right. The more the people keep repeating falsehoods while simultaneously demanding unquestioning obeisance, the less I'm persuaded by their position.

I also disagree with Mike's second point. I think there is a difference between how good teams play hard matches vs. easy ones, and their performance in hard matches is what should be preferentially considered. I certainly play more attentively and aggressively when I think the opposition poses a threat.
Marshall I. Steinbaum

Oxford University (2002-2005)
University of Chicago (2008-2014)

Get in the elevator.

User avatar
Matt Weiner
Sin
Posts: 8411
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Matt Weiner » Mon Mar 11, 2013 12:06 pm

It is what's happening. In the scenario of "Michigan 12-1, Chicago 12-1, Illinois 10-2" you are double-counting Chicago's victory over Michigan as opposed to their loss to Illinois. Your reasoning is "we have to count Chicago's game over Michigan because we're trying to figure out whether Chicago is better than Michigan because those are the teams being compared"...i.e., your proof of why head-to-head should be used contains the assumption that a head-to-head tiebreaker will be used and that bracketed results aren't played against the field! You're assuming the conclusion and creating a tautology!
Matt Weiner
Founder of hsquizbowl.org

User avatar
Skepticism and Animal Feed
Auron
Posts: 3182
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Skepticism and Animal Feed » Mon Mar 11, 2013 12:11 pm

If you are concerned that teams play up or down to their opponents, why not use the Litvak Slope-Intercept, a statistic that measures exactly that? True, it's an individual stat, but you can probably figure out a way to convert it into a team stat.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

fett0001
Tidus
Posts: 670
Joined: Wed Feb 27, 2008 11:50 am

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by fett0001 » Mon Mar 11, 2013 12:31 pm

I'm just wondering if anyone still has the script to pull out that data from the old thread.
Mike Hundley
PACE Member
Virginia Tech

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by grapesmoker » Mon Mar 11, 2013 1:05 pm

What is it with people determined to reinvent the wheel as a square and demand we all drive on catenary-shaped roads?

The double-counting is self-evident. You're giving extra weight to a game between two teams on the grounds that one team beat the other; why would you do this? Even if we were to accept the argument that you're trying to create a binary ranking between two teams, a single game would likely not be a sufficient sample size to do so. Which is why you sample over the much great number of teams that both of the tied teams play.

Matt's example is entirely correct. Taking this:
IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament
as a premise, nothing you have argued leads to this:
you should not be satisfied with the status quo.
In fact, people are taking great pains to point out that overall performance over the course of the tournament is a better indicator of which of two teams has done better at a given tournament, rather than which of two teams has done better at in a given game. You're engaged in basic question-begging here.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
Tees-Exe Line
Tidus
Posts: 622
Joined: Mon Apr 12, 2010 5:02 pm

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Tees-Exe Line » Mon Mar 11, 2013 1:09 pm

The very fact that there's a tie says that both teams did equally well against the field! Since it has to be broken somehow and we're operating from the agenda "produce a binary ranking," then the most relevant data (and hence the data deserving a higher weight) is from the head-to-head match(es).

I'm not sure what more there is to be said on this subject since we're all just repeating ourselves.
Marshall I. Steinbaum

Oxford University (2002-2005)
University of Chicago (2008-2014)

Get in the elevator.

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by grapesmoker » Mon Mar 11, 2013 1:15 pm

Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
No it doesn't. I can't believe you just said this. Look at the stats to find out how well they actually did against the field.
Since it has to be broken somehow and we're operating from the agenda "produce a binary ranking," then the most relevant data (and hence the data deserving a higher weight) is from the head-to-head match(es).
Your conclusion again fails to follow from your premise. You are, again, simply begging the question by advancing the data which will be used in the head-to-head tiebreaker to the status of "most relevant" data. The relevance of the data is in fact what's in dispute here.
I'm not sure what more there is to be said on this subject since we're all just repeating ourselves.
Here's a pro-tip: sometimes the majority consensus is wrong, to be sure. But sometimes the majority consensus is not only right, but right for such a simple and straightforward reason that arguing against it devolves into circular argumentation and tautology. Which horn is represented by this conversation is left as an exercise to the reader.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
Haaaaaaaarry Whiiiiiiiiiite
Auron
Posts: 1136
Joined: Tue Dec 04, 2007 8:46 pm
Location: Fairfax, VA
Contact:

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Haaaaaaaarry Whiiiiiiiiiite » Mon Mar 11, 2013 1:16 pm

Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
Actually, by your logic, the team that lost the head-to-head matchup did better against the field, as they had one fewer loss against common opponents.
Harry White
TJHSST '09, Virginia Tech '13, PACE Member
Tournament Database Search
Quizbowl Schedule Generator
Will run stats for food

User avatar
AKKOLADE
Sin
Posts: 15274
Joined: Thu Apr 24, 2003 8:08 am

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by AKKOLADE » Mon Mar 11, 2013 2:06 pm

Communi-Bear Silo State wrote:
Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
Actually, by your logic, the team that lost the head-to-head matchup did better against the field, as they had one fewer loss against common opponents.
This.

Put me down as another anti-H2H tiebreak guy.
Fred Morlan
PACE President, 2018-19
International Quiz Bowl Tournaments, co-owner
University of Kentucky CoP, 2017
hsqbrank manager, NAQT writer (former subject editor), former hsqb Administrator/Chief Administrator, 2012 NASAT TD

User avatar
Mewto55555
Tidus
Posts: 708
Joined: Sat Mar 13, 2010 9:27 pm
Contact:

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Mewto55555 » Mon Mar 11, 2013 8:21 pm

Maybe I'm thinking about it wrong, but isn't the head-to-head match already counted way more than any other match? If two teams are tied in record, then reversing the result of any other game will cause there to only be a one-game difference, whereas if you reverse the result of the H2H game, then that creates a two-game difference, so the H2H game (for purposes of determining whether A>B or B<A) is ALREADY being double-counted.
Max
formerly of Ladue, Chicago

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by No Rules Westbrook » Mon Mar 11, 2013 8:23 pm

Take it from someone who's more of a contrarian than even you, Marshall....you're overthinking this. H2H tiebreaking is double counting, and a really poor way to break ties.

On the other hand, even when the field plays a full round robin, I think PPG vs. PPB is a matter of personal preference about what you want to reward in this game. I'll admit that PPG is, as a strictly empirical matter, the better predictor of which team is more likely to win. But, I've always supported PPB because I feel like it puts a least a little bit of balance into this tossup-heavy game of ours - bonuses and bonus conversion are already the redheaded step-child of qb in that their importance has always been less important to team success than tossup conversion, and yet, bonus conversion is a much better measure of the breadth of a team's knowledge (which is what I'd rather reward in quizbowl, given the option).
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
prodski
Lulu
Posts: 47
Joined: Sun Jan 22, 2012 10:46 am

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by prodski » Mon Mar 11, 2013 8:27 pm

Put me down in the minority that doesn't understand why h2h isn't the first tiebreaker. If beating a team head to head doesn't matter, why even keep w/l records? Just rank everyone according to PPG? According to the logic, that is most important element, determining the best overall team at a tournament. What if you had two teams tie, and you played a best of 3. Team A wins the first game by 10 team B wins the 2nd game by 100, Team A wins the 3rd by 10. Surely you would crown Team A the champion, right? Since they won 2 of 3? You wouldn't reward Team B for having more points would you? I know quizbowl is different than sports and virtually every other competition I can think of that uses h2h, but I still don't understand the argument for not using h2h. I feel it is over-rewarding good teams, basically by saying that they lost to a team they shouldn't have, and it must have been a fluke, but they played better all day, so they deserve it more. Yes, the other team lost a match they shouldn't have either, but they beat the team they ended tied with. I would also point out that beating a team twice is tough - even in the NFL or NBA - hence the reason the previous data may not "predict" the winner of 2 close teams who play again in favor of the first winner more often. When I host a tournament, like I have over the past 17 years, and I can determine my own tiebreakers, it is simple, first tiebreaker is always h2h. If you feel your team is better than the other team, you should have beaten them when you had the chance. Really simple. Two fairly even teams, I don't care what you did to the last place team, all I care about is what you did when you had the chance to play the team you ended up tying with. It is harder to see with 1-loss teams, but lets extrapolate a bit: imagine 2 teams go 22-4 in a crazy tournament, and one team beats the other 4 times. The other loses 4 matches to other teams in the tournament. Are you still going to go to points? That is crazy. Maybe the first team just matches up better against the 2nd team? They deserve 1st place in my opinion out of the 2 tied teams. Yes, the other team is better against the rest of the field, but the rest of the field is no longer part of the discussion, we are down to 2 teams. Yes, I am double-counting those 4 wins since it is the best indicator of who I think is better in a hypothetical two team tournament. Let me also say that as long as everyone knows the rules going into a tournament, whatever they may be, is fine with me - I don't lose sleep over this. I just don't think it is so obvious that ppg is a better tiebreaker than h2h. If you are trying to crown the best overall team, PPG may be the best tiebreaker. If you are trying to crown the real champion of a tournament, whether it be luck, or a fluke, or an improbable helmet catch, you go head to head and determine it on the field like every other competition I can think of besides quizbowl.
Peter Rodski
Professor of Mathematics
CCCT Champions 2016
Jefferson Community & Technical College
Louisville, KY. 40272

User avatar
setht
Auron
Posts: 1177
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by setht » Mon Mar 11, 2013 9:10 pm

prodski wrote:Put me down in the minority that doesn't understand why h2h isn't the first tiebreaker. If beating a team head to head doesn't matter, why even keep w/l records? Just rank everyone according to PPG?
I was thinking about this earlier, and I think the question is, what is a tiebreaker supposed to do? Is it supposed to pick out the better of two teams in "a hypothetical two team tournament," as you suggest, or is it supposed to pick out the team that's done better against the full field of the tournament, or the team that's more likely to do well in a top bracket or finals match (or whatever the tiebreaker is being used to determine)? Given that finals matches should always be played, it seems to me that picking out the better of two teams in a hypothetical two team tournament pretty much never matches the actual in-tournament situation in which a tiebreaker arises: a tie in record at the end of prelims or playoffs that affects which teams have a shot at the title. In other words, at the end of a prelim round-robin, if teams A and B are tied in record, and you're trying to decide which one should advance to the top playoff bracket, I'm not sure that the question "which of these teams do I think would win a hypothetical two team tournament?" is the right one to ask. It seems like the more relevant question is something like "which of these teams do I think has the better shot at the title?" Or, if you're looking at a tiebreaker for a non-top bracket, the question could be rephrased as "which of these teams do I think will place higher in this bracket if they get in?" Once the question is phrased that way, I think it makes perfect sense to look at data beyond which team beat which other team. For instance, suppose team A beat team B by 5 points, lost to team C by 300 points, and beat all other teams by 5 points. Meanwhile, team B beat all teams except for team A by 150 points. Which of these teams would you guess has a better shot at success in the top bracket? In a finals match against (say) team C?

-Seth
Seth Teitler
Formerly UC Berkeley and U. Chicago
President and Chief Editor, NAQT
Emeritus member, ACF

User avatar
prodski
Lulu
Posts: 47
Joined: Sun Jan 22, 2012 10:46 am

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by prodski » Mon Mar 11, 2013 9:22 pm

I agree with you Seth, if you are looking to move a team up into a bracket, that makes perfect sense. I obvioulsy am looking at it for crowning a champion, or final placement at a tournament when it is over and there are no other tiebreakers or matches to be played. Your point is well taken. Thanks.
Peter Rodski
Professor of Mathematics
CCCT Champions 2016
Jefferson Community & Technical College
Louisville, KY. 40272

User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Post by Sen. Estes Kefauver (D-TN) » Mon Mar 11, 2013 11:23 pm

prodski wrote:I agree with you Seth, if you are looking to move a team up into a bracket, that makes perfect sense. I obvioulsy am looking at it for crowning a champion, or final placement at a tournament when it is over and there are no other tiebreakers or matches to be played. Your point is well taken. Thanks.
At a standard quizbowl tournament the final placement is already determined by your win-loss record within the playoff pool, and any leftover ties that need to be played off will be broken by a packet anyway.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Post by cvdwightw » Sat Apr 20, 2013 12:38 am

Apologies for necroing this thread. In my defense, I haven't posted anything here in over 6 months.
Tees-Exe Line wrote:As for people whose arguments actually deserve discussion, if the thread Jeff linked above is what established the consensus in favor of a PPG tiebreaker and against head-to-head, that consensus ought to be revisited. As I understand the analysis, the sample was taken from tournaments featuring repeated match-ups. Each of the candidate tiebreakers was calculated before the second (or third...) match between heretofore-tied teams, and the "prediction" about the outcome of that match was tested against the actual outcome, yielding statements like "PPG predicted the outcome of the repeat match 70% of the time." If I've misunderstood the procedure, please correct me.
This is largely correct, except that statistics were not calculated across the tournament, but rather only in games against common opponents and each other. Thus, a team did not benefit by playing a game against a poor team its opponent did not play.
That procedure has a number of flaws, but probably the most important one to address and resolve is one of interpretation about the nature of a tiebreaker. First, let me propose that the best way to break a tie is to play a complete packet between the two tied teams and award the tiebreaker to the winner of that match.
You will note that this proposition has been a commonly accepted proposition well before this thread (in fact, it was an axiom of the thread under review). I'm not going to respond to Andrew Hart's critique since I think that criticism is somewhat right (for instance, there may have been factors such as question quality and relative team strength that biased the results one way or another; for another, H2H was transformed to "H2H differential" if the teams had played multiple times such that there would be an actual H2H-based tiebreaker if H2H was split evenly between the teams). I don't feel that it's right "enough" to totally invalidate the collected data. If I understand your twist on the critique, however, you are arguing that a "regular" repeat match in a tournament between two teams of the exact same composition as the original match is subjectively different than a "tiebreaker" repeat match, which, furthermore, is subjectively different from a "finals" repeat match. I'm not sure what the subjective difference here is (we used all three types, IIRC). Question quality/difficulty? Should be the same. Distribution? I'd hope that "whether or not that third arts tossup is opera or architecture" would be unbiased as to regular vs. tiebreaker matches. Pressure? I haven't seen any objective or subjective evidence that teams play differently in tiebreaker matches compared to other matches.

Also, I don't understand what you mean by PPG being hegemonic. That shouldn't have any bearing on your (or anyone else's) ability to find a sufficiently large set of games that fit your preconceived notions of what games would be valid samples to collect to answer the question, "what statistical tiebreaker best predicts the results of a hypothetical tiebreaker?". That thread tested two common sense hypotheses - the "Andrew Hart common sense hypothesis" (PPB) and the "JR Barry common sense hypothesis" (H2H) - and found that the data didn't support JR Barry's common sense.
Second, the outcome in this case is a binary prediction: was the tiebreaker in question was "right" or "wrong" about the outcome of the repeat match? If, as I suspect, a linear probability model was used, than in some sense that is required to be incorrect by construction. At the very least we'd need corrected standard errors around the percentage-correct estimates. Since the inputs have a lot of variance empirically, it would be far better would to use probit or logit. (Were the PPG and PPB tiebreakers specified as binary or continuous variables in that procedure, ie "Team A has higher PPG" or "Team A's PPG - Team B's PPG" or some variation thereof?)
Marshall, I've already copped to using the wrong statistical test in this thread and re-analyzed the data using what I believe to be the correct test. The second test used a 2x2 matrix of outcomes (e.g. H2H better won/PPG better won, H2H better lost/PPG better won, H2H better won/PPG better lost, H2H better lost/PPG better lost). I did not make any corrections for the number of tests that were done but a one-tailed McNemar's test between PPB and H2H is still significant at the 5% level even using Bonferroni corrections.

Finally, I'll note that in both the original (wrong) and updated versions, all four statistical tiebreakers performed statistically worse than "who won the match" in predicting "who won the match" and that the only conclusion drawn from the whole thing was that H2H was significantly worse than each of the other three tiebreakers in predicting "who won the match." In fact, H2H (or H2H differential, where H2H was like 1-1) was statistically indistinguishable from a coin flip. I highly suspect that, if you were to run your own statistical analysis on a set of data you collected yourself and you felt assured that no one else could accurately point out flaws in your data collection, you would see a similar result (at least with respect to H2H). The reason for this is that H2H is predicting the result of a future match based on one game (or, best case scenario, 3) - and, at that, a game whose outcomes are extremely highly variable (there was a tournament a few years back where Chicago and Minnesota both beat each other by like 400+ points).
Finally, I should say that it's quite possible to use a function of several tiebreakers. That seems to be the philosophy behind the NAQT "D-value," and I don't see why it can't be done here. I imagine it wouldn't be hard to program such a function into the tournament statistics software, and we can discuss what the function would be.
As the person who largely combined the suggestions of Andrew Hart and others into the D-Value, I sought to accurately rank teams who had played wildly different sets of opponents on potentially different packet sets. The D-Value roughly translates into "how many points would we expect you to score against a hypothetical nationally-average team on the appropriate set of questions," which to me (and apparently NAQT) seems to be as good as any a way of ranking every college team that played SCT without making the calculations too arcane for the average quizbowler (there are some adjustments to the raw rankings to ensure that order of finish within a given tournament takes precedence over statistical measures). Of course a D-Value or equivalent future statistic can be calculated for any individual tournament. However, not knowing your experience, I am unsure whether the second sentence here is due to your inexperience TDing anything larger than ~10-15 teams. Reconstructing the results of an entire bracket, by hand, in a small time window, using scoresheets with borderline unintelligible writing is something that happens all the time in moderately-sized high school tournaments. The best statistical tiebreakers are things that can be computed relatively easily by hand if one of the thousand things that could go wrong with stats-entering does go wrong. I'm not saying it can't be done, but that it would be wholly impractical to do it.
Let me conclude by saying that IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament, you should not be satisfied with the status quo. To my mind, by rewarding teams for doing well in unimportant matches, it overvalues a lot of uninformative data.
No one is arguing that we should be satisfied with the status quo, or that a 4.5-year-old thread is somehow the final word on tiebreakers. However, the objective analysis done in 2008 (and corrected in 2011 due to being wrong) was broadly consistent with subjective quizbowl experience that:

(1) Ties should be broken by actually playing tiebreaker games when possible
(2) Statistical tiebreakers are not a valid substitute for actually playing the game, but are used due to expediency
(3) When teams play identical schedules excepting each other and have identical win-loss records, head-to-head is about as effective at predicting the result of the rematch as a coin flip.

(and if you really want to see an argument against H2H, consider which three teams advance from a bracket in which, entering the final round, A is 7-1 with a loss to B, B and C are 7-1 and playing each other, and D is 6-2 with a win over B and losses to A and C. You'll find that the result depends entirely on whether B beat C if you use H2H - D advances or does not advance based solely on the result of a game it did not play; furthermore, if B wins, the only reason you're breaking a two-way tie instead of a three-way circle of death is because B beat A - a match that involved neither of the teams in the current two-way tie).

Let me conclude by saying that while Matt is largely correct about the insufficiency of data in most quizbowl-related statistical measures (including most of mine), I think he's completely incorrect here. We don't care about who buzzed where or whether someone vultured a neg, which is what we have to approximate using "arithmetic wizardry" in order to obtain most of the statistics he doesn't like. Here, all we care about is (1) whether a team won a game against a team with an identical won-loss record against common opponents excepting each other and (2) how many points (or ppb) the two teams scored against that slate of opponents. These data are available for literally EVERY tournament that has full statistics. So this study can be replicated. Perhaps, given the number of people who have said "there were a lot of problems with that study" (most of whom never say what those problems are; Marshall, you are an exception here, and I hope I've addressed your posted concerns), it should be replicated to account for whatever the problems are. Because the problem isn't that there isn't enough data.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry

Locked