Page 1 of 1

Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Wed Mar 06, 2013 4:11 pm
by Matt Weiner
Tees-Exe Line wrote:Any tiebreaker to enter the final will first regard teams’ records in their head-to-head matches, and if that is insufficient (or the tournament is bracketed), points per bonus.
Is this a joke?

[Split from this thread--mgmt]

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Wed Mar 06, 2013 4:12 pm
by Tees-Exe Line
No. Is there some reason it's ridiculous?

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Wed Mar 06, 2013 4:13 pm
by Matt Weiner
Head-to-head is a stupid tiebreaker for reasons that have long been established. Why would you ever not use PPG in this situation? And are you seriously proposing using PPB instead of PPG even in an intrabracket tie? What?

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Fri Mar 08, 2013 4:34 pm
by Tees-Exe Line
grapesmoker wrote:Matt is right: h2h is a terrible tiebreaker (double-values a single game)
I'm not wedded to the format I posted here, but I must say I haven't heard a convincing argument against head-to-head tiebreakers. In particular, the proposition that it double-values a single game is not correct. Normalizing the weight on every other game to 1, the weight on the head-to-head game is in the range (1,2). If that game were in fact double-counted, there would be no tie. So the question is whether the head-to-head match should have a weight epsilon greater than the weight on every other match, for epsilon < 1, and I think there's a strong argument that it should. The tiebreaker is supposed to answer the question "which is the better of these two teams?" and the data from the match in which those teams met is more informative than data from other matches; hence a higher weight is justified.

More philosophically and perhaps quizbowl-relevant, matches between good teams feature more aggressive buzzing since one cannot afford to wait for additional clues, so I would argue doing well in that situation is demonstrative of better quizbowling than running up the score against crappy teams. That argues further against a PPG tiebreaker, or at least calculating PPG against all the teams rather than some selected subset.

I'm not going to divert attention from this tournament to a stupid argument about proper tiebreaking, but since there's such a consensus against the head-to-head tiebreaker I favor, I'd like to hear more than just assertions that up-weighting the head to head match is somehow obviously bad.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Fri Mar 08, 2013 4:47 pm
by Important Bird Area
Tees-Exe Line wrote: I'm not wedded to the format I posted here, but I must say I haven't heard a convincing argument against head-to-head tiebreakers.
Read this thread from 2008 and then get back to us.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Mon Mar 11, 2013 12:02 am
by Sen. Estes Kefauver (D-TN)
It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Mon Mar 11, 2013 1:09 am
by theMoMA
Horned Screamer wrote:It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
Not to mention the team that won the head-to-head matchup is also likely the team that took the worse loss. (In a 10-team round robin, consider two teams tied at 8-1. If the #1B team beat the #1A team, then the #1B team necessarily lost to a team at #3 or below, which the #1A team necessarily beat. Yet the #1B team gets the tiebreaker. This seems perverse to me.)

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Mon Mar 11, 2013 11:16 am
by Tees-Exe Line
Horned Screamer wrote:It's incredibly simple. The tie wouldn't exist in the first place if not for one team beating another, so it makes no sense to break the tie with a result that caused the tie in the first place.
Thanks for re-stating the very point I already addressed in my previous post. But when it comes to totally unnecessary, pointlessly hostile forum posts, we know whom to turn to. At least this time you didn't out and out lie.

As for people whose arguments actually deserve discussion, if the thread Jeff linked above is what established the consensus in favor of a PPG tiebreaker and against head-to-head, that consensus ought to be revisited. As I understand the analysis, the sample was taken from tournaments featuring repeated match-ups. Each of the candidate tiebreakers was calculated before the second (or third...) match between heretofore-tied teams, and the "prediction" about the outcome of that match was tested against the actual outcome, yielding statements like "PPG predicted the outcome of the repeat match 70% of the time." If I've misunderstood the procedure, please correct me.

That procedure has a number of flaws, but probably the most important one to address and resolve is one of interpretation about the nature of a tiebreaker. First, let me propose that the best way to break a tie is to play a complete packet between the two tied teams and award the tiebreaker to the winner of that match. This debate is about which second-best tiebreaker to adopt given time/packet/whatever constraints. The notion that the proper second-best tiebreaker should "predict" the outcome of that unplayed match is crucially different from actually playing it, however. I submit that the correct objective for the proper tiebreaker is to produce a binary ranking between two teams based solely on those teams' results in the tournament that produced the tie. The binary ranking is different from a ranking of all the teams in the tournament in which you would break in favor of which of two teams had the higher ranking in that overall list.

Aside from the mistaken notion that prediction is the goal of tiebreaking, there are (I believe) a number of problems with the 2008 thread. First, at the time Andrew Hart made a rather undefined but I think essentially correct critique of the sample. A random set of tournaments that happened to feature tied teams before the repeat match-ups is just not drawn from the relevant population, which is tournaments that featured actual tiebreakers. Unfortunately, since it seems like PPG is hegemonic we won't have enough variation in the sample to analyze the alternatives, so perhaps something like the 2008 sample is the best we can do. Second, the outcome in this case is a binary prediction: was the tiebreaker in question was "right" or "wrong" about the outcome of the repeat match? If, as I suspect, a linear probability model was used, than in some sense that is required to be incorrect by construction. At the very least we'd need corrected standard errors around the percentage-correct estimates. Since the inputs have a lot of variance empirically, it would be far better would to use probit or logit. (Were the PPG and PPB tiebreakers specified as binary or continuous variables in that procedure, ie "Team A has higher PPG" or "Team A's PPG - Team B's PPG" or some variation thereof?)

Finally, I should say that it's quite possible to use a function of several tiebreakers. That seems to be the philosophy behind the NAQT "D-value," and I don't see why it can't be done here. I imagine it wouldn't be hard to program such a function into the tournament statistics software, and we can discuss what the function would be.

Let me conclude by saying that IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament, you should not be satisfied with the status quo. To my mind, by rewarding teams for doing well in unimportant matches, it overvalues a lot of uninformative data.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Mon Mar 11, 2013 11:32 am
by Cheynem
But if you use head to head, does that not reward the team who did not do well in "unimportant" matches (i.e., losing to a team below them)?

Edit: However, as the World Baseball Classic demonstrates, breaking ties by PPG does lead to intense fistfights.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Mon Mar 11, 2013 11:37 am
by Tees-Exe Line
Cheynem wrote:But if you use head to head, does that not reward the team who did not do well in "unimportant" matches (i.e., losing to a team below them)?
I'm not exactly sure what you're saying, but I guess relative to other tiebreakers head-to-head DOES reward the team that did NOT do well in a less important match (which is downweighted by definition)--and that's a plus!

My point about unimportant matches is really about clubbing baby seals, though. The match in which the head-to-head winner lost is almost certainly against a reasonably good team albeit not one part of the relevant comparison for the tiebreaker, whereas the PPG differential is usually dominated by lopsided matches whose data I'm proposing is just not as important for the bilateral comparison at issue.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Mon Mar 11, 2013 11:44 am
by Matt Weiner
Marshall, speaking as a board staffer, you can't post about the tone of other people's posts or tell them to stop posting; don't do so.

Speaking as a quizbowl contributor, let me say that, like all other attempts to make up for a paucity of the actual data being analyzed by performing additional arithmetic wizardry, the infamous thread about tiebreaker correlations is not what caused or changed anybody's opinions on this issue. The reason everyone involved in good quizbowl does it this way (which is, in fact, a great reason to do it that way) is because of years and years of constant observation and discussion of the fairness of all the methods that used to be tried arriving in an evolutionary fashion at this very solid consensus. If the (completely correct, and presented over and over again to you for that reason and no other) point about arbitrary double-counting and rewarding losing to a worse team doesn't communicate this to you, just remember to look at the premise of what you are trying to evaluate: a bracket/round-robin is evaluating a team's strength against THE FIELD (the entirety of teams within the bracket), not "any one arbitrarily chosen opponent." That's why teams' overall record is the first ranking. PPG is superior to any head-to-head result or to PPB as a tiebreaker because it reflects the teams' performance on all questions played, against the entire field, and thus does not change the very frame of reference as to what is being ranked between the tie-creating and tie-breaking stat.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 11:50 am
by Cheynem
I guess I don't get the "clubbing baby seals" argument--especially when each team is playing the same schedule, which makes PPG that much better. The point of using PPG is to see how well this team is playing over a series of games, easily comparable since both teams are playing the same other teams. It factors in the scoring (the only way to determine who is playing well) from the various games--I don't think the point about "clubbing baby seals" is valid (more on that in a sec), but either way, it's fair to look at it because both teams had equal opportunity to "club seals," so to speak.

I don't think the "clubbing seals" analogy is valid for a few reasons:

1. If this tournament is like last year's, there are no baby seals. Even the teams at the bottom of the field last year pulled off some upsets and were competitive. I think Rob and D. David Seal were near the bottom last year; they aren't baby seals (they beat us), and on the right packet, putting up a lot of points on them is just as impressive as putting up a lot of points about Kurtis and Libo.

2. This isn't like a sport where when you get a big lead, you can empty your bench or just hold on to the ball. In quizbowl, you still try to get the question to the best of your ability and thus improve your PPG. The only way to "ease up" in a blowout would be to just not buzz, which is very patronizing to the other team.

I don't know if I'm going to change your mind on this and I'm obviously not super invested in a tiebreaking procedure for a side event, but I just wanted to share my thoughts.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 12:01 pm
by Tees-Exe Line
I used the clubbing seals analogy because it's relevant to the larger question of tiebreaking in most tournaments, obviously not really a good analogy for this one. Addressing your and Matt's points that the frame is to see how teams played against the field, the whole point is there's a tie so two or more teams did equally well. Now we have to decide which, in fact, did best, and it's appropriate to change that frame to one of bilateral comparison.

As for "double-counting," that just isn't what's happening, yet everyone keeps repeating it as though it decides the issue. For that reason I've concluded that the consensus is not correct and just because everyone agrees on something doesn't mean they're right. The more the people keep repeating falsehoods while simultaneously demanding unquestioning obeisance, the less I'm persuaded by their position.

I also disagree with Mike's second point. I think there is a difference between how good teams play hard matches vs. easy ones, and their performance in hard matches is what should be preferentially considered. I certainly play more attentively and aggressively when I think the opposition poses a threat.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 12:06 pm
by Matt Weiner
It is what's happening. In the scenario of "Michigan 12-1, Chicago 12-1, Illinois 10-2" you are double-counting Chicago's victory over Michigan as opposed to their loss to Illinois. Your reasoning is "we have to count Chicago's game over Michigan because we're trying to figure out whether Chicago is better than Michigan because those are the teams being compared"...i.e., your proof of why head-to-head should be used contains the assumption that a head-to-head tiebreaker will be used and that bracketed results aren't played against the field! You're assuming the conclusion and creating a tautology!

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 12:11 pm
by Skepticism and Animal Feed
If you are concerned that teams play up or down to their opponents, why not use the Litvak Slope-Intercept, a statistic that measures exactly that? True, it's an individual stat, but you can probably figure out a way to convert it into a team stat.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 12:31 pm
by fett0001
I'm just wondering if anyone still has the script to pull out that data from the old thread.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 1:05 pm
by grapesmoker
What is it with people determined to reinvent the wheel as a square and demand we all drive on catenary-shaped roads?

The double-counting is self-evident. You're giving extra weight to a game between two teams on the grounds that one team beat the other; why would you do this? Even if we were to accept the argument that you're trying to create a binary ranking between two teams, a single game would likely not be a sufficient sample size to do so. Which is why you sample over the much great number of teams that both of the tied teams play.

Matt's example is entirely correct. Taking this:
IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament
as a premise, nothing you have argued leads to this:
you should not be satisfied with the status quo.
In fact, people are taking great pains to point out that overall performance over the course of the tournament is a better indicator of which of two teams has done better at a given tournament, rather than which of two teams has done better at in a given game. You're engaged in basic question-begging here.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 1:09 pm
by Tees-Exe Line
The very fact that there's a tie says that both teams did equally well against the field! Since it has to be broken somehow and we're operating from the agenda "produce a binary ranking," then the most relevant data (and hence the data deserving a higher weight) is from the head-to-head match(es).

I'm not sure what more there is to be said on this subject since we're all just repeating ourselves.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 1:15 pm
by grapesmoker
Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
No it doesn't. I can't believe you just said this. Look at the stats to find out how well they actually did against the field.
Since it has to be broken somehow and we're operating from the agenda "produce a binary ranking," then the most relevant data (and hence the data deserving a higher weight) is from the head-to-head match(es).
Your conclusion again fails to follow from your premise. You are, again, simply begging the question by advancing the data which will be used in the head-to-head tiebreaker to the status of "most relevant" data. The relevance of the data is in fact what's in dispute here.
I'm not sure what more there is to be said on this subject since we're all just repeating ourselves.
Here's a pro-tip: sometimes the majority consensus is wrong, to be sure. But sometimes the majority consensus is not only right, but right for such a simple and straightforward reason that arguing against it devolves into circular argumentation and tautology. Which horn is represented by this conversation is left as an exercise to the reader.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 1:16 pm
by Whiter Hydra
Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
Actually, by your logic, the team that lost the head-to-head matchup did better against the field, as they had one fewer loss against common opponents.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 2:06 pm
by AKKOLADE
Communi-Bear Silo State wrote:
Tees-Exe Line wrote:The very fact that there's a tie says that both teams did equally well against the field!
Actually, by your logic, the team that lost the head-to-head matchup did better against the field, as they had one fewer loss against common opponents.
This.

Put me down as another anti-H2H tiebreak guy.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 8:21 pm
by Mewto55555
Maybe I'm thinking about it wrong, but isn't the head-to-head match already counted way more than any other match? If two teams are tied in record, then reversing the result of any other game will cause there to only be a one-game difference, whereas if you reverse the result of the H2H game, then that creates a two-game difference, so the H2H game (for purposes of determining whether A>B or B<A) is ALREADY being double-counted.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 8:23 pm
by No Rules Westbrook
Take it from someone who's more of a contrarian than even you, Marshall....you're overthinking this. H2H tiebreaking is double counting, and a really poor way to break ties.

On the other hand, even when the field plays a full round robin, I think PPG vs. PPB is a matter of personal preference about what you want to reward in this game. I'll admit that PPG is, as a strictly empirical matter, the better predictor of which team is more likely to win. But, I've always supported PPB because I feel like it puts a least a little bit of balance into this tossup-heavy game of ours - bonuses and bonus conversion are already the redheaded step-child of qb in that their importance has always been less important to team success than tossup conversion, and yet, bonus conversion is a much better measure of the breadth of a team's knowledge (which is what I'd rather reward in quizbowl, given the option).

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 8:27 pm
by prodski
Put me down in the minority that doesn't understand why h2h isn't the first tiebreaker. If beating a team head to head doesn't matter, why even keep w/l records? Just rank everyone according to PPG? According to the logic, that is most important element, determining the best overall team at a tournament. What if you had two teams tie, and you played a best of 3. Team A wins the first game by 10 team B wins the 2nd game by 100, Team A wins the 3rd by 10. Surely you would crown Team A the champion, right? Since they won 2 of 3? You wouldn't reward Team B for having more points would you? I know quizbowl is different than sports and virtually every other competition I can think of that uses h2h, but I still don't understand the argument for not using h2h. I feel it is over-rewarding good teams, basically by saying that they lost to a team they shouldn't have, and it must have been a fluke, but they played better all day, so they deserve it more. Yes, the other team lost a match they shouldn't have either, but they beat the team they ended tied with. I would also point out that beating a team twice is tough - even in the NFL or NBA - hence the reason the previous data may not "predict" the winner of 2 close teams who play again in favor of the first winner more often. When I host a tournament, like I have over the past 17 years, and I can determine my own tiebreakers, it is simple, first tiebreaker is always h2h. If you feel your team is better than the other team, you should have beaten them when you had the chance. Really simple. Two fairly even teams, I don't care what you did to the last place team, all I care about is what you did when you had the chance to play the team you ended up tying with. It is harder to see with 1-loss teams, but lets extrapolate a bit: imagine 2 teams go 22-4 in a crazy tournament, and one team beats the other 4 times. The other loses 4 matches to other teams in the tournament. Are you still going to go to points? That is crazy. Maybe the first team just matches up better against the 2nd team? They deserve 1st place in my opinion out of the 2 tied teams. Yes, the other team is better against the rest of the field, but the rest of the field is no longer part of the discussion, we are down to 2 teams. Yes, I am double-counting those 4 wins since it is the best indicator of who I think is better in a hypothetical two team tournament. Let me also say that as long as everyone knows the rules going into a tournament, whatever they may be, is fine with me - I don't lose sleep over this. I just don't think it is so obvious that ppg is a better tiebreaker than h2h. If you are trying to crown the best overall team, PPG may be the best tiebreaker. If you are trying to crown the real champion of a tournament, whether it be luck, or a fluke, or an improbable helmet catch, you go head to head and determine it on the field like every other competition I can think of besides quizbowl.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 9:10 pm
by setht
prodski wrote:Put me down in the minority that doesn't understand why h2h isn't the first tiebreaker. If beating a team head to head doesn't matter, why even keep w/l records? Just rank everyone according to PPG?
I was thinking about this earlier, and I think the question is, what is a tiebreaker supposed to do? Is it supposed to pick out the better of two teams in "a hypothetical two team tournament," as you suggest, or is it supposed to pick out the team that's done better against the full field of the tournament, or the team that's more likely to do well in a top bracket or finals match (or whatever the tiebreaker is being used to determine)? Given that finals matches should always be played, it seems to me that picking out the better of two teams in a hypothetical two team tournament pretty much never matches the actual in-tournament situation in which a tiebreaker arises: a tie in record at the end of prelims or playoffs that affects which teams have a shot at the title. In other words, at the end of a prelim round-robin, if teams A and B are tied in record, and you're trying to decide which one should advance to the top playoff bracket, I'm not sure that the question "which of these teams do I think would win a hypothetical two team tournament?" is the right one to ask. It seems like the more relevant question is something like "which of these teams do I think has the better shot at the title?" Or, if you're looking at a tiebreaker for a non-top bracket, the question could be rephrased as "which of these teams do I think will place higher in this bracket if they get in?" Once the question is phrased that way, I think it makes perfect sense to look at data beyond which team beat which other team. For instance, suppose team A beat team B by 5 points, lost to team C by 300 points, and beat all other teams by 5 points. Meanwhile, team B beat all teams except for team A by 150 points. Which of these teams would you guess has a better shot at success in the top bracket? In a finals match against (say) team C?

-Seth

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 9:22 pm
by prodski
I agree with you Seth, if you are looking to move a team up into a bracket, that makes perfect sense. I obvioulsy am looking at it for crowning a champion, or final placement at a tournament when it is over and there are no other tiebreakers or matches to be played. Your point is well taken. Thanks.

Re: Head-to-head as a tiebreaker (split from UCU announcement)

Posted: Mon Mar 11, 2013 11:23 pm
by Sen. Estes Kefauver (D-TN)
prodski wrote:I agree with you Seth, if you are looking to move a team up into a bracket, that makes perfect sense. I obvioulsy am looking at it for crowning a champion, or final placement at a tournament when it is over and there are no other tiebreakers or matches to be played. Your point is well taken. Thanks.
At a standard quizbowl tournament the final placement is already determined by your win-loss record within the playoff pool, and any leftover ties that need to be played off will be broken by a packet anyway.

Re: ANNOUNCEMENT: Urgent Call for Unity (Summer 2013)

Posted: Sat Apr 20, 2013 12:38 am
by cvdwightw
Apologies for necroing this thread. In my defense, I haven't posted anything here in over 6 months.
Tees-Exe Line wrote:As for people whose arguments actually deserve discussion, if the thread Jeff linked above is what established the consensus in favor of a PPG tiebreaker and against head-to-head, that consensus ought to be revisited. As I understand the analysis, the sample was taken from tournaments featuring repeated match-ups. Each of the candidate tiebreakers was calculated before the second (or third...) match between heretofore-tied teams, and the "prediction" about the outcome of that match was tested against the actual outcome, yielding statements like "PPG predicted the outcome of the repeat match 70% of the time." If I've misunderstood the procedure, please correct me.
This is largely correct, except that statistics were not calculated across the tournament, but rather only in games against common opponents and each other. Thus, a team did not benefit by playing a game against a poor team its opponent did not play.
That procedure has a number of flaws, but probably the most important one to address and resolve is one of interpretation about the nature of a tiebreaker. First, let me propose that the best way to break a tie is to play a complete packet between the two tied teams and award the tiebreaker to the winner of that match.
You will note that this proposition has been a commonly accepted proposition well before this thread (in fact, it was an axiom of the thread under review). I'm not going to respond to Andrew Hart's critique since I think that criticism is somewhat right (for instance, there may have been factors such as question quality and relative team strength that biased the results one way or another; for another, H2H was transformed to "H2H differential" if the teams had played multiple times such that there would be an actual H2H-based tiebreaker if H2H was split evenly between the teams). I don't feel that it's right "enough" to totally invalidate the collected data. If I understand your twist on the critique, however, you are arguing that a "regular" repeat match in a tournament between two teams of the exact same composition as the original match is subjectively different than a "tiebreaker" repeat match, which, furthermore, is subjectively different from a "finals" repeat match. I'm not sure what the subjective difference here is (we used all three types, IIRC). Question quality/difficulty? Should be the same. Distribution? I'd hope that "whether or not that third arts tossup is opera or architecture" would be unbiased as to regular vs. tiebreaker matches. Pressure? I haven't seen any objective or subjective evidence that teams play differently in tiebreaker matches compared to other matches.

Also, I don't understand what you mean by PPG being hegemonic. That shouldn't have any bearing on your (or anyone else's) ability to find a sufficiently large set of games that fit your preconceived notions of what games would be valid samples to collect to answer the question, "what statistical tiebreaker best predicts the results of a hypothetical tiebreaker?". That thread tested two common sense hypotheses - the "Andrew Hart common sense hypothesis" (PPB) and the "JR Barry common sense hypothesis" (H2H) - and found that the data didn't support JR Barry's common sense.
Second, the outcome in this case is a binary prediction: was the tiebreaker in question was "right" or "wrong" about the outcome of the repeat match? If, as I suspect, a linear probability model was used, than in some sense that is required to be incorrect by construction. At the very least we'd need corrected standard errors around the percentage-correct estimates. Since the inputs have a lot of variance empirically, it would be far better would to use probit or logit. (Were the PPG and PPB tiebreakers specified as binary or continuous variables in that procedure, ie "Team A has higher PPG" or "Team A's PPG - Team B's PPG" or some variation thereof?)
Marshall, I've already copped to using the wrong statistical test in this thread and re-analyzed the data using what I believe to be the correct test. The second test used a 2x2 matrix of outcomes (e.g. H2H better won/PPG better won, H2H better lost/PPG better won, H2H better won/PPG better lost, H2H better lost/PPG better lost). I did not make any corrections for the number of tests that were done but a one-tailed McNemar's test between PPB and H2H is still significant at the 5% level even using Bonferroni corrections.

Finally, I'll note that in both the original (wrong) and updated versions, all four statistical tiebreakers performed statistically worse than "who won the match" in predicting "who won the match" and that the only conclusion drawn from the whole thing was that H2H was significantly worse than each of the other three tiebreakers in predicting "who won the match." In fact, H2H (or H2H differential, where H2H was like 1-1) was statistically indistinguishable from a coin flip. I highly suspect that, if you were to run your own statistical analysis on a set of data you collected yourself and you felt assured that no one else could accurately point out flaws in your data collection, you would see a similar result (at least with respect to H2H). The reason for this is that H2H is predicting the result of a future match based on one game (or, best case scenario, 3) - and, at that, a game whose outcomes are extremely highly variable (there was a tournament a few years back where Chicago and Minnesota both beat each other by like 400+ points).
Finally, I should say that it's quite possible to use a function of several tiebreakers. That seems to be the philosophy behind the NAQT "D-value," and I don't see why it can't be done here. I imagine it wouldn't be hard to program such a function into the tournament statistics software, and we can discuss what the function would be.
As the person who largely combined the suggestions of Andrew Hart and others into the D-Value, I sought to accurately rank teams who had played wildly different sets of opponents on potentially different packet sets. The D-Value roughly translates into "how many points would we expect you to score against a hypothetical nationally-average team on the appropriate set of questions," which to me (and apparently NAQT) seems to be as good as any a way of ranking every college team that played SCT without making the calculations too arcane for the average quizbowler (there are some adjustments to the raw rankings to ensure that order of finish within a given tournament takes precedence over statistical measures). Of course a D-Value or equivalent future statistic can be calculated for any individual tournament. However, not knowing your experience, I am unsure whether the second sentence here is due to your inexperience TDing anything larger than ~10-15 teams. Reconstructing the results of an entire bracket, by hand, in a small time window, using scoresheets with borderline unintelligible writing is something that happens all the time in moderately-sized high school tournaments. The best statistical tiebreakers are things that can be computed relatively easily by hand if one of the thousand things that could go wrong with stats-entering does go wrong. I'm not saying it can't be done, but that it would be wholly impractical to do it.
Let me conclude by saying that IF you agree with me that the point of a tiebreaker is to choose which of two teams has done better at a given tournament, you should not be satisfied with the status quo. To my mind, by rewarding teams for doing well in unimportant matches, it overvalues a lot of uninformative data.
No one is arguing that we should be satisfied with the status quo, or that a 4.5-year-old thread is somehow the final word on tiebreakers. However, the objective analysis done in 2008 (and corrected in 2011 due to being wrong) was broadly consistent with subjective quizbowl experience that:

(1) Ties should be broken by actually playing tiebreaker games when possible
(2) Statistical tiebreakers are not a valid substitute for actually playing the game, but are used due to expediency
(3) When teams play identical schedules excepting each other and have identical win-loss records, head-to-head is about as effective at predicting the result of the rematch as a coin flip.

(and if you really want to see an argument against H2H, consider which three teams advance from a bracket in which, entering the final round, A is 7-1 with a loss to B, B and C are 7-1 and playing each other, and D is 6-2 with a win over B and losses to A and C. You'll find that the result depends entirely on whether B beat C if you use H2H - D advances or does not advance based solely on the result of a game it did not play; furthermore, if B wins, the only reason you're breaking a two-way tie instead of a three-way circle of death is because B beat A - a match that involved neither of the teams in the current two-way tie).

Let me conclude by saying that while Matt is largely correct about the insufficiency of data in most quizbowl-related statistical measures (including most of mine), I think he's completely incorrect here. We don't care about who buzzed where or whether someone vultured a neg, which is what we have to approximate using "arithmetic wizardry" in order to obtain most of the statistics he doesn't like. Here, all we care about is (1) whether a team won a game against a team with an identical won-loss record against common opponents excepting each other and (2) how many points (or ppb) the two teams scored against that slate of opponents. These data are available for literally EVERY tournament that has full statistics. So this study can be replicated. Perhaps, given the number of people who have said "there were a lot of problems with that study" (most of whom never say what those problems are; Marshall, you are an exception here, and I hope I've addressed your posted concerns), it should be replicated to account for whatever the problems are. Because the problem isn't that there isn't enough data.