SCT placement policies

Elaborate on the merits of specific tournaments or have general theoretical discussion here.
Post Reply
User avatar
CPiGuy
Tidus
Posts: 649
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

SCT placement policies

Post by CPiGuy » Tue Feb 12, 2019 2:12 pm

There's been some confusion over exactly how NAQT determines placement for SCT -- whether it is based on overall record, or playoff-only record. Jeff indicated that NAQT intends to solve this issue over the summer, but in my opinion it needs to be solved now, because it materially affects the D-values of several participating teams this year.

Any policy other than "count all games that all teams in the bracket have in common, and count zero games otherwise" is unfair, stupid, and should not be used. To wit:

-- If there is a single prelim pool which plays a round robin and then splits in half for playoffs (cf. Michigan, Canada SCT sites), then all games should count in the final standings (as all teams in the same bracket have played an identical schedule).

-- If there are multiple prelim pools which play round robins, and then play only crossover games for playoffs (cf. Michigan D2 SCT, presumably several others), then prelim games against teams who end up in the same playoff bracket should count for the final standings but prelim games against teams that end up in different playoff brackets should not (if you lose to the 5th-place team and top 4 make top bracket, that loss shouldn't carry over because none of the teams in the opposite bracket had to play that team).

-- If there are multiple prelim pools who play round robins, and then play a full round robin for the playoffs (I don't know if this happened at any SCT sites but it's not uncommon in tournaments in general), then only the playoff games should be counted, because those are the only games all the teams in a playoff bracket have in common.

If NAQT intends to deviate from what I have laid out above, they are going to need to justify that decision to provide a demonstrably less fair qualification procedure for ICT.

Also, as an SCT host, I will say this -- NAQT publishes an extensive, publicly-available, set of SCT guidelines (for formats, tiebreaker procedures, and such). None of those guidelines include "which games to count", so SCT hosts were entirely in the right in using the most fair method for this. Any anger about surprise changes in placement should be directed at NAQT, not SCT hosts (unless those hosts did actually botch the procedures, but in both cases I've seen, they did not.)
Conor Thompson (he/him)
Bangor HS (Maine) '16
Michigan '20

User avatar
Benin Rebirth Party
Yuna
Posts: 789
Joined: Sat Jun 12, 2010 8:46 pm
Location: Farhaven, Ontario

Re: SCT placement policies

Post by Benin Rebirth Party » Tue Feb 12, 2019 2:50 pm

I thought NAQT makes hosts rank teams by all games played.
Jonah in the 2017 thread wrote:The hosts of this tournament did not realize that NAQT's policy is to consider all games when ranking teams.
It's a dumb policy but it's definitely a policy.
Joe Su
Lisgar 2012, McGill 2015, McGill 20--

FINALIST -- 2017 ILQBM MEME OF THE YEAR

User avatar
CPiGuy
Tidus
Posts: 649
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

Re: SCT placement policies

Post by CPiGuy » Tue Feb 12, 2019 3:45 pm

Benin Rebirth Party wrote:
Tue Feb 12, 2019 2:50 pm
I thought NAQT makes hosts rank teams by all games played.
Jonah in the 2017 thread wrote:The hosts of this tournament did not realize that NAQT's policy is to consider all games when ranking teams.
It's a dumb policy but it's definitely a policy.
Well, they neither told me that policy nor included it on any of their several publicly-available tournament guidelines webpages.

That policy is also extremely stupid and screwed at least one team out of a spot at the top bracket of ICT last year, not to mention the teams it stands to potentially screw out of qualification to ICT this year.
Conor Thompson (he/him)
Bangor HS (Maine) '16
Michigan '20

User avatar
CPiGuy
Tidus
Posts: 649
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

Re: SCT placement policies

Post by CPiGuy » Tue Feb 12, 2019 3:55 pm

To wit, NAQT's approved formats page says:
NAQT provides questions for 16 rounds of play at SCT tournaments. The tournament format at SCT must meet these criteria:

Division I and Division II teams must have separate divisions whenever four or more teams are entered in each division.
Every team must be guaranteed the opportunity to play a minimum of nine games.
NAQT’s elimination policy and tiebreaker policy apply. These requirements may be overlooked only if a host is using all 16 packets provided, and therefore no additional NAQT questions are available for use in tiebreaker play.
The elimination policy has nothing to say on counting playoff vs. prelim games.

The tiebreaker policy states that "rebracketing and final results should be decided on the basis of won-lost records". It does not make specific whether those won-lost records must be overall won-lost records, playoff-only won-lost records, or some combination thereof.

It should not be incumbent on tournament hosts to extrapolate from the ICT format, or to wade through old forum posts from 2017, to determine how NAQT wants them to run their tournament -- especially when the way NAQT wants them to run their tournament is bad and wrong!
Conor Thompson (he/him)
Bangor HS (Maine) '16
Michigan '20

User avatar
jasongg17
Wakka
Posts: 138
Joined: Sat May 17, 2014 1:52 pm

Re: SCT placement policies

Post by jasongg17 » Tue Feb 12, 2019 8:51 pm

I would second Conor's post, and add that NAQT is placing an enormous, entirely unnecessary burden on teams playing and ESPECIALLY on teams hosting SCT by not taking the very, very, simple step of explicitly telling at the very least their SCT hosts clearly and precisely how they want placement (and also, whether a fully transitive ranking absolutely must result from the games and tiebreakers) to be determined. The amount of frustration I had to watch Emmett, JJ, and the entire DII field experience at Rutgers on Saturday purely on account of NAQT's lack of clarity on this subject could have been avoided by basic due diligence on NAQT's part. When most of the DII players from Princeton, which is literally a single train stop away from the site, decide that they would rather take a likely severe DII-value hit by not contesting a tiebreaker of whose necessity nobody is certain rather than stay any longer for additional games, I think it should be obvious that the cost of this oversight is entirely unjustifiable. Imagine the burden this placed on teams like Johns Hopkins and Maryland that actually had significant trips to make.
Jason Golfinos
Trinity School '13 (inexplicably in charge, 2011-13)
Princeton '17 (inexplicably in charge, 2015-16)
Cambridge '18
HLS '22

User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3080
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: SCT placement policies

Post by DumbJaques » Mon Feb 18, 2019 5:29 pm

That policy is also extremely stupid and screwed at least one team out of a spot at the top bracket of ICT last year, not to mention the teams it stands to potentially screw out of qualification to ICT this year.
Apropos of this, does NAQT have any plans to revamp how this is going to work? Needless to say, it was incredibly frustrating to be kept out of top bracket contention by two different extremely bad and poorly-justified rules.

The first of these was the aforementioned insistence on counting prelim losses against teams that other bracket-mates did not play against. There aren't really any good theoretical reasons to do this (and in fact, there are several such reasons NOT to - because the standard being applied to teams is not actually the same). Moreover, this is in stark contrast to best practices in all other tournaments (and has been so for some time).

The second issue was that the new ICT field size led to a system where ties are broken to get into the playoff round, and to get out of the superplayoff round (and into the final), but not to get out of the playoff round and into the superplayoff round. What is the possible motivation for this? Are ties more significant at one point than the other? I understand that there seemed to be a concern about tournament time and packet availability, but it's beyond unclear to me why just randomly picking one of the cut-points to be suboptimal is the best way to go about this. (For instance, if you can't solve this problem in a better, perhaps it would be better to... not use this ICT format?). It would be great to hear NAQT address this.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE

User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5522
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: SCT placement policies

Post by Important Bird Area » Wed Feb 20, 2019 6:19 pm

Here are the ranking policies used by NAQT to compute the D-values for qualification to the 2019 ICT.

1. If the entire tournament field plays a full round-robin (formats marked "RR" for 4 through 14 teams on the approved SCT format list), then teams are ranked within each playoff bracket by overall record.

2. If the prelims are a split-field round-robin followed by bracketed playoffs that do not repeat any opponents from the preliminary pools (formats marked "SFRR" for 15 through 24 teams on the approved SCT format list), then teams are ranked within each playoff bracket by overall record.

3. If the prelims are a split-field round-robin followed by bracketed playoffs that do repeat one or more opponents from the preliminary pools, then teams are ranked within each playoff bracket by playoff record only. (Please note that no formats of this type are currently on our list of default approved formats, so SCT hosts wishing to use this sort of schedule need to contact sct@naqt.com for permission.)

4. Note that it is possible for teams to have different rank orders in the Undergraduate and Overall rankings for a given SCT (or the ICT). (This is because not all teams have access to a potential Undergraduate final, and we do not want such finals to change the Overall ranking of non-Undergraduate teams.)


Our apologies to the quizbowl community for not previously posting the complete details of our policy, for not adequately communicating our policies about both team-ranking and tiebreakers to SCT hosts, and for the miscommunications from NAQT that marred the tournament experience for some teams at this year's SCT sites.
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

User avatar
CPiGuy
Tidus
Posts: 649
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

Re: SCT placement policies

Post by CPiGuy » Wed Feb 20, 2019 6:26 pm

Thanks for the transparency!

Points 1, 3, and 4 are all well-founded in my opinion, and I at least have no issue with them.
Important Bird Area wrote:
Wed Feb 20, 2019 6:19 pm
2. If the prelims are a split-field round-robin followed by bracketed playoffs that do not repeat any opponents from the preliminary pools (formats marked "SFRR" for 15 through 24 teams on the approved SCT format list), then teams are ranked within each playoff bracket by overall record.
This is the part that's bad -- teams should be ranked within each playoff bracket by record against teams in that playoff bracket -- not playoff-only record, not overall record. Ranking teams within a playoff bracket by overall record is unfair when some of those teams played different teams in the prelims. This goes against well-established quizbowl best practices that every other tournament I've been to has used.
Conor Thompson (he/him)
Bangor HS (Maine) '16
Michigan '20

User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2315
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: SCT placement policies

Post by Cody » Wed Feb 20, 2019 6:54 pm

Important Bird Area wrote:
Wed Feb 20, 2019 6:19 pm
2. If the prelims are a split-field round-robin followed by bracketed playoffs that do not repeat any opponents from the preliminary pools (formats marked "SFRR" for 15 through 24 teams on the approved SCT format list), then teams are ranked within each playoff bracket by overall record.
This method for computing records is unfair.

Let us consider the simplest possible crossover format: a field of 16 teams is split into 2 brackets of 8 (A & B) to play a round-robin in the prelims (7 games) and the top 4 teams (A1-A4, B1-B4) & bottom 4 teams (A5-A8, B5-B8) in each bracket play a crossover (4 games). The only games that should count are the games between teams in the same playoff bracket (all games between teams in the set A1-A4 and B1-B4 or all games between teams in the set A5-A8 and B5-B8) for a total of 7 games for all teams.

By counting all games, you are necessarily imputing that the quality of teams in each bracket is consistent*. This is empirically (and theoretically) false**. Ike's post in the 2017 thread is a fine illustration of the problem. Tweak the numbers however realistically you like and you will find the same result.

I again refer to the prelims of 2012 ICT: Virginia and Harvard made the top bracket with Harvard carrying over a loss to UCSD, who went to the second bracket as a result of a 3-way tiebreaker. For Harvard, the game they played against UCSD counted the same as Chicago A's game against the 3rd seed in their bracket, either Columbia or WUSTL. UCSD finished tied for 9th with a prelim PPB of 17.26 because they were a much better team than WUSTL (tied for 12th, prelim PPB of 15.92) or Columbia (16th, prelim PPB of 15.62). It is not implausible that UCSD could've beaten Chicago A had they been placed in Maryland / Chicago A's bracket instead of Virginia / Harvard's. If the skill of the teams between brackets is so different, how then is it fair to count all games as equal?

(No slight to any of the teams mentioned here, by the way! Y'all happen to make a convenient example.)

Choosing between the two methods of counting games is not a natural constraint of quizbowl. There is a right way to count games (games against a common field) and a wrong way to count games (count all games regardless of context). The community has long recognized the correct way; NAQT should follow suit.

* Consider any team A1 - A4 vs any team B1 - B4. Consistent as used here would mean that teams A5 - A8 are comparable in skill to teams B5 - B8, i.e. the variance in skill between the two sets of teams is low enough to be acceptable for modern quizbowl standards, the same way we accept variance in subdistributions between packets, or imperfect seeding for bracketed prelims, or other natural constraints of quizbowl.

** If this were true, one implication is that PPG would be better than PPB at comparing teams in different prelim brackets. I don't see that argument gaining traction any time soon.

postscript: see this post of mine in the 2017 thread for a primer on what bracketed prelims are supposed to accomplish, which is part of the theoretical underpinnings for only counting games against a common field.

edit to add: the Harvard / Chicago A comparison is but one way to probe the fundamental unfairness of this scheme. It is unfair to all teams, including teams that go to upper brackets and those that go to lower brackets.
Cody Voight, VCU ‘14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ‘13-‘17. HSAPQ President ‘15-16.
Hero of Socialist Quizbowl Labor (NSC ‘14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ‘16).
Quizbowl at VCU

User avatar
setht
Auron
Posts: 1177
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: SCT placement policies

Post by setht » Thu Feb 21, 2019 12:21 am

Jeff posted our ranking policy for the 2019 SCT, but I wanted to clarify that we are planning on reassessing the policy this summer. If you disagree with any part of the policy, it would be helpful to do as Cody has done: post arguments and data that seem especially germane. (That is, I don't think there's much use in getting 100 people to post along the lines of "I don't like point 2 of the policy posted by Jeff." I do think it will be useful if people point out past instances where this policy [or ones very much like it] have produced results that seem sub-optimal, at SCT and ICT. Or if people have other convincing arguments beyond what Cody has written/cited, those would be useful to collect as well. And I suppose if people dislike other parts of the policy, or do like point 2 and what to argue in favor of retaining it, it would be useful to collect arguments and data in support of those positions as well.)

We are also going to reassess our ICT ranking policy, and I want to highlight a detail of the 2018 (36-team) DI format: DI teams play 5 prelim games, then 5 playoff games, then 3 super-playoff games. The 6 teams that wind up in the top super-playoff brackets will not have played against the same opponents in the playoffs. What games should/should not be used in the final rankings? In particular, do people think it is best to throw out all non-common-opponent matches and rank the top 6 super-playoff teams on the basis of their 5 games against each other? Or does the sorting of teams into playoff brackets reduce the "variance in skill between . . . two sets of teams" (as Cody puts it) sufficiently that we stand to gain by determining final rankings based on 9 games (1 prelim game + 5 playoffs + 3 super-playoffs), all played against top-playoff-bracket opponents? (E.g. do we think that ranking based on 5 games will boost the probability of circles of death that might need to be broken on half-packets?)

Also note that it is possible that two teams who play each other in the prelims can wind up playing each other again in the super-playoffs. It seems a little weird to me to say "we need to throw out all games against non-common opponents; as a result, the result of your prelim match will not count towards the final rankings, but the result of your super-playoff match will."
Seth Teitler
Formerly UC Berkeley and U. Chicago
President and Chief Editor, NAQT
Emeritus member, ACF

person361
Lulu
Posts: 35
Joined: Fri Sep 28, 2018 12:18 am

Re: SCT placement policies

Post by person361 » Thu Feb 21, 2019 12:28 am

Important Bird Area wrote:
Wed Feb 20, 2019 6:19 pm
2. If the prelims are a split-field round-robin followed by bracketed playoffs that do not repeat any opponents from the preliminary pools (formats marked "SFRR" for 15 through 24 teams on the approved SCT format list), then teams are ranked within each playoff bracket by overall record.
If the preliminary pools do not have the same number of teams, how are teams ranked by overall record? Are teams that have played fewer games given automatic wins?
Eric Lu
Montgomery Blair HS '16
MIT '20

User avatar
setht
Auron
Posts: 1177
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: SCT placement policies

Post by setht » Thu Feb 21, 2019 9:19 am

person361 wrote:
Thu Feb 21, 2019 12:28 am
Important Bird Area wrote:
Wed Feb 20, 2019 6:19 pm
2. If the prelims are a split-field round-robin followed by bracketed playoffs that do not repeat any opponents from the preliminary pools (formats marked "SFRR" for 15 through 24 teams on the approved SCT format list), then teams are ranked within each playoff bracket by overall record.
If the preliminary pools do not have the same number of teams, how are teams ranked by overall record? Are teams that have played fewer games given automatic wins?
If you're asking about formats like 15-team SFRR (8/7), then the answer is yes: everyone in the 7-team prelim pool gets one automatic prelim win.
Seth Teitler
Formerly UC Berkeley and U. Chicago
President and Chief Editor, NAQT
Emeritus member, ACF

User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2315
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: SCT placement policies

Post by Cody » Thu Feb 21, 2019 10:03 am

setht wrote:
Thu Feb 21, 2019 12:21 am
DI teams play 5 prelim games, then 5 playoff games, then 3 super-playoff games. The 6 teams that wind up in the top super-playoff brackets will not have played against the same opponents in the playoffs. What games should/should not be used in the final rankings? In particular, do people think it is best to throw out all non-common-opponent matches and rank the top 6 super-playoff teams on the basis of their 5 games against each other?
Yes. I will develop this train of thought further below.
setht wrote:
Thu Feb 21, 2019 12:21 am
Or does the sorting of teams into playoff brackets reduce the “variance in skill between . . . two sets of teams” (as Cody puts it) sufficiently that we stand to gain by determining final rankings based on 9 games (1 prelim game + 5 playoffs + 3 super-playoffs), all played against top-playoff-bracket opponents? (E.g. do we think that ranking based on 5 games will boost the probability of circles of death that might need to be broken on half-packets?)
I do not think it does (re: variance). (The probability of circles of death is guaranteed to increase, but I do not think it renders the placement policy worse than the alternative.)

I want to note that the variance / skill (team strength) argument is, in my opinion, a precondition for accepting a ranking based on overall record. If you cannot make that argument, then there is (again, in my opinion) no point in further discussion because it is clear that using overall record is fundamentally unfair compared to the alternative. I use it because the implications are quite plain and it’s easy to see that the argument is not true; the disparity is often huge, even with great seeding. (And even if it were true, this variance is easily eliminated, which is a positive.)

However, variance / skill (team strength) is not the only argument against counting games between teams not in the field of common opponents. See below.
setht wrote:
Thu Feb 21, 2019 12:21 am
Also note that it is possible that two teams who play each other in the prelims can wind up playing each other again in the super-playoffs. It seems a little weird to me to say “we need to throw out all games against non-common opponents; as a result, the result of your prelim match will not count towards the final rankings, but the result of your super-playoff match will.”
I don’t see it as weird / see it as sensible due to the following chain. The alternative would need to be that the super-playoff match against that opponent be a bye. If you played the super-playoff match and counted the prelim match then you would be counting extra games for some teams but not others and/or you are counting extra games against different opponents for some teams but not others. I think such an alternative scheme (not replaying the match in the super-playoffs) is actually perfectly fine, but it would be suboptimal for a few reasons.

First, it’s a bit of a logistical nightmare for no gain (you don’t save a round or eliminate a room) and puts a lot of pressure on the tournament staff to make sure they have identified all these matches since it would propagate through all 6 super-playoff brackets. Second, tabula rasa, it’s accepted that certain inefficiencies of these types may occur that require you to replay matches and that this is a perfectly fair compromise that can be worthwhile (for example, because teams will be more in tune with the set in the afternoon than in 5 rounds in the morning). I say tabula rasa because once the results of the game are known, I imagine some teams would strenuously disagree with that statement :)

____________________________________________________________

There is a more fundamental argument to be had, which I mentioned in my postscript previously and which I have referenced in the previous two quotes: what is the purpose of bracketed, round-robin preliminary rounds? I would like to develop this from a low-level as follows. (To be clear, I understand this is a regurgitation of known or intuited knowledge for many people and especially you, Seth.)

We accept that the simplest and fairest possible format is a single round robin of all the teams in a tournament because every team plays every other team. But, when you have more than 14 teams, this format imposes significant logistical hurdles and starts to affect the fairness of other aspects of quizbowl (consistent packet difficulty and quality being an obvious example).

The community standard of bracketed round-robin preliminary & playoff rounds (or any other extensions, including super-playoffs) are a compromise that significantly improve on other formats (single/double/x-elimination, whatever NHBB’s monstrosity is) while being much easier to execute than the other fair compromise format we have accepted (power matching, which excels at large field sizes). A properly executed bracketed format meets many of the fundamental standards of fairness embodied by a whole-field round robin: a single loss will not eliminate a team from winning the tournament (hence also the “advantaged final”), teams play as much of the field as feasible, teams play as many other teams of similar strength as feasible, etc.

A key to the bracketed format is seeding / re-seeding. You cannot take all the teams from one bracket to a given tier in the playoffs, else we would not need to use a bracketed format. So, we snake seed teams for preliminary rounds based on a value judgement of their strength to spread them out and try to ensure that the appropriate teams make each tier of the playoffs. (Imperfections in seeding are the critical weakness of a bracketed format because they block teams from making the tier they may have “deserved” to make based on their strength. It’s a small but significant reason that power matching excels for large field sizes, circa 150+ teams. In the presence of perfect seeding, bracketed formats would be almost as perfect as a whole-field round robin.) At the conclusion of preliminary rounds, teams are reseeded based on win-loss and tiebreakers and put into playoff brackets where they play other teams of similar strength.

The critical take-away from the bracketed format scheme is that the preliminary rounds are designed specifically to seed teams for the playoffs in the fairest format we can devise, accepting that some imperfections will come into play (mitigated by proper design of the playoff format). The playoffs, against teams of similar strength, are how we determine the final standings. By this understanding, there is no tension in dropping games from the preliminary rounds in determining placement because they have served their purpose in slotting teams into playoff brackets.

At some point, someone recognized that throwing out all the games from the preliminary rounds might be suboptimal. A game played between two teams in the same playoff bracket has no less legitimacy because it occurred in the preliminary rounds. (Else there would be bigger problems!) Hence: the crossover. If you carry those games over and count them in the playoffs, you do not need to replay them. As a consequence, you can take more teams into a given playoff bracket and keep more teams in contention for a given tier of placement. (This results because the number of games for a crossover bracket with an even number of teams is {#teams / 2} instead of {#teams – 1} as in a round robin.) (In the example from my previous post for 16 teams, you can keep teams in contention for places #1 – #8 and #9 – #16. If you did not carry over games and instead had everyone play a strict round robin in the playoffs you would split into places #1 – #4, #5 – #8, #9 – #12, #13 – #16.)

Using a crossover becomes more complicated for larger tournaments, which is why NAQT is using a prelims -> playoffs -> super-playoffs format for 36 teams. (I am disregarding the prelims -> playoffs tiebreaker because it is the same in both formats.) A prelims -> playoffs crossover would take 5 + 10 rounds because you would need to take the top 2 teams from every bracket in order to not eliminate teams for a single loss. (The current ICT format only takes 13 rounds due to the use of statistical tiebreakers from the playoffs -> super-playoffs stage.) And so, the preliminary rounds are designed to slot teams into a playoff bracket within a playoff flight. (Playoff flights contain two or more brackets in “parallel contention”, e.g. at ICT there are two brackets in the top playoff flight for places #1 – #12.) And the playoff rounds are designed to slot teams into a super-playoff bracket half the size (in ICT’s case) of the playoff flights. Similarly to the argument for dropping games as you transition from preliminary rounds to playoff rounds, dropping games from the playoffs is fine because the playoffs served their purpose in slotting teams into the super-playoffs as fairly as can be devised. (Except that this is not the case at ICT currently because overall record and overall statistics are used at the playoff -> super-playoff stage.)
Cody Voight, VCU ‘14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ‘13-‘17. HSAPQ President ‘15-16.
Hero of Socialist Quizbowl Labor (NSC ‘14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ‘16).
Quizbowl at VCU

User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3080
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: SCT placement policies

Post by DumbJaques » Thu Feb 21, 2019 11:36 pm

Also note that it is possible that two teams who play each other in the prelims can wind up playing each other again in the super-playoffs. It seems a little weird to me to say "we need to throw out all games against non-common opponents; as a result, the result of your prelim match will not count towards the final rankings, but the result of your super-playoff match will."
As a far less impressive supplement to Cody's discussion of this, I will add: If we are forced to make choices between two imperfect options, I'd much rather choose this one, which seems to have a meaningfully lower chance of actually happening (whereas the current system caused its problem at the very first ICT that employed it!). The by-far most likely scenario here is that the 2-seed in a bracket upsets the 1-seed, which then goes on and does 1-seed stuff in the opposite playoff bracket. If both teams made it to the super playoffs, yes, you'd need to replay a result and arbitrarily discount the prelims. But the team that won in prelims wouldn't be totally without benefits - they'd only have had to face 2 other 1-seeds to make the top bracket, whereas the team that lost in prelims would've had to face three.

This brings up what I see as another flaw in the current ICT format: Winning your prelim bracket (as opposed to finishing second in it) confers a far more significant advantage than the single game it ought to. If you finish 2nd in the prelim bracket, you must then play 3 1st-place finishers in the playoff round (while, again, the 1st-place team only has to play two). Whereas losing a single game in the prelims under the 32-team format wouldn't restrict your chances to play the rest of the top bracket, under the 36-round format, it does.

Well, ok - that's not SO bad ("win your prelim games!," we might jeer). But - to again use OSU's uniquely unfortunate 2018 experience as an example - what if you drop a random game to a lower-placing team during prelims, beat the team that's going into the playoffs with you, but finish with a slightly lower ppg than the team with the same record? We don't play that off - even though tiebreaker rounds may be occurring - but it has a major impact on what teams you'll play in the playoffs (and thus, your chance to be in contention for the top spot). Under the 32-team format in such a scenario, one team will be 1-0, the other 0-1, and both have the same shot at the same opponents. Under this format, the formally 1-0 team now gets ranked lower, which in this case is quite significant. And that issue is compounded because the loss to the non-playoff team (which no other accepted tournament format in our game would count!) continues to haunt you throughout the playoffs/superplayoffs. Again, sure, losing games should have consequences - but it's hard to see the justification for these consequences.

And honestly, I just really don't see much of a justification for the 36-team format at all. Is expanding DI ICT by 4 teams worth this? I'd say no, not without fixing the arbitrary tiebreaker procedure and addressing the prelim losses issue. [Ideally I'd love to see NAQT fix those issues and even keep expanding ICT, but until that happens, I think it's clearly a bad call to run a suboptimal format in the name of adding the 33rd- through 36th-best team in the country to go 1-12 at nationals. I'm sorry if this is disappointing to any oligarchs, but there you are.]
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE

User avatar
ryanrosenberg
Auron
Posts: 1355
Joined: Thu May 05, 2011 5:48 pm
Location: Chicago, Illinois

Re: SCT placement policies

Post by ryanrosenberg » Fri Feb 22, 2019 12:48 am

DumbJaques wrote:
Thu Feb 21, 2019 11:36 pm
I think it's clearly a bad call to run a suboptimal format in the name of adding the 33rd- through 36th-best team in the country to go 1-12 at nationals. I'm sorry if this is disappointing to any oligarchs, but there you are.
Because of how ICT bids are assigned, these four teams given bids by the field expansion are often D-value qualifiers who just missed out on the top 20; last year they were MIT A, Harvard A, NYU A, and Amherst. MIT A and Harvard A both made the second bracket, and NYU A made the top bracket (Amherst, while in the bottom bracket, still finished a respectable 6-7).
Ryan Rosenberg
North Carolina '16 | Ardsley '12
PACE | ACF

User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2315
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: SCT placement policies

Post by Cody » Fri Feb 22, 2019 2:02 am

As an addendum to Chris's supplement, regarding games having consequences, here's a section of a post of mine from the 2017 thread.
Cody wrote:
Thu Mar 02, 2017 11:19 am
Separately from the intrinsic unfairness, I don’t understand the contention that including non-common games does more to “reward” or “punish” teams than the current prelim system. Teams are “rewarded” for winning prelim games by going to a higher playoff bracket (and “punished” for losing prelim games by going to a lower playoff bracket). To again take an example from the 2012 ICT, Carleton College’s “reward” for beating the top-bracket-Brown-team (a “better” team) in the prelims would’ve been that they got to play a tiebreaker with Alabama for entrance into the 2nd bracket—except they were “punished” for losing to Arizona State (a “worse” team) and got put in the 3rd bracket. Brown’s “punishment” for losing to Carleton College was that they had to play a tiebreaker for entrance into the top bracket against a very good Penn team (and won).
In the case of OSU last year, their punishment was that they had to win out to make the top bracket (pretty big consequence, I'd say!). In the case of Chicago B, they were rewarded by going 2-3 instead of 1-4, which allowed them to make the second playoff flight without having to play a tiebreaker.

The re-seeding mechanism works as designed to enforce the consequences of games; trying to layer overall record on top of that is messed up.
Cody Voight, VCU ‘14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ‘13-‘17. HSAPQ President ‘15-16.
Hero of Socialist Quizbowl Labor (NSC ‘14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ‘16).
Quizbowl at VCU

Post Reply