ICT format and seeding discussion

Elaborate on the merits of specific tournaments or have general theoretical discussion here.
User avatar
CPiGuy
Tidus
Posts: 605
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

ICT format and seeding discussion

Post by CPiGuy » Sun Apr 07, 2019 10:15 pm

See title. The ICT format is bad. It does at least two things objectively wrong, sometimes in a manner in which they are literally the only people in good quizbowl to do them wrong. These things are not weird hypothetical degeneracies, but meaningful errors that have led to multiple teams being fucked out of a spot in the top superplayoff bracket in the last two years alone. Given the number of these issues and the directness and magnitude of some of those fuckings-over, I'm honestly baffled as to why there hasn't been greater community blowback against NAQT about this. The two issues I've heard about that need to get fixed are:

Carrying over incomparable prelim games into the playoffs (and the corollary: carrying over incomparable playoff games into the superplayoffs)
This is by far the most egregious issue with the ICT format. Carrying over incomparable prelim games into the playoffs is bad and unfair because it means that teams are punished for having a tougher prelim bracket, a factor entirely outside their control. It also magnifies any potential seeding errors (which is likely the reason that there's usually so much anger about the ICT seeds, which in my opinion are not actually egregiously bad -- sure there are one or two head-scratchers, but seeding is hard -- that's why it's so important to come up with a format that doesn't make "prelim seeding" a major factor in a team's performance!)
I don't really feel like writing a longer explanation of why this is bad, especially when Cody Voight has made several excellent posts explaining both why NAQT's system is unfair and why the better system does not actually erase consequences for losses that don't carry over (a common argument against this).
If you're 2018 OSU, 2019 Berkeley A, or 2019 Chicago A, you have been materially screwed by this.
2018 OSU: were in the same prelim bracket as Chicago B, who they lost to. Despite the fact that no other teams in their playoff bracket had to play Chicago B, they were punished for that loss by missing out on top bracket in favor of Michigan A on a statistical tiebreaker. (If nothing else, I hope this episode proves that I'm not impartial; as I'm arguing that my school ought to have missed a shot at a national title). OSU deserved the chance to play for a national title last year, and they got screwed out of it.
2019 Berkeley A: same deal but replace Chicago B with WUSTL. They actually lost their tiebreaker game with Maryland, but they should never had to play that game at all!
2019 Chicago A: took a playoff loss to Penn that should not have carried over. This deprived them of the opportunity to play Columbia in a play-in to the final. (In this case their loss was actually to a common opponent with Columbia, but it should not have carried over nonetheless; it could easily lead to a situation where they get a lower placing than they deserved.)

It's possible something like this has happened in D2 as well, or before 2018, but I haven't bothered to look. Three issues in the last two years where teams were denied a shot at a national title for reasons outside of their control is too many. Speaking of reasons out of their control:

The other major issue here is:

Literally using the FUCKING ALPHABET as a tiebreaker for an ACTUAL MEANINGFUL RANKING! (holy fuuuuuck this is so stupid it's actually funny)
(I edited this section shortly after posting bc i had bad information about who had been given the 1 card from one bracket)
https://cdn.discordapp.com/attachments/ ... nknown.png
What the fuck, NAQT? Seriously, this is really bad. Doing this for tournaments where there is no difference between first and second is fine, but at this tournament there is such a difference -- the first place team has to play only two first-place teams and three second-place teams! Because "Berkeley" (not even the name that UC Berkeley is listed as on the stats!) is ahead of "Florida" in the standings, Berkeley got the 1 card. Amherst also got the 1 card over Maryland for similar reasons, giving both those teams an objectively easier schedule. This is, to put it mildly, complete and utter bullshit, especially since NAQT actually used PPG as a tiebreaker to determine playoff-superplayoff rankings (which don't actually affect the tournament at all!), and they didn't have an entire lunch period to figure shit out during that! Is NAQT really so determined to give teams their playoff cards before lunch starts that they nontrivially compromise the tournament's integrity?

NAQT is the only major tournament host I know of that goes against the norm of "do not carry over imcomparable losses", and they are almost certainly the only tournament host that's going to give teams a materially harder schedule because they had the misfortune to represent a school whose name lies closer to Z. I would like to see NAQT, at the very least, explain why they continue to buck the norms of good tournament planning, and ideally actually change the ICT logistics for next year so teams don't get left out of top bracket -- or a final -- again because of poor alphabetical luck.

Sorry, I still can't believe that even minor determinations of seeding at a national championship tournament are determined by the alphabet. It's fucking bush league. NAQT is capable of seeding 352 teams at HSNCT without resorting to alphabetical order. I think they can handle 36.
Last edited by CPiGuy on Thu Apr 11, 2019 4:21 pm, edited 1 time in total.
Conor Thompson
Bangor HS (Maine) '16
Michigan '20

User avatar
Fuddle Duddle
Memerator
Posts: 720
Joined: Tue Aug 02, 2016 12:31 pm
Location: East Lansing, MI or Naperville, IL

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Fuddle Duddle » Mon Apr 08, 2019 1:02 am

I agree with this post. I would also add that the sheer number of tiebreakers it was necessary to play after prelims indicates that the number of prelim rounds currently on offer does not have sufficient resolving power to fairly sort teams into brackets. More transparency on a seeding method that produced prelim brackets like McGill's would also be welcome.
Jakob Myers
MSU '21, Naperville North (IL) '17
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal

User avatar
Periplus of the Erythraean Sea
Auron
Posts: 1879
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Periplus of the Erythraean Sea » Mon Apr 08, 2019 8:30 am

If NAQT wants to maintain its consistent position that all games are a measure of a team's strength, I think that isn't the most unreasonable decision they could make. Losing to a playoff team in particular should always count against you IMO - it sure does in the top bracket of ACF Nationals! The alphabet thing, however, is ridiculous.

In addition, why were the 2 and 3 seeds paired in round 1 again, despite there being a discussion about this before? I think it's reasonable to have a five game prelim schedule if it's seeded well, since in the previous seven game format you still wouldn't get a chance at top bracket if you had two losses (most of the time at least) and that's generally true here as well barring a 3-2 / 5-0 situation. But if your most important game is round one without getting a chance to warm up to the tournament, that's insane. Let's fix that.

That being said, I really don't like the ingratitude of the tone on display here. The ICT took a lot of extra work to put together this year specifically so it could have enough tiebreaker packets available - and it seems like these tiebreakers generally shook out in favor of the better teams. Extra packets take extra money - Joel made a point of saying that LetterOne sponsorship enabled them to have extra tiebreakers this year at the staff meeting. A lot of these format mistakes are pretty egregious, but we would do well to give NAQT a lot of credit for sinking extra resources into a tournament that almost certainly loses them money, in response to complaints posted on an internet forum. I also suspect such a tone is likely to have a much more positive impact on the leadership.
Last edited by Periplus of the Erythraean Sea on Mon Apr 08, 2019 8:57 am, edited 1 time in total.
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
NAQT Writer and Subject Editor

jonah
Auron
Posts: 2294
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jonah » Mon Apr 08, 2019 8:47 am

Periplus of the Erythraean Sea wrote:
Mon Apr 08, 2019 8:30 am
In addition, why were the 2 and 3 seeds paired in round 1 again, despite there being a discussion about this before? I think it's reasonable to have a five game prelim schedule if it's seeded well, since in the previous seven game format you still wouldn't get a chance at top bracket if you had two losses (most of the time at least) and that's generally true here as well barring a 3-2 / 5-0 situation. But if your most important game is round one without getting a chance to warm up to the tournament, that's insane.
I don't remember any discussion about this from last year, but I have fixed it in the template for next year's Division I schedules (if the preliminary format remains basically the same, which I don't mean to imply will necessarily be the case).

Other than that, I'm not in a position to say much about the other issues raised in this thread, but speaking for myself, I basically agree with Will. I don't remember hearing any issues about the alphabetical aspect last year, which is why—to the best of my knowledge—we didn't even contemplate changing it.
Jonah Greenthal
National Academic Quiz Tournaments

User avatar
Fuddle Duddle
Memerator
Posts: 720
Joined: Tue Aug 02, 2016 12:31 pm
Location: East Lansing, MI or Naperville, IL

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Fuddle Duddle » Mon Apr 08, 2019 8:59 am

Periplus of the Erythraean Sea wrote:
Mon Apr 08, 2019 8:30 am
That being said, I really don't like the ingratitude of the tone on display here. The ICT took a lot of extra work to put together this year specifically so it could have enough tiebreaker packets available - and it seems like these tiebreakers generally shook out in favor of the better teams. Extra packets take extra money - Joel made a point of saying that LetterOne sponsorship enabled them to have extra tiebreakers this year at the staff meeting. A lot of these format mistakes are pretty egregious, but we would do well to give NAQT a lot of credit for sinking extra resources into a tournament that almost certainly loses them money, in response to complaints posted on an internet forum. I also suspect such a tone is likely to have a much more positive impact on the leadership.
I agree that having tiebreakers when ties exist is better than not having them, and your point about tone is well taken. Credit is in fact due for the modifications NAQT made this year and the speed with which Jonah responded to the 2v3 complaint in this thread. Tone, however, shouldn't detract from the issues Conor raises, and a further question that Jonah's post brings to mind for me: why was this alphabet thing a policy in the first place?
Jakob Myers
MSU '21, Naperville North (IL) '17
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal

jonah
Auron
Posts: 2294
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jonah » Mon Apr 08, 2019 9:20 am

Fuddle Duddle wrote:
Mon Apr 08, 2019 8:59 am
why was this alphabet thing a policy in the first place?
It's a carryover from the 32-team format (still used for Division II), in which the order of certain tied teams truly doesn't matter (it only affects the order, not the identities, of their opponents), we just need some way to assign their playoff schedules. I think it just didn't occur to us that that wasn't the case for the 36-team format.

Did I miss this issue being pointed out (in a place we could reasonably be expected to see) after last year's ICT? I cannot guarantee it, but I suspect we would have addressed it if we had known about it. If we missed it, I'm sorry.
Jonah Greenthal
National Academic Quiz Tournaments

User avatar
Periplus of the Erythraean Sea
Auron
Posts: 1879
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Periplus of the Erythraean Sea » Mon Apr 08, 2019 9:32 am

I don't think the alphabet thing was noticed before - if I recall correctly it didn't really come up, since there were not any tied records going into playoffs last year.
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
NAQT Writer and Subject Editor

User avatar
Kasper Kaijanen
Lulu
Posts: 93
Joined: Mon Jun 23, 2014 7:00 pm

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Kasper Kaijanen » Mon Apr 08, 2019 9:57 am

RE: the 2v3 matchup in round 1, Caleb and I both specifically raised that issue in the ICT discussion thread last year after we had to play WUSTL round 1

viewtopic.php?f=280&t=21208
Finn Bender
Edmond Memorial '15
OU '19
Colorado '21

jonah
Auron
Posts: 2294
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jonah » Mon Apr 08, 2019 10:06 am

Kasper Kaijanen wrote:
Mon Apr 08, 2019 9:57 am
RE: the 2v3 matchup in round 1, Caleb and I both specifically raised that issue in the ICT discussion thread last year after we had to play WUSTL round 1

viewtopic.php?f=280&t=21208
Guess I blew it (didn't notice it), then. Sorry! It really has been fixed for next year, assuming the folder name 2020 ICT DI use this one because it fixes something from last year is a sufficient reminder to myself.
Jonah Greenthal
National Academic Quiz Tournaments

User avatar
jonpin
Forums Staff: Moderator
Posts: 2027
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jonpin » Mon Apr 08, 2019 11:16 am

Just wanna put it out there that if you are not a member of the board staff, you should not be spending the bulk of your post telling someone you don't like the manner in which they are complaining about a quiz bowl thing. It is right at the top of the board rules:
As a result of the above, and of the need to reserve the privilege of moderating the forums for the designated staff members, anything that looks like telling other people not to discuss quizbowl is prohibited. This includes but is not limited to: telling someone that you do not like the “tone” of their post...
---

In terms of the substance:
CPiGuy wrote:
Sun Apr 07, 2019 10:15 pm
If you're 2018 OSU, 2019 Berkeley A, or 2019 Chicago A, you have been materially screwed by this.
2018 OSU: were in the same prelim bracket as Chicago B, who they lost to. Despite the fact that no other teams in their playoff bracket had to play Chicago B, they were punished for that loss by missing out on top bracket in favor of Michigan A on a statistical tiebreaker. (If nothing else, I hope this episode proves that I'm not impartial; as I'm arguing that my school ought to have missed a shot at a national title). OSU deserved the chance to play for a national title last year, and they got screwed out of it.
2019 Berkeley A: same deal but replace Chicago B with WUSTL. They actually lost their tiebreaker game with Maryland, but they should never had to play that game at all!
2018 Ohio State was absolutely screwed by the format, but the 2019 Berkeley situation is a little more complicated, as Maryland also had a carryover loss from the morning that shouldn't have counted. In truth, the results for that round-robin were Columbia 4-1, Chicago 3-2, Maryland 3-2, Berkeley 3-2, Penn 1-4, Michigan State 1-4. So there should've been a two-leg playoff where (from what I can tell from the numbers) Chicago and Maryland would play first, and then the loser would play Berkeley.
It's possible something like this has happened in D2 as well, or before 2018, but I haven't bothered to look. Three issues in the last two years where teams were denied a shot at a national title for reasons outside of their control is too many.
It's definitely been brought up in the past, repeatedly, and even with examples, this one from 2005.
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov

jagluski
Wakka
Posts: 124
Joined: Tue Nov 11, 2003 1:59 am

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jagluski » Mon Apr 08, 2019 1:04 pm

I just want to add one thing.

I don't disagree with anyone about the alphabet thing for the playoffs round (for the superplayoffs, it doesn't matter because whether you're A1 or A2, you still play the same three teams). Honestly, this was a mistake borne out of the fact that we want to give as many teams as possible their playoff card before they go to lunch. I already have a plan to do this differently. I'm sorry for anyone this affected negatively.

That said, I want to make sure that everyone understands that we never have (and never will) break a tie that determines which playoff bracket you go to alphabetically. The only tie that was broken alphabetically would be one where in the morning bracket, two teams tie for first (or 3rd, 5th, 7th). I understand that A1 has a slightly easier schedule in the playoff round than A2 and we shouldn't break that tie alphabetically. However, if there is a tie between A2 and A3 to determine who is still alive for the championship, alphabetically has never happened.


Joel Gluskin
NAQT VP of Logistics
Joel Gluskin
WUSTL '04
NAQT Vice President for Logistics

bradleykirksey
Wakka
Posts: 158
Joined: Sat Nov 12, 2011 5:09 pm

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by bradleykirksey » Mon Apr 08, 2019 1:18 pm

Periplus of the Erythraean Sea wrote:
Mon Apr 08, 2019 8:30 am
If NAQT wants to maintain its consistent position that all games are a measure of a team's strength, I think that isn't the most unreasonable decision they could make. Losing to a playoff team in particular should always count against you IMO - it sure does in the top bracket of ACF Nationals!
OK, I don't think that's actually what's happening though (I think). I'm still not 100% sure how the 2012 DII ICT thing that happened to my team actually happened. If we play a playoff team before lunch, I agree that that should count as a playoff game. In the case of our 2012 game, we played Michigan before lunch. If Delaware and MIT and Harvard counted their Michigan game, then so should we.

But through round 14 (when tiebreakers were announced)
Harvard had 7 wins against playoff teams
NYU had 4 wins against playoff teams
UCF had 4 wins against playoff teams
Michigan State had 3 wins against playoff teams
MIT had 3 wins against playoff teams
Delaware had 3 wins against teams

I expected a one-game playoff against NYU to play Harvard, but NYU played Harvard direcly. Leaving the other 4 teams to play a 4-game playoff for third. Because NYU had a better overall record (and incidentally, an easier prelim bracket) they went to the championship while we didn't.

Now, that's because we lost to Valencia B, and I can't, with a straight face, argue that that shouldn't come back to bite us. NYU didn't lose to any bottom bracket teams.

But we had the 4 and 5 seeds, by D-Value, in our bracket, as well as us. Plus, one the second best second bracket team. If a loss to the 5-seed in prelims had kept us out of championship contention, when NYU's bracket only had one team in the top-14, that would have struck me as really really unfair.

I'm not super plugged in, so I don't know if it still works like this, or if it were ever fixed, or if it ever happened to a DI team, but...

tl;dr it seems unfair to punish one team in the playoffs for coming out of a strong bracket. And then reward another team for coming out of a strong bracket. It seems, to me, and I'm not math expert, like wins against common opponents should be the determining factor in the playoffs.
Bradley Kirksey
Mayor of quiz bowl at the University of Central Florida (2010-2015)

User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3073
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by DumbJaques » Mon Apr 08, 2019 1:47 pm

jagluski wrote:
Mon Apr 08, 2019 1:04 pm

That said, I want to make sure that everyone understands that we never have (and never will) break a tie that determines which playoff bracket you go to alphabetically. The only tie that was broken alphabetically would be one where in the morning bracket, two teams tie for first (or 3rd, 5th, 7th). I understand that A1 has a slightly easier schedule in the playoff round than A2 and we shouldn't break that tie alphabetically. However, if there is a tie between A2 and A3 to determine who is still alive for the championship, alphabetically has never happened.
To be clear, I'm pretty sure we all understand the latter case doesn't happen, so apologies if my language was unclear.

However, I really want to highlight a couple things here. First, the idea that "A1 has a slightly easier schedule in the playoff round than A2" is not remotely correct. It's MUCH easier. The playoff round essentially requires you to win 3 games (barring goofiness, etc), without which you probably cannot advance to have a shot at the title. A1 plays two 1 seeds, while A2 plays three. So, the difference is actually a really big deal.

Second and relatedly, the statement "we never have (and never will) break a tie that determines which playoff bracket you go to alphabetically" is not correct. This would be true if you meant "playoff tier" (which I definitely understand you do). But I'm pointing out this difference because actually, it literally does determine which BRACKET you go to, and the distinction really matters. You end up in the same playoff tier, but have very different brackets, and very different games to determine placement.

This is a point that we made at length in the ICT schedule thread, and I do have to believe had NAQT officials spent more time there, this point would be better understood (and nobody would presume we were misinformed about whether teams were denied tiebreakers to sort between playoff tiers, which we've never thought).
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE

jagluski
Wakka
Posts: 124
Joined: Tue Nov 11, 2003 1:59 am

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jagluski » Mon Apr 08, 2019 2:04 pm

DumbJaques wrote:
Mon Apr 08, 2019 1:47 pm
jagluski wrote:
Mon Apr 08, 2019 1:04 pm

That said, I want to make sure that everyone understands that we never have (and never will) break a tie that determines which playoff bracket you go to alphabetically. The only tie that was broken alphabetically would be one where in the morning bracket, two teams tie for first (or 3rd, 5th, 7th). I understand that A1 has a slightly easier schedule in the playoff round than A2 and we shouldn't break that tie alphabetically. However, if there is a tie between A2 and A3 to determine who is still alive for the championship, alphabetically has never happened.


Second and relatedly, the statement "we never have (and never will) break a tie that determines which playoff bracket you go to alphabetically" is not correct. This would be true if you meant "playoff tier" (which I definitely understand you do). But I'm pointing out this difference because actually, it literally does determine which BRACKET you go to, and the distinction really matters. You end up in the same playoff tier, but have very different brackets, and very different games to determine placement.
You are correct here, my mistake on verbiage.
Joel Gluskin
WUSTL '04
NAQT Vice President for Logistics

User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3073
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by DumbJaques » Mon Apr 08, 2019 2:06 pm

Here I will also note that all of these issues are exacerbated by NAQT's unique desire to count prelim games against unshared opponents; OSU actually could have won our last-tossup game against Yale in 2018, gone 4-1 in the playoffs, and *still not made the super-playoffs because we lost to Chicago B in the prelims*. Thanks to folks like Joel, we would now at least have a chance to play for that spot on tiebreakers, but they are tiebreakers that shouldn't have to be played in the first place.

This is my larger issue when people like Will defend the "count prelim games" practice, even though nobody else in quizbowl does this. Ok, sure, I understand there's a theoretical case for it (though it seems weaker than the theoretical case for the alternative, something reflected by the fact that nobody else does it. But let's say it's arguable in the abstract - fine.
In the reality of the current ICT though, the downsides to counting prelim losses exacerbate other issues: The treatment of tied A1/A2 teams, the fact that a single loss can so powerfully gate-keep which teams you'll play and how you'll have to do to advance, the fact that the top 10 teams will not all play against one another to begin with, etc.

Even in a world where NAQT fixes all those issues, there's still the fact that seeding has been a consistent issue at ICT since approximately forever. I know for a fact that many of the NAQT brass who most endorse the "count all prelim losses" position also hold the opinion that "it's not the NAQT sucks at seeding, it's just that seeding is really hard" (because I have heard those people voice both opinions). And honestly, I don't fully know how hard seeding is - it might well be harder than I make it out to be.
But here's the thing, though. If seeding is indeed inherently hard, and this is the main reason for the agony of prelim brackets year after year at this tournament, why would you favor a schedule that places extra weight on prelim seeds???

And like, hey, if Chad and R. and the other folks who love the rule want to show up here and talk about how they just deliberately chose to ngaf about properly seeding the tournament for the last decade, I'll happily concede the issue here (though, you know, others would be raised...). But as I've said, I I absolutely think those folks DO care, and do try. And whatever the reason, seeding has been a demonstrable issue at this tournament, so it puzzles me that people would endorse a format that makes seeding mistakes even more consequential than they need to be.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE

jagluski
Wakka
Posts: 124
Joined: Tue Nov 11, 2003 1:59 am

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jagluski » Mon Apr 08, 2019 2:11 pm

DumbJaques wrote:
Mon Apr 08, 2019 1:47 pm
This is a point that we made at length in the ICT schedule thread, and I do have to believe had NAQT officials spent more time there, this point would be better understood (and nobody would presume we were misinformed about whether teams were denied tiebreakers to sort between playoff tiers, which we've never thought).

All ties in the prelim brackets to decide which playoff tier a team went to were broken by gameplay only. If you can't find them online, I am happy to show you the preliminary round standings which you can crosscheck with all the Round 6 games. I understand and agree that the method to decide how to sort teams that were tied that went to the same tier was not good and it will not happen again.

I think we're on the same page, but I just want to make sure.

edit by JP to fix quote syntax
Joel Gluskin
WUSTL '04
NAQT Vice President for Logistics

User avatar
Aaron's Rod
Sec. of Cursed Images, Chicago SJW Cabal
Posts: 565
Joined: Wed Nov 27, 2013 7:29 pm

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Aaron's Rod » Mon Apr 08, 2019 2:24 pm

As a forums moderator that tries to be very responsive to community concerns, I've split this thread--posts about how NAQT responds to community concerns can go here, and the ICT format discussion can stay here.
Alex D.
ACF
http://tinyurl.com/qbmisconduct

"You operate at a shorter wavelength and higher frequency than most human beings." —Victor Prieto

User avatar
1.82
Rikku
Posts: 291
Joined: Thu Feb 05, 2015 9:35 pm
Location: Atlanta area
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by 1.82 » Mon Apr 08, 2019 2:26 pm

Without touching questions of tone and whatnot (and also without discussing carrying over losses against non-common opponents, since that has been talked to death), it seems that the issue is that Division I ICT is a 36-team tournament where teams are limited to 13 games before finals, not counting tiebreakers. I don't believe that there's a good tournament format that works with those constraints.

As it stands, a six-team round-robin is not a very robust way of selecting teams to advance, as we saw this year from the number of tiebreakers. When three teams are tied at 4-1, as is often the case, one of those teams will draw the short straw and have their fate determined by a half-packet. (Any system that involves rebracketing will potentially face this issue, but it seems anecdotally that this year's Division I ICT was particularly struck by it.) Beyond that, the current method of moving from prelims to playoffs and playoffs to superplayoffs is clearly opaque, since nobody realized the problems that would result from breaking ties by alphabetical order. Given that (unlike in nearly all tournament formats used in quizbowl) the difference between finishing first in your prelim bracket and second in your prelim bracket has a substantial effect beyond just the order of playoff matches, the importance of those five prelim games is further magnified. Moreover, I haven't been to ICT since the new format was adopted, but I would imagine that rebracketing twice poses more logistical complications than rebracketing once would.

This could be fixed in what seems to me a pretty straightforward way if ICT were longer. If teams played 15 games instead of 13, then the tournament format could be either 3×12 prelims to 6×6 playoffs (11 prelim games and 4 playoff games) or 6×6 prelims to 3×12 playoffs (5 prelim games and 10 playoff games). Furthermore, given that there would only be need to rebracket once instead of twice, one fewer tiebreaker packet would be required, since there would be only one break in the tournament to break ties instead of two, so instead of 13 regular packets and two tiebreaker packets there would be 15 regular packets and one tiebreaker packet.

I'm not sure what exactly the factors are that cause NAQT to determine the maximum possible length of ICT, but at its current level and given the current field size, devising a solution that doesn't upset competitors seems like squaring the circle.
Naveed Chowdhury
Maryland '16
Georgia Tech '17

jagluski
Wakka
Posts: 124
Joined: Tue Nov 11, 2003 1:59 am

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jagluski » Mon Apr 08, 2019 3:23 pm

1.82 wrote:
Mon Apr 08, 2019 2:26 pm
Without touching questions of tone and whatnot (and also without discussing carrying over losses against non-common opponents, since that has been talked to death), it seems that the issue is that Division I ICT is a 36-team tournament where teams are limited to 13 games before finals, not counting tiebreakers. I don't believe that there's a good tournament format that works with those constraints.

As it stands, a six-team round-robin is not a very robust way of selecting teams to advance, as we saw this year from the number of tiebreakers. When three teams are tied at 4-1, as is often the case, one of those teams will draw the short straw and have their fate determined by a half-packet. (Any system that involves rebracketing will potentially face this issue, but it seems anecdotally that this year's Division I ICT was particularly struck by it.) Beyond that, the current method of moving from prelims to playoffs and playoffs to superplayoffs is clearly opaque, since nobody realized the problems that would result from breaking ties by alphabetical order. Given that (unlike in nearly all tournament formats used in quizbowl) the difference between finishing first in your prelim bracket and second in your prelim bracket has a substantial effect beyond just the order of playoff matches, the importance of those five prelim games is further magnified. Moreover, I haven't been to ICT since the new format was adopted, but I would imagine that rebracketing twice poses more logistical complications than rebracketing once would.

This could be fixed in what seems to me a pretty straightforward way if ICT were longer. If teams played 15 games instead of 13, then the tournament format could be either 3×12 prelims to 6×6 playoffs (11 prelim games and 4 playoff games) or 6×6 prelims to 3×12 playoffs (5 prelim games and 10 playoff games). Furthermore, given that there would only be need to rebracket once instead of twice, one fewer tiebreaker packet would be required, since there would be only one break in the tournament to break ties instead of two, so instead of 13 regular packets and two tiebreaker packets there would be 15 regular packets and one tiebreaker packet.

I'm not sure what exactly the factors are that cause NAQT to determine the maximum possible length of ICT, but at its current level and given the current field size, devising a solution that doesn't upset competitors seems like squaring the circle.
Naveed, speaking for myself here, I think the two biggest factors that affect the ICT length are 1) Length of the day overall and 2) Number of Packets produced. To briefly touch on those, Length of Day: From my perspective, if we added any additional rounds, it would cause us to need to add a dinner break, which would greatly prolong the day (instead of ending at 730, we end at 930, for example). In my role as logistics director, I'm also concerned about requiring staff to work that long and staff that many rounds. In terms of number of packets produced, there's a question of whether we can produce additional packets (which I'm not qualified to answer) along with the fact that producing additional packets costs additional money (as would a dinner break).

I think your proposal is interesting, but I'm going to reserve forming an opinion until a day where I didn't have to wake up at 4am to fly home. I do appreciate the suggestion though.
Joel Gluskin
WUSTL '04
NAQT Vice President for Logistics

User avatar
Periplus of the Erythraean Sea
Auron
Posts: 1879
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Periplus of the Erythraean Sea » Mon Apr 08, 2019 3:31 pm

Personally I'd love to see a 40-team ICT with 15 rounds - 7 prelims and 8 playoffs. Presumably teams are willing to pay more for the guaranteed extra rounds. Given that each round takes about 30 minutes, particularly considering the generally excellent logistics of the ICT, you could maybe start prelims at 8:30 and playoffs at 2, and you'd finish gameplay around 6-6:30. Then, a snack and cleanup break (I think teams would be OK not eating a full meal before finals, and most staff would be able to head out) and finals could start around 7:30.

Such a schedule I think would require 19 or 20 packets to cover all scenarios. However, this year's format already required nearly that many because packet 12 had to be saved for tiebreakers before rebracketing. Thus, you need 15 main packets, two tiebreaker packets (one to break prelims ties, one for play-ins to the finals), and two finals packets, plus tiebreaker tossups. This comes out to 19 full packets and a packet of spare tossups, which I actually think is what got produced this year.

ICT is already much cheaper than HSNCT, which is good and all, but the gap could be narrowed a bit there to compensate, and there'd be four additional paying teams at such a tournament as well. A $25 fee increase, plus four more teams, would give a couple thousand dollars in additional revenue or so.
Last edited by Periplus of the Erythraean Sea on Mon Apr 08, 2019 3:39 pm, edited 1 time in total.
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
NAQT Writer and Subject Editor

User avatar
Haaaaaaaarry Whiiiiiiiiiite
Auron
Posts: 1114
Joined: Tue Dec 04, 2007 8:46 pm
Location: Fairfax, VA
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Haaaaaaaarry Whiiiiiiiiiite » Mon Apr 08, 2019 3:35 pm

One mitigating factor of going to 3x12 -> 6x6 is that one fewer tiebreaker game has to be played, which means over a round's worth of time doesn't have to be spent confirming records and playing lunchtime tiebreakers. You'd only need one additional packet's worth of questions and time, not two.
Harry White
TJHSST '09
Virginia Tech '13
Tournament Database Search by Team
Will run stats for food

User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2283
Joined: Sun Nov 15, 2009 12:57 am
Location: Richmond

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Cody » Mon Apr 08, 2019 3:37 pm

I think a 6x6 to 2/2/2 crossover format is the better format for the future given this year's format. (It is better than 3x12 because of the location of lunch / tiebreakers.)

In 2018, using superplayoffs saved 2 packets because there was no tiebreaker from the playoff to superplayoff stage. In 2019, it only saved one packet.

From a length of day perspective, I think the crossover will perform equal to the superplayoff because the afternoon games will be played straight through instead of needing a reseeding break. Because rooms typically finish ahead of schedule, the reseeding break is longer than it needs to be. If you budgeted 27 minutes per round in D1 for the afternoon instead of 30, it would work out to the same.
Cody Voight, VCU ‘14. I wrote lots of science and am an electrical engineer.
VCU Tournament Director ‘13-‘17. HSAPQ President ‘15-16.
Hero of Socialist Quizbowl Labor (NSC ‘14). “esteemed colleague” of Snap Wexley, ca. 2016. Stats Hero (Nats ‘16).
Quizbowl at VCU

User avatar
jonpin
Forums Staff: Moderator
Posts: 2027
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jonpin » Mon Apr 08, 2019 4:17 pm

I would think 6x6 -> 3x12 is much preferable to the reverse, because it will allow the rebracket to happen at lunchtime, and because it allows more TOP. TEAMS. to play directly against each other.
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov

User avatar
John Ketzkorn
Wakka
Posts: 181
Joined: Tue Oct 08, 2013 2:54 pm

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by John Ketzkorn » Mon Apr 08, 2019 5:30 pm

jonpin wrote:
Mon Apr 08, 2019 4:17 pm
I would think 6x6 -> 3x12 is much preferable to the reverse, because it will allow the rebracket to happen at lunchtime, and because it allows more TOP. TEAMS. to play directly against each other.
Considering the inherent variability with NAQT format (timed / distro by set / trash / general knowledge / shorter questions / etc), I worry about this format because it doesn't fix the tiebreaker issue in prelims. This format only fixes (most of) the problem of losses carrying over for the upper bracket. For those of us in a rough group, we still have to win our half-packet (or two) to even see those games.
Michael Etzkorn
Illinois Mathematics and Science Academy '16
UIUC '21

User avatar
CPiGuy
Tidus
Posts: 605
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by CPiGuy » Mon Apr 08, 2019 5:31 pm

A format which requires 14 rounds but only 1 tiebreaker (as opposed to 13 rounds and two tiebreakers, as the current format does), without expanding the field, is:

4x9 crossover brackets (so, each team in pool A plays each team in pool B and none from their own; likewise for pools C and D)

Split to 8/8/8/8/4, play five playoff games so that:

Top four brackets complete a single round robin, the first two games of which carry over from the prelims
Bottom bracket completes a double round robin, one game of which carries over from the prelims.

This format is kind of complicated but I think it would be preferable to the current format for the reasons that people have explained.
Conor Thompson
Bangor HS (Maine) '16
Michigan '20

User avatar
sephirothrr
Quizbowl Detective Extraordinaire
Posts: 205
Joined: Sun Mar 23, 2008 11:00 pm
Location: Louisville, KY

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by sephirothrr » Mon Apr 08, 2019 5:42 pm

CPiGuy wrote:
Mon Apr 08, 2019 5:31 pm
A format which requires 14 rounds but only 1 tiebreaker (as opposed to 13 rounds and two tiebreakers, as the current format does), without expanding the field, is:

4x9 crossover brackets (so, each team in pool A plays each team in pool B and none from their own; likewise for pools C and D)

Split to 8/8/8/8/4, play five playoff games so that:

Top four brackets complete a single round robin, the first two games of which carry over from the prelims
Bottom bracket completes a double round robin, one game of which carries over from the prelims.

This format is kind of complicated but I think it would be preferable to the current format for the reasons that people have explained.
while neat, (and I'm certainly a huge fan of paired brackets) this has the risk of falling apart due to bad seeding, which is already a problem - to wit, it's hypothetically possible that all the teams in one bracket are 0-9 and all the teams in another are 9-0. While that extreme level is unlikely, it creates situations where it removes some of a team's own agency in making top bracket.
Ramapriya Rangaraju
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote:
Fri Apr 04, 2014 1:52 pm
If you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.

User avatar
CPiGuy
Tidus
Posts: 605
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ann Arbor, MI

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by CPiGuy » Mon Apr 08, 2019 5:47 pm

sephirothrr wrote:
Mon Apr 08, 2019 5:42 pm
while neat, (and I'm certainly a huge fan of paired brackets) this has the risk of falling apart due to bad seeding, which is already a problem - to wit, it's hypothetically possible that all the teams in one bracket are 0-9 and all the teams in another are 9-0. While that extreme level is unlikely, it creates situations where it removes some of a team's own agency in making top bracket.
I'm not sure it removes any more of a team's own agency than non-paired seeding, it just makes situations where teams have been incorrectly seeded more obvious. The only factor impacting a team's ability to make top bracket is still "their performance against the same set of opponents, as compared to the other teams in their bracket". I don't think there are any situations where a team that plays well enough to make top bracket in a paired prelim situation wouldn't make top bracket in a traditional prelim situation or vice versa -- but maybe I just haven't thought enough about it.

It is considered fair enough that ACF approved it as a format for Regs 2018, for what it's worth.
Conor Thompson
Bangor HS (Maine) '16
Michigan '20

User avatar
sephirothrr
Quizbowl Detective Extraordinaire
Posts: 205
Joined: Sun Mar 23, 2008 11:00 pm
Location: Louisville, KY

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by sephirothrr » Mon Apr 08, 2019 5:55 pm

I think they're usually fine - I've used them at quite a few tournaments in the past, but the problems are in rebracketing - either you pick a fixed number of teams from each bracket to advance and run the risk of rising to a higher bracket than a team with a better record than you, or you just pick the highest records, which slightly magnifies the already extant issues of brackets being unevenly seeded.
Ramapriya Rangaraju
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote:
Fri Apr 04, 2014 1:52 pm
If you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.

touchpack
Rikku
Posts: 310
Joined: Tue Sep 20, 2011 12:25 am

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by touchpack » Mon Apr 08, 2019 8:07 pm

Periplus of the Erythraean Sea wrote:
Mon Apr 08, 2019 3:31 pm
Personally I'd love to see a 40-team ICT with 15 rounds - 7 prelims and 8 playoffs. Presumably teams are willing to pay more for the guaranteed extra rounds. Given that each round takes about 30 minutes, particularly considering the generally excellent logistics of the ICT, you could maybe start prelims at 8:30 and playoffs at 2, and you'd finish gameplay around 6-6:30. Then, a snack and cleanup break (I think teams would be OK not eating a full meal before finals, and most staff would be able to head out) and finals could start around 7:30.

Such a schedule I think would require 19 or 20 packets to cover all scenarios. However, this year's format already required nearly that many because packet 12 had to be saved for tiebreakers before rebracketing. Thus, you need 15 main packets, two tiebreaker packets (one to break prelims ties, one for play-ins to the finals), and two finals packets, plus tiebreaker tossups. This comes out to 19 full packets and a packet of spare tossups, which I actually think is what got produced this year.

ICT is already much cheaper than HSNCT, which is good and all, but the gap could be narrowed a bit there to compensate, and there'd be four additional paying teams at such a tournament as well. A $25 fee increase, plus four more teams, would give a couple thousand dollars in additional revenue or so.
The 2019 ICT had 19 packets, and a packet of spare tossups, that is correct. However, you're forgetting that one of the packets is a backup/emergency packet in case someone reads a wrong round, or a major question security issue happens, etc, so that 40-team format would require one more packet than the current format does. However, I still like the idea--assuming it's logistically and financially feasible for NAQT, I think 40 teams would be a great ICT size.
Billy Busse
Illinois '14
President, ACF
Writer/Subject Editor/Set Editor, NAQT

User avatar
Benin Rebirth Party
Yuna
Posts: 777
Joined: Sat Jun 12, 2010 8:46 pm
Location: Farhaven, Ontario

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Benin Rebirth Party » Mon Apr 08, 2019 8:36 pm

How much more money would it cost to run the proposed 40 team format?
Joe Su
Lisgar 2012, McGill 2015, McGill 20--

FINALIST -- 2017 ILQBM MEME OF THE YEAR

User avatar
jonpin
Forums Staff: Moderator
Posts: 2027
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by jonpin » Tue Apr 09, 2019 12:45 am

John Ketzkorn wrote:
Mon Apr 08, 2019 5:30 pm
jonpin wrote:
Mon Apr 08, 2019 4:17 pm
I would think 6x6 -> 3x12 is much preferable to the reverse, because it will allow the rebracket to happen at lunchtime, and because it allows more TOP. TEAMS. to play directly against each other.
Considering the inherent variability with NAQT format (timed / distro by set / trash / general knowledge / shorter questions / etc), I worry about this format because it doesn't fix the tiebreaker issue in prelims. This format only fixes (most of) the problem of losses carrying over for the upper bracket. For those of us in a rough group, we still have to win our half-packet (or two) to even see those games.
But the bubble is happening lower. A team that is seeded 9th overall is now right in the middle of the second line and just needs to beat teams that are significantly seeded lower in order to make the top bracket. In a 3x12 system, the team seeded 9th has to beat either the #3 or #4 overall seeds, or they're out. So if that team was underseeded and should rightfully be 6th or 7th, they're screwed.
On the other hand, if a host team is placed in the 5th overall seed, and that turns out to be an over-estimate of their ability, now the 6th and 7th seeds (which maybe should've been 5th and 6th overall) are facing off in the prelims, and one of them is going to be shut out of the top group.

Also, having larger playoff groups makes it more likely that the top UG teams will be in the same playoff group and thus the selection of teams for the final is more likely to be decided by direct competition, rather than "Team A finished 4th in an easy prelim group, Team B finished 5th in a hard prelim group, so Team A has the advantage in the UG final"
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov

User avatar
btressler
Tidus
Posts: 612
Joined: Fri Mar 26, 2004 7:23 pm
Location: West Chester, PA
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by btressler » Tue Apr 09, 2019 10:11 am

John Ketzkorn wrote:
Mon Apr 08, 2019 5:30 pm
Considering the inherent variability with NAQT format
Because teams don't split rounds when they play in other formats and see the result swing 330 points in their favor the second time around.

If the results are supposed to exactly follow the seedings, then why bother to play the matches?

This discussion isn't new. When Penn Bowl was 4 x 16, some team who finished 5th in their group argued they would have made the octofinals if they would have been in a different prelim pool. I felt the same way the very first time I played at a tournament with playoff brackets. My team was in a circle of death, got relegated, and won the next four matches by over 200 points each.

With any size brackets, at some point you have to draw a line, and someone is going to think they belonged on the other side of said line. It doesn't matter if it happens after round 5 or round 17.
Bill Tressler,
Dickinson ('97) Carnegie Mellon ('99) Delaware ('06)
Seen moderating at various SE Pennsylvania events.

User avatar
Fuddle Duddle
Memerator
Posts: 720
Joined: Tue Aug 02, 2016 12:31 pm
Location: East Lansing, MI or Naperville, IL

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Fuddle Duddle » Tue Apr 09, 2019 11:47 am

btressler wrote:
Tue Apr 09, 2019 10:11 am
John Ketzkorn wrote:
Mon Apr 08, 2019 5:30 pm
Considering the inherent variability with NAQT format
Because teams don't split rounds when they play in other formats and see the result swing 330 points in their favor the second time around.

If the results are supposed to exactly follow the seedings, then why bother to play the matches?
While interesting, I don't think this match result does too much to rebut Mike's point, which is that the lack of a set distribution or number of questions in any given ICT packet adds quite a bit to this already existing variability. You're correct, results shouldn't be expected to exactly follow seedings, but that's not what anyone is saying here. Results like McGill being relegated to the middle bracket are demonstrably a result of bad seeding (you might argue that they should've beaten Virginia, but a larger point is that they shouldn't have had to) and had the effect of denying what was arguably a top-6 team an opportunity to contend for the title. I agree that we shouldn't expect perfection out of the seeding process, but adopting a laxer attitude about this is not the solution.
Jakob Myers
MSU '21, Naperville North (IL) '17
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal

User avatar
Mike Bentley
Auron
Posts: 5743
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Mike Bentley » Tue Apr 09, 2019 12:00 pm

Fuddle Duddle wrote:
Tue Apr 09, 2019 11:47 am
btressler wrote:
Tue Apr 09, 2019 10:11 am
John Ketzkorn wrote:
Mon Apr 08, 2019 5:30 pm
Considering the inherent variability with NAQT format
Because teams don't split rounds when they play in other formats and see the result swing 330 points in their favor the second time around.

If the results are supposed to exactly follow the seedings, then why bother to play the matches?
While interesting, I don't think this match result does too much to rebut Mike's point, which is that the lack of a set distribution or number of questions in any given ICT packet adds quite a bit to this already existing variability. You're correct, results shouldn't be expected to exactly follow seedings, but that's not what anyone is saying here. Results like McGill being relegated to the middle bracket are demonstrably a result of bad seeding (you might argue that they should've beaten Virginia, but a larger point is that they shouldn't have had to) and had the effect of denying what was arguably a top-6 team an opportunity to contend for the title. I agree that we shouldn't expect perfection out of the seeding process, but adopting a laxer attitude about this is not the solution.
Didn't McGill lose 4 of their 8 middle bracket games? That doesn't strike me as a team that was robbed of a top bracket spot.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

User avatar
Periplus of the Erythraean Sea
Auron
Posts: 1879
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Periplus of the Erythraean Sea » Tue Apr 09, 2019 12:48 pm

I think a lot of these brackets that people are complaining about are the sort of thing you'd expect from a normal snake seeding schedule across six brackets, with some seeding that over-weighted D values.

01 02 03 04 05 06
12 11 10 09 08 07
13 14 15 16 17 18
24 23 22 21 20 19
25 26 27 28 29 30
36 35 34 33 32 31

In such a seeding scheme, it doesn't seem unlikely that Minnesota and MSU, or OSU and Columbia, would end up in the same bracket. Perhaps Minnesota and Columbia were both under-seeded, considering how they performed.

Assuming that the 2 and 3 seeds were paired in round 1 in every prelim bracket, I would surmise that the top seeds were as follows, in this order:

Seeds 1 through 6: Chicago, Yale, MSU, Northwestern, OSU, Berkeley A
Seeds 7 through 12: Florida, Columbia, Maryland, Minnesota, Chicago B, McGill
Seeds 13 through 18: Virginia, Penn, Stanford, Amherst, Berkeley B, Toronto

Based on this, I'd have to conclude that the seeding was primarily based on D-values, with a few corrections made here and there (i.e. MSU was the top D-value, but clearly was not given the one seed at this tournament). This resulted in some very bizarre one-seeds - I'm guessing Maryland was given a two-seed because of their low D-value, in part because Weijia was not on their SCT team because he had since graduated and moved to California. Indeed, it's possible Maryland - a team which is definitely among the best six in the country, based on fairly common knowledge and performance on this year's hardest tournament (PIANO) - was even given the three-seed, because their D-value was lower than Amherst! Indeed, three of the teams in the top six were underranked two-seeds. Columbia I think can be explained by the fact that they didn't really play together much this year, but the other under-seedings seem quite strange indeed.

In NAQT's defense, I don't think it's truly unreasonable that, say, Penn and Chicago B, or Virginia and McGill, got put in the same bracket. That seems like a normal outcome of a normal snake-seeding scheme - and in NAQT's further defense, seeding Virginia and McGill around 12 and 13 doesn't seem wrong either considering these teams' end performance, and in particular the fact that Virginia is a much better mACF team. What sucks is that these teams had to play their most important games of the tournament in round 1. Thankfully, the latter issue seems like it's going to get addressed, but this seeding really needs to be improved next year.

EDIT: minor changes
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
NAQT Writer and Subject Editor

User avatar
setht
Auron
Posts: 1168
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by setht » Wed Apr 10, 2019 5:43 pm

I commented on an early draft of the DI brackets. Jeff made some adjustments based on my comments, but the final brackets are somewhat different from the version adjusted for my comments so evidently Jeff made further tweaks. I'm guessing this was based on feedback from veteran circuit observers, but I don't know.

I believe the draft I commented on was based largely on the midseason poll. My commentary was mostly based on comparing the draft brackets against the D-value list (and to a much lesser extent the A-value list) and trying to identify large discrepancies.

We're definitely interested in trying to improve our seeding procedures. To that end, it would be helpful to hear if there are any consensus opinions on teams that were mis-seeded, and what data we could/should have looked at to avoid those mis-seedings*. So far in this thread, Jakob has suggested that McGill was under-seeded, and Will has suggested that Columbia, Minnesota, and Maryland were under-seeded (and perhaps as a corollary, that Northwestern and/or Michigan State was over-seeded?). Do people generally agree that some/all of these teams were mis-seeded? Do people think there were any other seeding mistakes? (I'm focusing here on DI, but if people have thoughts on DII seeding that feedback would also be welcome.)

* Or perhaps there's a consensus that there were seeding problems, but there's no consensus on what the specific problems were. That might suggest simply that "seeding is very difficult," in which case it would be better to focus efforts on identifying formats that are more robust against seeding mistakes (presumably at the cost of offering fewer [super-]playoff games between teams of similar skill).

Let's look quickly at McGill, Columbia, Minnesota, and Maryland, and ignore for a moment how they did at ICT (since that information is sadly not available when trying to seed brackets).

McGill: 11th on the midseason poll, 12th by D-value (or 13th, if we assume that Florida at full strength would have had a higher D-value), 4th by A-value. Based on those numbers it doesn't seem crazy to me that they'd be a low 2-seed (close to 12th overall) and have to face a strong 3-seed. Are there other data that should have been factored in that would suggest bumping up their seed? (Or do Jakob/other people think upon further review that McGill's seed was more or less appropriate?)

Columbia: 8th on the midseason poll, 8th by D-value (or 9th, if we assume that full-strength Yale would have outperformed Columbia at SCT, or 10th if we assume full-strength Florida also would have outperformed them), 8th by A-value. Looking at those numbers, "a high 2-seed" seems sensible. (And I gather there weren't too many full-strength Columbia results we could have consulted—but note that the D-value result was obtained by the full-strength Columbia A team.)

Minnesota: 6th on the midseason poll, 10th by D-value (or 11th if we assume full-strength Yale leapfrogs them, or 12th if we assume full-strength Florida also would have had a higher D-value), 11th by A-value. Looking at these numbers, "a mid 2-seed" seems reasonable to me.

Maryland: 4th on the midseason poll, 13th by D-value (or 14th if full-strength Florida leapfrogs Maryland), 14th by A-value. Maryland was short-handed for SCT (and I assume ACF Regionals but I haven't checked). Will suggested that the results from PIANO might have made it clear that Maryland should have been higher than a mid 2-seed. Having just looked up the stats from the JHU mirror of PIANO, I gotta say I at least would not be bold enough to say "the results from UMD A at this event convince me that I should bump them from ~10th to ~5th on the list of ICT seeds." Let me hasten to clarify that I don't mean that UMD A didn't play well! But I find it hard to be confident in making national cross-comparisons based on an open tournament played across multiple sites with strongly varying field strengths.
Seth Teitler
Formerly UC Berkeley and U. Chicago
Member and Chief Editor, NAQT
Emeritus member, ACF

User avatar
ryanrosenberg
Auron
Posts: 1299
Joined: Thu May 05, 2011 5:48 pm
Location: Chicago, Illinois

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by ryanrosenberg » Wed Apr 10, 2019 5:46 pm

I think that Northwestern was over-seeded as a 1 seed, but given that we were in the bracket with Maryland it evened out.
Ryan Rosenberg
North Carolina '16 | Ardsley '12
PACE | ACF

User avatar
Periplus of the Erythraean Sea
Auron
Posts: 1879
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Periplus of the Erythraean Sea » Wed Apr 10, 2019 6:49 pm

It's probably worth noting that the community also under-ranked Columbia on the assumption that their full A team, as it showed up to ICT, was not going to exist in any form. In any case, this same Columbia A team put up better results than OSU at both Nationals last year and nonetheless got a lower seed than OSU. While players can certainly improve a lot year over year, it seems to also take a serious presumption of decline on Columbia's part to give their A team a lower seed than a lot of teams seeded higher than them (particularly Northwestern). Making a similar guess for Maryland, though, would have been pretty tough because their top scorer wasn't on the trap last year.

In general I think a lot of this may be a bit of a garbage in, garbage out problem. The polls didn't have clear data, and predicting team strength at hard Nationals from results at lower difficulty tournaments is going to inevitably over value teams whose coverage across the distribution doesn't scale well. I'm not really sure how this can be corrected for formulaically, as opposed to just making some inferences and hoping for the best, but I'm sure other folks have some better ideas.
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
NAQT Writer and Subject Editor

User avatar
vinteuil
Auron
Posts: 1319
Joined: Sun Oct 23, 2011 12:31 pm

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by vinteuil » Wed Apr 10, 2019 8:18 pm

setht wrote:
Wed Apr 10, 2019 5:43 pm

McGill: 11th on the midseason poll, 12th by D-value (or 13th, if we assume that Florida at full strength would have had a higher D-value), 4th by A-value. Based on those numbers it doesn't seem crazy to me that they'd be a low 2-seed (close to 12th overall) and have to face a strong 3-seed. Are there other data that should have been factored in that would suggest bumping up their seed? (Or do Jakob/other people think upon further review that McGill's seed was more or less appropriate?)
I don't think the McGill seeding was that crazy, given that their handicap on NAQT's Americana-heavy distribution and house style was fairly predictable.
Jacob Reed
Chicago ~'25
Yale '17, '19
East Chapel Hill '13
"...distant bayings from...the musicological mafia"―Denis Stevens

User avatar
Fuddle Duddle
Memerator
Posts: 720
Joined: Tue Aug 02, 2016 12:31 pm
Location: East Lansing, MI or Naperville, IL

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Fuddle Duddle » Wed Apr 10, 2019 9:33 pm

Yeah I'll withdraw my complaint about McGill's seeding; with the data available where they were seeded is definitely fine.
Jakob Myers
MSU '21, Naperville North (IL) '17
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal

User avatar
setht
Auron
Posts: 1168
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by setht » Wed Apr 10, 2019 10:44 pm

ryanrosenberg wrote:
Wed Apr 10, 2019 5:46 pm
I think that Northwestern was over-seeded as a 1 seed, but given that we were in the bracket with Maryland it evened out.
With the benefit of hindsight (specifically, seeing how things played out at ICT) I would agree that Northwestern probably shouldn't have been a 1-seed, but look at what was known going into ICT: you guys put up the 4th-highest D-score at SCT (probably 5th after accounting for Yale playing SCT short-handed; maybe 6th depending on where you rate full-strength Florida). And you did that playing shorthanded! Looking at the ICT stats I can see that your 4th player for ICT didn't change your team strength all that much, and in fact didn't play a large chunk of the tournament. (I hope everything's okay there.) I also see that you were ranked a good bit lower on the midseason poll (19th), and had a lower ranking by A-value (13th). Of those results, I'd give the most weight to the D-score. I don't see any sign that Northwestern played PIANO; what other results should have factored into the seeding here? (Or do you feel that the poll and/or A-value results should have been enough to tip Northwestern over to a 2-seed? I could buy that, but I think it would be a bit of a bold move.)
Periplus of the Erythraean Sea wrote:
Wed Apr 10, 2019 6:49 pm
It's probably worth noting that the community also under-ranked Columbia on the assumption that their full A team, as it showed up to ICT, was not going to exist in any form. In any case, this same Columbia A team put up better results than OSU at both Nationals last year and nonetheless got a lower seed than OSU. While players can certainly improve a lot year over year, it seems to also take a serious presumption of decline on Columbia's part to give their A team a lower seed than a lot of teams seeded higher than them (particularly Northwestern). Making a similar guess for Maryland, though, would have been pretty tough because their top scorer wasn't on the trap last year.
In Columbia's case, we don't have to make any presumptions about year-to-year declines: we can look at what the full Columbia A team did at 2019 SCT. They came in 8th (9th after bumping up Yale; probably 10th after bumping up Florida), comfortably behind OSU. If anything, I would think that looking at Columbia's results from SCT and then giving them a 1-seed would constitute a questionable presumption (that Columbia was rusty at SCT but would work off that rust in 2 months, or whatever).
Seth Teitler
Formerly UC Berkeley and U. Chicago
Member and Chief Editor, NAQT
Emeritus member, ACF

User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Sen. Estes Kefauver (D-TN) » Thu Apr 11, 2019 12:06 am

Will's "garbage in, garbage out" analogy is calling the D-value a not great ranking for determining how teams will do at nationals, and I think that in particular something is wrong with the way the "strength of schedule" calculation works, and the fact that there isn't much of a statistical reward for winning games. For example, the raw D-value puts Johns Hopkins A above Columbia A even though the barest eyeballing of the stats results should obviously put Columbia ahead - undefeated with 2 more wins, higher ppb, higher ppg, more powers despite playing 1 fewer game. I walked out of sectionals feeling like beating every team in a strong field that ultimately produced 3 ICT top bracket teams was a statement (a "we might just make the finals of the ICT" kind of statement), and was bemused when I saw what the D-value thought of it. When I did the similar eyeballing of statistics after sectionals I found it strange that Chicago A, who using similar criteria seemed to me to have done the best job at sectionals overall (undefeated in a perfectly strong field, highest ppb, slightly fewer powers per game than MSU but my gut tells me that's a less important number when scaling up to ICT), was not the top D-value, and I'm sure the reason why is because the fact that MSU played OSU 5 out of 15 games ramped up their strength of schedule without any number to correct for the fact that both MSU and OSU sustaining losses to each other suggests perhaps some weaknesses that going undefeated doesn't. I suspect Northwestern's D-value was a byproduct of this strength-of-schedule wonkiness, because I remember going through the stats at the time and finding similar problems further down the list.

Columbia has played 3 tournaments where Rafael, Ben, and I were there, including 1 with a nationals difficulty set where we played MSU, OSU, and an open team of former national champions, and we've done OK at the other tournaments we've showed up to, so I'm unsure what people mean by a lack of statistics to judge us. I do suggest avoiding using the polls, at least as long as there's no good set of criteria people use and cases of glaring knowledge omissions - perhaps a behind the scenes expert poll would be a smarter move if one doesn't feel they have the skills to seed a national championship correctly.
Last edited by Sen. Estes Kefauver (D-TN) on Thu Apr 11, 2019 1:17 am, edited 1 time in total.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White

User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5471
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Important Bird Area » Thu Apr 11, 2019 12:26 am

Quick format announcement for the 2020 ICT:

The alphabetical tiebreaker was a mistake, and we would like to apologize to the community for accidentally introducing an element of unfairness into one of our national championships. Going forward for 2020 and beyond, if we choose to use the 36-team format from the 2018 and 2019 ICTs, we will at least use a statistical tiebreaker of points per tossup heard to break these ties (and may choose to break some or all such ties by further gameplay instead).

Notes:

-We have not finalized the format or field size for the 2020 Division I ICT, so this note is just "if it remains at 36 teams, these are the potential parameters of our tiebreaker policy."

-This does not change tiebreaker policy for 32-team formats with four prelim pools of eight teams each (such as the 2019 Division II ICT, or most Division I ICTs prior to 2018). (The alphabetical tiebreaker works fine there because the teams involved play exactly the same playoff opponents; we mistakenly copied that into the 36-team format and neglected to make the tiebreaker change required by the additional playoff pools.)
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5640
Joined: Wed Jul 26, 2006 11:46 pm
Location: Columbia, MO

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Sen. Estes Kefauver (D-TN) » Thu Apr 11, 2019 12:28 am

I'd be FASCINATED to find out what the D-value predicts the finishes at ICT should have been based on its statistics.
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White

User avatar
Periplus of the Erythraean Sea
Auron
Posts: 1879
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Periplus of the Erythraean Sea » Thu Apr 11, 2019 10:52 am

I think one possible takeaway from this discussion, as others have suggested, is that D-value (and A-value at ACF Regionals) are acceptable measures for determining qualification - since you have to use something and the statistics will still allow good teams to qualify - but that extrapolating from them to determine success at a higher difficulty set is likely to not be successful down to the level of precision required to determine an accurate seeding.

I wonder how we can consider how will teams scale. Going above, any statistic only drawn from a weaker tournament seems to be a poor measure of how teams scale. What if we looked at some other numbers? How about a PPB ratio? Let's take a look at just PPBs for some top teams that were complete at both SCT and ICT and played the D1 SCT set, listed here by order of finish at ICT:

Columbia: 21.09 at SCT, 18.79 at ICT (-2.3)
Chicago: 21.91 at SCT, 18.04 at ICT (-3.87)
OSU: 21.76 at SCT, 16.18 at ICT (-5.58)
MSU: 20.87 at SCT, 17.05 at ICT (-3.82)
Northwestern: 19.51 at SCT, 16.14 at ICT (-3.37)
McGill: 21.13 at SCT, 17.08 at ICT (-4.05)
Illinois: 19.93 at SCT, 15.82 at ICT (-4.11)

What about powers? I think raw power numbers are not a great metric because because being able to buzz early/for power consistently is far from the only game-winning skill as Andrew Hart points out here. I'm not sure any statistic can capture this besides PPG, but comparing PPG across different SCT fields is hard. Power rates are similarly hard to compare, but I think they hold a bit better. That all being said, maybe the ability to maintain your SCT power rate at ICT is also a decent measure of scaling:

Columbia: 17.7% TUs powered at SCT, 9.0% at ICT (49% decrease)
Chicago: 22.7% at SCT, 19.8% at ICT (13% decrease)
OSU: 18.2% at SCT, 9.9% at ICT (46% decrease)
MSU: 26.6% at SCT, 14.8% at ICT (44% decrease)
Northwestern: 17.8% at SCT, 9.7% at ICT (45% decrease)
McGill: 17.8% at SCT, 6.3% at ICT (65% decrease)
Illinois: 15.7% at SCT, 9.1% at ICT (42% decrease)

If I had to guess, the PPB scaling is a decent rough measure of "breadth-wise scaling" and the power rate scaling is similar for "depth-wise scaling." Taken in combination, I think they probably could be a way of extrapolating from performance at lower difficulties. The obvious problem, of course, is that we don't have many hard tournaments to measure team performance on and calculate such scale factors from. The best we'd be able to do is maybe to compare stats across several regular-difficulty tournaments versus the pre-nats open. Alternatively, we're stuck guessing scaling based on data far in the past, which is bound to run us into a Lucas critique type of problem.

That being said, I do think these scale factors might let us build a few narratives that roughly correspond with how teams did:

Columbia scales very well breadth-wise, but their depth-wise scaling is about average or slightly worse than the field. You might expect such a team to win with a lot of clutch mid to late buzzes.
Chicago scales incredibly well depth-wise, but their breadth-wise scaling is about average. You'd expect such a team to beat you by buzzing early.
OSU doesn't scale that well breadthwise compared to the other listed teams (I actually find this quite surprising, since Chris has a really, really broad knowledge base) but they start out as a very, very good team on regular difficulty as predicted by their high D-value and are able to keep on punching into the top bracket of ICT
MSU looks on paper like it scales a bit better than OSU, and indeed from D-value you might predict that they'd have done better. I think the crucial factor here at ICT was MSU's neg rate - they negged a lot more times than any other team in the top bracket except for Penn and Amherst, both of which placed lower than MSU did.
Northwestern scales decently across both spectra, but not nearly as good as either Chicago depth-wise or Columbia breadth-wise, and doesn't have as good of a baseline as OSU or MSU.
McGill scales mediocre breadth-wise and poorly depth-wise, and suffered on account of this.
Illinois is a similar story to Northwestern.

Obviously there are lots of idiosyncracies that this analysis does not touch on, but maybe it's a start.
Last edited by Periplus of the Erythraean Sea on Thu Apr 11, 2019 10:59 am, edited 1 time in total.
Will Alston
Bethesda Chevy Chase HS '12, Dartmouth '16, Columbia Business School '21
NAQT Writer and Subject Editor

User avatar
ryanrosenberg
Auron
Posts: 1299
Joined: Thu May 05, 2011 5:48 pm
Location: Chicago, Illinois

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by ryanrosenberg » Thu Apr 11, 2019 10:55 am

setht wrote:
Wed Apr 10, 2019 10:44 pm
ryanrosenberg wrote:
Wed Apr 10, 2019 5:46 pm
I think that Northwestern was over-seeded as a 1 seed, but given that we were in the bracket with Maryland it evened out.
With the benefit of hindsight (specifically, seeing how things played out at ICT) I would agree that Northwestern probably shouldn't have been a 1-seed, but look at what was known going into ICT: you guys put up the 4th-highest D-score at SCT (probably 5th after accounting for Yale playing SCT short-handed; maybe 6th depending on where you rate full-strength Florida). And you did that playing shorthanded! Looking at the ICT stats I can see that your 4th player for ICT didn't change your team strength all that much, and in fact didn't play a large chunk of the tournament. (I hope everything's okay there.) I also see that you were ranked a good bit lower on the midseason poll (19th), and had a lower ranking by A-value (13th). Of those results, I'd give the most weight to the D-score. I don't see any sign that Northwestern played PIANO; what other results should have factored into the seeding here? (Or do you feel that the poll and/or A-value results should have been enough to tip Northwestern over to a 2-seed? I could buy that, but I think it would be a bit of a bold move.)
Adam edited PIANO, so results from that wouldn't have been very helpful. The main thing in D-value overranking Northwestern is that Adam (and me, to a much lesser extent) is a very good regular-difficulty generalist who doesn't scale all that well outside of science (or history, in my case). So Northwestern will look much stronger on sets easier than Nationals. That's not something super-easy to tell from stats, but the large discrepancy between D-value ranking, midseason poll ranking, and last year's ICT finish I think should have been enough to push NU down to a 2-seed.
Ryan Rosenberg
North Carolina '16 | Ardsley '12
PACE | ACF

User avatar
vinteuil
Auron
Posts: 1319
Joined: Sun Oct 23, 2011 12:31 pm

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by vinteuil » Thu Apr 11, 2019 11:03 am

ryanrosenberg (paraphrased) wrote:
Thu Apr 11, 2019 10:55 am
D-value overranks very good regular-difficulty teams who don't scale all that well.
This is a well-known problem trying to evaluate teams from regular season stats, and I'm not sure there's a good statistical way to resolve it in the absence of detailed stats.

I don't think using the midseason poll would resolve this problem, given that its results inevitably reflect exactly the same bias!

And yet, ACF Nationals hasn't seemed to have seeding problems like ICT's. (At least as far as I remember?) Given the overlap between the two organizations, it probably wouldn't be crazy-difficult to coordinate in discussions of seeding, which would probably benefit both.
Jacob Reed
Chicago ~'25
Yale '17, '19
East Chapel Hill '13
"...distant bayings from...the musicological mafia"―Denis Stevens

User avatar
Cheynem
Sin
Posts: 6511
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Cheynem » Thu Apr 11, 2019 11:11 am

I would suspect that ACF might be slightly easier to seed since the rounds lend themselves to less variable "upsets" and the distribution is less unique (so looking at every other tournament probably gives a better idea of how teams will do). But even at ACF, there's still quirks--in 2011, the #1 and #2 finishing teams at the tournament started in the first prelim bracket; in 2012, the teams that finished #1 and #2 at ICT were placed in the first prelim bracket. Seeding is hard.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Mike Bentley
Auron
Posts: 5743
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Mike Bentley » Thu Apr 11, 2019 12:49 pm

Cheynem wrote:
Thu Apr 11, 2019 11:11 am
I would suspect that ACF might be slightly easier to seed since the rounds lend themselves to less variable "upsets" and the distribution is less unique (so looking at every other tournament probably gives a better idea of how teams will do). But even at ACF, there's still quirks--in 2011, the #1 and #2 finishing teams at the tournament started in the first prelim bracket; in 2012, the teams that finished #1 and #2 at ICT were placed in the first prelim bracket. Seeding is hard.
ACF Nationals is usually after ICT, right? If so, ACF has the big benefit of using ICT as a data point in doing seeding.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

User avatar
Mike Bentley
Auron
Posts: 5743
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: The ICT format is awful and unfair (or: why I now play for the "Ann Arbor Campus, University of Michigan")

Post by Mike Bentley » Thu Apr 11, 2019 12:53 pm

I guess I'll also mention that it's worth putting into perspective how much quizbowl already puts into fairness. I'd be curious to learn if there are other team-based competitive activities out there that don't use single-elimination playoffs, guarantee all teams pretty much the same number of games as the top teams, and lack things that teams further behind late in the competition can do to increase their odds of coming back (i.e. like fouling in basketball). This isn't to say that we shouldn't work towards being even more fair, but at some point there's probably a limit on how far one can go. In the case of ICT, I suspect that seeding is always going to be really hard to do.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

Post Reply