2023 ICT general discussion

[Important Bird Area]

This is your general discussion thread for the 2023 NAQT ICT.

[theMoMA]

A quick word of thanks to my co-editors on the DI set: Matt Bollinger, Billy Busse, and Rob Carson. Big thanks as well to the writers and subject editors who helped the DI set come together, especially Will Alston, Alston Boyd, Mike Cheyne, Kurtis Droge, Auroni Gupta, Rahul Keyal, Joseph Krol, Ben Miller, Will Nediger, Benji Nguyen, Tejas Raje, Jonathen Settle, Seth Teitler, Jason Thompson, Danny Vopava, Andrew Wang, Matt Weiner, Josh Xiong, and Andrew Yaphe. Without the hard work of dozens of very talented people, a huge collective endeavor like the DI ICT set could never come together. Thank you.

I'm also grateful for the hard work of the DII editors, tournament logistics and technical support, and all the moderators and staffers. Finally, big thanks to all the teams who braved bad travel conditions and made it to ICT this weekend. I'm sure I speak for everyone who worked on ICT when I say that your competitive excellence and enjoyment of the tournament experience is what makes all the hard work worthwhile.

We're interested to hear as much feedback on the set (specific or general) as you'd like to provide.

[Valefor]

Following up on Andrew's post, I want to thank Matt Weiner, Auroni Gupta, and Joseph Krol for all of their work on the DII set, as well as the finals packet for IPNCT.

[Mahavishnu]

Really enjoyed the set. The eye test and some basic math makes me think that bonuses were slightly harder while powers were slightly easier to attain than last year (my feelings playing the set conformed with the second of these assertions and were contrary to the first). Thought the set did a good job of feeling like a uniquely "fun" set under the classic NAQT question constraints.

In this thread from some years ago Seth expressed opposition to posting seeds prior to the tournament. What's NAQT's position on posting them now that the event has concluded? To be clear I think this year's seeding was quite bad (regarding teams nominally in contention for top bracket; I have no opinion regarding the bottom portion of the field), and increased transparency through this process would help to avoid a similar occurrence in the future.

[theMoMA]

My focus on bonus difficulty and consistency has gotten somewhat stronger in the past couple of years, which is likely reflected in the (marginally) increased difficulty of bonuses at SCT and ICT. Essentially, I think ensuring a distinct easy, medium, and hard part at a regular or nationals-level college tournament requires cutting down on soft medium and (especially) hard parts, and thinking carefully about how to modulate clue and answer line difficulty to avoid easy outlier bonuses as much as possible.

I think reducing easy outlier bonuses, especially in close matches decided by the margin of a tossup (or possibly even bonus conversion), is one of the easiest ways to reduce detrimental variance. At ICT, for instance, where even very good teams are usually getting either 10 or 20 points on a given bonus, a bonus that misses its targets on the easy side can essentially make the associated tossup randomly have about twice as many points at stake. For this reason, I think it's better to make the bonuses "tough but fair" than to have more generous bonuses.

In golf, there's a concept of trying to limit your miss to one direction: good players try to make it so that, when they hit a bad shot, they either "miss left" or "miss right," but eliminate the miss in the other direction. That way, you know what side of the course you'll be playing, and you can navigate around the worst hazards without too much worry because you know what miss is in play. I think there's a similar concept in quizbowl writing, and it involves eliminating "missing up" while being ok with "missing down." (By "missing up," I mean missing in a generous direction, and by "missing down," I mean missing in the more-difficult direction.) In other words, you still aim for the ideal conversion target, but you modulate your clues and answers so that you expect any deviations from the ideal, especially on the hard part, are in the downward direction.

The nice thing about missing down is that, numerically speaking, it's a much less costly miss. If we're striving for something like a fieldwide 90/40/10 H/M/E conversion target, and our miss down takes the ideal 10 or 15 to a 0, there are only 10-15 percentage points of miss in play. If we miss up on a hard part, then we've brought much bigger miss into play, potentially 50 or 60 percentage points with a bad miss on the hard part alone. I think writing for the miss to be down is almost guaranteed to make the bonuses more consistent and fair (although certainly more challenging to play). Another way of looking at it is that pretty much every team at this level is going to have a bonus conversion closer to 10 than to 30, so reducing the amount of soft 30s available, and ensuring that teams really have to earn their sweeps, is going to make the tournament more consistent and more fair, because (for a hypothetical team averaging 17 PPB) the misses will mostly be -7s and +3s instead of +13s.

Ultimately, I think the bonus profile at this ICT looked how I would've wanted it to look, with Chicago's statistical excellence corresponding to its ability to get more 30s than 10s, and the rest of the top teams averaging something in the 16-19 range, and bonus conversion also mostly corresponding to order of finish. Anecdotally, I saw plenty of evidence that the hard bonus parts were gettable, even though they were hard, although I'm sure there were some parts that were not converted. (Although, as a double caveat, not every team plays every bonus, so there was almost certainly some phantom conversion out there with the unconverted parts.)

Of course, all that makes bonus calibrating sound much more scientific than it really is. All I'm really saying is that I think it makes a tournament tighter and fairer to have truly difficult hard parts, because difficult hard parts have a tighter window of conversion. Difficulty predictions are obviously difficult and based purely on conjecture, so there I'm sure we still missed up many times, and I'm sure there were a few bonuses that played very easy and a few that played very hard.

[benchapman]

I had a lot of fun playing this set (despite the unideal weather circumstances that forced me to solo for the first time ever); thanks to all the writers and editors for their work putting this together.

With regards to Andrew's discussion of bonus difficulty, I felt like most of the Es were pretty on target: they were mainly straightforward questions about a germane fact to the subject that didn't devolve into a "guess the most obvious answer" or "touch your butt" style part, while still remaining gettable for weaker teams (such as myself).

What was the music subdistribution with regards to opera? It felt like there were a lot of opera tossups, and those tossups tended to be primarily on specific hard-ish titles. Opera is somewhat of a blind spot for me so the questions may well have been reasonable; I just felt like there was more opera at this tournament than others by a fairly noticeable margin.

[Votre Kickstarter Est Nul]

I was reading but just wanted to say that I enjoyed this set alot. I told Mike already, but the baseball was great (the rest of the pop culture was good too, baseball is just the best pop culture). I also thought the history was excellent and had alot of perfect ICT answerlines (that is, not someone you'd want to hear an 8 line tossup on, but would love to hear a 5 line tossup on).

I don't know much about the production process, but in terms of subdistribution, a few people mentioned to me (which I half-noticed while reading) what felt like a dearth of east asian history. Subjectively, it also felt funny that I believe the first 7 history tossups were "this leader," but that's not really a real issue. If the former impression is correct, though, I wonder if specifically for ICT, since its a nats, there could be a more thorough packetization review. Again, since I don't know anything about the production process, I don't want to impugn anything about it here (especially since non-NAQT sets presumably do packetization differently and yet end up with weird subdistro quirks all the time!).

As is presumably expected, the readability of this set was excellent and I don't remember really at any point tripping up over any awkward phrasings, etc.

EDIT: one other funny subdistro thing: it felt amusing that there were back to back packets with TUs on pre WWII comedy films. I'd think 1 is enough, but like, fair enough, they both seemed worthy of a TU and well-executed.

[theMoMA]

I don't know much about the production process, but in terms of subdistribution, a few people mentioned to me (which I half-noticed while reading) what felt like a dearth of east asian history. Subjectively, it also felt funny that I believe the first 7 history tossups were "this leader," but that's not really a real issue. If the former impression is correct, though, I wonder if specifically for ICT, since its a nats, there could be a more thorough packetization review. Again, since I don't know anything about the production process, I don't want to impugn anything about it here (especially since non-NAQT sets presumably do packetization differently and yet end up with weird subdistro quirks all the time!).

There shouldn't be a dearth of East Asian history. It's possible that a question or two might have ended up at the end of a packet, or a bonus corresponded to a harder tossup, so that some of those questions weren't heard as much. (I didn't check for this.) But it looks to me that, in addition to reserved spaces for Japanese and Chinese history, there were also appropriate numbers of East Asian questions in the general "Asian history" category. NAQT's distribution does have fewer spots for "world history" relative to circuit quizbowl.

Seth did very thoroughly read through the DI set in a final packetizing review, but I'll push back very gently on the idea that this is an extremely important part of set editing, for a couple reasons. First, it's basically impossible to see the set along every vector and get rid of every strange emergent coincidence that someone might notice. Second, most of these convergences are just aesthetically strange and do not affect gameplay at all. (For instance, the fact that there were back-to-back packets with tossups on old comedy movies struck me as odd when reading, but of course would be very unlikely to cause any gameplay issues.) Reading through the set takes a long time, and even a thorough reader won't notice every emergent coincidence in the set.

[Votre Kickstarter Est Nul]

I don't know much about the production process, but in terms of subdistribution, a few people mentioned to me (which I half-noticed while reading) what felt like a dearth of east asian history. Subjectively, it also felt funny that I believe the first 7 history tossups were "this leader," but that's not really a real issue. If the former impression is correct, though, I wonder if specifically for ICT, since its a nats, there could be a more thorough packetization review. Again, since I don't know anything about the production process, I don't want to impugn anything about it here (especially since non-NAQT sets presumably do packetization differently and yet end up with weird subdistro quirks all the time!).

There shouldn't be a dearth of East Asian history. It's possible that a question or two might have ended up at the end of a packet, or a bonus corresponded to a harder tossup, so that some of those questions weren't heard as much. (I didn't check for this.) But it looks to me that, in addition to reserved spaces for Japanese and Chinese history, there were also appropriate numbers of East Asian questions in the general "Asian history" category. NAQT's distribution does have fewer spots for "world history" relative to circuit quizbowl.

Seth did very thoroughly read through the DI set in a final packetizing review, but I'll push back very gently on the idea that this is an extremely important part of set editing, for a couple reasons. First, it's basically impossible to see the set along every vector and get rid of every strange emergent coincidence that someone might notice. Second, most of these convergences are just aesthetically strange and do not affect gameplay at all. (For instance, the fact that there were back-to-back packets with tossups on old comedy movies struck me as odd when reading, but of course would be very unlikely to cause any gameplay issues.) Reading through the set takes a long time, and even a thorough reader won't notice every emergent coincidence in the set.

I completely agree with the latter point. I hope my tone was light enough that it suggested I was more amused than having a qualm with these minor things. My point about the film thing is moreso that I think 1 pre WWII comedy film is enough in a tournament’s film distribution, not that hearing them back to back would cause any issues (and I strongly agree with your point that if it doesn’t cause a gameplay issue, minor things like the fact that these were back to back is not an issue worth managing, if it’s even an issue in the first place). I hope pointing these out didn’t feel nit picky, since it was not my intention to do so as criticism.

As for the Asian history, fair! My impression was very light, since when I’m reading more questions just wash over me. Happy to retract that.

[Important Bird Area]

In this thread from some years ago Seth expressed opposition to posting seeds prior to the tournament. What's NAQT's position on posting them now that the event has concluded? To be clear I think this year's seeding was quite bad (regarding teams nominally in contention for top bracket; I have no opinion regarding the bottom portion of the field), and increased transparency through this process would help to avoid a similar occurrence in the future.

OK, let's talk about ICT seeding again. I'll go ahead and post the top half of the Division I seeding for the 2023 ICT. (This is not a commitment to continue posting this information in future years. Furthermore, we continue to believe that it would not be a good idea for us to post seeding for public discussion prior to the tournament itself.)

Here are the seedings for the four prelim pools of the 2023 ICT. Each group is listed in order 1/2/3/4.

Chicago A
Maryland
Columbia A
MIT

Ohio State A
WUSTL A
Harvard
North Carolina A

Virginia
Georgia Tech A
Illinois A
UC Berkeley A

Cornell A
Brown
Florida A
Chicago B

Do others agree with Tracy that this year's seeding was suboptimal? Which of these groups, if any, seems stronger or weaker than the others in ways that would have a meaningful impact on the outcome of the ICT? Are there general lessons from observing this group of teams that we could apply when ranking teams for the 2024 ICT and beyond? Commentary welcome.

[theMoMA]

We've talked about this to some degree before, but the ICT seeds have a few structural challenges, the main one being that ICT happens before Nationals and does not have the benefit of incorporating real results from a national-level tournament into the seeding.

We've talked before about how the circuit could help with ICT seeding by having an ICT-only poll as a component of the pre-nationals team poll. As far as I can tell, we've gone completely in the opposite direction, with no collegiate team polls running this year at all. I don't want to come off too harshly, but the team poll is a basic function that the circuit should be providing, and it puts whoever's seeding ICT/Nats in a tough position to not have that valuable input.

[Cheynem]

Historically, even when I was doing polls, the "pre-Nats" poll garnered the least amount of interest probably because people just assumed there wasn't much point until the nationals season was over. This also ensured that the poll would really be driven by the personal opinions of perhaps a small amount of people.

I know that NAQT solicited opinions on seeding before ICT, but I wonder if perhaps they might be better off privately contacting unaffiliated people who have run/staffed tournaments throughout the season and gotten them to weigh in like a blue ribbon committee.