What is a collegiate nationals middle part?

Old college threads.
Locked
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

What is a collegiate nationals middle part?

Post by naan/steak-holding toll »

I had a fairly short, but illuminating exchange with John Lawrence at this past weekend's ACF Nationals during the prelims in which we were both discussing how the tournament was "eta" harder than previous ACF Nationals. (Personally, I thought it was the hardest I've played/moderated). John offered the initial thought that the bonuses were not as much the issue, but that the tossups were a bit hard to penetrate. I actually thought the opposite - tossups were playing out similarly to what I'd seen at previous nationals, but middle-bracket caliber teams were struggling mightily with bonuses. What we agreed on, though, was that the middle parts were fairly inconsistent. John stipulated that this middle-part identity crisis is one of the defining features of ACF Nationals and I'm very much inclined to agree with him.

Let's look at the stats from this year's ACF Nationals. Ten teams scored more than 15 points per bonus, with an additional 11 teams in the 13-15 range. This suggests to me that most teams are really struggling with the middle parts, and that a vast majority of bonuses are getting 10d, with very occasional 20s and 30s.

As a point of contrast, I'd like to offer the ICT stats. With the notable exceptions of Chicago, a team that is decidedly stronger on mACF sets, and OSU, most teams had a higher bonus conversion on ICT - even Columbia and Penn, which both fielded stronger teams at Nationals! In a smaller field, seventeen teams scored more than 15 PPB, and few teams got fewer than 10 PPB, indicating that even the weakest teams were able to pick up middle parts decently often. I personally think this is about what nats should look like - most tournaments aim for 50% middle part conversion, and these are the kinds of PPB stats you'd expect to see with that.

Since ICT isn't clear, I'm not going to launch into too much of a comparison here. What I want to do is start a discussion. I think the community generally agrees that easy parts should be a real test of basic knowledge, and that lower and middle bracket teams shouldn't be able to fumble their way to a 10. It's also generally agreed that hard parts should be gettable, allowing for a reasonable margin of error since hard parts at the Nationals level are often a bit of a guess. This leaves middle parts fairly unresolved, as John pointed out. What should middle parts be doing at Nationals? Who is our audience for them? What makes a middle part too hard?
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What is a collegiate nationals middle part?

Post by naan/steak-holding toll »

I spoke with Jacob Reed regarding this issue, since without commenting on the easy and hard parts, I think PIANO middle parts generally did a good job of hitting a similar happy medium of difficulty to this past year's ICT. Jacob offered a vision of middle parts as [paraphrased] "something you think a decent specialist, or an elite generalist, would be likely to get." I think the emphasis here should be on the word likely - i.e. it should be more probable than not that a team with a decent specialist in a category at the Nats level will get your middle part. After all, it's still very possible that such a specialist might fumble, and teams that are weak in that category will be helpless and get a 0 or 10 (as they should).
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Sima Guang Hater
Auron
Posts: 1958
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Re: What is a collegiate nationals middle part?

Post by Sima Guang Hater »

I guess the meta-question I have is - what kind of stats distribution do you want to see at nats?

Having been privy to a lot of conversations among non-championship flight players and teams at this years nationals, there is at least a significant plurality that thinks that a finals that ends with neither team getting to 300 points and several questions going dead is sub-optimal, as are the large number of low-scoring bottom bracket games with lots of questions going dead and bonuses being zeroed.

I'm glad someone's asking the question about middle parts and who should get them, but the conversation doesn't really end there. It's also worth seriously asking: what kind of conversion you want on hard parts/how much should you have to know to get a hard part? How many tossups should be going dead?
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Smuttynose Island
Forums Staff: Moderator
Posts: 614
Joined: Wed Oct 21, 2009 9:07 pm

Re: What is a collegiate nationals middle part?

Post by Smuttynose Island »

Sima Guang Hater wrote: Fri Apr 19, 2019 1:13 am I guess the meta-question I have is - what kind of stats distribution do you want to see at nats?
Something that I have always found interesting is how different the community's answer to this question is at the high school level versus the college level. It seems as if the community believes that, relative to their respective sets, high school teams should perform significantly better at nationals than college teams.

Here are the five number summaries for PPBs from the 2016-18 ACF Nationals and PACE NSCs:
ACF Nationals
Min: 5.85
1Q: 11.33
Median: 13.5
3Q: 15.35
Max: 20.34

PACE NSC
Min: 2.08 (6.08 if we exclude this data point)
1Q: 13.79
Median: 16.09
3Q: 18.5175
Max: 24.39

Overall, it appears as if the PPB distribution at PACE NSC is shifted to the right of the ACF Nationals PPB distribution by roughly 2-3 PPB. This suspicion is borne out by looking at a couple of histograms: Image

I will be upfront, I'm not sure if this is a valid comparison. If it is, I am not entirely sure what conclusion to draw from it. At a minimum, it could suggest that there is still room to distinguish between the top teams at ACF Nationals while dialing down the bonus difficulty. As a community, we certainly believe that a more generous PPB distribution works at the High School level.

How might we achieve a more generous PPB distribution at ACF Nationals? Looking at the five number summary, the easiest way is probably to make middle parts easier. As Will mentioned, 15ppb is probably an ideal median for bonus conversion. At yet, nearly 75 percent of teams playing ACF Nationals fail to meet that benchmark. The fact that the first quartile value is 11.33ppb suggests that many teams are converting easy parts (and enough middle and hard parts to balance out missed easy parts). On the other hand, the fact that the third quartile value is 15.35 suggests that many teams are failing to convert enough middle parts (and hard parts) to smooth out the distribution. Easing up on the difficulty of middle parts should increase the PPBs of many of the upper middle bracket and lower upper bracket teams, thus smoothing out the distribution a bit.
Daniel Hothem
TJHSST '11 | UVA '15 | Oregon '??
"You are the stuff of legends" - Chris Manners
https://sites.google.com/site/academicc ... ubuva/home
User avatar
sephirothrr
Quizbowl Detective Extraordinaire
Posts: 210
Joined: Sun Mar 23, 2008 11:00 pm
Location: Louisville, KY

Re: What is a collegiate nationals middle part?

Post by sephirothrr »

Smuttynose Island wrote: Fri Apr 19, 2019 3:19 am
Something that I have always found interesting is how different the community's answer to this question is at the high school level versus the college level. It seems as if the community believes that, relative to their respective sets, high school teams should perform significantly better at nationals than college teams.
I mean this is just a function of the canon, right? No matter how you shake it, four high schoolers are more capable of covering the smaller HS canon than four college students are for the college one. I wouldn't necessarily say that's indicative of anything.
Ramapriya
Kentucky Quizbowl Alliance
University of Louisville
duPont Manual High School
Birdofredum Sawin wrote: Fri Apr 04, 2014 1:52 pmIf you don't want to be regarded as a "raving lunatic," it might be advisable to rave less, or at least to do so in a less loony manner.
User avatar
naan/steak-holding toll
Auron
Posts: 2515
Joined: Mon Feb 28, 2011 11:53 pm
Location: New York, NY

Re: What is a collegiate nationals middle part?

Post by naan/steak-holding toll »

Rama's very right that it's super hard, even for a team of four elite players, to cover the entire college canon. Based on what I've heard in some Discord and private conversations, it seems the general consensus is that we could stand to have PPBs be one to two points higher - I haven't really seen anyone argue for PACE NSC type stats.

I've never edited ACF Nationals, so I'm wondering what the ACF Nationals thought process usually is for middle parts, if any editors care to speak to that. For the most part, I've generally used a standard similar to the one discussed by Jacob Reed above (perhaps a bit less forgiving) when writing middle parts for my ICT bonuses.
Will Alston
Dartmouth College '16
Columbia Business School '21
User avatar
Stained Diviner
Auron
Posts: 5085
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: What is a collegiate nationals middle part?

Post by Stained Diviner »

sephirothrr wrote: Fri Apr 19, 2019 10:32 am
Smuttynose Island wrote: Fri Apr 19, 2019 3:19 am
Something that I have always found interesting is how different the community's answer to this question is at the high school level versus the college level. It seems as if the community believes that, relative to their respective sets, high school teams should perform significantly better at nationals than college teams.
I mean this is just a function of the canon, right? No matter how you shake it, four high schoolers are more capable of covering the smaller HS canon than four college students are for the college one. I wouldn't necessarily say that's indicative of anything.
It's a function of decisions by the writers and editors, not the canon. National middle parts are going to be things that are at the edge of the canon--answers that are used rarely in regular sets or things that are usually clues in regular sets but are now answers in nationals sets. Writers and editors can decide how far out on that edge to go. If the writers and editors of nationals at either level made an effort to move conversion up or down, then they could do so. I respect the fact that editors do not know exactly how conversion will play out, but there is no reason to think that there is some iron rule that makes the third quartile of ACF Nationals PPB close to 15 and the median of NSC PPB close to 16.

There may be legitimate reasons for the numbers to be different. It is possible that the college editors are more focused on challenging the best teams and the high school editors are more focused on encouraging the median teams, and it is possible that those different focuses are both good for the game. These issues are worthy of discussion, and things can be changed if they should be changed.
David Reinstein
Head Writer and Editor for Scobol Solo, Masonics, and IESA; TD for Scobol Solo and Reinstein Varsity; IHSSBCA Board Member; IHSSBCA Chair (2004-2014); PACE President (2016-2018)
User avatar
Smuttynose Island
Forums Staff: Moderator
Posts: 614
Joined: Wed Oct 21, 2009 9:07 pm

Re: What is a collegiate nationals middle part?

Post by Smuttynose Island »

Deviant Insider wrote: Fri Apr 19, 2019 11:32 am
sephirothrr wrote: Fri Apr 19, 2019 10:32 am
Smuttynose Island wrote: Fri Apr 19, 2019 3:19 am
Something that I have always found interesting is how different the community's answer to this question is at the high school level versus the college level. It seems as if the community believes that, relative to their respective sets, high school teams should perform significantly better at nationals than college teams.
I mean this is just a function of the canon, right? No matter how you shake it, four high schoolers are more capable of covering the smaller HS canon than four college students are for the college one. I wouldn't necessarily say that's indicative of anything.
It's a function of decisions by the writers and editors, not the canon. National middle parts are going to be things that are at the edge of the canon--answers that are used rarely in regular sets or things that are usually clues in regular sets but are now answers in nationals sets. Writers and editors can decide how far out on that edge to go. If the writers and editors of nationals at either level made an effort to move conversion up or down, then they could do so. I respect the fact that editors do not know exactly how conversion will play out, but there is no reason to think that there is some iron rule that makes the third quartile of ACF Nationals PPB close to 15 and the median of NSC PPB close to 16.
This is a good post. The Nats PPB distribution is not some inevitable conclusion of an expanded Nationals canon. Rather, it is at least partially a result of how writers and editors treat that expanded canon. If that isn't believable, here is the five number summary for the 2016-2018 ICTs:

Min: 5
1Q: 12.085
Median: 14.73
3Q: 16.9325
Max: 20.34

I also want to be clear. I am not advocating for ACF Nationals to have the same PPB distribution as the NSC. For that to occur, you would probably have to run ACF Nationals on ACF Regionals questions. I merely brought up the NSC as a potentially interesting comparison that demonstrates the belief that a legitimate national championship can be run on a much more generous PPB distribution.

EDIT: Added five number summary.
Daniel Hothem
TJHSST '11 | UVA '15 | Oregon '??
"You are the stuff of legends" - Chris Manners
https://sites.google.com/site/academicc ... ubuva/home
Locked