HSNCT Then and Now

Tell your tales of bygone days and rank historical things here.
Post Reply
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

HSNCT Then and Now

Post by cvdwightw »

Inspired by Jerry's recent posts about how the 2003 ICT field was so weak compared to modern quizbowl, I looked at the HSNCT stats for the first two years of power-matching (2002 and 2003) and the most recent two years (2010 and 2011).

I found some quite interesting results.

First, bonus conversion:

The chart attached below roughly shows the distribution of bonus conversions in each year as % of the field converting at least (5, 7.5, 10, etc., up to 20) PPB. What you can see is that the 2002 and 2003 bonuses appeared to be much easier for the middle of the field. This muddle was especially noticeable in 2002 (e.g., the four teams with a bonus conversion between 17 and 17.25 finished 7-3, 7-3, 5-5, and 4-6; the team with the second-highest bonus conversion missed the playoffs entirely).
HSNCT_BC.JPG
(17.19 KiB) Downloaded 189 times
One-way ANOVA showed a significant main effect of YEAR on bonus conversion. Post-Hoc Tukey HSD test indicated that the 2002 BCs were significantly higher than those of 2010 and 2011 (each p<0.01) and that the 2003 BCs were significantly higher than those of 2010 (p<0.01).

I also looked at measures of tossup performance: in particular, powers per 20 tossups heard. The chart attached below roughly shows the distribution of powers per 20 tossups heard in each year as % of the field converting at least (0, 0.5, 1, etc., up to 3) powers per 20 TUH. This graph is pretty surprising: the 2003 and 2010 graphs are pretty close, with the 2011 power rate being much higher and the 2002 rate being lower (only 1 team in the whole field powered more than 2 tossups per 20 heard). Perhaps the most striking difference: over half the 2011 HSNCT field averaged at least 1 power per 20 TUH; less than a quarter of the 2002 field did.
HSNCT_PowP20H.JPG
(18.3 KiB) Downloaded 156 times
One-way ANOVA showed a significant main effect of YEAR on powers per 20 TUH. Post-Hoc Tukey HSD test indicated that the 2011 power rate was much higher than in 2002 and 2003 (each p<0.01) and 2010 (p<0.05).

Yes, I know there are some procedural anomalies, like ANOVA being used on basically a population of HSNCT teams when it should be reserved for samples; also, that the 2010 and 2011 data contains playoff matches (which, if anything, should actually depress the power statistics for the top teams) while the 2002 and 2003 data does not. You can also argue that I should be using something like APPK to make my point, but I think this is a fairly comprehensive picture of what it was like for the middle 50% of HSNCT teams 10 years ago and today.

The standard narrative is that high school questions have steadily been getting easier at every level, while the top high school players at every level have been getting better as well. Instead, this data shows that, contrary to the narrative, HSNCT bonuses are actually a lot harder to convert than they were 10 years ago, and HSNCT tossups were not significantly easier to power in 2010 compared to 10 years ago.

One possible narrative is that the much smaller field kept weaker HSNCT qualifiers from attending. This is empirically not the case - the 2002 and 2003 HSNCTs featured all sorts of second/third/etc. teams that would not have qualified under today's qualification policies.

If I have the time and willpower, I'll go back through and look at the missing years (2004-2009) to see if there is any sort of trend in HSNCT performance over time. I nevertheless think that this is an interesting way to look back on the evolution of high school quizbowl.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry
evilmonkey
Yuna
Posts: 964
Joined: Mon Sep 25, 2006 11:23 am
Location: Durham, NC

Re: HSNCT Then and Now

Post by evilmonkey »

Dwight - I love the work. Did you run any tests on negs? I ask because among the many possible reasons for increased power rate, a decreased neg rate is one of the few that can be meaningfully tested.
Bryce Durgin
Culver Academies '07
University of Notre Dame '11
Texas A&M '15
User avatar
btressler
Tidus
Posts: 612
Joined: Fri Mar 26, 2004 7:23 pm
Location: West Chester, PA
Contact:

Re: HSNCT Then and Now

Post by btressler »

I believe that the 2011 data was on purpose -- the set's difficulty was brought down a notch.
Bill Tressler,
Dickinson ('97) Carnegie Mellon ('99) Delaware ('06)
Seen moderating at various SE Pennsylvania events.
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: HSNCT Then and Now

Post by cvdwightw »

evilmonkey wrote:Dwight - I love the work. Did you run any tests on negs? I ask because among the many possible reasons for increased power rate, a decreased neg rate is one of the few that can be meaningfully tested.
I did not run any tests on neg rate but I do not understand why a decreased neg rate would positively correlate with an increased power rate - if anything, I would expect power rates and neg rates to go up simultaneously.

Also, here's this cool area chart I made of HSNCT bonus conversions across the years (excluding 1999, for which no data appears to exist):
HSNCT_BC_AreaChart.JPG
HSNCT Bonus Conversions Through the Years
(42.44 KiB) Downloaded 211 times
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry
User avatar
Maxwell Sniffingwell
Auron
Posts: 2164
Joined: Sun Feb 12, 2006 3:22 pm
Location: Des Moines, IA

Re: HSNCT Then and Now

Post by Maxwell Sniffingwell »

Is that little green notch up top TJ 2005?
Greg Peterson

Northwestern University '18
Lawrence University '11
Maine South HS '07

"a decent player" - Mike Cheyne
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: HSNCT Then and Now

Post by cvdwightw »

cornfused wrote:Is that little green notch up top TJ 2005?
Yep. The NAQT website's stats for that year include points but not tossups heard/answered during the playoff rounds, so the listed conversion stats are all wrong for playoff teams, but I calculated TJ's prelim PPB to be 23.66. The only other team to get over 22.5 PPB was State College last year.
Dwight Wynne
socalquizbowl.org
UC Irvine 2008-2013; UCLA 2004-2007; Capistrano Valley High School 2000-2003

"It's a competition, but it's not a sport. On a scale, if football is a 10, then rowing would be a two. One would be Quiz Bowl." --Matt Birk on rowing, SI On Campus, 10/21/03

"If you were my teammate, I would have tossed your ass out the door so fast you'd be emitting Cerenkov radiation, but I'm not classy like Dwight." --Jerry
jrbarry
Tidus
Posts: 690
Joined: Mon Apr 21, 2003 10:22 pm
Location: Baton Rouge, Louisiana

Re: HSNCT Then and Now

Post by jrbarry »

I remember the very first HSNCT at the University of Oklahoma. Rob, Eric, et al did their usual good job from the very beginning. The only one I missed was 2003 in Myrtle Beach, SC. 2012 in ATL will be my last one as a coach.
J.R. Barry
Retired teacher and former coach
User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: HSNCT Then and Now

Post by Mechanical Beasts »

cvdwightw wrote:
evilmonkey wrote:Dwight - I love the work. Did you run any tests on negs? I ask because among the many possible reasons for increased power rate, a decreased neg rate is one of the few that can be meaningfully tested.
I did not run any tests on neg rate but I do not understand why a decreased neg rate would positively correlate with an increased power rate - if anything, I would expect power rates and neg rates to go up simultaneously.
My best hypothesis is that--since we're starting fairly far back in history--the occurrence of hoses should decrease over time, making powers... possible in the first place. But yeah, over time the dominant trend should be an increase in aggressive buzzing and thus both outcomes.
Andrew Watkins
Post Reply