ACF Nationals 2019 discussion: Logistics/format

Old college threads.
Locked
User avatar
VSCOelasticity
Rikku
Posts: 256
Joined: Sun Nov 20, 2016 7:05 pm

ACF Nationals 2019 discussion: Logistics/format

Post by VSCOelasticity »

[This thread has been split from the discussion on question content, which you can find here. -Mods]

I very much enjoyed this set and its approach to the sciences. There was a good amount of creativity (electrides and atmospheric entry come to mind immediately, but there is other stuff I am forgetting). I am looking forward to reading over the packets.

I will echo the consensus about the excess CS. I think having 1/1 of the math approach CS topics (DAGs and encryption respectively) that came up later in the CS category (Bellman-Ford graph algorithm and consensus in Bitcoin) made the tournament feel like there was no math. I'm not sure how I feel about "pure" CS like DAGs and cryptography taking up slots in the math distro. Admittedly, this feeling is probably heightened by the packetization. Apart from the sub-distributional issues, I thought the execution of the CS was excellent. I will particularly single out the networking content: the TU on Cisco and the earlier clues for Bellman-Ford were well done.

Purely format-wise stuff: As a player, I would like qb to move away from the "superplayoff" model at national tournaments. With only three games on Sunday I believe prelim games were done around 11:30AM. The format from 2018 ACF Nationals would have required three more games (not counting tiebreakers), most likely taking us to 1PM. The counterpoint to this as an editor is, of course, that this is more questions to write. So I guess the question is: how many packets should we expect the editors to produce? I am not really in a place to comment on this because it took me an inordinate amount of time just to create a relatively easy tournament. I imagine the hourly wage per ACF Nationals question is quite low, so I think it is fair to suggest there is a limit on how much time we expect the editors/writers to sink into this tournament.

Also, I think rounds ended at around 8/8:30PM on Saturday, while the e-mailed schedule allocated time until 10PM for games. Ending early was nice, but this leads me to believe that there was time to play off tiebreakers.

(Please correct me if I got any of the times wrong, as that would definitely change my arguments)

Finally, thank you to the editors for all of your hard work on the set, and thank you to all of the tournament staff who made the tournament run smoothly!
Eleanor
they/she
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: ACF Nationals 2019 discussion

Post by Cody »

settlej wrote: Mon Apr 15, 2019 2:03 pmPurely format-wise stuff: As a player, I would like qb to move away from the "superplayoff" model at national tournaments. With only three games on Sunday I believe prelim games were done around 11:30AM. The format from 2018 ACF Nationals would have required three more games (not counting tiebreakers), most likely taking us to 1PM. The counterpoint to this as an editor is, of course, that this is more questions to write. So I guess the question is: how many packets should we expect the editors to produce? I am not really in a place to comment on this because it took me an inordinate amount of time just to create a relatively easy tournament. I imagine the hourly wage per ACF Nationals question is quite low, so I think it is fair to suggest there is a limit on how much time we expect the editors/writers to sink into this tournament.
Speaking purely schedule-wise, I think Sunday was a good length and one that accommodates teams the best because it isn't always possible to get later flights at a good price point. I do not think it is wise to have the rounds for most of the teams end past 12:00 PM, which means you can only ever offer 4 or 5 games on Sunday (and 5 is very iffy, requiring an 8:00 AM start and no delays). The reason is that finals and awards take time, and anything ending past 1:30 PM - 2:00 PM is dangerous, even if it is announced in advance.

Specifically, this year, the finals and award ceremony didn't conclude until close to 2 PM (maybe 1:45 PM?), and multiple teams had to leave or readjust their plans in order to make it to the awards ceremony. The logistics team for national tournaments needs to be cognizant of time constraints in order to allow teams to take appropriately priced flights, and I think 3 rounds on Sunday (whether tethered to a 3-stage format or scheduled that way from a 2-stage format) is actually good.
Cody Voight, VCU ’14.
A Dim-Witted Saboteur
Yuna
Posts: 973
Joined: Tue Aug 02, 2016 12:31 pm
Location: Indiana

Re: ACF Nationals 2019 discussion

Post by A Dim-Witted Saboteur »

Cody wrote: Mon Apr 15, 2019 3:49 pm Speaking purely schedule-wise, I think Sunday was a good length and one that accommodates teams the best because it isn't always possible to get later flights at a good price point. I do not think it is wise to have the rounds for most of the teams end past 12:00 PM, which means you can only ever offer 4 or 5 games on Sunday (and 5 is very iffy, requiring an 8:00 AM start and no delays). The reason is that finals and awards take time, and anything ending past 1:30 PM - 2:00 PM is dangerous, even if it is announced in advance.

Specifically, this year, the finals and award ceremony didn't conclude until close to 2 PM (maybe 1:45 PM?), and multiple teams had to leave or readjust their plans in order to make it to the awards ceremony. The logistics team for national tournaments needs to be cognizant of time constraints in order to allow teams to take appropriately priced flights, and I think 3 rounds on Sunday (whether tethered to a 3-stage format or scheduled that way from a 2-stage format) is actually good.
Speaking specifically for MSU and other teams with long drives, we appreciated the early end of rounds on Sunday; getting back home before midnight was delightful.
Jakob M. (they/them)
Michigan State '21, Indiana '2?
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal
User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3109
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: ACF Nationals 2019 discussion

Post by DumbJaques »

I want to thank the editors for the considerable effort they put into the set, which I enjoyed a great deal of. As someone who has made critiques of editors for indulging in their own tendencies in the past, I want to be very clear that I though Auroni did a phenomenal job specifically addressing a great deal of the feedback nats-level events have received in recent years. This is what it takes to produce a top-quality event, and it really does deserve to be lauded.

As much as I generally enjoyed and greatly appreciated the set, I was incredibly frustrated by the playoff-superplayoff format, and I'll add my voice to those calling for it to be retired. I understand there's a limit to what can be done, but last year's schedule, for instance, seemed to work just fine to me. My understanding is that this would have only required 2 more packets (since you'd need one fewer tiebreaker in this case). Of course, "just" 2 more nationals packets isn't insignificant, and I understand there were specific challenges this year (I certainly identify with Auroni's feelings of editor burnout).

But, on the other hand, it's ACF Nationals. I have to believe that if the editors had sent up a flare two months ago and requested some help, they could have brought in two packets worth of freelance contributions. If we need to pay a bit more, or submit a bit sooner, etc, that's ok. This year we ended up with a situation where the winning team lost 3 games - to at least 2 and quite possibly 3 teams that we'd all agree are top-12 this year. The 2nd-place team, on the other hand, lost only two games. That's a really unfortunate outcome, if it's at all preventable, and in this case I think it is. It was also particularly frustrating, as a team that caught a few rough packets in the playoffs and some great ones in the superplayoffs, to have the relative importance of the latter restricted by the former. Not to mention that, if a single game at this tournament had gone two tossups differently, Columbia's placement at this set would have been 7th rather than 1st. (Of course, you might say, those games are part of every tournament! Except that normally, this only happens in prelim brackets, and it is probably super unlikely that a top-5 teams will miss the 10- or 12- playoffs entirely; with a superplayoff format, this is demonstrably no longer true. And, even if you don't agree with that, you're still doubling the opportunity for such situations to occur, which seems prima facie bad).

I'd like to see us move to a community consensus where not having a full round-robin among playoff teams is seen as a clear sub-optimal outcome in tournament scheduling. It helps us alleviate the fundamental quizbowl issue of packet-to-packet variation, and it helps us alleviate the unavoidable difficulty of accurate tournament seeding. In cases like this, it does require a few more rounds (produced and played), but I don't think we have to go further than comparing last year to this year to see that this is entirely viable. It's unfortunate that the takeaway from this event has to be the format concerns, rather than simply the excellent work on the set or Columbia's dramatic and impressive run.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: ACF Nationals 2019 discussion

Post by Cody »

Fuddle Duddle wrote: Mon Apr 15, 2019 3:58 pmSpeaking specifically for MSU and other teams with long drives, we appreciated the early end of rounds on Sunday; getting back home before midnight was delightful.
Oh, yes, I don't know why I forgot to mention this. You can expect some teams to have up to 10 hour drives (and sometimes more) after a tournament and likewise the early end helps with that.
Cody Voight, VCU ’14.
User avatar
Auroni
Auron
Posts: 3145
Joined: Thu Nov 15, 2007 6:23 pm

Re: ACF Nationals 2019 discussion

Post by Auroni »

Thank you for your kind words about the set, Chris.
DumbJaques wrote: Mon Apr 15, 2019 5:16 pm But, on the other hand, it's ACF Nationals. I have to believe that if the editors had sent up a flare two months ago and requested some help, they could have brought in two packets worth of freelance contributions. If we need to pay a bit more, or submit a bit sooner, etc, that's ok.
I’d like to note that this is precisely what we did — so that we would be able to complete the planned number of packets (actually 20, in imitation of ACF Nationals 2017, but we added a 21st after Ryan informed me that we would need one for a 48 team format). We did receive significant help, but those questions, like all submissions, had to be edited to fit the quality and vision of the rest of the set — rarely was a freelance question just a gift wrapped present we could immediately put into the set. Despite heroically pooling together everyone’s efforts, we finished with barely enough time for proofreading and basic error catching, resulting in the issues mentioned in the thread (though this is my fault for not realizing before last Sunday that we’d need to start proofreading in tandem with finishing up the final questions).

I realize that the life circumstances of this year’s editing and writing team are not completely generalizable, but many of the game’s longest tenured writers and editors are represented within it, and the general sense is that due not just to burnout, but life commitments and careers it’s extremely unlikely for a small crew to produce 25 packet sets like they have in the past. This constrains the formats for future iterations of Nationals at a time when the tournament is more popular than ever. Something’s got to give.
Last edited by Auroni on Mon Apr 15, 2019 6:00 pm, edited 1 time in total.
Auroni Gupta (she/her)
User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3109
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Re: ACF Nationals 2019 discussion

Post by DumbJaques »

Auroni wrote: Mon Apr 15, 2019 5:43 pm

I’d like to note that this is precisely what we did — so that we would be able to complete the planned number of packets. We did receive significant help, but those questions, like all submissions, had to be edited to fit the quality and vision of the rest of the set — rarely was a freelance question just a gift wrapped present we could immediately into the set.

I realize that the life circumstances of this year’s editing and writing team are not completely generalizable, but many of the game’s longest tenured writers and editors are represented within it, and the general sense is that due not just to burnout, but life commitments and careers it’s extremely unlikely for a small crew to produce 25 packet sets like they have in the past. This constrains the formats for future iterations of Nationals at a time when the tournament is more popular than ever. Something’s got to give.
So, I might just be misinformed a bit about the specifics of this set; did you write 23 packets? My understanding is that there 8 prelim games, 7 playoff/superplayoff games, 3 tiebreakers, 2 finals packets, and perhaps 1 emergency packet. This would bring us to 21. Since one fewer tiebreaker is required by removing the playoff/superplayoff split, this means that only two more packets would be required (so, 23, not 25). But again, correct me if this isn't correct, as you obviously know better than I.

More generally, I'm glad to hear that you were proactive about asking for help; this further underscores the good job you did on the set. My points were very much not meant to be a specific indictment of what happened with the production of the 2019 set. Rather, I'm more saying that establishing, as a community, a baseline expectation of what is needed is important going forward with future nationals (and would perhaps have allowed you to send up the flare farther and sooner, etc).

Of course, there is some debate about whether the work required for such a baseline is practical. I think that it is, and demonstrably has been in the past. If other institutional changes need to be made - with money, or submission schedules, etc. - let's talk about those. But I would push back on the argument that there's something different that makes this project no longer possible. This might be true for specific people involved in the production of nationals, but this group has been in flux for the entire history of the event. As some folks transition to lesser involvement, others are ready to step up for more - if, that is, we're doing our job developing writers and editors, as an organization. If this is the story behind the inability to produce the nationals packet count we want, then it's a different discussion; we should instead be focusing on why that gap has cropped up, and what we can do about it. (Maybe it's time for ACF to start just paying people to churn out a pool for nationals questions? Yes, growth puts added pressures on production, but also offers added resources).
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE
User avatar
Auroni
Auron
Posts: 3145
Joined: Thu Nov 15, 2007 6:23 pm

Re: ACF Nationals 2019 discussion

Post by Auroni »

DumbJaques wrote: Mon Apr 15, 2019 5:59 pm
Auroni wrote: Mon Apr 15, 2019 5:43 pm

I’d like to note that this is precisely what we did — so that we would be able to complete the planned number of packets. We did receive significant help, but those questions, like all submissions, had to be edited to fit the quality and vision of the rest of the set — rarely was a freelance question just a gift wrapped present we could immediately into the set.

I realize that the life circumstances of this year’s editing and writing team are not completely generalizable, but many of the game’s longest tenured writers and editors are represented within it, and the general sense is that due not just to burnout, but life commitments and careers it’s extremely unlikely for a small crew to produce 25 packet sets like they have in the past. This constrains the formats for future iterations of Nationals at a time when the tournament is more popular than ever. Something’s got to give.
So, I might just be misinformed a bit about the specifics of this set; did you write 23 packets? My understanding is that there 8 prelim games, 7 playoff/superplayoff games, 3 tiebreakers, 2 finals packets, and perhaps 1 emergency packet. This would bring us to 21. Since one fewer tiebreaker is required by removing the playoff/superplayoff split, this means that only two more packets would be required (so, 23, not 25). But again, correct me if this isn't correct, as you obviously know better than I.
You replied before I clarified, but I meant our plan to write 21 packets was in trouble before we reached out to others. I mentioned 25 packets as a common size of many past ACF Nationals sets.
Auroni Gupta (she/her)
User avatar
ryanrosenberg
Auron
Posts: 1890
Joined: Thu May 05, 2011 5:48 pm
Location: Palo Alto, California

Re: ACF Nationals 2019 discussion

Post by ryanrosenberg »

settlej wrote: Mon Apr 15, 2019 2:03 pm Also, I think rounds ended at around 8/8:30PM on Saturday, while the e-mailed schedule allocated time until 10PM for games. Ending early was nice, but this leads me to believe that there was time to play off tiebreakers.

(Please correct me if I got any of the times wrong, as that would definitely change my arguments)
The playoff rounds ended at about 8:30, but the the three-way tiebreaker between Columbia, Stanford, and Penn didn't end until about 9:30. I'd estimate that playing off all tiebreakers on Saturday would have added about 30-45 minutes between the two tiebreakers. To me, that time savings is in the range where I'm not wholly sure that it's worth it to not play the extra tiebreakers off, but I'd like to hear how teams and staff would feel about potential Saturday ending times.

More generally, I think there should be a community discussion about ACF Nationals size and format. We're getting to the point where the expansion of ACF Nationals comes at the expense of the everyone-plays-everyone, full-playoff-round-robin formats that characterized previous ACF Nationals. Producing enough packets to do that for 48 teams requires an amount of packets that is a lot for any editing team. Having such a long and intense tournament also makes it more difficult for ACF to recruit staff, as well.
Ryan Rosenberg
North Carolina '16
ACF
User avatar
theMoMA
Forums Staff: Administrator
Posts: 5993
Joined: Mon Oct 23, 2006 2:00 am

Re: ACF Nationals 2019 discussion

Post by theMoMA »

In my opinion, we should be finding ways to make Nationals shorter in schedule and require fewer packets, not trying to push things in the other direction. The tournament is already incredibly long and requires what I think are borderline unreasonable amounts of staffer and editor time.
Andrew Hart
Minnesota alum
Banana Stand
Wakka
Posts: 158
Joined: Thu Apr 26, 2012 4:38 pm

Re: ACF Nationals 2019 discussion

Post by Banana Stand »

Personally, I care more about the integrity of ACF Nationals as a “players championship” than I do about its expansion. Obviously, I’m not a member of ACF and don’t knows its priorities, but if the field needs to be 40 teams instead of 48 to ensure a full RR in playoffs which is almost unanimously considered a better format, then I think that should take priority every single time. Also, in terms of expansion, what’s the end goal? Teams outside the top 40 are already pulling sub-10 PPB’s(and sometimes lower than 8-9) and I question whether 10-20 more teams would be willing to subject themselves to that unless the tournament became substantially easier(which is a whole different discussion). I don’t think a team with more playoff losses should win the tournament(not to take away from Columbia, they played an amazing tournament) if it can be avoided.

As of now, packets are being wasted in prelims that could be used to differentiate the best teams. Any time a 1 seed plays a bottom bracket team, that packet is essentially being thrown away. There’s “that’s why you play the games” and then there’s mathematical certainties. I don’t know if anybody has any interest in changing this, or think it goes against the spirit of the game to change it, but that’s one avenue where better determining the top teams and decreasing stress on writers and editors could be done simultaneously.
Jack Mehr
St. Joe's NJ '14
UVA '19
User avatar
vinteuil
Auron
Posts: 1454
Joined: Sun Oct 23, 2011 12:31 pm

Re: ACF Nationals 2019 discussion

Post by vinteuil »

Banana Stand wrote: Mon Apr 15, 2019 6:44 pm Personally, I care more about the integrity of ACF Nationals as a “players championship” than I do about its expansion. Obviously, I’m not a member of ACF and don’t knows its priorities, but if the field needs to be 40 teams instead of 48 to ensure a full RR in playoffs which is almost unanimously considered a better format, then I think that should take priority every single time. Also, in terms of expansion, what’s the end goal? Teams outside the top 40 are already pulling sub-10 PPB’s(and sometimes lower than 8-9) and I question whether 10-20 more teams would be willing to subject themselves to that unless the tournament became substantially easier(which is a whole different discussion). I don’t think a team with more playoff losses should win the tournament(not to take away from Columbia, they played an amazing tournament) if it can be avoided.

As of now, packets are being wasted in prelims that could be used to differentiate the best teams. Any time a 1 seed plays a bottom bracket team, that packet is essentially being thrown away. There’s “that’s why you play the games” and then there’s mathematical certainties. I don’t know if anybody has any interest in changing this, or think it goes against the spirit of the game to change it, but that’s one avenue where better determining the top teams and decreasing stress on writers and editors could be done simultaneously.
I know that Matt Weiner, among others, has in the past advocated breaking out Nats into "divisions." As a stopgap, would it be unreasonable to do that even without using two different question sets? (After all, the lowest-placing teams often come to test their mettle against the toughest questions.)

(This is to say: I fully agree with Ryan and Andrew.)
Last edited by vinteuil on Mon Apr 15, 2019 6:52 pm, edited 1 time in total.
Jacob R., ex-Chicago
User avatar
VSCOelasticity
Rikku
Posts: 256
Joined: Sun Nov 20, 2016 7:05 pm

Re: ACF Nationals 2019 discussion

Post by VSCOelasticity »

ryanrosenberg wrote: Mon Apr 15, 2019 6:20 pm
settlej wrote: Mon Apr 15, 2019 2:03 pm Also, I think rounds ended at around 8/8:30PM on Saturday, while the e-mailed schedule allocated time until 10PM for games. Ending early was nice, but this leads me to believe that there was time to play off tiebreakers.

(Please correct me if I got any of the times wrong, as that would definitely change my arguments)
The playoff rounds ended at about 8:30, but the the three-way tiebreaker between Columbia, Stanford, and Penn didn't end until about 9:30. I'd estimate that playing off all tiebreakers on Saturday would have added about 30-45 minutes between the two tiebreakers. To me, that time savings is in the range where I'm not wholly sure that it's worth it to not play the extra tiebreakers off, but I'd like to hear how teams and staff would feel about potential Saturday ending times.

More generally, I think there should be a community discussion about ACF Nationals size and format. We're getting to the point where the expansion of ACF Nationals comes at the expense of the everyone-plays-everyone, full-playoff-round-robin formats that characterized previous ACF Nationals. Producing enough packets to do that for 48 teams requires an amount of packets that is a lot for any editing team. Having such a long and intense tournament also makes it more difficult for ACF to recruit staff, as well.
Oh okay, I didn't hear about that tiebreaker. I'm personally torn on the time/tiebreaker situation. Getting out with enough time to eat a sit-down meal and also get a good rest is really appreciated, but playing off tiebreakers is ideal.

I really like ACF's inclusiveness in inviting 48 teams, but maybe somewhere between 40 and 48 (like 2016 and 2017) would be more ideal from a fairness and writing standpoint. Having finished in 41st before, it would suck to be bubbled out of playing ACF nats, but I think logistics has to take precedence at some point. I think an easier load for editors/staffers outweighs teams who are upset about being bubbled at 40+. And (as has been mentioned, I believe) with a different format there would only be one rebracket so tiebreakers could be played off.

Could something be done to make editing submissions less taxing? For example, should citations be required so that fact checking is easier? Should teams receive more specific instructions for submission (I'm thinking of stuff like "submit 1/1 with 1 methodological-focused and 1 linguistics" instead of "submit 1/1 social science"?

Also I just double checked the announcement thread and didn't see a mention of a line cap. Maybe having one less line per TU/bonus could be achieved without sacrificing resolving power between teams.
Eleanor
they/she
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: ACF Nationals 2019 discussion

Post by Cheynem »

How would you determine who goes in what division?
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
Mike Bentley
Sin
Posts: 6461
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: ACF Nationals 2019 discussion

Post by Mike Bentley »

settlej wrote: Mon Apr 15, 2019 6:51 pm Could something be done to make editing submissions less taxing? For example, should citations be required so that fact checking is easier? Should teams receive more specific instructions for submission (I'm thinking of stuff like "submit 1/1 with 1 methodological-focused and 1 linguistics" instead of "submit 1/1 social science"?
I think some tournaments in the past have had the editors pick answer lines for teams submitting packets. That helps reduce repeats and questions that get thrown out because they're too hard or bad ideas. But there are downsides of that as well. Picking answers is hard in itself. It also reduces the fun of submitting a packet because you need to write questions on assigned topics rather than writing about what you want to write about. This model may work a bit better for freelance submissions. (Also, teams may still submit pretty unusable questions even if they've been told what to write.)
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
Banana Stand
Wakka
Posts: 158
Joined: Thu Apr 26, 2012 4:38 pm

Re: ACF Nationals 2019 discussion

Post by Banana Stand »

vinteuil wrote: Mon Apr 15, 2019 6:50 pm I know that Matt Weiner, among others, has in the past advocated breaking out Nats into "divisions." As a stopgap, would it be unreasonable to do that even without using two different question sets? (After all, the lowest-placing teams often come to test their mettle against the toughest questions.

(This is to say: I fully agree with Ryan and Andrew.)
A division system would probably be ideal, even without changing the set (you could, but since we’re talking about ways to reduce editor workload I won’t consider that). Pragmatically, it would be very difficult as teams on the bubble of certain divisions would be angry. In general, I think regular season results should matter more to determine what you’re playing for at Nats in order to avoid the sort of packet wasting I mentioned. Top teams should be playing as many games as possible given packets and time constraints, and figuring out some way to predetermine divisions would be helpful for that. I don’t foresee this happening any time soon, but I think it’s at least worth talking about.
Jack Mehr
St. Joe's NJ '14
UVA '19
User avatar
VSCOelasticity
Rikku
Posts: 256
Joined: Sun Nov 20, 2016 7:05 pm

Re: ACF Nationals 2019 discussion

Post by VSCOelasticity »

Mike Bentley wrote: Mon Apr 15, 2019 6:58 pm
settlej wrote: Mon Apr 15, 2019 6:51 pm Could something be done to make editing submissions less taxing? For example, should citations be required so that fact checking is easier? Should teams receive more specific instructions for submission (I'm thinking of stuff like "submit 1/1 with 1 methodological-focused and 1 linguistics" instead of "submit 1/1 social science"?
I think some tournaments in the past have had the editors pick answer lines for teams submitting packets. That helps reduce repeats and questions that get thrown out because they're too hard or bad ideas. But there are downsides of that as well. Picking answers is hard in itself. It also reduces the fun of submitting a packet because you need to write questions on assigned topics rather than writing about what you want to write about. This model may work a bit better for freelance submissions. (Also, teams may still submit pretty unusable questions even if they've been told what to write.)
Yeah, picking answerlines is challenging in and of itself, but I think limiting a question to a specific subdistribution is manageable. Most writing teams lay out subdistribution requirements ahead of time, so it seems feasible. Editors could also do some sort of compromise where, for each 1/1, teams must submit one on a specific subdistribution and one on something the team wants to write about. I always look forward to submitting packets with creative/fresh clues but there's also an argument that you're doing something for a discount. I think the editors are free to impose some restrictions on team submissions. Do people think that is a worthwhile idea to implement, especially for tournaments that are difficult to produce like ACF Nationals?

Unusable submissions will probably always happen, and I'm sure many of my early submissions were absolutely garbage (hell, probably even ones I submitted this year), but by constraining the writing space editors may get garbage that is easier to polish.
Eleanor
they/she
User avatar
VSCOelasticity
Rikku
Posts: 256
Joined: Sun Nov 20, 2016 7:05 pm

Re: ACF Nationals 2019 discussion

Post by VSCOelasticity »

Banana Stand wrote: Mon Apr 15, 2019 7:08 pm
vinteuil wrote: Mon Apr 15, 2019 6:50 pm I know that Matt Weiner, among others, has in the past advocated breaking out Nats into "divisions." As a stopgap, would it be unreasonable to do that even without using two different question sets? (After all, the lowest-placing teams often come to test their mettle against the toughest questions.

(This is to say: I fully agree with Ryan and Andrew.)
A division system would probably be ideal, even without changing the set (you could, but since we’re talking about ways to reduce editor workload I won’t consider that). Pragmatically, it would be very difficult as teams on the bubble of certain divisions would be angry. In general, I think regular season results should matter more to determine what you’re playing for at Nats in order to avoid the sort of packet wasting I mentioned. Top teams should be playing as many games as possible given packets and time constraints, and figuring out some way to predetermine divisions would be helpful for that. I don’t foresee this happening any time soon, but I think it’s at least worth talking about.
I think a division system is ideal in a situation with perfect information, but given the distributed nature of quiz bowl it seems ill-advised. Canada and Florida circuits, for example, do not interact with most of the qb world until ICT/Nats. If people think seeds are unjust, I can only imagine what the response to this division system would be.
Eleanor
they/she
User avatar
Fado Alexandrino
Yuna
Posts: 834
Joined: Sat Jun 12, 2010 8:46 pm
Location: Farhaven, Ontario

Re: ACF Nationals 2019 discussion

Post by Fado Alexandrino »

Where does the estimate of 30-45 minute increase if you played off all rr-playoff tiebreakers come from? I don’t think you’d save much time if, for example, at ICT only the top bracket ties were played off.
Joe Su, OCT
Lisgar 2012, McGill 2015, McGill 2019, Queen's 2020
User avatar
ryanrosenberg
Auron
Posts: 1890
Joined: Thu May 05, 2011 5:48 pm
Location: Palo Alto, California

Re: ACF Nationals 2019 discussion

Post by ryanrosenberg »

Benin Rebirth Party wrote: Mon Apr 15, 2019 7:44 pm Where does the estimate of 30-45 minute increase if you played off all rr-playoff tiebreakers come from? I don’t think you’d save much time if, for example, at ICT only the top bracket ties were played off.
Unlike ICT, ACF Nationals games are fairly variable in timing due to not being on the clock. It takes time to arrange and get tiebreakers going, more so if there are more tiebreakers.
Ryan Rosenberg
North Carolina '16
ACF
User avatar
CPiGuy
Auron
Posts: 1070
Joined: Wed Nov 16, 2016 8:19 pm
Location: Ames, Iowa

Re: ACF Nationals 2019 discussion: Logistics/format

Post by CPiGuy »

I made a bunch of comments on the logistics and format in the other thread in the same post as question feedback. Gonna put them here instead.

Thoughts on the weekend's logistics:

-- I didn't actually staff this year but I saw that the staff grids for Sunday were finalized at the beginning of the tournament, which I appreciate. I also liked the addition of nametags for staff -- it not only added a little bit of "officialness" to the tournament but made it easier to identify the many staff from around the country, many of whom i had met never or only a couple times.

-- Speaking of the staff, the staff did a great job -- thanks to all of them! Reading at ACF Nats is not easy and I appreciate everyone who gave up their weekend to do so.

-- The logistics themselves were really good. I was concerned about having the auditorium in a completely different building but it turned out to be not an issue at all, and the tournament in fact ran ahead of schedule. Literally the only logistic hiccup I can think of was that some of the prelim rooms were mis-labeled on the schedule, but this was easily fixed in about five minutes (and also fixed on the playoff schedules!)

-- I also appreciated the staff's actually printing out playoff schedules with the teams filled in rather than using "A6", etc.

-- Andrew Feist maintaining the standings on his website was really nice and easy to follow -- probably the best following-a-tournament-online experience other than Neg5 (for which there are presumably obvious reasons that ACF didn't use it)!

Thoughts on the scheduling discussion:

-- I was disappointed to only play three rounds on Sunday, but I understand that editing a tournament like this is hard and do not want to ask the editors to produce more questions (they already produced 21 packets!).

-- I do think that perhaps they should return to the model of mandatory packet submission, since if I recall correctly it was dropped because ACF didn't actually need that many packets. Since ACF has now had to shorten Nats for lack of packets, it might be a good idea to revert that decision.

-- I really, really, really like Jonathan's suggestion to make more specific subdistributional requirements of submitting teams. It's not unreasonable to expect teams to produce, say, "American poetry, pre-1900" rather than just "American literature". Perhaps this and the above suggestion, taken together, might be unreasonable; in that case I think this suggestion is better -- presumably the yield from teams that are only submitting packets because they're required to is low, but the yield from good subdistributional management is high.

-- I think having separate divisions is a terrible, terrible idea. Speaking as someone who is actually in the lower half of the Nats field (since so far everyone offering opinions on this are on very good teams that wouldn't be affected at all), I would not be interested in spending considerable amounts of time and planning effort to attend a tournament to determine the 31st-place team in the country while playing against zero of the best 30 such teams. I doubt I'm alone in this -- we attend ACF Nationals with the express purpose of competing against the country's best, and with full knowledge that we're going to get the shit knocked out of us in several of those game. If you want the average team quality at Nats to be higher in order to enable a more round-robin-y format, just reduce the qualification threshold and don't bother with organizing a second concurrent consolation tournament that will inevitably feature almost exclusively teams within easy driving distance of the host.
Conor Thompson (he/it)
Bangor High School '16
University of Michigan '20
Iowa State University '25
Tournament Format Database
User avatar
Sam
Rikku
Posts: 338
Joined: Sat Nov 07, 2009 2:35 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Sam »

I don't know enough about tournament direction or production to say anything about the format, but I was really impressed with the overall logistics of the tournament. Conor already said most of this, but the general tournament infrastructure (working buzzers set up on time, schedules printed in real time, name tags) made going through the tournament very simple. The moderators and scorekeepers were also superb. Thanks to Ryan and his colleagues for putting it together.

I'm not sure if this belongs in the Nationals discussion thread or not, but, like John Lawrence, I would like a little more clarity about ACF's rule on players changing their answers before a moderator says anything. Especially at a tournament where moderators may be unfamiliar with many of the answer lines, it can be a few seconds between when a player says something and when the moderator officially responds, and the pause can be informative to the player on whether they should adjust their answer. There's also a fair amount of variation between moderators in how quickly they signal a neg (though I suppose this is less important as long as they are consistent across teams).
Sam Bailey
Minnesota '21
Chicago '13
User avatar
TheIronLadyOfQuizbowl
Kimahri
Posts: 4
Joined: Sun Jun 19, 2016 8:35 pm
Location: Norman, Oklahoma

Re: ACF Nationals 2019 discussion: Logistics/format

Post by TheIronLadyOfQuizbowl »

What is the rationale between not using head-to-head as a tiebreaker?
Thatcher Chonka
Jenks High School '16
University of Oklahoma '19
User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4295
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Auks Ran Ova »

TheIronLadyOfQuizbowl wrote: Tue Apr 16, 2019 6:15 pm What is the rationale between not using head-to-head as a tiebreaker?
Doing so double-counts the game in question.
Rob Carson
University of Minnesota '11, MCTC '??, BHSU forever
Member, ACF
Member emeritus, PACE
Writer and Editor, NAQT
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Cody »

Auks Ran Ova wrote: Tue Apr 16, 2019 6:34 pm
TheIronLadyOfQuizbowl wrote: Tue Apr 16, 2019 6:15 pm What is the rationale between not using head-to-head as a tiebreaker?
Doing so double-counts the game in question.
For a longer discussion, refer to this post.
Cody Voight, VCU ’14.
User avatar
VSCOelasticity
Rikku
Posts: 256
Joined: Sun Nov 20, 2016 7:05 pm

Re: ACF Nationals 2019 discussion: Logistics/format

Post by VSCOelasticity »

Cody wrote: Tue Apr 16, 2019 6:48 pm
Auks Ran Ova wrote: Tue Apr 16, 2019 6:34 pm
TheIronLadyOfQuizbowl wrote: Tue Apr 16, 2019 6:15 pm What is the rationale between not using head-to-head as a tiebreaker?
Doing so double-counts the game in question.
For a longer discussion, refer to this post.
I'm confused about this then. The decision to not play off tiebreakers was based on time constraints, but those threads establish the idea that H2H is a less fair tiebreaker than PPG in a round robin format. If this is the case, why are tiebreakers being played between potential top bracket teams? Shouldn't we just use PPG for everyone (if it is a community consensus that PPG is more fair)?
Eleanor
they/she
User avatar
Cheynem
Sin
Posts: 7219
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Cheynem »

I think you're confusing head to head with playing off the tie. "Head to head" is simply just counting how the two teams played each other in their previous match up, so if Illinois finished 6-1 and Michigan State finished 6-1, then you would break the tie by who beat who. Playing off the tie has them play each other again, which most people would judge to be fairer.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger
User avatar
VSCOelasticity
Rikku
Posts: 256
Joined: Sun Nov 20, 2016 7:05 pm

Re: ACF Nationals 2019 discussion: Logistics/format

Post by VSCOelasticity »

Cheynem wrote: Tue Apr 16, 2019 7:10 pm I think you're confusing head to head with playing off the tie. "Head to head" is simply just counting how the two teams played each other in their previous match up, so if Illinois finished 6-1 and Michigan State finished 6-1, then you would break the tie by who beat who. Playing off the tie has them play each other again, which most people would judge to be fairer.
Oh, my bad. Thanks for clearing that up!
Eleanor
they/she
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Cody »

Sam wrote: Tue Apr 16, 2019 11:09 amI'm not sure if this belongs in the Nationals discussion thread or not, but, like John Lawrence, I would like a little more clarity about ACF's rule on players changing their answers before a moderator says anything. Especially at a tournament where moderators may be unfamiliar with many of the answer lines, it can be a few seconds between when a player says something and when the moderator officially responds, and the pause can be informative to the player on whether they should adjust their answer. There's also a fair amount of variation between moderators in how quickly they signal a neg (though I suppose this is less important as long as they are consistent across teams).
Rule F.2*, as presented in its entirety at the staff meeting, requires that there be no unnatural pauses in amending an answer, so the situation as written-up would not allow the player to gain any advantage. If the player answers and the moderator pauses to rule on the answer, then the player has stopped answering (an "unnatural pause" has occurred) and any additional information doesn't count towards the "first given answer".

(By the way, it would honestly be great if players in the situation Sam lays out would stop and give the moderator a chance to work out the correct course of action. You can always answer correctly when prompted, but if you're ruled wrong you've potentially given your opponent the answer. This is probably one of the top-5 most annoying things I encounter as a moderator.)

* "When a player or team has the opportunity to give an answer, the first given answer which the moderator recognizes is the only answer that will be ruled on. Giving an answer involves completing at least one word that could uniquely identify a potential answer and then, if applicable, completing all other words of that potential answer without any unnatural pauses or pauses of longer than a second between words within that potential answer (disregarding embellishing; See F.8). A player who wishes to amend their given answer (e.g. to correct the pronunciation of a word, to change from one given answer to another) must do so, without any unnatural pause, before the moderator has ruled on any answer that player has given."
Cody Voight, VCU ’14.
User avatar
Sam
Rikku
Posts: 338
Joined: Sat Nov 07, 2009 2:35 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Sam »

Cody wrote: Wed Apr 17, 2019 4:02 pm ]Rule F.2*, as presented in its entirety at the staff meeting, requires that there be no unnatural pauses in amending an answer, so the situation as written-up would not allow the player to gain any advantage. If the player answers and the moderator pauses to rule on the answer, then the player has stopped answering (an "unnatural pause" has occurred) and any additional information doesn't count towards the "first given answer".
That makes sense. Thanks for the clarification!
Sam Bailey
Minnesota '21
Chicago '13
User avatar
a bird
Wakka
Posts: 164
Joined: Sun Feb 26, 2012 3:50 pm
Location: College Park, MD

Re: ACF Nationals 2019 discussion: Logistics/format

Post by a bird »

I agree with Chris and Jack that this year's crossover playoff felt less complete than a full round robin in the top bracket. In addition to the point about playoff losses not carrying over, we should remember that this year's format can also erase playoff wins against higher finishing opponents (e.g. OSU's win over Columbia). Maybe this year's super playoff/crossover format is the best compromise between fairness, completeness, and scheduling considerations, but I think ACF should consider the advantages of a top bracket round robin for future championships.
Graham R.

Maryland
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Cody »

a bird wrote: Thu Apr 18, 2019 11:05 amI agree with Chris and Jack that this year's crossover playoff felt less complete than a full round robin in the top bracket. In addition to the point about playoff losses not carrying over, we should remember that this year's format can also erase playoff wins against higher finishing opponents (e.g. OSU's win over Columbia). Maybe this year's super playoff/crossover format is the best compromise between fairness, completeness, and scheduling considerations, but I think ACF should consider the advantages of a top bracket round robin for future championships.
I really don't like arguments along the lines of "erasing wins" / "erasing losses" because I think it relies on a misconception about how a proper 2-stage or 3-stage format is supposed to work. (No one has said similar things about Stanford's prelim losses / wins, and for good reason -- that point of acceptability is ingrained into quizbowl.)

To quote myself from the SCT placement policies thread (click the asterisks to follow to the original posts):
Cody wrote: Thu Feb 21, 2019 10:03 amA key to the bracketed format is seeding / re-seeding. You cannot take all the teams from one bracket to a given tier in the playoffs, else we would not need to use a bracketed format. So, we snake seed teams for preliminary rounds based on a value judgement of their strength to spread them out and try to ensure that the appropriate teams make each tier of the playoffs. (Imperfections in seeding are the critical weakness of a bracketed format because they block teams from making the tier they may have “deserved” to make based on their strength. It’s a small but significant reason that power matching excels for large field sizes, circa 150+ teams. In the presence of perfect seeding, bracketed formats would be almost as perfect as a whole-field round robin.) At the conclusion of preliminary rounds, teams are reseeded based on win-loss and tiebreakers and put into playoff brackets where they play other teams of similar strength.

The critical take-away from the bracketed format scheme is that the preliminary rounds are designed specifically to seed teams for the playoffs in the fairest format we can devise, accepting that some imperfections will come into play (mitigated by proper design of the playoff format). The playoffs, against teams of similar strength, are how we determine the final standings. By this understanding, there is no tension in dropping games from the preliminary rounds in determining placement because they have served their purpose in slotting teams into playoff brackets.
Cody wrote: Wed Feb 20, 2019 6:54 pmpostscript: see this post of mine in the 2017 thread for a primer on what bracketed prelims are supposed to accomplish, which is part of the theoretical underpinnings for only counting games against a common field.
It must be understood, then, that the job of the playoffs in the 3-stage format this year is primarily to take 12 teams from the prelims and allow some sorting so they can be grouped into the 6 team brackets for the superplayoffs, which is very different from the role of playoffs in a 2-stage format where the playoffs are the concluding stage. Half the games for the superplayoff sorting must be discarded because half the teams wind up in different superplayoff brackets.

In effect, the playoffs in a 3-stage format are no different than the prelims except that we expect that playoff games count "more" because that's how it usually works and they are further into the tournament with the games being between competition closer in talent.
____________________________________________________________

Abstract defense aside, I personally prefer a full playoff round robin and think that it is one of the special attractions of ACF Nationals, but I think the scheduling and packet concerns are extremely serious issues that offer a reasonable basis for pursuing a superplayoff schedule.
Cody Voight, VCU ’14.
User avatar
jonpin
Auron
Posts: 2266
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: ACF Nationals 2019 discussion: Logistics/format

Post by jonpin »

Essentially, this is the classic "pick 2 out of 3" situation:
* Large ACF Nationals, with more than 40 teams
* Championship bracket with 10+ teams playing a full round-robin
* Tournament of a length that writers and editors can produce at a high quality; that moderators can read without losing their voice; and that ends at a reasonable time on Saturday and Sunday
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov
User avatar
heterodyne
Rikku
Posts: 427
Joined: Tue Jun 26, 2012 9:47 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by heterodyne »

jonpin wrote: Thu Apr 18, 2019 2:25 pm Essentially, this is the classic "pick 2 out of 3" situation:
* Large ACF Nationals, with more than 40 teams
* Championship bracket with 10+ teams playing a full round-robin
* Tournament of a length that writers and editors can produce at a high quality; that moderators can read without losing their voice; and that ends at a reasonable time on Saturday and Sunday
Which of these was not satisfied by ACF Nationals 2018?
Alston [Montgomery] Boyd
Bloomington High School '15
UChicago '19
UChicago Divinity '21
they
User avatar
ryanrosenberg
Auron
Posts: 1890
Joined: Thu May 05, 2011 5:48 pm
Location: Palo Alto, California

Re: ACF Nationals 2019 discussion: Logistics/format

Post by ryanrosenberg »

heterodyne wrote: Thu Apr 18, 2019 5:03 pm
jonpin wrote: Thu Apr 18, 2019 2:25 pm Essentially, this is the classic "pick 2 out of 3" situation:
* Large ACF Nationals, with more than 40 teams
* Championship bracket with 10+ teams playing a full round-robin
* Tournament of a length that writers and editors can produce at a high quality; that moderators can read without losing their voice; and that ends at a reasonable time on Saturday and Sunday
Which of these was not satisfied by ACF Nationals 2018?
ACF Nationals 2018 had 25 packets; more generally, you'd need at least 24 packets to have a 42-48 team tournament with a two-stage format. I can't speak to the difficulty of producing a 24-packet Nats set each year, but I'm given to understand that it's quite rough.
Ryan Rosenberg
North Carolina '16
ACF
User avatar
Bosa of York
Rikku
Posts: 362
Joined: Sun Jun 15, 2014 9:16 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Bosa of York »

ryanrosenberg wrote: Fri Apr 19, 2019 9:05 am
heterodyne wrote: Thu Apr 18, 2019 5:03 pm
jonpin wrote: Thu Apr 18, 2019 2:25 pm Essentially, this is the classic "pick 2 out of 3" situation:
* Large ACF Nationals, with more than 40 teams
* Championship bracket with 10+ teams playing a full round-robin
* Tournament of a length that writers and editors can produce at a high quality; that moderators can read without losing their voice; and that ends at a reasonable time on Saturday and Sunday
Which of these was not satisfied by ACF Nationals 2018?
ACF Nationals 2018 had 25 packets; more generally, you'd need at least 24 packets to have a 42-48 team tournament with a two-stage format. I can't speak to the difficulty of producing a 24-packet Nats set each year, but I'm given to understand that it's quite rough.
Would this be easier to do with a greater number of submissions (since fewer editor questions would be needed for prelims)? Would ACF consider reinstating mandatory packet submission for Nationals (half or full) if it could lead to a greater number of editor's packets and teams were willing to go along with it?
Eric Wolfsberg
Bethlehem Central High School 2016
University of Delaware 2020
Stanford 2025 or whatever
User avatar
The King's Flight to the Scots
Auron
Posts: 1645
Joined: Mon Jan 26, 2009 11:11 pm

Re: ACF Nationals 2019 discussion: Logistics/format

Post by The King's Flight to the Scots »

Milhouse wrote: Fri Apr 19, 2019 9:24 am
ryanrosenberg wrote: Fri Apr 19, 2019 9:05 am
heterodyne wrote: Thu Apr 18, 2019 5:03 pm
jonpin wrote: Thu Apr 18, 2019 2:25 pm Essentially, this is the classic "pick 2 out of 3" situation:
* Large ACF Nationals, with more than 40 teams
* Championship bracket with 10+ teams playing a full round-robin
* Tournament of a length that writers and editors can produce at a high quality; that moderators can read without losing their voice; and that ends at a reasonable time on Saturday and Sunday
Which of these was not satisfied by ACF Nationals 2018?
ACF Nationals 2018 had 25 packets; more generally, you'd need at least 24 packets to have a 42-48 team tournament with a two-stage format. I can't speak to the difficulty of producing a 24-packet Nats set each year, but I'm given to understand that it's quite rough.
Would this be easier to do with a greater number of submissions (since fewer editor questions would be needed for prelims)? Would ACF consider reinstating mandatory packet submission for Nationals (half or full) if it could lead to a greater number of editor's packets and teams were willing to go along with it?
Not speaking for ACF here, but as an editor, getting a submitted packet up to our standards is still a lot of work. In both 2017 and 2019, it was often pretty disappointing to see what extremely qualified writing teams would submit to us. I understand that everyone has their own stuff going on and I've definitely sent off a packet just to make the deadline before, but I would just ask that, if you know you can do better, please take the extra time or even ask for an extension so that you can submit work in which you can take pride. Getting more packets this year probably wouldn't have taken much load off the editors, but receiving a couple more good packets absolutely would have.
Matt Bollinger
UVA '14, UVA '15
User avatar
Cody
2008-09 Male Athlete of the Year
Posts: 2891
Joined: Sun Nov 15, 2009 12:57 am

Re: ACF Nationals 2019 discussion: Logistics/format

Post by Cody »

Packet submission was not mandatory last year and 16 packets were submitted. This year, 21 packets were submitted.

In 2017, with mandatory packet submission, only 21 of 36 packets were submitted prior to 4 weeks before the tournament. (Although 6 of those 15 late packets were used*.)

In any case, packet submission only solves problems for certain rounds because you need 2 submitted packets per round in the playoffs. (Which is why the playoffs have been purely editors packets of late.) So, for the usual scheduling reasons** and the round issues, packet submission can't solve much in relieving the burden on the editors.

* per my referencing of the schedule I created; I didn't check the packet archive.
** producing more of the tournament closer to the tournament is not the recipe for success
Cody Voight, VCU ’14.
Locked