A serious discussion of ACF packet submission

[gyre and gimble]

As a current ACF Fall editor, it seems to me that the way to resolve this would be to group the massive number of packet submissions we receive into a consolidated final packet of 3-4 submissions. It would give us much more flexibility, reduce the number of replacements we have to write, and throw a bone to the teams feeling left out by not having their submissions even considered. We can still choose to write the vast majority or even the entire final packet based on the "good" packets we receive. Plus, this system would give teams a greater incentive to write questions that are actually good, as good questions from those packets are much more likely to make it into the final set than under the current system.

Isn't that what you should be doing anyway? That's usually how these things work.

When I edited WIT, Auroni combined all the packets we received. At least for this year's edition of ACF Fall, only two of the final packets will be combination packets, whereas 15 of the final packets are exclusively the submission of one team.

Not that I'm telling you guys what to do, but this is quite surprising to me. In general, packet combination has been the norm for most tournaments, but I assume if you're doing it this way you have good reasons for it.

I have to admit there weren't good reasons for it, but by the time I realized this it was a bit too late so we just combined packets for the last two. We didn't combine packets last year for ACF Fall so I just sort of assumed that's how it works.

[Tees-Exe Line]

Something that I was thinking of would be to have an expanded editing team and create two sets from the packets, creating an ACF Fall I and ACF Fall II like a month apart that teams would only need to submit the one packet for the ACF Fall I deadlines in order to play both tournaments.

Economics says that this is a good idea.

[Tale of Mac Datho's Pachycephalosaur]

Something that I was thinking of would be to have an expanded editing team and create two sets from the packets, creating an ACF Fall I and ACF Fall II like a month apart that teams would only need to submit the one packet for the ACF Fall I deadlines in order to play both tournaments.

Economics says that this is a good idea.

Is there enough space in a typical fall tournament schedule? It's always a little bewildering how many housewrites spring up, so I'd be a little concerned about schedule crowding. That said, if people will commit to running fewer housewrites (or at least not more), then perhaps this will work well. I, for one, would like to be on an ACF editing team next year, and the level of editing supervision that ACF tournaments seem to provide strikes me as a good way of training more editors.

One issue I see is that of timing. Would ACF Fall I then run in late Oct, and II in late November? That seems the only way to do this without setting packet deadlines (especially discount deadlines) unreasonably early.

[Frater Taciturnus]

Something that I was thinking of would be to have an expanded editing team and create two sets from the packets, creating an ACF Fall I and ACF Fall II like a month apart that teams would only need to submit the one packet for the ACF Fall I deadlines in order to play both tournaments.

A month after the current ACF Fall weekend is when the plurality of college teams are going to be in finals.* The first packet deadline is already the first week of September; if you move the first fall any earlier in order to put the second fall in late November you are moving the packet deadlines up so that the packets will start being due before anyone is even on campus yet.

*My biggest objection here is that this is a tournament for new players, almost all of whom are college freshmen, who not having had a college finals week before would be even less likely to want to spend part of it playing a tournament somewhere than the standard quizbowler.

[Susan]

Something that I was thinking of would be to have an expanded editing team and create two sets from the packets, creating an ACF Fall I and ACF Fall II like a month apart that teams would only need to submit the one packet for the ACF Fall I deadlines in order to play both tournaments.

Economics says that this is a good idea.

Practicality says that it's a more difficult idea than it may appear at first glance (which is not to say it's a bad or unworkable idea). The issue of when the two tournaments could occur has been covered above. Other issues--The questions that make it into the set were (as a group) the ones that were easier to edit--they came in earlier and/or they had acceptable answerlines and/or they were better-clued to begin with. The questions that did not make it into the set will (on average) take more time to edit. I've never edited ACF Fall, but I have seen a fair few ACF Fall submissions from Chicago D-n* teams over the years, and it's entirely plausible that large chunks of some unused submitted packets are simply unusable. This really starts to look like a gargantuan task, and I'm not sure giving an extra month (or even an extra two months) to get the second set done makes it feasible. Adding editors might help, but: 1) we usually try to introduce people to editing at Fall, and I'm not sure that would work as well with this scheme and 2) as anyone who has worked on a tournament with lots of editors knows, adding editors SOMETIMES makes things work more quickly but can often result in organizational morasses and less oversight by the head editor.

Requiring teams to stick to one lineup for two tournaments is also a potential source of complication (probably not as big an issue, though?).

*Speaking of this--back in the day at Chicago we used to have older, non-ACF-Fall-consuming players review newer players' packets before sending them on to ACF (or Illinois, for their novice tournament, or whatever). This was--I think--a helpful thing in that I was usually able to give the team whose packet I was reviewing a lot more feedback than anyone on the editing team would have time to do, and it (hopefully) made the editors' job a little bit easier in that I got to cut the annual tossup on Osamu Dazai from Chicago E's Fall packet. I realize not all teams are in the position to do this, but if you have people around who could provide some peer review, I really recommend trying it.

[Golran]

Perhaps a late January tournament and revive ACF Winter? There will be enough packets with enough accidental difficulty variation that the slightly more difficult packets can be put into ACF Winter, while the easiest packets can be kept in Fall.

[Tees-Exe Line]

Something that I was thinking of would be to have an expanded editing team and create two sets from the packets, creating an ACF Fall I and ACF Fall II like a month apart that teams would only need to submit the one packet for the ACF Fall I deadlines in order to play both tournaments.

Economics says that this is a good idea.

Practicality says that it's a more difficult idea than it may appear at first glance (which is not to say it's a bad or unworkable idea). The issue of when the two tournaments could occur has been covered above. Other issues--The questions that make it into the set were (as a group) the ones that were easier to edit--they came in earlier and/or they had acceptable answerlines and/or they were better-clued to begin with. The questions that did not make it into the set will (on average) take more time to edit. I've never edited ACF Fall, but I have seen a fair few ACF Fall submissions from Chicago D-n* teams over the years, and it's entirely plausible that large chunks of some unused submitted packets are simply unusable. This really starts to look like a gargantuan task, and I'm not sure giving an extra month (or even an extra two months) to get the second set done makes it feasible. Adding editors might help, but: 1) we usually try to introduce people to editing at Fall, and I'm not sure that would work as well with this scheme and 2) as anyone who has worked on a tournament with lots of editors knows, adding editors SOMETIMES makes things work more quickly but can often result in organizational morasses and less oversight by the head editor.

Requiring teams to stick to one lineup for two tournaments is also a potential source of complication (probably not as big an issue, though?).

All of the issues you bring up are obviously important and need to be addressed. My point was merely that there's both excess demand for Fall as evidenced by the number of teams writing packets that aren't used, as well as excess supply since it seems like there's more than one set of people willing to edit it. ACF obviously has a good record and reputation for quality, so it seems like various of the objections to this idea could be handled by annexing the place of one of the Autumn tournaments that happens already. I'm merely saying that the world seems to be clamoring for a large number of easy tournaments in the Autumn, and specifically, for ones put on and overseen by ACF.

* Note: reviving ACF Winter also seems reasonable but for whatever BS reason there doesn't seem to be as much appetite for slightly harder tournaments. But the same idea goes: there are plenty of tournaments being written in contemporary quizbowl; thus it would seem at least one additional one of them could be an ACF tournament.

[Adventure Temple Trail]

I don't think it'll be good for the health of regular tournaments if we indulge the seemingly insatiable demand for more easy tournaments in this way. Eventually, people need to move on from easier events and start playing the many regular tournaments throughout the calendar year. More teams, in fact, are doing regular tournaments right now than before, and I don't want us to send a signal that that progress ought to be reversed by adding yet another easy tournament to the already-large stable of Collegiate Novice, existing ACF Fall, frequent easier-than-normal fall housewrite (EFT, IFT, MFT), DII SCT, and MUT. It just indulges the fallacy that REGULAR IS IMPOSSIBLE and results in more separation between the events that Top Teams play and the events for everyone else - which the standardization of regular difficulty and expansion of quizbowl over the past 5-6 years have all attempted to successfully counteract. Zach is right that the schedule is maximally crowded right now, so the addition of Fall II would almost certainly displace an existing tournament. The thing about economically good ideas is that people might want things that are worse for the system as a whole, and "ACF Fall Rejects File: The Tournament" seems squarely in that category.

And you're all super-mistaken if you think that the burden of editing 15 or more new packets (or more - think about how much people would complain if their packet weren't used in the second ACF Fall either!) is in some way trivial - even at Fall, a large number of submitted packets are filled with straight up terrible questions, as any experienced editor will tell you, and Susan's right that the effort is only compounded after the 15-20 best submissions have already been eaten up. I'll also add that the idea of making an "ACF Fall" rejects set out of the unused packets isn't new; I've been hearing about it since like 2009, and it's been hard to take seriously that whole time.

Re: Susan's mentorship suggestion: I have mentored Fall packets for my team in the past and encourage non-Fall-consuming veteran players at other schools to consider doing so, provided that they have the experience to be helpful and respect question security.

reviving ACF Winter also seems reasonable

Since someone brought it up: It seems not-terrible to consider reviving ACF Winter in future years to displace other events in the late January window. I've found from the schedule-wrangling discussions of the past few years that putting a good tournament in that late January spot can be pretty difficult; it took a while for SUBMIT to fall into place this year, for example, and in 2012 and 2011 the tournaments in this spot (BARGE and TIT) were both rollovers from the fall. Even when a tournament does emerge then, it's usually due to the large efforts of a small number of people. So I guess it's worth asking: Is there a larger crowd of experienced editors than there was four years ago? And if so, does the arrangement of labor in which 4-6 editors from different institutions come together to edit a tournament (ACF style) result in a better product for that late-January tournament spot than a model which involves a smaller number of schools/editors work on a slightly rougher product?

[Victor Prieto]

The first packet deadline is already the first week of September; if you move the first fall any earlier in order to put the second fall in late November you are moving the packet deadlines up so that the packets will start being due before anyone is even on campus yet.

Why is this an issue? Teams generally know which people will have to write before the season begins, so I don't see why it's unreasonable to set the earliest deadline sometime in August. Writing during the summer gives people more free time to write the packet, too.

I agree that having two ACF Fall tournaments is a good idea, but this obviously needs much more investigation by the ACF editors + officers. I'd really like to hear from current ACF Fall editors what they think about this (since I don't think anybody else has dealt with as many packets as they did this year).

Of note: according to Stephen's post in the global thread, the packets used in the tournament went as follows, sorted by deadline:

-$50
Caltech A
Dartmouth A
Haverford
LASA A
Rice

-$25
Chattahoochee
Northmont
Washington A
Wellesley
Yale A
Yale B

$0
Michigan A
Maryland B
St. John's

+$25
Ottawa A

+$50
Maryland A (combined)
Stanford A (combined)
Stanford B (combined)
Chicago A (combined)

Eleven packets that will be used were already submitted by September 22nd. That's about two-thirds of the tournament already. It would appear that the rest were simply picked at leisure, depending on which ones had the least repeats. Of course, it's entirely possible that most of the later ones were mostly discarded because they were hopelessly unsalvageable, but I would hope that repeats were the main issue. The editors can shed more light on this after the packets have been released.

I wasn't sure about reviving ACF Winter at first, but after thinking about it... Perhaps two separate editing teams can sort incoming packets based on difficulty. The easier packets would go to ACF Fall, while the harder ones go to ACF Winter (Ian's idea). Repeats that are submitted late, but are still usable, can go to the other tournament. Doing this may not be possible with even ~50 packets being submitted, this may have to wait until the pool of submitted packets reaches sixty or so.

This plan would be dependent on a large number of packets submitted being actually usable.* I don't personally have any editing experience, I'm just proposing an option based on my observations. I'm sure there are more problems with this idea, can anybody point out more foreseeable flaws?

*I wrote most of this before seeing Matt Jackson's post, which apparently kills this idea. If submissions actually start getting better, than I stand by my idea. But again, we'd have to wait to see what the newest edition of ACF Fall is like. Hopefully, somewhere down the line, there will be a large enough number of decent submissions for this to be possible.

Also, what's the big deal about having more tournaments than there is available in the schedule? Isn't there an abundance of question sets in the high school circuit? I don't actually know, because I never played in high school, but it seems that way.

[Tees-Exe Line]

Just wanted to jump in and say that the idea of a "rejects" Fall is indeed terrible. I am proposing two different tournaments with a similar ethos to Fall and where you only have to submit to one to play both. The idea that even if there's excess demand you still need to make people eat their vegetables instead of endless dessert is odd, but perverse
enough that I could get behind it.

[Adventure Temple Trail]

Yeahno, if ACF Fall is an easy tournament and ACF Winter is a regular tournament, it makes no sense to have teams submit one packet to cover both. (Also, teams play really different lineups at easy tournaments. Think of Hypothetical College P, who might not play their elite player "Schmeric Schmukherjee" at Fall, but would gladly add him to their lineup for Winter and employ his packet-writing services in doing so.)

Re: Victor's last point: I presume we're more concerned about economizing question sets in the college circuits because college sets really are a lot more labor-intensive to produce and have a smaller audience, so it matters that each one gets maximal use and there aren't people putting forth redundant effort.

[Tees-Exe Line]

To clarify once again, what I mean is TWO Fall-style tournaments. I agree that if the compromise proposal is to revive Winter, that should be a separate packet submission.

[Victor Prieto]

Yeahno, if ACF Fall is an easy tournament and ACF Winter is a regular tournament, it makes no sense to have teams submit one packet to cover both. (Also, teams play really different lineups at easy tournaments. Think of Hypothetical College P, who might not play their elite player "Schmeric Schmukherjee" at Fall, but would gladly add him to their lineup for Winter and employ his packet-writing services in doing so.)

I didn't consider different lineups for easy/regular tournaments... I suppose my idea might still be possible for holding two Fall-level tournaments. I thought ACF Winter was intended to be Fall+ difficulty, anyway, not regular.

Another negative that I hadn't considered is that it may preclude some editors that may want to play the set, but I think a potential Fall or Winter editor would probably not play easy level tournaments anyway.

[gyre and gimble]

Eleven packets that will be used were already submitted by September 22nd. That's about two-thirds of the tournament already. It would appear that the rest were simply picked at leisure, depending on which ones had the least repeats. Of course, it's entirely possible that most of the later ones were mostly discarded because they were hopelessly unsalvageable, but I would hope that repeats were the main issue. The editors can shed more light on this after the packets have been released.

I can shed some light now: First, we have to consider that there were more packets that came in by the no penalty deadline than there were afterward. Second, we have to consider that even if I received a bunch of bad packets early on, I couldn't just sit around and wait, hoping for better ones to come in later. I picked the best ones I had and sent them to the other editors so that the editing team could pace itself instead of editing everything after the last deadline. So the packets weren't really picked "at leisure" and we made pretty heavy edits on many of them.

I guess the dumb part is that I didn't really think about combining packets until pretty late in the game, but that doesn't help the case for two ACF Fall's from the same set of packets.

[grapesmoker]

Before you suggest another ACF tournament, please consider the fact that our humanpower resources are already stretched quite thin. It's damn hard to produce three quality sets per year, and I don't think another set is a plausible suggestion given the work all our editors are already doing. Assuming that enough editor-hours are going to materialize from our editor corps is a bit of magical thinking, in my opinion.

[Cheynem]

I think another ACF Winter like tournament is a decent idea (or a Fall like tournament) but it wouldn't (or shouldn't) be produced by ACF.

[Sun Devil Student]

putting a good tournament in that late January spot can be pretty difficult

Might it be easier to put a novice tournament (possibly the above-mentioned "second ACF Fall" though not necessarily an official ACF tournament) in that late January spot rather than a varsity/"regular" tournament?

Since more difficult tournaments require higher levels of knowledge and experience on the editor's part, there might be more people capable of contributing editing hours (and thus, compensating for whatever intrinsic difficulty that schedule spot seems to have historically had in drawing enough editing power to make it work) if that particular spot was filled by a novice-level event. Also, clubs often do get new members joining at the start of the spring semester (coinciding with freshmen's schedule changes making them available, etc), and having a novice-level tournament to take those new members to right away (and then again in late spring - MUT has filled that slot nicely) might be a good thing.

Also, after reading the long thread above: If vast portions of the submitted packets are typically unusable (wrong difficulty, non-pyramidal, etc) - isn't that another argument in favor of asking for only half-packet submissions? Yes, these teams need practice writing packets, but making them write 24/24 when they are still doing it wrong is just ingraining their existing bad writing practices further, as they repeat their poor-quality writing over and over to get to 24/24. What these teams need is to write a few questions and get feedback so that they start to change their bad writing practices (before they get engrained) and get better. Why not instead have them write 12/12 covering the same subjects, so that with fewer total submitted questions the ACF editors have more time to provide feedback on each poorly written question, thus helping those teams to improve their writing next time (instead of having made the same mistakes twice as many times, making those neural pathways harder to change)?

edit: For teams that clearly have this whole pyramidal packet-writing thing down (e.g. top-25 high school teams, other very experienced teams), I would say making those teams write 24/24 makes more sense, as their packets are probably not the ones editors are describing as "unusable" - and since those teams do twice as much work they can be given a correspondingly larger discount/smaller late-penalty for their packets

[Adventure Temple Trail]

Do people really hate regular-difficulty tournaments this much? I'm a bit taken aback. There are only like five per year anyway, and one is bundled with a DII tournament to boot! This proposal would functionally cut down the number of standalone regular events to three per school year by replacing one.

[Cheynem]

Everyone likes regular difficulty tournaments.

[Sun Devil Student]

Do people really hate regular-difficulty tournaments this much? I'm a bit taken aback. There are only like five per year anyway, and one is bundled with a DII tournament to boot! This proposal would functionally cut down the number of standalone regular events to three per school year by replacing one.

Well I thought the point was that the circuit always has a really hard time making that late January slot be the fifth regular tournament, and I think having a late January tournament works nicely in many academic schedules, and maybe making it a novice tournament rather than a regular tournament would make it easier to have that late January tournament. I kind of hope the fifth regular tournament would still happen somewhere else in the calendar (perhaps the summer if editors were more available then?).

On that note, why do regular tournaments need to be standalone? Parallel tournaments like D1/D2 SCT allow clubs with veteran and novice members to send everyone to the same place on the same day, which may make scheduling and logistics easier than going to two separate tournaments. This actually just gave me an idea: What if one of the mACF events used an "overlapping question set" like NAQT does, where every bonus has 4 difficulty-graded parts, the harder 3 of which are read to the D1 field and the easier 3 of which are read to the D2 field, with the middle 2 parts duplicated; and the tossups are the same in each field except with the lead-in clue removed for the D2 field. But that's for another discussion.

That said, if there is really no room for more tournaments and changing late January to a novice tournament actually wipes out the fifth regular tournament completely, it does seem to me that the hypothetical set of 4 novice + 4 regular + 1 bridging (MUT) tournaments per year has two advantages over the existing 3 novice + 5 regular + 1 bridge setup:
1) reduces the annual demand on the circuit's limited available elite-editor-person-hours
2) better accommodates the rapid expansion of quizbowl nationwide that is probably what's causing the number of novice teams to grow much faster than the number of varsity teams

When the circuit reaches a stable point where varsity teams outnumber novice teams and the inflow of new players is just balancing the departure of old ones, that would be a good time to switch one novice tournament back to a regular tournament. (At that point hopefully the expanded base of varsity teams will have increased the elite editing hours available to the circuit and made that switch feasible.)

Actually, it would be really nice to have 2 MUT-level "bridges" each year (with 3 novice and 4 or 5 regular events) but again that takes more editing work which might make it harder to do.

Everyone likes regular difficulty tournaments.

Unfortunately, not yet... but that may be an unavoidable side effect of rapid growth adding lots of novice teams that are still getting <10ppb and <18 tossups vs. empty chairs. Things will be different once we've reached that distant day when every English-speaking college has a well-established quizbowl team (all fielding 1-2 varsity teams and an occasional novice team each year).

[grapesmoker]

Please stop proposing unworkable ideas that only mean more work for editors and are not going to materially change tournament attendance. I guarantee you that we are not going to turn ourselves inside out to provide yet another novice tournament.

[Sun Devil Student]

Please stop proposing unworkable ideas that only mean more work for editors and are not going to materially change tournament attendance

I know, I know, just wishful-thinking-out-loud It would be nice to add one more tournament to the calendar, but yes, the circuit doesn't have enough editing capacity to do so right now and it would just displace something already present.

However, based on the impression given up-thread that editors are currently stretched to the limit trying to produce 5 regular tournaments per year (with the late January one being the most difficult), I just thought keeping the same number of tournaments and replacing a regular tournament with a novice or bridge-level tournament would make less work for the editors and make more people qualified to contribute to editing. It seems intuitive to me that the amount of knowledge and experience required to judge the relative difficulty of early and middle tossup clues at regular-difficulty is much greater than the amount required to do the same at novice-difficulty, and therefore the pool of potential editors who are qualified to edit (or subject-edit) a regular-difficulty tournament is much smaller. The large number of "good" high school packet sets compared to the small number of "good" college sets, for example. So I would think a novice-tournament editor should be able to find more subject editors to help than a regular-tournament editor, all other things equal.

That said, not being one of the potential elite editors myself, I might be missing something that you think should be obvious. If there is some special reason why it takes *more* editing hours to write a novice tournament than a regular tournament, please do enlighten me.

[Sun Devil Student]

Also, while I've been considering this primarily from a practical viewpoint, I do recognize there's an ideological component too (which maybe should get its own thread, actually, but it's a good discussion to have too) - namely, this:

Eventually, people need to move on from easier events and start playing the many regular tournaments throughout the calendar year.

In my mind, if it's easier to produce a high-quality novice tournament than a high-quality regular tournament, then this becomes the major objection to having the novice/varsity tournament balance reflect the novice/varsity team population balance.

While I personally hold myself to this standard (and have refrained from playing ACF Fall for this reason), I don't think it's reasonable to generalize this assumption to everyone. Not every new (or old) quizbowl player or team is serious about quizbowl. There are many casual players who enjoy playing at a level that challenges them appropriately but don't love the game enough to take it very seriously and dedicate themselves to improving. I would treat the novice tournaments as the equivalent of intramural sports - a more relaxed, less-skilled level of competition that's suitable for people just trying it out, or people who like it but aren't good enough (aka aren't willing to work hard enough) to play for real championships.

People will move on from the easier events when they're ready. When ACF Fall isn't challenging anymore, those who still want a challenge will go to Regionals. If they do that, but then continue going to ACF Fall and getting 24ppb while beating up on opponents who aren't worthy of them, then that's an issue for ACF to address. (For the record, I disapprove of those behaviors, but that's only my personal opinion.)

But entire teams who are getting only 15ppb at ACF Fall and 7ppb at ACF Regionals aren't going to enjoy Regionals (with few rare exceptions), and nothing we do can force them to. Believe me, I've tried, I know from personal experience. And if our circuit is growing so fast that we have a lot more of these novice teams than we do varsity-level teams, then I think a bigger intramural division (so to speak) is good. Some of them will outgrow the recreational level and join us in playing competitively, and those who don't are still contributing both tournament entry fees and tournament staffing to the overall circuit. The main practical difficulty then (again, if I'm seeing this correctly and not missing something big) is getting the best competitors to contribute enough editing power to the circuit to make it all work.