This year, I have somehow worked on multiple high school housewrites. Additionally, I had the privilege of being the head editor of the second incarnation of mRNA VACCINE, an editor and significant contributor to the IQBT Undergraduate Championship, and an editor for ACF Winter 2022.
This document is born out of the experiences, challenges, and frustrations with these various tournaments, some or all of which required frantic last-minute work to complete. It is intended as a logistics guide for anyone wishing to work on a housewrite (though much of it will apply to packet-submission tournaments as well).
II. THE GOLDEN RULE
Like everything else, quizbowl obeys Murphy’s Law – what can go wrong, will go wrong. For quizbowl writing and editing, this manifests in a few different ways.
A. Corollary 1 (the DART Law)– One or more of your editors will bail or lag behind in a way that affects set production. Inevitably. If you are a head editor, prepare for this eventuality. Named for DART, a 2022-3 set in which multiple editors failed to complete their work in a timely manner, some were completely incommunicado during the whole process, and at least one stated they were completely unable to edit their category the week before the first mirror of the tournament, despite being on the server for months at a time (but saying very little).
B. Corollary 2 – Everything takes much longer than you expect. For example, packetizing inevitably takes several hours at best, and errors are often made during the process. Do not save any actual editing or assembly for the week before the tournament – the set should be assembled, in packets, and formatted for the first mirror weeks in advance, and the most that should happen in the week or two before the tournament is some proofreading.
C. Corollary 3 – There are always more errors than you expect. Repeats (small or big) inevitably crop up during editing, necessitating replacements. This is particularly bad across categories (e.g. having both history and literature questions on Napoleon) because those are often covered by different editors. Factual errors, big and small, often sneak into a set, particularly when questions are written quickly. I am guilty of this. This is one of many reasons why writing early, diligent surveillance by a head editor, and playtesting are vital.
III. PRE-PLANNING
A. Assembling your team
A housewrite takes at least 8 months to write, after assembling a team. Generally, the editing team is assembled anywhere between 9 months to 1 year before a set is due to be played. A set needs a dedicated editor for each 1/1 of the distribution. Ideally, no editor would be responsible for more than 4/4 of the distribution, as this can become unwieldly. Editors should ideally have previous experience editing a well-received set. Most college sets have switched to an application process, which is reasonable. It is also reasonable to directly solicit help from people you would like to work with, who you know are capable of editing.
Writers would ideally also have previous experience, though for high school housewrites, this may not be possible and editors should be willing to teach and/or intervene heavily to fix a novice writer’s questions. In my opinion, applications for writers (including a writing sample) are very useful. And remember that the less experience a team has, the more important playtesting becomes and the more time should be allotted.
There is a common pitfall when selecting writers and editors to inevitably pick people who are very good at their categories. While this is reasonable at first glance, very good players often write questions far too difficult for their categories, and should be closely watched.
If possible, also consider having a logistician (separate from the editors) whose job it is to solicit mirrors and deal with payment. This can often be a full-time job on its own, and it can reduce a lot of headaches to have one person dealing with that entirely. Pay your logistician 5-10% of the total haul – it’s worth every penny. Make sure it’s a person who knows how to deal with W-9 forms from schools in particular, as these are often required for payment.
B. Selecting a Format
Before beginning to write, a few things must be decided.
- The difficulty of the tournament. In addition to saying “regs”, “regs+”, “novice”, etc., pick two or three previous tournaments whose difficulty you are attempting to replicate, to give a clear idea of what is being aimed for. Also - don't make the finals packets harder, that's a deprecated practice at this point, because it changes the nature of the game in the middle of a tournament.
- Tossup and bonus length. A strict line or character cap is important to keep everyone on the same page. NAQT IS-A sets use 300 characters per tossup. NAQT IS sets use 425. DART used 650, and PACE NSC uses 775.
- The distribution of the tournament, including geographical, temporal, and subject sub-distributions within each 1/1. This can be hammered out in a meeting between the editors, but it should be set early.
- How people are being paid. Set the percentage for head editing, logistics (if you have a separate logistician), writing, and editing at the beginning. Additionally, decide on a mirror fee. $15-20 is typical for high school housewrites, whereas $40-50 is typical for college, with ACF sets going for 60-70$ at the top end.
Google Drive is standard for writing sets, so I’ll use that as the example moving forward. Some people have access to QEMS2, which is an entirely different beast, and which I will not cover here.
Make sure to have one Google Drive folder for your tournament, rather than having stray documents on your drive – that way the whole folder can be shared and updated as needed. The folder should contain a spreadsheet of answer claims, documents for each subject (Science, History, Literature, Art, RMPSS, Trash/Geo/CE), and any other ancillary documents needed (a sample tournament announcement, a spreadsheet for accounting for mirror fees and payment, etc.). The name of each document, spreadsheet, and folder should contain the name of the tournament - when your collaborators search their drives, a document titled "Penn Bowl 2017 - History" is much more easily understood than "History".
The answer spreadsheet should have separate ways of indicating when a question is, at the very least, claimed, written, ready for playtesting, edited, and proofread. The sheet should be able to count these separately. A tournament I worked on did not distinguish between “claimed” and “written” on their spreadsheet, giving an inflated completion percentage due to claims counting as written questions. Additionally, try to put as much of the answerlines on one sheet as possible, so repeats can be searched for easily. At the very least, do not separate the tossups and bonuses of the same category on different sheets.
There should be one document per subject. For one tournament I’ve written, all of the unedited questions are pasted into one sheet, and then they are cut and pasted to subject documents when I edit them. However, your mileage may vary. A key thing, however, is that the documents should only contain one version of any given question. Do not keep older versions, and do not have multiple copies of any given question, as this can become confusing during editing and packetizing (and this is an issue that has cropped up multiple times). If an editor or writer insists on keeping older copies of a question, have them save them offline, rather than in the Google Drive folder.
The documents should include a formatted example of a TU and bonus, including:
- How to indicate prompts, alternate answers, and rejects
- How pronunciation guides are to be formatted
- How the easy, medium, and hard part of a bonus should be indicated (whether [e] or [10e])
- How author, subject, and sub-distribution tags are to be formatted
D. Communication
There should be a discord server created for the editors and writers. Some editors (John Lawrence, notably) prefer to keep the editors in one discord and have writers siloed off from the discord server and the tournament documents, so that they are blind for playtesting. However, this is somewhat nonstandard. I will assume that all the editors and writers are on one discord server.
There should be separate channels for each subject, where questions can be pasted for the group’s consideration. An editors-only channel should also be created. Separate channels for introductions, payment, mirror logistics, and off-topic are also reasonable. The schedule for writing and editing should be clearly posted (more on this later).
A pinned message for writers should be present in one channel, with clear instructions on the workflow. Writers should know how to claim a question, where to put completed questions, and how to use the spreadsheet. Editors should know what to do with edited versions of their questions and how to indicate that a question is edited, both in the document and on the spreadsheet.
IV. WRITING AND EDITING
After you have assembled your team, a schedule should be determined. My suggestions are these:
- Every question should be written 6 weeks before the first mirror
- Every question should be edited 4 weeks before the first mirror (and should be done alongside writing)
- The tournament should be packetized completely 2 weeks before the first mirror
- A read-through and final edit of the set should be done in the final two weeks before the first mirror
A minimum number of questions per week or two-week period for the group, divided among individual editors, is the best way to ensure your set is written on time and avoid a mad dash at the end. For a group of 15 writers, a reasonable assignment of 1/1 per week gives a total of 30 questions, and beats a schedule of 20 questions per week. Having per-person weekly assignments that add up to more than the group’s weekly assignment is critical – this gives allowances for writers not meeting deadlines or assignments. Note also that people tend to run out of steam near the end, for various reasons, and this should be noted going in.
Editors should similarly have deadlines overlapping with writing deadlines. If an editor is editing 4/4 or less (the maximum I outlined earlier), having them edit up to 25% of their distribution every month (about 3 packets worth for a 13-packet set) is similarly reasonable. That gives 2 months for the writers to get a bit of a lead and then 4 months for editing. However, as an editor I prefer, and highly recommend, editing the questions in my category as soon as they are written, or at least notating them with the changes I want to make.
As questions are being written and edited, they should be posted in the corresponding channel in the writing discord for commentary. This ensures that everyone has a chance to comment on questions and focuses the group on individual questions. Mistakes can be caught and easy edits can be done more quickly this way. It also creates a sense of solidarity among the writing team and encourages people to keep pace. Another way to do this is to have writing parties or contests to take a big chunk out of the set.
The head editor should be keeping an eye on all of the written and edited questions to ensure they are meeting difficulty standards as they are being edited. This should be done throughout the process, rather than at the end. Particularly with high school housewrites, it is common to overshoot difficulty, as the average high school writer is significantly better at quizbowl than the average high school player, and newer writers do not want to put “easy” clues early in their questions. The head editor should also note if any subject is lagging behind, and redirect the group’s efforts should be refocused on any deficient areas of the set.
If writers or editors are missing assignments, they should be approached (diplomatically) by the head editor. If it becomes habitual, editors should be fired and replaced early. Do not wait to confront delinquent writers and editors in the last month. Editors and writers should also be communicative about their schedule early – if you are an editor, and you have a major deadline, vacation, major event, another tournament, etc., tell your head editor beforehand and adjust your schedule accordingly. While I don’t presume to know everyone’s lives, it’s safe to say that we’re all very busy, but not so busy that writing 1/1 a week should be an issue.
V. PLAYTESTING
There are two forms of playtesting – hosting a playtesting mirror, or playtesting before your first mirror. This section will cover the latter. If you decide to host a playtest mirror, treat it like your first mirror and schedule your set to be entirely finished by then.
Playtesting is an excellent way to catch factual errors, fix the flow of your questions, and help iron out difficulty issues. To maximize the utility of playtesting, a few things should be done.
A. Pick the Right People. Selecting playtesters who are familiar with the difficulty being written for is vital. For high school housewrites, college players with experience writing and editing at that level are particularly helpful.
B. Read Questions Out Loud. This is the best way to catch tortuous language or grammatical errors. Paste your questions after they are read out loud for an additional check. Do not just paste the text of questions for playtesters to read them without reading them out loud first.
C. Clarify the Difficulty. Playtesting should be done in the mindset of an average team in the field. For tossups, make sure they are powerable by a knowledgeable player at the level they are being played. For a bonus, make sure that easy parts are sufficiently easy that 90% of the field will get them, and that hard parts are gettable and not too hard for the field. Note that these are also things you should be doing while editing, but playtesting is particularly helpful for this.
D. Follow-up on Any Comments. Subsequent edits should be shown to the group after playtesting, to ensure nothing is missed and that feedback was implemented appropriately.
Scheduling playtesting can be difficult. Some sets, like ACF Nationals and UG Nationals, had weekly playtesting sessions. This is a good strategy. If your set is writing 20 questions a week and editors are keeping up, one can playtest one packet worth of questions a week, and thus the whole set in 13 weeks, fairly easily. Complete playtesting and editing at least 4 weeks before the tournament is set to run, and make sure editors are making changes suggested by the playtesters as time goes on, rather than at the end.
Some sets are playtested after they’ve already been packetized – this can be advantageous for picking up feng shui issues, but requires packetizing the questions earlier than the above schedule. One could packetize the set with 2 weeks left and then playtest two packets a night over a week. However, for beginning writers, playtesting often reveals extensive edits that need to be done, and I would not recommend this approach. Also, this can be very demanding on the playtesters’ schedules and may not be feasible.
VI. FINAL STEPS
Now your set is written, playtested, and edited. Hooray! But you’re still not done. The last steps, packetizing and formatting, can often be very difficult. Leaving two weeks just for packetizing and feng shui considerations is useful for this reason.
A. Proofreading
Some sets are fortunate to have separate proofreaders. If this is your situation, then make sure the proofreaders are working on questions as they are being edited, as this saves a huge amount of time on the backend. Proofreading also should include adding interpuncts (particularly for protein names and abbreviations) and pronunciation guides – though an astute head editor should be making sure that subject editors are doing that as well.
If you are not so fortunate, proofreading can also be done in the last two weeks after the packets are assembled, though I would recommend leaving three weeks if possible, as it can be a laborious process. Each question should be read out loud, interpuncts should be added, pronunciation guides should be checked, and formatting should be finalized. Packetizing and proofreading can occur at the same time, if and only if you make sure that exactly one version of each question exists in the documents.
This is a head-editor checklist I created for mRNA VACCINE. You can create something similar for your set, to ensure every question gets clear scrutiny.
FOR EACH QUESTION
[ ]Check each clue for ambiguity in isolation
[ ]Expand answerline to ensure completeness
[ ]Pronunciation guides
[ ]Interpuncts
[ ]Make sure the indicator is as early as possible in the leadin
[ ]Minimize or eliminate compound and garden-path sentences
[ ]Use they/them pronouns
[ ]Make sure bonuses are two lines per part and TUs are <7.5 lines
[ ]Read out loud to catch mistakes
[ ]Check writer/subject tag
B. Feng Shui
Before the tournament is edited, the head editor or their designee should create randomized packet templates. The principles are usually that each half is balanced (e.g. 2 science in the first half, 2 in the second), and the trash question isn’t in the last five. There are various scripts available for doing this. Henry Atkins has one that was used for mRNA.
During editing, the head editor should also make feng shui notes on a separate sheet, i.e. Question X should not be in the same packet as Question Y, or Question Z should appear before Question W. Some editors have access to a complex feng shui tag system that allows packetizing while considering these constraints (this was used for UG Nats and ACF Nationals 2022) but I have no access to it.
C. Packetizing
After the questions are completed, if a more advanced method isn’t available, a group of editors should schedule a meeting and decide which questions go in what packet. This can be done in a spreadsheet in which each column is a packet and each row is a subject. Be particularly careful about ensuring that there isn’t a geographic or temporal skew to each packet (e.g. having a bunch of China content in one packet, having a bunch of modern literature in another, etc). This can take about two hours to do properly.
After the questions are assigned to packets, the final step is to move the questions from the subject documents to the packet templates. Create a subfolder in your google docs folder for the final packets and upload your templates to it. Then divide up the work among the editors – each editor can packetize their own questions. Remember, cut and paste, do not copy and paste! If you would like to save a backup of the question documents at this stage, that is reasonable. Remember, the online version should only have one copy of any given question at any given time. This prevents repeats from appearing.
Your set should be packetized 2 weeks before your first mirror. In the last two weeks, the editors should be proofreading at all of the final packets, reading them out loud to make sure that there are no grammatical or feng shui issues. One or two packets a day is reasonable for this.
D. Other Considerations
If you are planning to use an advanced statistics program like MODAQ, which requires conversion to another format, do not wait until the day before the tournament to do this – do it after packetization. Sometimes random errors and formatting issues affect the conversion. For MODAQ in particular, converting early and clicking through all the packets is vital (and is actually a good way to catch errors). With UG Nats this year, MODAQ did some odd things with spacing that were not caught before the actual tournament – it is vital to be able to deal with this kind of thing.
Some tournaments will have passwords for each packet, to ensure that the wrong packet isn’t read in any given round. I strongly recommend this method. Do this a few days before the tournament as well.
VII. AFTERMATH
Your set has run. Now people will want to talk about it. While my generation strongly prefers the HSQB forum (and I advocate making a thread if possible, particularly for college sets), most high schoolers prefer a discord server. For the server, make sure that participants have to identify themselves and what mirror they played before they can see the rest of the channels, and create separate channel for each subject.
Feedback on sets is often hit or miss, and you don’t necessarily have to listen to everything said. However, things like factual errors and ambiguities should ideally be corrected between mirrors, if nothing else.
Finally, there’s the consideration of payment. Invoices should be sent to schools promptly and payments should be collected as quickly as possible. When soliciting mirrors, the head editor or logistician should figure out what paperwork is needed to ensure this is done quickly, before the mirror occurs. Even given this, many schools (particularly colleges) are delinquent or at the very least slow to pay, and it may take persistent efforts to make sure you are paid.
VIII. CONCLUSION – SEVEN LAWS
1. Do everything early
2. Make sure your set has consistent progress and communicate extensively
3. Write, edit, and packetize the whole set two weeks before the first mirror
4. Fire people who aren’t keeping up, and stop shirking responsibilities
5. The docs should only have one version of any given question
6. Playtest your questions on people who know the subject
7. Leave enough time for proofreading
APPENDIX. COMMON PITFALLS
1. Editing only enough packets for the first mirror and putting off the rest. This inevitably leads to all kinds of issues during the mirror itself. Extra questions are often needed, or some format issue arises. Additionally, the lack of inertia means that the head editor has to repeatedly chase down the delinquent editor before the subsequent mirror to finish a set that should have already been finished. As an editor, do not do this. As a head editor, do not tolerate it. The tournament should be done and complete for the first mirror.
2. Not firing people. If one of your editors is not keeping up with the schedule, or not communicating, despite your entreaties to do so, fire them and find someone else. And make it clear that firing is on the table from the beginning.
3. Not calling for help. If you cannot meet the writing schedule, call for help earlier rather than later. Preferably around 2 months before your first mirror, rather than 2 weeks. Help may not be available unless you leave enough time.
4. Not accounting for tiebreakers. Your tournament is not done until every last required question is written, edited, and packetized. By required questions, I mean every single tossup and bonus that could be necessary in every possible world, including tiebreakers in each packet and any extra packets that are needed. There is often a temptation to put these off until later, or not account for them while writing. Do not do this.
5. Not accounting for life events. If you are on a flight, or college visit, or vacation, during the production of a set, do not assume that you will be able to write or edit during that time. This is particularly important in the weeks leading up to a set, when communication is vital, particularly from the head editor.
6. Taking feedback too seriously. This is especially true if a very good player says that a question was too easy. Check your stats – chances are they’re probably wrong. Also don’t automatically assume that a neg means that a question was wrong or confusing.
