2021 WORKSHOP: Thanks and General Discussion

[reindeer]

WORKSHOP is a huge project and this year’s iteration would not have been possible without the work done by everyone involved: the 27 writers, 11 editors, and 5-person logistics team. Thanks also to everyone who played, staffed, playtested, or generally helped out.

I’ll leave it to the subject editors to specifically thank and highlight the writers who excelled in their categories. For my part, I would like to thank all the members of the logistics team, particularly Will Grossman, who led it. The 2020 WORKSHOP wrap-up post said to hire Will Grossman and I really can’t put it any better than that. He was thoughtful, responsive, conscientious, and capable throughout the entire process; I cannot imagine this set coming together without his work. Special thanks also to Em Gunter, who helped organize the playtest mirror, and Jacob Egol, who worked on packetization with unflagging energy.

All of the editors — Adam Fine, Athena Kern, Caroline Mao, Chris Sims, JinAh Kim, Jordan Brownstein, Kevin Wang, Will Holub-Moorman, Wonyoung Jang, and Zach Knecht — in addition to their subject-editing, also helped plan, make decisions, and solve problems on a wide array of production tasks. This was a team effort and they were a great team. In particular, thanks to my co-head-editors, Kevin and Wonyoung, and to Caroline, Chris, and Jordan for stepping up in critical moments to write outside their categories. Extra shout out to Caroline, who despite being the only editor responsible for a full 4/4, also somehow found leftover energy to write arts questions.

I also want to thank Ophir for the work he has done, both for WORKSHOP specifically and in all the years he has been developing advanced stats and other tools. Ophir thinks deeply and carefully about important questions that are often overlooked, and this set, and many others, are better for his work.

I very much hope to see this set happen again next year. There will be another public application form for writers in the spring or early summer, so please look out for that if you are interested in writing for WORKSHOP 2022! And if you’d like to contribute in some other form, let me or another editor know, or email quizbowlworkshop at gmail dot com.

This thread is for general discussion - have at it! I'd like for the public discussion to be as constructive as possible; keep in mind that many of these questions were written and/or proposed by new and learning writers.

[caroline]

I was the literature editor for this tournament. I had sixteen (!) writers: Raymond Wang, Henry Goff, Vittal Bhat, Jack van Nostrand, David Bass, Kevin Jiang, Kiara Pornan, Io Gilman, Payton Schubel, Aruna Das, Elizabeth Grace Ho, Christopher Shin, Milan Fernandez, Ananya Tadigadapa, Felix Wang, and Manu Sundar. Whether you wrote one question or seventeen, I enjoyed reading and giving feedback on your work. You all had such a wide variety of ideas and interests, and it was a delight to get to work with such a diversity of writers.

My only regret is that I couldn’t give you all more individual attention. If you wrote 4 or more questions, I will be emailing individualized feedback to you at some point in the future (after I have had some actual sleep and maybe a few mirrors); if you wrote less than that but still want feedback, hit me up and we can talk.

A few shoutouts:

• Payton: on top of being an incredibly talented writer with a passion for everything you write about and a stellar work ethic, you are likely the most proactive writer I have ever met, from responding to suggestions so fast I had to ask Olivia “did you leave a comment on this? I saw it in my inbox but now it’s gone” to adding several PGs to leaving helpful comments on others’ questions.

• Raymond: many thanks for your consistently brilliant ideas that make me go “ugh, I wish I thought of that,” high level of technical polish and thematic coherence, and agreeing to guest-edit 5 questions (!) and then doing it so perfectly I was like, “Why am I here?! Is there anything he can’t do?” Candide is still boring, though.

• Henry: I was absolutely floored when I counted how many lit questions you’d written; it felt like a million more because I’d learned so much from them. Always entertaining and creative ideas drawing from an interesting variety of sources—if you ever wondered why I took embarrassingly long to send feedback, it was because I was busy reading the works you clued.

Many thanks to everyone on the WORKSHOP team, especially Olivia Murton for her diligence and patience and Will Grossman for keeping our logistics organized! An extra big thank you to Ophir, who is so absolutely lovely and talented and thoughtful. I don’t think I can put into words how incredible he is. And thank you to my predecessor, Olivia Lamberti, for laying down the work that allowed me to be here today and inspiring me to do for others what she did for me. I would like to apologize that I did not honor Olivia’s 2020 WORKSHOP farewell post, as I failed to include a single “the x from y” answerline.

I hope you enjoyed playing 2021 WORKSHOP! If you have any feedback, questions, comments, et cetera, I encourage you to post it here. My DMs are also open.

[cwasims]

Despite the roughness around the edges, I hope people enjoyed the European and Other history questions today! Many thanks are in order to the writers, who did an exceptionable job.

My first thanks has to go to Kevin Thomas, who wrote an astounding 9/9 in these two categories, including almost the entirety of the ancient history and prehistory/archaeology questions. His deep knowledge of those categories as well as a willingness to (I gather) also venture a bit outside of his comfort zone topic-wise were both very appreciated by me. He also managed to pay attention to several editing/feng shui issues in the questions that I might’ve missed otherwise.

Marcell Maitinsky saw grew a lot in his writing abilities over the course of writing for this tournament. I ended up also editing several questions he’d written for world history, and he has areas of deep knowledge that he did a good job of working into tossups that ventured into some more unusual territory in the history distribution.

Jack Van Nostrand wrote several excellent questions that piqued my interest in topics I knew relatively little about while very consistently producing well-written questions that generally required little editing on my part.

Henry Goff, Elaijah Lapay, Victor Pavao, Felix Wang, and Raymond Wang all made smaller contributions to my history categories that were very welcome – all of them touched on interesting themes and topics and everyone was very receptive to feedback.

Please feel free to reach out if you have comments on the categories I edited or on the questions I wrote, both in history and in other categories.

[Crazy Khan Tech]

I hope everyone enjoyed the American and world history! Huge thanks to Chris and Jordan, who helped out when I was falling behind with the American and world, respectively.

All the writers for these two categories submitted solid questions with a high level of quality right out the gate. Kevin Thomas deserves special praise for writing a huge quantity of consistently solid questions across multiple time periods and geographic locations.

Please feel free to DM me with any comments on these categories or questions you might have about anything. Thank you to the writers, the logistics team, my co-editors, and playtesters!

[wjg]

I served as the logistics coordinator for this set. I was fortunate to have a team of 4 extremely motivated and enthusiastic logistics team members: Jacob Egol, Em Gunter, John Nienajadlo, and Kiara Pornan, all of whom made extremely valuable contributions to the logistics for this set. I would particularly like to highlight Em's excellent work in helping to organize the playtest mirror, Jacob and John's excellent packetizing contributions, and Jacob and Kiara's ability to manage logistics contributions while also writing for the set. The efforts that all of the logistics team members made goes beyond that, however, and I believe they will all be great contributors to future quizbowl sets.

I would also like to thank all the writers and editors of this set for their excellent questions and hard work. I had worked with many of the editors on sets in the past, WORKSHOP 2020 or otherwise, but I greatly enjoyed getting to work more closely with all of you on this year's set and am continually impressed by your question writing and editing. I'd particularly like to thank Olivia Murton and Kevin Wang, whose responsibilities most directly overlapped with mine, for all of their logistical contributions. I'd also like to thank Ophir for his excellent contributions in setting this set up for his advanced stats, and all of the mirror hosts who I've communicated with (and continue to communicate with) for hosting their upcoming regional mirrors. Lastly, I'd like to thank Alex Damisch, who was my predecessor as WORKSHOP logistics coordinator and my mentor on the set last year. I learned much of what I know about running the logistics of a quizbowl set from working with Alex last year, and I likely would not have been able to lead the logistics team this year had I not had the experience of working with Alex in the past.

I hope to you all enjoyed the set, and I hope that anyone who is considering it applies to be a part of WORKSHOP 2022!

[34 + P.J. Dozier]

Hey all! I just want to say a few words –– some of gratitude and acknowledgement and some of my own writing/editing philosophy –– before discussion gets underway.

I co-sign everything Olivia said in her introductory post –– all the editors, logistics team members, and writers were vital to the successful production of this set, which was obviously borne out of less-than-ideal circumstances. I want to first thank Olivia and Kevin immensely for serving as co-head editors with me. This was my first time serving in any head editor role in the production of a set, and, while I certainly was not perfect in a lot of ways, Olivia and Kevin were patient, kind, fun to work with, and overflowing with wisdom and guidance that I will take away and apply to any future projects I work on.

As far as the other editors go, I want to especially thank Caroline, Jordan, and Chris for stepping up and writing a lot of arts questions in crunch time when I had a lot of things on my plate that prevented me from doing as much as I wanted to. I also want to thank all my writers –– anyone who has written for me can tell you that I am extremely fastidious to a fault in regards to my standards and workflow for question writing, and they still managed to push through the learning curve, take my feedback, and grow extraordinarily as writers, while also teaching me a lot of really cool things. Jacob Egol, Raymond Wang, Henry Goff, and Eric Gunter in particular produced some amazing questions that required very little editing, and they were extremely communicative, punctual, and open to feedback; you would be lucky to have any of them on your writing team.

My general philosophy as the arts editor was to build on my past subject editing experience –– which often skewed more adventurous and novel –– by finding a better mix between “meat-and-potatoes” content and the spicier fare that I am wont to write about. All throughout trying to meet an optimal ratio, I also wanted and pushed my writers to include as much art made by and for womxn, POC, members of the LGBTQ+ community, etc. and expand the canon’s conception of “art” –– this is, for example, where the above-average amount (read: greater than 0/0) of fashion/cosmetics questions as well as the questions on artists such as Esperanza Spalding and Kanye West came from. Hopefully this approach made for an enlightening and eclectic experience –– I would love to hear more on what was executed well and what could’ve been executed better.

[TaylorH]

This was a fun and well made set. Thanks to all the writers, editors and logistic people that made it happen! I think WORKSHOP can and should be a staple tournament for the community going forward.

I will give my general thoughts on the tournament by category here, and note questions that made me pause in the other thread. Subjectively, this tournament felt a just a tad easier than ACF Regionals last week (though our teams bonus conversion at WORKSHOP was a little worse, suggesting otherwise). This feels like a good difficulty target to me. The answerlines and clue selection in most categories felt a little less adventurous than other recent tournaments (with the exception of the OArts), which understandable for having many first-time writers.

Literature: For the most part I liked the lit in this set, but there were a few types of questions that didn't work for me. There were a few questions on more common answerlines that felt more or less like high school questions with a very challenging extra textual leadin tacked on. The Frankenstein question comes to mind with this: a drama adaptation followed by a clue about a character climbing a glacier, which is, to me, one of the most memorable plot clues about Frankenstein. I will note a few more of these in the other thread, but in general I think that many of the toss ups on high school-level answerlines could be made harder to make them feel more natural next to harder TUs like Burger's Daughter or Antoinette Cosway.

The other minor quibble I had was the somewhat frequent use of character names for the hard part of bonuses. Off the top of my head, this happened in the Long Day's Journey bonus, the Moliere bonus, The Crucible bonus, as well as one or two I'm forgetting. Now, you could argue that these are all important works so the strong lit player should be able to remember these names and score the points, but the fact is that multiple people on our teams have read the works in question, and could tell you the general plot and memorable scenes in each, but character names tend to leak the brain faster than post other elements of literature, at least for our team. It feels a bit like the question are being made artificially hard in these cases. Those hard parts would function better if you asked for things relating directly to the plot. For the O'Neill bonus, "wedding dress" or "morphine" or even "Swinburne" feel like more natural and memorable parts to me than trying to remember which one is Jamie or Eugene. Maybe there are lit players out there who like remembering character names, but in my experience in this tournament and elsewhere, these bonus parts really just elicited groans.

History: I don't really know enough to say much about the history, but it seemed pretty solid, though a little standard. There were a few difficulty spikes that have been noted elsewhere, like Eritrea and Goryeo. It seemed to be testing knowledge well, as the the good history players in the tournament were buzzing early and often.

Science: I'm useless on 60% of the science, but in general I liked hearing the questions I understood. Again, a few difficulty spikes (venom, proton decay, the antifreeze protein bonus), but mostly things felt appropriate and interesting.

Arts: The art in this set was very strong and a highlight for me. The OArts stood out from most of the rest of the set as both more interesting and more challenging than most of the other 1/1s. This largely was a result of Wonyoung's approach mentioned above, and I think it was largely a success. The painting and Oart felt like it skewed quite modern/contemporary, but this is an editorial choice I can respect. The music and painting felt a little more standard than Oarts, but there were certainly a number of delightful moments (the "nails" art bonus comes to mind).

RM: I don't remember much of the myth in this set, but it didn't seem to cause any problems except for occasion transparence issues (volcano/mountain comes to mind). The religion felt like it had a slight skew towards practice over texts/theology/historical stuff.

PSS: I really liked the thought in this set, and especially the social science. I appreciated the numerous contemporary books/studies that were clued in lieu of the more boring usual historical fare. There were a lot of pretty great new ideas in these categories that I look forward to reading up about latter.

Other stuff: I don't remember much of the "other" content, though I remember liking the L'Oreal and the New Mexico questions.

I'll say more in the specific question thread, but I do think this set was very successful and can be made even better with some small adjustments.

[VSCOelasticity]

I thought this set was across the board fun and interesting! Thank you to all the writers, editors, staffers, logistics people, and anyone not in those roles that helped make the set & playtest mirror happen. I hope to play more questions written by some of the younger writers who worked on this! I'll post the more detailed feedback I've left on Discord for those that are not in the playtest server in the other thread.

[Santa Claus]

I was the biology, other science, and myth/legends editor for this set. I was very pleased with the contributions from our writers in my categories, which brought in a lot of good ideas. The result of the many good submissions from our writing crew is a distribution that has many original ideas (though unfortunately a few were scooped by Regs) and some number fewer retroddings of better known content.

I was also co-head editor for the set, and would like to thank my fellow editors for their work on getting everything done, as well as our excellent logistics team for their work handling our mirrors and our first site. Olivia has already expressed many of my thoughts better than I could, but I'm happy to have worked with her and Wonyoung on what was also my first time working as the head editor of a set, and with all of the editors and writers.

I would like to highlight Kevin Thomas, who ended up contributing a majority of the myth and many of the biology questions as well (to the point that those distributions have been shaped by his work), and David Bass, who provided many good other science questions. Both these writers were very open to discussion and feedback, and the set is better for it.

I also received many good questions from Adi Basu-Dutta, Alex Li, Annie Lin, Brandon Weiss, Christopher Shin, Eric Gunter, Jacob Egol, Kevin Jiang, and Raymond Wang. I greatly enjoyed editing this year's set and hope to see much more work from these writers in the future.

[AGoodMan]

First, I want to thank all the editors and (especially) the new writers for creating a very enjoyable set! I think overall, I had a better experience on this year's edition than last year. I also really appreciate the passion and the workrate of the new writers, which should not be glossed over.

I believe this year's set was more successful than last in toning down overall difficulty. I think part of why I did not enjoy last year's set as much was because the set overshot its intended difficulty (and thus I guess I had misplaced expectations going in) and it seemed to have relatively stingy powermarking. This year's WORKSHOP generally had better control in terms in-question clue difficulty, and also had a slightly more generous powermarking philosophy, which I support. I think the pros of more powers outweigh the cons, at least from a player's POV.

I will comment more extensively on singular questions noteworthy of praise in the Specific Questions discussion, but I generally really liked the beliefs category. I think Kevin and Athena, along with the writers, did a great job of asking about fresh/creative ideas with good difficulty control.

On the other hand, I think the relatively large variation in answerline difficulty remains to be a point of improvement for this set. From what I understand, this year's WORKSHOP sought to emulate ACF Regionals in difficulty. I believe the set probably achieved that vision, but did so with a greater variation in answerline difficulty than Regionals. Specifically, the TU difficulty in history felt "jagged" to me in a fashion similar to (or maybe even a little bit more than) last year's history, for which I was pretty surprised to hear a TU on Malawian history. Some of these have already been mentioned by others, but some tough TU answerlines that come to mind from last week include Goryeo, Ilkhanate, and Trinidad & Tobago. In the recent iterations of ACF Regionals, I would think Ilkhanate is probably on the upper end of acceptable answerline difficulty and would not expect to find TUs on Goryeo or T&T. I think such an approach of "large answerline difficulty variety" isn't wrong, but it also probably carries a greater risk of frustrating players as it's more difficult to calibrate what difficulty the set is aiming for. Smoothing out difficulty, especially in history (I say this because I feel most confident to comment on it as opposed to science or literature), would improve player experience.

Generally though, I want to reiterate that I am really impressed by the work put in by new writers who found some really cool and interesting clues (and editors who recognized such clues and included them)! One specific example is the leadin to the North Carolina (history) TU.

[Votre Kickstarter Est Nul]

I don't have a ton to say that isn't sorta sussed out in my specific questions post, but I'll try and add something. First, as others have already said, thanks for the fun set! Despite playing with just Darren, and therefore combining to punt a bunch of stuff, I was never really irritated, downtrodden, or overly fatigued.

History: I thought the history was well executed, pretty standard (I don't mean that as an insult, just as a stylistic note, and it's what I think is probably logical to expect from a set production format like this one), though I thought it skewed a bit easy in the middle parts at times (Philadelphia, Poitiers, Afrikaans, etc). That's almost certainly better than going too hard. I don't recall there being many hard parts that seemed unreasonable. I'll try and think of subdistro issues, though pardon my laziness in relying half on my notes half on my memory. I think we heard a single east asia TU, including none about China, yesterday, which seems subpar. We heard Korea and some East Asia adjacent content (Malaysia, Dutch Indonesia) though the latter was European History. I think the only East Asia content was the Song Dynasty bonus and the Kenmu Restoration bonus. I'd probably repacketize to move some stuff up.

It also didn't seem like there was a ton of social history, at least in us history, where I recall feeling this. Felt very political-leaning, except the tossup on Wells and disabled people. If I'm remember (I have them written but could be mistaking the content) there were political/war TUs on Vietnam, Jackson, North Carolina, Japanese-Americans, China, airlines (though I suppose that's business). Maybe that's enough, maybe there was more in the bonuses, I don't quite recall. I just like it a bit more is perhaps all this opinion boils down to.

Arts: I liked the increased fashion. The other arts felt good in general. It isn't clear in the forum post, but I assume there was scaled back classical music here (which I assume was combined with jazz?). I'm unsure how I feel about that, though I don't buzz on these things, so my opinion isn't really based in much except sensing a change during the game. The visual arts was enjoyable.

SS: I know so little of this, but, I agree with Taylor that it was well done and enjoyable.

EDIT: added the SS stuff.

[Smuttynose Island]

Thank you WORKSHOP writers, editors, and logistics people for putting together a solid set! It was an enjoyable experience.

Overall I thought that the technical execution of the visual art in this set was done well. The tossups were well-written, well-constructed, and interesting. Nonetheless I wish there had been more questions in the first 10 packets in the vein of the Netherlands and the Wilde/Perseus/mannerism bonus. Questions that clue pre-1850s influential non-"Old White Men" artists or clue modern responses to pre-1850s art. I think that such questions are a great way increase the representation of historically underrepresented groups in the "Arts" while maintaining a balanced temporal sub-distribution. In particular, it felt like there was an overabundance of visarts questions on the 1840s-1920s in the first 10 packets perhaps at the detriment of pre-modern and classical art*.


*I'm a big fan of Impressionism and the interplay between Japanese and Impressionist art, but it seemed odd to have two questions (bathing and Berthe Morisot) built around/drawing on this theme.

[meebles127]

It was a pleasure to do logistics for this set. Thanks to everyone who made the first open mirror a great experience.

[TaylorH]

I moderated this set yesterday and I just want to say that I appreciate all the improvements made based on feedback from the playtest mirror. I noticed quite a few changes that improved an already solid set and made it even better

[Zealots of Stockholm]

I thought this was a strong set with lots of interesting content. I think it hit its difficulty goal pretty well overall. Powers felt pretty stingy to me, but that was to be expected based on last year, and though I think bonuses in this set were overall harder than regs this year, I think they were in line with regs 18-20 pretty well, so I would say job well done on that. I'm going to give some general thoughts on the categories I'm somewhat competent in. Sometime tomorrow (Monday), I hope to go through the packets we played and give individual question feedback in the other thread.

Literature: I have some complaints here, but I want to preface them by saying that I thought throughout the set that the literature questions were interesting and well written from a content perspective, which I appreciated and is important. I also understand how much work is put into a set, and Caroline editing 4/4 is no small task. I thought the tossups were strong overall and had some really interesting ideas (Eatonville comes to mind!), and generally felt in line with the rest of the set difficulty wise. With that said, I really felt that throughout the day, literature bonuses seemed out of line difficulty wise with the other categories that I'm competent in, both in middle and hard parts. I feel that my team is pretty good at literature (I would personally call it one of our better categories, especially on bonuses), but we often struggled to 20 consistently. When its a sub-area that I'm weaker in, like Ancient lit, I expect this, but it felt like there were consistently middle parts that were hard to pull even in areas/subcategories I feel pretty confident in. Some of the hard parts felt quite out of line as well, and wouldn't have been out of place at nats in my opinion. But in general I didn't expect to get a ton of 30s at this difficulty so most were probably fine. There were a few bonuses that felt on the easier side (the Baldwin bonus in R10 comes to mind), but I think it was hard to tell if they felt easier compared to the rest of the set/the target difficulty, or just easier than the other literature bonuses. Again, I thought this category had lots of interesting stuff I'm excited to read/learn about, and I'm happy to provide examples in the specific question discussion and/or be proven wrong by the majority opinion of others or the conversion statistics which were being tracked afaik.

Arts (mostly visual): I thought this category was quite strong, and a highlight throughout the day. I do think some of the tossups trended quite hard, and that a few were sort of confusing/frustrating at game speed, but I'll discuss those in the other thread. I agree with Daniel's comments on the subdistribution, and I also found it kinda weird that both Cassatt and Morisot came up in separate questions. Many of the questions left me thinking "wow, that's a great idea." I also appreciated the increased fashion content. Even though I'm not very good at these questions, I found them consistently interesting in this set (and often in other sets as well), and think this is an important area for canon expansion in the arts distribution going forward. Also, while I appreciated the increased jazz (I'm bad at these questions and music, but jazz is a lot more interesting to listen to), virtually all of the jazz answerlines we heard seemed on the upper end/pushing what is acceptable. While all are acceptable as your hardest jazz answerline in a vacuum, my opinion is that you need a range of answerline difficulty, and this isn't quite achieved in the jazz tossups we heard (rounds 1-10). Also, keep asking Kanye as fine arts, that's based.

American history: I thought this was a strong and interesting category, with lots of good ideas. I studied (mostly US) history in undergrad but don't really study anything "for qb" and found the questions engaging. I don't know as much about other history categories but found several other questions interesting across the board.

Thought: I've recently started getting into this category, but I thought the questions were pretty interesting. Lots of stuff I'm excited to learn about. I'll mention this in the other thread, but our team really wasn't happy with a protest resolution on the "charity" question. Other than that, fun category throughout the day.

[Shahar S.]

I want to praise how tightly edited the science on this set was in terms of difficulty. This was the first set I played this year where I felt like the science was engaging without being inaccessible. It made my experience playing this set as a developing science player really enjoyable, so thank you.

[Santa Claus]

I want to praise how tightly edited the science on this set was in terms of difficulty. This was the first set I played this year where I felt like the science was engaging without being inaccessible. It made my experience playing this set as a developing science player really enjoyable, so thank you.

<3

[db0wman]

I thought this set was quite well-done. I had a lot of fun playing.

The art content was enjoyable, especially the world art, although it tended hard at points. The two jazz tossups that I heard both seemed very difficult ("Art" as a first name and Esperanza Spalding). I'd be interested in hearing the buzzpoint data on how many buzzes on Art came before the giveaway and how much Spalding was converted. In general, the music felt hard at times. I also don't think I heard any piano classical music.

I think that moving jazz to the auditory arts 1/1 was a good idea. However, (unless jazz in auditory instead of other arts is a norm with which I'm unfamiliar) I wish that this had been announced beforehand.

[Sam]

This was a great set. Thank you to the writers and editors who made it possible. There were a few instances where I think questions could have been clearer. Examples that come to mind are calling the Vietnam War "an event," calling Mars "a region," the question about revolutions named for flowers, and the question on lesbianism, for the reasons Auroni gives in the other thread. All of these had interesting clues and were well-structured but the phrasing led players to second-guess themselves.

The range of questions was very good. Several people have brought up the Spalding question as being very hard. I don't disagree, but the fact we're all pointing to the same question is a good sign: a whole distribution of questions at that level is not great, but having one or two extra tough questions is a good way to introduce players to these topics. (For what it's worth, Spalding is someone I've heard of not as someone who follows jazz [as I do not] but as someone who will read magazine interviews in waiting rooms. Just to say she's difficult, especially for an entire tossup on her, but not impossibly esoteric.)