Resetting [pt. 3 of 6]: The Problem of Flakeouts

Anything that's on topic but doesn't fit elsewhere, including related events that might be of interest to quizbowl players.
Post Reply
Adventure Temple Trail
Auron
Posts: 2617
Joined: Tue Jul 15, 2008 9:52 pm

Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Adventure Temple Trail »

Needless to say, Matt Weiner isn't the only person who has dropped the ball on completing a question set. If someone had to make a movie about the quizbowl experience tomorrow, the montage of every flakeout even of the good quizbowl era alone would be long enough to become its own feature film easily. Even if everything had carried on as normal with NHBB writing during this past year we would still need to have this thread now, because that wasn't even the only prominent, community-damaging example this year. Charlie Dees's disappearance during the ostensible editing of the DEES college set -- for which we still have no explanation -- forced just under a dozen other people to create the set from nothing but the raw submissions in just three weeks. Before that, Andrew Hart's decision not to oversee a 2014 edition of Collegiate Novice led many people to compile ICCS on very short notice. Tom Moore disappeared rather than admit that the Fall Kickoff Tournament set wasn't going to happen. And that is, of course, to say nothing of burnouts and flakeouts among editors on sets that did get to completion, or writers who claimed they'd do a bunch and then didn't. That movie could keep a theater open all day.

Too many sets go under. Organizations and teams need last minute bailouts all the time. This is obviously unsustainable and each instance undermines our ability to be a serious activity in the realm of intellectually rewarding extracurricular activities. We have to readjust our norms so that this happens less.

This isn't new -- people have been making "quizbowlers never get anything done" jokes since I started playing seriously about eight years ago, and irresponsibility has been quite high for much longer than that. Why is it so bad? Is it that much worse in quizbowl than elsewhere? If so, what are the factors making it worse and how do we deal with those factors? I don't know if anybody has earnestly asked (or attempted to answer) these questions, either in historical or in comparative perspective. And I don't have a ready-made answer available, so if others have thoughts about this, please share.

I suspect that one reason is the sheer self-fulfilling prophecy that results from years and years of flakeouts. Failure to finish is so common among well-regarded editors that it's hard to have it affect someone's reputation -- it's practically expected. (In fact, many prominent people wouldn't consider disappearing just due to failure to finish one set, as careers are so long factoring in HS, college, and post-college afterlife that on a long timescale of 10+ years, almost everyone will have >1 failure.) It ties in with the generally ad-hoc / "no rules" attitude that has prevailed in much of college quizbowl and some pockets of elite high school quizbowl over the past few years, an attitude which I have consistently opposed.

Another is that virtually all sets I've ever been a part of -- in part due to overcrowding of responsibilities, in part due to the amateurish way in which many students operate by default? -- are completed as close to the last minute as is practicable. (This sometimes means "before dawn the same Saturday as the tournament" and sometimes means a few more hours before that; by extension, many sets are completed days in advance if they need to be printed on paper as part of a large print run, but in effect that is its own last minute.) So much so that I suspect it's relatively common for people to go "eh, we'll just get it done at the last minute" as crisis mounts in the weeks and days prior to an event, and let otherwise-manageable crises get unmanageable due to poor time management.

Another still is that there is currently no real way to enforce people getting projects done. This is in part because nobody has any money, so expecting people to pay compensation for failure to render services is unrealistic. (Beyond that, many question writers aren't making anything at all for their efforts -- consider a college student ostensibly making money for their team by contributing questions to their set, who gets no personal pay either way; it's hard to make that person care unless they have a lot of intrinsic motivation to begin with.)

How to react

This far it seems like the main responses among people who have messed up on a set have been (a) pretend that everything's fine / that nothing happened or (b) disappear. I would like us to have some procedure for contrition and reinstatement which isn't among those two options, though I'm hard pressed to figure out what that'd be in the absence of a governing organization and suspect that we want to instill more of a sense that abandoning ship is not okay as part of it. (I really liked it when the Penn team made amends for the bad logistics and incomplete writing/editing of 2012 Penn Bowl by writing Penn-ance -- a much better-written and better-run event -- the next fall, kicking off a commitment to competency and quality which lasts to this day. I'm not sure what other examples come to mind.)

Part of the process of doing this is that we just have to do it. We have to get less lenient with people about their failures to catch up. We have to get more people involved so that we don't have to keep relying on unreliable people, and reduce the workloads on everybody to minimize the chances of unreliability. (It should be assumed that good players will try their hands at wiring, for example.) We should get realistic about who is going to actually care enough to write a lot of questions for little to no compensation, and perhaps be less demanding that the entire club pitch in or whatever.

We also need to inculcate a more accurate attitude as to what writing a set entails, and what failing to write a set entails. When you fail to deliver a set you don't just impact yourself; you affect the ability of teams to fundraise who agreed to run your set, you threaten the prospects of competitive opportunities for teams in new circuits, etc. etc. -- tons of "knock-off effects". If you really don't care about any of that, you shouldn't be writing a set to begin with. (This isn't to accuse anybody specific of not caring -- I think most people do care and earnestly feel quite bad when they can't get a set together. Though there's no script for expressing it and making things right again at present.) At some level we need to both be better about giving more people a chance and better about keeping people who really blew it when given a chance away from future responsibilities. And "Top writers" need to be more honest and forthcoming with one another about who has been unreliable in the past.

Some people flake out because they're just not as committed as they thought they were, or realized their lack of interest only once the process has started, or develop a conflict, or lose Internet access for some mandatory trip -- if you do this, it's often not a big deal; just BE COMMUNICATIVE EARLY and BE HONEST with your co-writers and editors. It's far better to cut out with lots of time to go, and be honest about why, than keep stringing people along or become non-responsive -- this gives them plenty of time to do contingency planning..

The responsibility runs two ways. If you're head-editing a project, communicate with your writers. (If you're a person who oversees a million local events, such as Dave Madden, communicate with your writers!!!). It's ridiculous to assume everything's going fine when you have no evidence of that being true. Use a real-time software such as Google Drive (or the forthcoming QEMS2) so the progress bar is visible to all. Email people regular reminders as benchmarks and milestone dates pass by.

If I have to give only one piece of advice through all of this, it's the following: No matter who you are, GET HELP WHEN YOU SENSE YOU NEED HELP, NOT WHEN IT'S TOO LATE. Nobody gains from you trying to preserve your pride! By contrast, a lot of people gain from sets being finished. Despite the general crunch in writers, there are many people around who will be able to help you if you run into trouble, and all of them will be more able to do so with lots of lead time.

Sidebar: Can question writing be a job/career?
(I had a conversation with Mike Cheyne, Bruce Arthur, and Eric Mukherjee about this topic a few weeks ago; my thoughts here are distilled largely from that.)

I do not believe it is a worthwhile idea to ever entrust a full question set to a single person again. As I've said elsewhere, I believe the ideally-sized writing team consists of 3 to 5 dedicated people who produce the vast majority of a set, with additional people chipping in as those lead writers deem necessary/useful.

Furthermore, I also want to use this post to caution pretty much everybody against the prospect of making a full-time job or primary source of income out of writing a huge bunch of questions. (Not that I think anybody's going to up and replicate the arrangement Mattw had going ever again.) David Madden's insistence that this is a job that one person could realistically do within reasonable full-time-job parameters is -- I say as someone with extensive experience writing a lot of questions that David Madden has not had -- wildly optimistic at best and utterly misguided at worst. Churning out a ton of questions is a far more draining and involved task mentally than many kinds of more rote adminstrative or office work. Writing questions without constantly recycling ideas or clues takes creativity, and doing a lot of it in short bursts can be quite deadening to one's creativity. Without other people to bounce ideas off of (or time to read & listen to podcasts and such), one can easily get myopic about which things are askable due to lack of external stimuli to make one go "huh, that's something I'd never heard of" or "huh, that's something I don't know a lot about." In effect, this is because many of the issues of trying to be a full-time question writer are the same issues that affect attempts to make a full-time career of being a creative writer: writer's block, overreliance on stale phrases, and burnout can be utterly ruinous. (As an aside, this is why I've enjoyed both of my experiences editing packet submission events, for 2014 and 2015 ACF Regionals -- the many submissions from different teams forced me to use question ideas that I wouldn't have been able to generate if I sat there trying to select answers out of my own head, and gave me many good ideas for middle parts and early clues that I wouldn't have assumed were gettable since I didn't know them beforehand.)

This is not to say that I dislike or in any way frown upon the rise of the "quizbowl professional". We're now reaching the point where the number of people whose full-time or primary source of employment is in academic quizbowl exceeds the ability to be counted on two hands. The fact that people like Jonah Greenthal and Nick Clusserath and Jeff Hoppes (and Brad Fischer, now) are willing and able to make a career in quizbowl work is a great testament to how far we've come. But all of those people are able to "detox" by varying up the kind of work they do pretty significantly from day to day and week to week. I know I've talked with Jeff in the past about how he's capable of recharging his batteries, as it were, by switching up from administrative stuff to editing to low-level questions to high-level questions and back again. By contrast, maintaining a steady drumbeat (or death march) of high school-level questions as one's sole responsibility does not seem conducive to one's long-term mental health or creativity. And if that's what one wants to do, then embark on it with a team of other people checking in on you, helping spitball ideas, and the like rather than being a solitary reaper. It looks like Brad Fischer's arrangement is much better-constructed on this score than was the 2014-15 arrangement on this score, so that's all well and good.

Much more about NHBB specifically in the next thread.
Matt J.
ex-Georgetown Day HS, ex-Yale
member emeritus, ACF

Try my original crossword puzzles

User avatar
Cheynem
Sin
Posts: 6654
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Cheynem »

I'll have more to say to Matt's overall post, which I more or less agree with, but I find it a bit terrible that Andrew Hart's decision not to write/edit Collegiate Novice is being lumped in with the DEES, NHBB, and other flakeouts. Andrew, as far as I know, never announced or promised to write Collegiate Novice for Fall 2014 aside from some unofficial musings on the subject (as opposed to DEES, which had an announcement, mirrors, and call for packets). He simply chose not to do a tournament he had been doing for a few years; it would be roughly equivalent to if MUT was suddenly off the books but we didn't hear about it until January. Now, perhaps Andrew should have revealed his plans earlier, but I wanted to refute notions that this was a "flakeout."
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Cheynem
Sin
Posts: 6654
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Cheynem »

I agree wholeheartedly with Matt's points about not taking on too much, admitting you need help, and recognizing the problems that come flying solo as a "career."

I am not sure what to make of his contention that we are too "easy" on flakeouts and that some form of contrition/repenting is necessary. I'm glad Penn wrote Penn-ance, which was an excellent set, but is that the model for contrition that we would like to see? To make amends for DEES, should we expect Charlie Dees to write DEESENT? Shoudl Matt Weiner have to write his 1,000 tossup purgatory? I don't think many quizbowlers who react to a flakeout just "don't care"; it's just that we understand that our reactions and our powers are a bit limited. Getting mad, getting angry, demanding repentance isn't very healthy, productive, or in many cases, feasible. We could have gotten mad at Charlie Dees until we were blue in the face, but that wouldn't have saved DEES. We could choose to "ban Matt Weiner from quizbowl," but is that really the proper solution either?

There are a couple ways to respond to flakeouts in more concrete ways that perhaps would be a break from what Matt is describing (the 'everything's okay, let's just finish it' approach):

1. Ending the Superman Syndrome. Note, I do not present this as the right approach, but as an approach. In talking with a notable flake out, post flake experience, he was genuinely surprised that so many quizbowlers came in to bail out his tournament and thought it would have simply been cancelled (or written by "other people," which is a curiously optimistic belief). Indeed, that was a possibility, but for a variety of reasons, including the desire to save the quizbowl experience for others (Superman Syndrome), many people chose to write and bail out the project. This in itself is an admirable, nice thing--quizbowl was saved--but perhaps it is what Matt is suggesting is a problem--it creates the veneer that "everything is okay." I was reading an Bill O'Reilly article in which he said the problem with George W. Bush was his belief that "everything would work out because everything always has." I wonder sometimes if a project that is just allowed to fail, completely and utterly fail, would be a better wake-up call for quizbowl. Then again, College History Bowl had completely and utterly failed in 2013, and...well.

I don't inherently agree with this above point, but I thought I'd throw it out here.

2. Not Penance, but Contrition. I can't speak for Dave Madden and his legal crew, but if Matt Weiner returns to quizbowl, I honestly don't care if he writes a NHBB question again or pays him $10,000 or whatever. What I do want from Matt and any other flakeout is not so much penance--doing something to make things up--but rather a honest, contrite overview of where they went wrong and how they can do better. Let me use Matt as an example since he's fresh on my mind. A returning Matt Weiner should be, in my opinion, not concerned about "how I do make up for what I did wrong?" (which has the temptation to translate into MOAR PROJECTS), but rather "what can I do better?" Matt has many, many strengths. He can staff things, TD small scale things, write a few freelance packets/questions, collaborate with others, write predictions, etc. If he did all (or some) of those things, combined with perhaps a feasible project, that would be great. Again, I don't think this contradicts Matt per se, but I wanted to clarify what I think our attitudes should be from flakeouts. Sometimes a person bites off more than they can chew, flakes out, and realizes that question writing is not their bag, that they should stick to playing and staffing. With some degree of contrition, cool.

Finally, I think quizbowl should avoid enabling less than contrite flakeouts. If Matt Weiner comes back to quizbowl next month and announces COLLEGE HISTORY BOWL 2016, completely written by him, I think most of us would realize that would be a bad idea. But there would also be a lively corps of quizbowlers who would play it or who would express interest in it. That is an unhealthy attitude to have. I realize we can't expect every quizbowler/program everywhere to be up to task on who's a flake and what's a feasible project, but "those who know better" should not encourage or support projects that seem dubious at best.
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Sima Guang Hater
Auron
Posts: 1861
Joined: Mon Feb 05, 2007 1:43 pm
Location: Philadelphia, PA

Re: Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Sima Guang Hater »

Cheynem wrote:1. Ending the Superman Syndrome. Note, I do not present this as the right approach, but as an approach. In talking with a notable flake out, post flake experience, he was genuinely surprised that so many quizbowlers came in to bail out his tournament and thought it would have simply been cancelled (or written by "other people," which is a curiously optimistic belief). Indeed, that was a possibility, but for a variety of reasons, including the desire to save the quizbowl experience for others (Superman Syndrome), many people chose to write and bail out the project. This in itself is an admirable, nice thing--quizbowl was saved--but perhaps it is what Matt is suggesting is a problem--it creates the veneer that "everything is okay."
What you say has some merit, but I see it completely differently. I have bailed out several projects, and have been bailed out/assisted in turn with some of mine. In an ideal world, sets wouldn't need bailing out (and that's something I'm working on fixing with my more recent work by pacing myself), but that's kind of a reality of set production. Plus, I don't think leaving every set that needs bailing out to "die" is necessarily the right approach, especially when travel plans have been made and so on.

Also, pictured: Matt Jackson and I

Image
Eric Mukherjee, MD PhD
Washburn Rural High School, 2005
Brown University, 2009
Medical Scientist Training Program, Perelman School of Medicine at the University of Pennsylvania, 2018
Intern in Internal Medicine, Yale-Waterbury, 2018-9
Dermatology Resident, Vanderbilt University Medical Center, 2019-

Member Emeritus, ACF
Member, PACE
Writer, NAQT, NHBB, IQBT

"The next generation will always surpass the previous one. It's one of the never-ending cycles in life."

User avatar
Cheynem
Sin
Posts: 6654
Joined: Tue May 11, 2004 11:19 am
Location: Grand Rapids, Michigan

Re: Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Cheynem »

I'm not saying it's the right approach, but it's an approach, an approach that theoretically would disrupt the assumption that bailouts and flaking are the "reality of set production." To return to the sadly not-the-worst-incident College History Bowl 2013, if...somehow, Matt Weiner had managed to cobble together, through bailouts and format changes, a set of some sorts, he would probably get some bashing but not much else. The very finality of it, combined with travel plans and schedule rearrangements, was more impactful, and would hopefully have led to greater reforms/changes (it didn't, of course, so maybe this approach doesn't work either!).
Mike Cheyne
Formerly U of Minnesota

"You killed HSAPQ"--Matt Bollinger

User avatar
Skepticism and Animal Feed
Auron
Posts: 3190
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Skepticism and Animal Feed »

There is no shame in asking people for help writing a tournament. I feel this is a norm we should promote.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

User avatar
Lagotto Romagnolo
Tidus
Posts: 533
Joined: Wed Jan 23, 2008 12:43 pm
Location: Alexandria, VA

Re: Resetting [pt. 3 of 6]: The Problem of Flakeouts

Post by Lagotto Romagnolo »

Skepticism and Animal Feed wrote:There is no shame in asking people for help writing a tournament. I feel this is a norm we should promote.
I'd like to extend Bruce's point to a few other tasks. Warning: the following is also a personal attempt to exorcise old demons, so apologies for schmaltz or preachiness.

There is no shame in asking people for help editing a tournament, provided you maintain question security and keep it all under control.
There is no shame in asking people for help proofreading a tournament. The grammatical errors a person is most likely to miss are his or her own. So there's no point in pretending to be a superhuman who is immune to such concerns
There is no shame in admitting that you wrote a subpar question.
There is no shame in asking your fellow editors/playtesters/etc. for feedback on why they believe a question is bad. Conversely, there is no reason, when giving feedback, to tell someone that they shouldn't be writing a given category, especially if they made a significant effort. We're not here to put people down. Of course, editors are still free to choose their writers, but it's a jump from simply not-giving-an-invitation to going-out-of-your-way to tell someone not to write period. No quizbowler is a god; no quizbowler is the ultimate arbiter of question quality, in any category. Which brings me to a point which I believe is not said enough:

The primary goal of writing questions is not to prove that you're a great writer. It is to produce great questions that ask about interesting material in ways people find enjoyable. Thus, when writing and (especially) editing, your primary focus should be on producing solid questions, not on convincing yourself or the world that you are, by virtue of identity, a great writer/editor. Certainly not by going the solo route when asking for help leads to a better product. To put it another way, consider this: are people more likely to respect you as an editor because you can write a tournament solo, because you're an expert in your field, or because you have actually edited well-received tournaments?

Talk to people. Trade ideas. Don't take it personally. If you need a competitive motivation to write well, consider letting it be to write better and more enjoyable questions, not to fuel your own ego or become the greatest writer.
Aaron Rosenberg
Langley HS '07 / Brown '11 / Illinois '14
PACE

Post Reply