WIT discussion

[Birdofredum Sawin]

I don't know when the tradition of performing tournament post-mortems died down, but I've noticed that most circuit tournaments just come and go nowadays without any comment. In any event, I haven't seen any discussion of last weekend's Berkeley tournament, the WIT. I have a few things to say about it, most of them (to be honest, all of them) negative. Before I do some hating, I want to say that these remarks are meant to be constructive, for the most part.

First, a minor gripe: The tournament lacked a coherent schedule. When we arrived, we learned that the TD had decided to have us play a double round-robin, even though there weren't enough packets for each team to play every other team twice. Frankly, I was amazed that anyone would have thought that this would be a fair or reasonable format. Fortunately, Seth Teitler stepped in and made the persuasive argument that it would be better to switch to a single round robin followed by a split-tier round-robin playoff. While this represented the optimal use of the packets, it was problematic for reasons I'll discuss below.

Much more important than the scheduling snafu was the problem of question quality. To wit: The packets were terrible. I won't cite specific flaws, though if anyone doubts my claim I'm sure any number of people at the tournament would be able to provide egregious examples. Obviously, there were inexperienced teams at the tournament, whom I don't blame for writing atrocious packets. That's only to be expected. Nor do I blame the team that wrote a ridiculously easy high-schoolish packet, since they thought that the tournament was supposed to be super-easy, or the teams that wrote questions that were much more difficult than the norm. Young players write bad questions; experienced players write questions that aren't quite right for the tournament's advertised difficulty level. These things happen. Over the years, the quizbowl community has evolved a surprisingly effective mechanism for dealing with such confusions. We call it "editing."

The most egregious fault of this tournament is that nobody seems to have edited it. As far as I could tell, the only serious editing done for WIT was the sprinkling of extra economics questions added by the TD. The early rounds were untouched but good, because they were written by competent teams. The later rounds -- perforce, the playoff rounds for the div 1 teams -- were also untouched, but horrendous. It's really ridiculous that Berkeley a) didn't write any house packets, b) didn't seriously edit the packets they got, and yet c) fielded three house teams.

This, really, is the heart of my complaint about the WIT. Teams invest time and money to attend tournaments. What did the Berkeley people do in their capacity as hosts? They cobbled together a poorly-conceived schedule which had to be reworked by competitors on the morning of the tournament. They bought about $5 worth of used books to serve as prizes. They didn't even bother printing out the packets, which unsurprisingly resulted in a potential crisis when the wrong round was read off somebody's laptop. They wrote no packets, nor did they appreciably edit the packets they received. On top of all that, they fielded three house teams, one of which was composed of veteran players who could presumably have done some of the work the tournament so desperately needed.

It's extremely important for the continuing viability of the circuit that there be a number of independent (i.e. non-NAQT, non-ACF) tournaments. These things aren't really all that difficult to run. You write a couple of house packets so you can have a playoff (or so you can avoid using the worst of the submitted rounds). You do some work on the packets you receive so that they are more or less comparable in difficulty and quality. Then you draw up a schedule based on the number of packets and teams, get some moderators together, and play the questions. Berkeley did the last two, but pretty thoroughly failed to accomplish the (at least equally vital) previous tasks. The only charitable explanation is that nobody at Berkeley understands what is entailed in running a tournament, which is the "constructive" aspect of this post. I'm posting it rather than emailing it to the Berkeley TD because I suspect that these points, as basic as they seem, need to be made more clear.

Andrew

[Rothlover]

I haven't read this post, but saw the subject and thought it important to point out to people to hold off on specific question discussion until after Harvard's WIT mirror this Saturday.

[plujan]

First of all, let me point out that while Berkeley fielded three house teams, two were newbies playing in their first tournament, and the third was David Farris playing solo. (All three contributed packets.) So the implication that the veterans were shirking their duty so they could play is inaccurate.

On to the larger issues. The basic issue is that Paul Reverdy volunteered to packet edit and TD without really knowing what he was getting himself into, and so he ended up not asking us (where by "us" I mean the more veteran members of the club) for help when he could have used it. This is our fault as much as his, and I don't mean to suggest otherwise. In a better world, we would have stepped in to help out without waiting to be asked and before it was too late. Unfortunately, the lack of communication meant that we ended up with the result you saw.

Anyway, that said, rest assured that we have been talking about this quite a bit internally and have taken steps to ensure it won't happen again. And we offer our apologies for it happening at all.

[setht]

Much more important than the scheduling snafu was the problem of question quality. To wit: The packets were terrible. I won't cite specific flaws, though if anyone doubts my claim I'm sure any number of people at the tournament would be able to provide egregious examples. Obviously, there were inexperienced teams at the tournament, whom I don't blame for writing atrocious packets. That's only to be expected. Nor do I blame the team that wrote a ridiculously easy high-schoolish packet, since they thought that the tournament was supposed to be super-easy, or the teams that wrote questions that were much more difficult than the norm. Young players write bad questions; experienced players write questions that aren't quite right for the tournament's advertised difficulty level. These things happen. Over the years, the quizbowl community has evolved a surprisingly effective mechanism for dealing with such confusions. We call it "editing."

I agree with Andrew that inexperienced teams should not be hammered about writing bad packets, but by my count, there were at most 3 teams out of 12 at WIT that had no players with at least 2 years' worth of playing and writing experience. I'm guessing some of the more egregiously bad questions came from those teams, but I know that some of the real clunkers also showed up in packets from teams with at least one experience writer. Is it too much to ask that experienced writers on a team take some time to look over their team's packet before sending it out, especially when they know that some of their teammates are relatively new to writing?

The most egregious fault of this tournament is that nobody seems to have edited it. As far as I could tell, the only serious editing done for WIT was the sprinkling of extra economics questions added by the TD. The early rounds were untouched but good, because they were written by competent teams. The later rounds -- perforce, the playoff rounds for the div 1 teams -- were also untouched, but horrendous. It's really ridiculous that Berkeley a) didn't write any house packets, b) didn't seriously edit the packets they got, and yet c) fielded three house teams.

It may have felt like the packets were unedited, but that is not the case. The only packet Andrew and I are familiar with in both the unedited and edited states is the packet we submitted. From what I can remember, it seemed largely untouched, but I didn't feel it needed much (perhaps any) editing. However, I have heard from people on two teams now who said that their packets had some questions replaced and others moved to the extras. In one case, the packet was written by a fairly experienced team, and relatively few questions were replaced; in the other case, the packet was written primarily by inexperienced players (but with one experienced player), and several questions were replaced. It's pretty hard to tell how much editing was done without having the edited and unedited packets at hand. However, I am convinced that Andrew's assertion that the packets were "untouched," with the exception of the occasional replacement econ question, is incorrect. Whether Andrew (or anyone else) liked the questions written or edited by the Berkeley editors is another question.

This, really, is the heart of my complaint about the WIT. Teams invest time and money to attend tournaments. What did the Berkeley people do in their capacity as hosts? They cobbled together a poorly-conceived schedule which had to be reworked by competitors on the morning of the tournament. They bought about $5 worth of used books to serve as prizes. They didn't even bother printing out the packets, which unsurprisingly resulted in a potential crisis when the wrong round was read off somebody's laptop. They wrote no packets, nor did they appreciably edit the packets they received. On top of all that, they fielded three house teams, one of which was composed of veteran players who could presumably have done some of the work the tournament so desperately needed.

I think the main problem with figuring out a viable schedule was the small number of packets. I would suggest that the people at Berkeley (and any other people planning on hosting a packet-submission tournament) seriously consider trying to arrange for packet swaps, multiple packets from mirror sites, and/or multiple packets written by experienced club members. I don't think it was reading off of laptops that caused the reading snafu; rather, it was the rather complicated reading schedule, which was a result of the small number of packets. Paul Lujan has already discussed the need for better communication between the (relatively new) head editor/TD and the more experienced members of the club. Another suggestion for people at Berkeley or any other club planning on hosting a packet-submission tournament: if your club has a pool of experienced players/editors/writers, use it. Ask the more experienced players to write 5/5, or 10/10, or whatever, in those subjects they know well. Ask them to read over packets. Get all the help you can--editing a full packet-submission tournament is much, much more difficult than writing the same number of packets for various tournaments throughout the year.

It's extremely important for the continuing viability of the circuit that there be a number of independent (i.e. non-NAQT, non-ACF) tournaments. These things aren't really all that difficult to run. You write a couple of house packets so you can have a playoff (or so you can avoid using the worst of the submitted rounds). You do some work on the packets you receive so that they are more or less comparable in difficulty and quality. Then you draw up a schedule based on the number of packets and teams, get some moderators together, and play the questions. Berkeley did the last two, but pretty thoroughly failed to accomplish the (at least equally vital) previous tasks. The only charitable explanation is that nobody at Berkeley understands what is entailed in running a tournament, which is the "constructive" aspect of this post. I'm posting it rather than emailing it to the Berkeley TD because I suspect that these points, as basic as they seem, need to be made more clear.

Andrew

I will agree with Andrew that the logistics of running a packet-submission tournament are not too complicated. However, I think Andrew is glossing over the amount of work involved in writing several blind packets and bringing all the packets to a comparable level of difficulty and decent quality, particularly when the input material is of wildly varying difficulty and quality. For anyone considering editing a tournament: it's doable and it can be very rewarding, but (unless you have already done it many times before) I heartily recommend that you ask for as much help as possible from other club members, and from other clubs--ask for packet swaps, mirrors, freelance packets, and anything else you can think of to reduce the burden.

Finally, focusing back on Berkeley, I am confident that if steps are taken to ensure better communication between TDs and the rest of the club, the large number of experienced people at Berkeley will be more than capable of running high-quality tournaments.

-Seth

[cvdwightw]

there were at most 3 teams out of 12 at WIT that had no players with at least 2 years' worth of playing and writing experience

There is a difference between experience and competence. In addition to the above teams, I can recall at least two packets written or partially written by players who have experience writing questions but who have rarely submitted anything above a medium-quality packet. Obviously a packet swap or soliciting extra freelance packets would have alleviated the problem to some degree, but Aztlan Cup did both of these two years ago and there were still complaints about several clunkers.

I am also led to believe the following things about WIT:
1. Several packets were submitted well after the no-penalty deadline. (I know that our team was the only one West Coast one to submit a packet more than 3 weeks before the tournament)
2. Several packets were also submitted with barely half the packet usable. It's quite probable that the set of packets referred to in (1) significantly overlaps with that referred to in (2).
3. At least one packet contained no extra questions, inexplicably causing an 18-tossup match after buzzer malfunctions (couldn't the moderator have just read blind extras from someone else's packet?)
4. Chris (the other editor) spent all of Friday night editing one packet to near-decency, only to have it kept blind to Division I by the packet schedule.
5. At least two packets had ultimately nonconsequential factual errors in questions.
6. One packet completely did away with the notion of including the phrase "for ten points" in tossups.

I expect many of these problems to continue on the West Coast since the number of quality question writers has steadily declined the past couple of years due to retirement and people going elsewhere for grad school. So far there haven't been many people replacing them. The fact is that right now the distribution of "good" writers on the West Coast and possibly in other areas is very uneven; many schools may have the experience writing questions but not the experience making them good ones.

That having been said, I apologize for my team's blatant disregard of the suggested difficulty, and in particular for my overzealous underlining.

[Matt Weiner]

Dare I ask, but did the DI/DII split exacerbate the scheduling, and thus the packet, problem? With all the other problems caused by putting DII in normal tournaments, it seems like Berkeley took unnecessary work upon itself here, on top of everything else.

[setht]

I don't think the split divisions caused a huge problem. There were 12 teams at the tournament, so I think it made sense to split into two groups of 7 and 5. I believe there were precisely 5 all-undergrad teams and 7 teams with 1 or more grad students, so that's how the groups were split.

-Seth

[Skepticism and Animal Feed]

One of the more bizarre things for me was the fact that they did not read the same packets at the same time. I'm fairly certain that both DI and DII read the same packets, but not simultaneously. A packet, say, that D1 read near the beginning of the tournament, was read to DII at the end.

Now, the fact is, players will fraternize inbetween rounds even if they are not on the same team. I didn't see or hear it happen, but it was possible that somebody could have inadvertently spilled an answer to a member of the other division.

At, say, ACF Regionals, when the game breaks into consolation and championship brackets, they hear the same questions at the same time. Why isn't this done for DI and DII?

[plujan]

That's an easy question to answer. Several of the D2 packets had to be combined with other packets in order to get 20 usable questions. When you do that, though, you create a situation where you have to burn 2 packets to get through 1 round, since you can't read that packet in all game rooms in a single round (only one team has a bye).

However, since the packets are clean to D1, you can read them in separate rounds in D1. In essence, you're trading off the cross-division packet security for an extra round in D1. I think this is a worthwhile tradeoff.

Formats in which different questions appear at different times for D1 and D2 are hardly unheard-of. It's less than ideal, but if you warn players thoroughly (and we did), it shouldn't create any problems.

[Rothlover]

For those who care (since it hasn't been posted yet), 12 teams showed up for the Harvard mirror of WIT, with the top 5 being Williams, Brown, Harvard, Brandeis, Williams B and the top 3 scorers being Jerry, David, and Dan. SQBS was used, so hopefully stats will be hosted online somewhere.

I think all tournaments using the WIT set have been used, correct? If so, this opens the door for comments on individual questions, right?

[Susan]

Wasn't it once standard practice to post an announcement of tournament results shortly after the tournament was held? Whatever happened to that?

[cvdwightw]

Evidently this only got sent to the West Coast schools.

Preliminary rounds only:
http://www.ocf.berkeley.edu/~quizbowl/t ... dings.html

Full standings (including playoffs):
http://www.ocf.berkeley.edu/~quizbowl/t ... dings.html

[Susan]

I understand that the stats were posted on Berkeley's website, but no public announcement of the results were made.