Problems with Packets

Old college threads.
Locked
User avatar
Mr. Kwalter
Tidus
Posts: 617
Joined: Tue Nov 18, 2003 1:48 am
Location: Houston, TX

Problems with Packets

Post by Mr. Kwalter » Mon Apr 28, 2008 11:46 am

So, overall, I was pretty disappointed in the packets we got. Let me be clear that while I want the newer teams who submitted packets to understand these criticisms and learn from them, I am not upset that their packets needed a lot of work. It's to be expected that younger players aren't necessarily going to write questions at the same level as those who have been doing this for a while. Unfortunately, we had some real problems that occurred almost across the board, and the more experienced teams should know better. The people at whom these criticisms are directed can tell who they are simply by glancing at the packet they submitted.

1) Difficulty. ACF nationals is not your chance to submit tossups on laughably difficult things. When we expand the canon we do so gradually, which means maybe you shouldn't write a tossup on something that has rarely if ever been a bonus prompt, even. We had one packet from a team that really should have known better in which I believe four of six lit tossup answers had never to my knowledge been tossed up before. Also, while it's OK to have a common link tossup or a tossup on something like "Mark Twain" in your packet, you shouldn't have three or four tossups on things that could be tossed up at ACF Fall. Though I know some people really get a kick out of writing the whole "easy answer with 7 of 8 lines being full of clues no one has heard of" questions, I personally don't.

2) Subdistribution - literature. OK, people, how long have you been doing this? There should be at least one American literature tossup, one British literature tossup, one European literature tossup, and one World literature tossup in each packet, with the same going for bonuses. For the final 1/1, they are generally supposed to be British or American lit but should not both be of one nationality. Additionally, there should be 1/1 poetry and 1/1 drama in every packet. How hard is this to do?

3) Subdistribution - arts. Jesus, people. 1/1 painting, 1/1 pure music, and 1/1 other arts. The two other arts questions should not be on the same kind of art. Almost every packet had problems with this. Did we assume too much in not writing that out explicitly in the announcement? Is "every other ACF and mACF tournament has expected this of you" not sufficient? I know many editors let this kind of thing slide, but this is ACF nationals. When rushing to finish the arts questions I did my best to even that out, but it meant cutting a lot of things and adding certain kinds of questions where none were provided. It's a lot easier to edit rather than write from scratch if you follow directions

4) Weak distribution. Please don't submit a packet in which all of the lit questions are from the 20th century or all the arts questions are from the 19th etc. This is a lesser objection, obviously, but we did make some decisions based on balancing the time distribution within packets.

5) Question quality. There's not much I can say here except that a lot of the questions looked hastily or lazily written. When you're writing painting questions, you should think about to how many other paintings your early clues apply (how many tossupable paintings do you know with a green-blue sky?). If you're writing a tossup on an opera, try to determine which arias (or duets/trios/etc) are the most well-known and be sure to include those, but later in the question. It's not that hard; search youtube to see if there are performances of any of the arias posted there, or look to see if any of them are included in compilation CDs (youtube has performances of "E sogno? o realta? and "L'onore! Ladri!" and googling them gives telling results). Use sources like Grove (to which almost everyone who is enrolled in school should have access through their university's library) instead of wikipedia etc. Additionally, if possible you should make sure you have the most well-known name for those arias. Sometimes different recordings list the pieces in a certain manner based on where they cut the tracks off, even though they have a more common name.

Nobody is expected to be an expert on every subject, but there are plenty of resources out there from which enough information can be gleaned to write a good question. If you don't know of any good resources for a given subject, consider asking around. A lot of us who have been doing this for a while have useful sources we've found that we'd be happy to share.

This post is only on the literature and arts. I know my fellow editors have things to say about their areas as well, so I'll let them say them. My final thoughts are these: submitting imperfect packets to most tournaments is inevitable and, even though it still sucks and should not be done, ultimately OK, since the editors of, say, Penn Bowl probably aren't going to be as concerned with the fine print on the subdistributions. But this was ACF nationals, and people should have done better. I know sometimes you assign various subjects to various people and thus you may not immediately realize, if you realize it at all, that your packet violates these rules. This can be avoided by having someone coordinate the packet and police potential tossup answers to make sure you have the right balance. I am not saying, "You submitted a sub-par packet so you can't complain about the tournament," because that's stupid and unproductive. In the future, though, I hope we see more packets that reflect a more meticulous attention to details that, though they might seem trivial at first glance, ultimately affect in a non-trivial manner how much the editors have to do to clean those packets up.
Eric Kwartler
Alumnus, University of Texas School of Law

User avatar
Mike Bentley
Auron
Posts: 5808
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Problems with Packets

Post by Mike Bentley » Mon Apr 28, 2008 12:15 pm

I think some clearer guidelines would be helpful. From what I remember, there weren't any subdistribution or difficulty guidelines provided like there were for tournaments like Penn Bowl this year. While writing a packet is pretty much consistent from tournament to tournament, there are usually a few deviations in subdistribution and especially difficulty.

I think if you specifically said things like "plan for tossups to be answered by 75% of teams" and "easy parts to be answered by 80% of teams" or something it would help out a lot. A lot of people, especially people not putting up a lot of points at ACF Nationals, have a hard time distinguishing between what is hard but not unreasonable and what it is way too hard. I know I wrote a few tossups that were probably way too hard for this tournament (like the one on Bertel Thorvaldssn) and would have probably reconsidered this if better guidelines were in place.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

User avatar
Mr. Kwalter
Tidus
Posts: 617
Joined: Tue Nov 18, 2003 1:48 am
Location: Houston, TX

Re: Problems with Packets

Post by Mr. Kwalter » Mon Apr 28, 2008 2:02 pm

And next year's guidelines will be more specific. I just don't think asking teams to induce that we want the same kind of variety in our questions that has been expected for every other ACF/mACF tournament ever is too much. Why would ACF nationals simply punt the obviously logical idea of adhering to subdistributions within each subject? Not to mention some of the teams that submitted packets, including D2 team and best packet winner Minnesota, Brown, and Stanford to name a few (though they had other problems), did manage to think about this, and when I told some of them of this problem they were baffled that anyone wouldn't.

As for difficulty, give me a break. Too easy I understand, maybe, given lack of guidelines, and we'll correct that next year, but "don't write on things you really know are too hard to be tossupable" shouldn't be something we need to tell people. If you're not sure whether it's too hard, either just bet that it is and find something else or search other packets for questions on it. ACF nationals is not the time for self-indulgence. Save it for CO.

Edit: Also, I specifically said these criticisms were largely directed at people who should know better. People who are relatively new to question writing are excused from these admonitions, as you're right, many times they just don't know.
Eric Kwartler
Alumnus, University of Texas School of Law

User avatar
Birdofredum Sawin
Rikku
Posts: 400
Joined: Tue Nov 09, 2004 11:25 pm
Location: Mountain View

Re: Problems with Packets

Post by Birdofredum Sawin » Mon Apr 28, 2008 2:23 pm

While I can completely sympathize with Eric's posts here, I'm also taken aback by them. ACF nationals, much like all other ACF tournaments, much like all other packet-submission tournaments, invariably suffers from poorly written and/or tardily submitted packets. If I had wanted to do so, I could have posted similar angry critiques each of the last three years after slaving over nats. (Incidentally, I think most of the current editors have submitted packets to past ACF nats which suffered from the flaws Eric is singling out in this year's submissions.)

Of course, it would be nice if people started writing their ACF nationals packets the summer prior to the tournament, and really worked to bring each question to perfection, and never let their question-writing duties get pushed aside in favor of other priorities. But that has NEVER been the case. Which means that editors ALWAYS have to pick up the slack, if the tournament is going to come off. I don't think there has ever been a time in the game when things were any different.

Which brings me to another point. In theory, at least, there should be a social compact between teams attending a packet-submission tournament and the editors of that tournament. In the case of the teams, the obligations are sort of obvious: they should write packets in a timely manner, to the best of their abilities, etc. But the editors also have obligations. They need to assemble a team of people who will definitely do their work. If any of those people have a history of flaking, then the editors need to take that into account and have back-up plans in place (rather than throwing up their hands in shock and scrambling for makeshifts when people with a proven record of dubious conduct live up to that record by blowing off work). And obviously, they need to edit the tournament in a consistent and timely manner.

I'm not saying that any of these statements are applicable to this year's tournament. I'm just pointing out that the obligations run both ways. With the past few ACF nationals, at least, I think the editors have been better about fulfilling their obligations to the tournament than the teams have been. From what Eric says, it sounds as if this year may have seen flawed work on the part of both the teams and the editors (to varying extents). I don't know what more can be said, save once again to point out that these mutual obligations exist, and express the hope that people will live up to them better in the future.
Andrew

Ex-Virginia, Ex-Chicago, Ex-Stanford

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Problems with Packets

Post by No Rules Westbrook » Mon Apr 28, 2008 2:59 pm

Well, everyone knows what theoretical jobs the editors and packet writers for a tourney have. It's unproductive to point them out; the question is whether they get done or not. And editors "picking up the slack" is quite an understatement.

More to the point, I'm not sure what you're supposed to do to provide insurance against people "flaking out" in this qb world we live in. Sadly, there are not competent and trustworthy editors falling out of trees out there. As a result, the same people work their asses off over and over because they have to, so we can have a series of good events. Lots of people aren't really willing to do much editing work unless there's a dire emergency, and the fate of the world is at stake. Don't get me wrong, what happened with editors falling out at nats this years is confusing and irritating as hell to me, and another example of me feeling like quizbowl just isn't a priority for a lot of people the way it should be (if we want this game to be good). But, I don't think there was any "throwing of hands up in shock," at least not on my part anyway - more like a sigh and an "okay, let's fix this as best we can." Overall, I think we did a pretty good job at fixing it and produced a good nats set, but that doesn't mitigate my irritation with the way lots of people go about their qb duties.

That said, I will note that (at least in my areas) - I wasn't quite as disappointed with the submissions as Eric appears to be (maybe just because I'm more grimly realistic, I don't know). I did have some important issues with submitted stuff, and some helpful suggestions for the future, and I'll post about them in this thread later. You know, I think there were some good questions and more bad questions, and it was about what I expected.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
Mr. Kwalter
Tidus
Posts: 617
Joined: Tue Nov 18, 2003 1:48 am
Location: Houston, TX

Re: Problems with Packets

Post by Mr. Kwalter » Mon Apr 28, 2008 3:14 pm

Birdofredum Sawin wrote:Of course, it would be nice if people started writing their ACF nationals packets the summer prior to the tournament, and really worked to bring each question to perfection, and never let their question-writing duties get pushed aside in favor of other priorities. But that has NEVER been the case. Which means that editors ALWAYS have to pick up the slack, if the tournament is going to come off. I don't think there has ever been a time in the game when things were any different.
Great. Next year let's write the tournament, ok? That sound good? Let's just be prepared for a complete fuckout on the part of the teams and the editors by writing 20 editor packets. We all did write a substantial number of editor questions in anticipation of problems. Then there were so many problems that in many cases those questions just went away.
But the editors also have obligations. They need to assemble a team of people who will definitely do their work. If any of those people have a history of flaking, then the editors need to take that into account and have back-up plans in place (rather than throwing up their hands in shock and scrambling for makeshifts when people with a proven record of dubious conduct live up to that record by blowing off work). And obviously, they need to edit the tournament in a consistent and timely manner.
Where the hell are these other editors that can be relied on and are just DYING to edit nationals instead of playing? Please, point them out. Also, let's think about the Frankel situation. I had faith in Frankel because I knew he really wanted to do this and if at all possible wouldn't fuck out. When he did, he was coughing up blood and being forced to work more shifts than he could handle. But you're absolutely right, regardless of sickness/work he did get morose and useless and I should have expected it. Oh, wait, I did! But I had a contingency plan.

Ezequiel Berdichevsky (months ago): "If you need me to, I can edit arts in my sleep."

Ezequiel Berdichevsky (just over a week ago): "I have the flu, sorry dude, but I just can't help."

I of course don't blame Zeke for not being able to help, but don't give me shit for fucking this up by letting Frankel edit because I had a reasonable expectation that I wouldn't have to work my ass off to finish it in the case that Frankel did quit.
From what Eric says, it sounds as if this year may have seen flawed work on the part of both the teams and the editors (to varying extents). I don't know what more can be said, save once again to point out that these mutual obligations exist, and express the hope that people will live up to them better in the future.
Of course there was flawed work by the editors, even taking into account the mass fuckout by some of our editing staff. All of us will freely admit what we did wrong when editing our subjects. But, just as an aside, it's not like your nationals were completely pristine and not at all characterized by absurd self-indulgence. I'm kind of tired of the sanctimonious "it was better in our day" attitude that you tend to take in discussions like these. I'm not saying you don't have grounds to levy criticisms against this tournament and its editors, but stop acting like you all were gods and we are mere mortals struggling and failing to mimic you.

I want to hear every single criticism of the set people have to offer, because it matters to me what people think. If I ever do this again I want to have learned from my mistakes, and the other editors feel the same way. But don't tell me things like "you should have been prepared" and "you have no right to criticize teams that finished in the top five for writing questions that are either way too hard or just shitty." God knows I've committed my share of packet sins, but it's not like spewing ad hominem attacks is going to make my critiques any less justified.
Eric Kwartler
Alumnus, University of Texas School of Law

User avatar
Birdofredum Sawin
Rikku
Posts: 400
Joined: Tue Nov 09, 2004 11:25 pm
Location: Mountain View

Re: Problems with Packets

Post by Birdofredum Sawin » Mon Apr 28, 2008 4:09 pm

Kit Cloudkicker wrote: Of course there was flawed work by the editors, even taking into account the mass fuckout by some of our editing staff. All of us will freely admit what we did wrong when editing our subjects. But, just as an aside, it's not like your nationals were completely pristine and not at all characterized by absurd self-indulgence. I'm kind of tired of the sanctimonious "it was better in our day" attitude that you tend to take in discussions like these. I'm not saying you don't have grounds to levy criticisms against this tournament and its editors, but stop acting like you all were gods and we are mere mortals struggling and failing to mimic you.

I want to hear every single criticism of the set people have to offer, because it matters to me what people think. If I ever do this again I want to have learned from my mistakes, and the other editors feel the same way. But don't tell me things like "you should have been prepared" and "you have no right to criticize teams that finished in the top five for writing questions that are either way too hard or just shitty." God knows I've committed my share of packet sins, but it's not like spewing ad hominem attacks is going to make my critiques any less justified.
Hey, I get that editing tournaments can be a tiring and irritating process, but maybe it's time to chill out a bit. Saying that I'm "taken aback" at a tournament editor excoriating teams at length for how "disappointed" he was in their submissions isn't exactly tantamount to saying "we were all gods and you are mortals." Nor is it even to imply that my nationals were "completely pristine," which they weren't, as I would often note in my posts following the tournaments. I did think that there was some useful commentary in those post-nationals discussions (e.g. about the place of science history or common-link tossups in the game); it's not clear that those discussions would have been as productive if I had prefaced each one with an angry screed about how the teams had let me down. Or maybe the mere act of comparing this year's discussion with that of 2005, 2006, or 2007 constitutes sanctimonious "it was better in our day"-ism in your eyes?
Andrew

Ex-Virginia, Ex-Chicago, Ex-Stanford

User avatar
AKKOLADE
Sin
Posts: 15269
Joined: Thu Apr 24, 2003 8:08 am

Re: Problems with Packets

Post by AKKOLADE » Mon Apr 28, 2008 4:29 pm

Hey, let's not turn into a "my method of ACF editing is better than your method" thing, ok?
Fred Morlan
PACE President, 2018-19
International Quiz Bowl Tournaments, co-owner
University of Kentucky CoP, 2017
hsqbrank manager, NAQT writer (former subject editor), former hsqb Administrator/Chief Administrator, 2012 NASAT TD

User avatar
theMoMA
Forums Staff: Administrator
Posts: 5686
Joined: Mon Oct 23, 2006 2:00 am

Re: Problems with Packets

Post by theMoMA » Mon Apr 28, 2008 4:37 pm

Maybe this is getting lost somewhere, but pretty much everyone loved the set. It was particularly touching to see Jerry talking to Ryan about how enjoyable the questions were after the heartbreaking finals match. I don't think that it's fair to say that a defining feature of this tournament is flaws, editor or otherwise.
Andrew Hart
Minnesota alum

User avatar
Skepticism and Animal Feed
Auron
Posts: 3178
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Problems with Packets

Post by Skepticism and Animal Feed » Mon Apr 28, 2008 4:43 pm

I have to say I am a bit taken aback by the tone that Eric has taken to Andrew in this thread. I find it inappropriate given who Andrew is and what he has done.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Problems with Packets

Post by No Rules Westbrook » Mon Apr 28, 2008 4:49 pm

I have to say I am a bit taken aback by the tone that Eric has taken to Andrew in this thread. I find it inappropriate given who Andrew is and what he has done.
See, ACF isn't elitist!
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Problems with Packets

Post by grapesmoker » Mon Apr 28, 2008 4:58 pm

DJ Shadow wrote:I have to say I am a bit taken aback by the tone that Eric has taken to Andrew in this thread. I find it inappropriate given who Andrew is and what he has done.
The revolution devours its own children. Let's put this behind us and discuss the tournament.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4047
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: Problems with Packets

Post by Auks Ran Ova » Mon Apr 28, 2008 5:02 pm

grapesmoker wrote:
DJ Shadow wrote:I have to say I am a bit taken aback by the tone that Eric has taken to Andrew in this thread. I find it inappropriate given who Andrew is and what he has done.
The revolution devours its own children. Let's put this behind us and discuss the tournament.
So, I had a great time at the tournament and I loved the questions. Other than a few clunkers (hilariously hard science bonuses, for example), it was a great set.
Rob Carson
University of Minnesota '11, MCTC '??
Member, ACF
Member, PACE
Writer and Editor, NAQT

User avatar
millionwaves
Auron
Posts: 1360
Joined: Sun Feb 25, 2007 12:35 pm
Location: Urbana, Illinois
Contact:

Re: Problems with Packets

Post by millionwaves » Mon Apr 28, 2008 5:08 pm

Ukonvasara wrote:So, I had a great time at the tournament and I loved the questions. Other than a few clunkers (hilariously hard science bonuses, for example), it was a great set.
I also really enjoyed this set. It was a great time, and I look forward to playing it again next year.
Trygve Meade
Illinois, ACF

Above the Star-Apple Kingdom

User avatar
naturalistic phallacy
Auron
Posts: 1368
Joined: Tue May 01, 2007 12:03 am
Location: Minneapolis, MN
Contact:

Re: Problems with Packets

Post by naturalistic phallacy » Mon Apr 28, 2008 5:30 pm

Gengangere wrote:
Ukonvasara wrote:So, I had a great time at the tournament and I loved the questions. Other than a few clunkers (hilariously hard science bonuses, for example), it was a great set.
I also really enjoyed this set. It was a great time, and I look forward to playing it again next year.
Agreed. There were some awesome questions (the homosexuality TU comes to mind), and even though it was a difficult tournament for me especially, I am no longer afraid of ACF Nationals.
Bernadette Spencer
University of Minnesota
MCTC
Event Manager, PACE

Father, among these many souls / Is there not one / Whom thou shalt pluck for love out of the coals?

User avatar
Chris Frankel
Rikku
Posts: 369
Joined: Mon Apr 21, 2003 12:52 pm
Location: Houston, TX

Re: Problems with Packets

Post by Chris Frankel » Mon Apr 28, 2008 10:27 pm

I've got a longer, more substantive post in the making, but I'm quite sleep deprived at the moment and will save it for a few days later. First off, I apologize profusely to Eric, the rest of the editors, and everyone who attended Nationals, for my failure to live up to my commitments. Some of the things that brought that result about were beyond my control (e.g. the getting called to work a 14 hour double shift at my job while coughing up blood bit), and the rest was foolishness on my part (not being more prompt in communicating my difficulties and allowing time for alternate arrangements to be made).

That said, I will defend Eric (and myself, to a lesser extent) by saying that things would have gone a lot better as far as the arts go had the many experienced writers submitting followed some of the basic question writing tenets he mentioned. Nearly all of the "editing" work I contributed to the set consisted of writing questions from scratch on brand new answers (FWIW, I believe I did end up writing well over half of the arts questions that were used) to replace questions that were summarily deemed unusable due to poor answer selection and inattention to major subdistribution/balance issues. I have enough editing experience to know that comes with the territory, but it was immensely irritating to watch the submissions trickle in and realize that anything less than a substantial razing of the submitted material would have resulted in an arts distribution that was nothing but esoteric 20th Century art and composer biography, opera, and architecture.
"They sometimes get fooled by the direction a question is going to take, and that's intentional," said Reid. "The players on these teams are so good that 90 percent of the time they could interrupt the question and give the correct answer if the questions didn't take those kinds of turns. That wouldn't be fun to watch, so every now and then as I design these suckers, I say to myself, 'Watch this!' and wait 'til we're on camera. I got a lot of dirty looks this last tournament."

User avatar
Mike Bentley
Auron
Posts: 5808
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Problems with Packets

Post by Mike Bentley » Mon Apr 28, 2008 10:35 pm

Kit Cloudkicker wrote:As for difficulty, give me a break. Too easy I understand, maybe, given lack of guidelines, and we'll correct that next year, but "don't write on things you really know are too hard to be tossupable" shouldn't be something we need to tell people. If you're not sure whether it's too hard, either just bet that it is and find something else or search other packets for questions on it. ACF nationals is not the time for self-indulgence. Save it for CO.
I was personally under the opinion that CO was actually an easier tournament than ACF Nationals. It certainly seemed that way to me playing both tournaments last year. With ACF Nationals pretty much the cap on tournament difficulty (besides stuff like Westbrook's doubles), writing some tossups that were reasonably canon expanding seemed not an outrageous thing to do.

Teams not adhering to a distribution is less excusable, but, again, there weren't really clear guides on what you wanted. Obviously having people write 4 literature tossups on 20th century American novels is not acceptable, but I find it hard to criticize people for submitting things like two opera questions for their fine arts requirement.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

User avatar
Chris Frankel
Rikku
Posts: 369
Joined: Mon Apr 21, 2003 12:52 pm
Location: Houston, TX

Re: Problems with Packets

Post by Chris Frankel » Mon Apr 28, 2008 10:59 pm

Bentley Like Beckham wrote: Teams not adhering to a distribution is less excusable, but, again, there weren't really clear guides on what you wanted. Obviously having people write 4 literature tossups on 20th century American novels is not acceptable, but I find it hard to criticize people for submitting things like two opera questions for their fine arts requirement.
Little things like this can add up to become problems that have a noticeable negative effect on the quality of the tournament set. Your example is actually one that can be visibly demonstrated. A musically-inclined poster in the discussion thread commented that the music distribution suffered from a glut of esoteric composer biography questions. Well, the main reason for that problem was that many submitted packets either outright ignored the classical instrumental/vocal music requirement in favor of opera/ballet or half-assed the music distribution and just cranked out formulaic composer biography questions. As a result, there was a major scramble to fill this significant gap in usable music questions, and "editing music" de facto turned into "writing the whole music distribution." If the answer spreadsheet holds true, I wrote (from scratch) a few more than half the music questions used, Matt Nance wrote (from scratch) a few more, and pretty much all the rest of the packets had pure music tossups/bonuses left out or had lower quality ones left in because of lack of time/Eric's admitted lack of music specialization. Obviously the final blame lies on myself for not doing my job, but the point is the neglect given to that category by the submissions helped the problem snowball and turn into a problem that damaged the entire distribution and not just a small subset of packets.
"They sometimes get fooled by the direction a question is going to take, and that's intentional," said Reid. "The players on these teams are so good that 90 percent of the time they could interrupt the question and give the correct answer if the questions didn't take those kinds of turns. That wouldn't be fun to watch, so every now and then as I design these suckers, I say to myself, 'Watch this!' and wait 'til we're on camera. I got a lot of dirty looks this last tournament."

User avatar
grapesmoker
Sin
Posts: 6365
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Re: Problems with Packets

Post by grapesmoker » Mon Apr 28, 2008 11:48 pm

Regarding music, writing music tossups is pretty hard if you're more or less tone-deaf like me. I do try to write them to adhere to the distribution, but if someone doesn't play an instrument or know some theory, it's tough going.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
code ape, loud voice, general nuissance

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Problems with Packets

Post by No Rules Westbrook » Tue Apr 29, 2008 12:22 am

I agree with Jerry. I find writing those theory/pure music tossups really tough too, and we're both pretty skilled at writing, so that's some indication. On the other hand, I like operas and obscure composers/their works perfectly well, and they're easy to write. Of course, I agree that there should be some tus that are "pure music" in the set, but I don't really think they should have a preferred status. Arvo Part isn't obscure at all, and I'd rather hear a tossup on him or some crazy obscure dude whose works I can study. And I'm sure there are plenty of people like me who can't make heads or tails of those lead-ins that blabber on about transitions from A minor to D major and how famous they are.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

vandyhawk
Tidus
Posts: 584
Joined: Sat Dec 13, 2003 3:42 am
Location: Seattle

Re: Problems with Packets

Post by vandyhawk » Tue Apr 29, 2008 12:47 am

Ryan Westbrook wrote:I agree with Jerry. I find writing those theory/pure music tossups really tough too, and we're both pretty skilled at writing, so that's some indication. On the other hand, I like operas and obscure composers/their works perfectly well, and they're easy to write. Of course, I agree that there should be some tus that are "pure music" in the set, but I don't really think they should have a preferred status. Arvo Part isn't obscure at all, and I'd rather hear a tossup on him or some crazy obscure dude whose works I can study. And I'm sure there are plenty of people like me who can't make heads or tails of those lead-ins that blabber on about transitions from A minor to D major and how famous they are.
I just posted comments similar to Ryan's in the main thread too...

As for difficulty, I tend to write tossups a little bit on the easier side, so perhaps I'm a culprit on that front. I think that ACF Nats tossups can be harder than lower levels, but don't need to be systematically. Of course, a whole round of tossups at regionals level would be out of place, as would an entire round of tossups that can only be done at nats. Dead tossups are the bane of people enjoying quizbowl, though, so I personally like to err on the easier side, with bonuses hopefully at the right level.

I think every team attending this tournament who wrote a round should know enough to submit packets with the proper sub-distribution. More specific guidelines, like Matt W. did for Penn Bowl and Regionals, would probably be good to ensure it is done, but I'll have to side with Eric on that particular point.

I made a small post about the quality of submitted rounds after regionals, but I think it got buried as the last post on page 1 or something. Anyway, as much as editors would love it, people just don't spend as much time writing questions for submission vs. editing. I try not to fall prey to this myself, but it's tough, especially in areas where I would hardly consider myself an expert. It's a delicate situation of whether you should ask people to re-do things or just suck it up and change it yourself.

User avatar
setht
Auron
Posts: 1176
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: Problems with Packets

Post by setht » Tue Apr 29, 2008 1:13 pm

I would strongly encourage the editors (especially those that plan on working on ACF Nats next year) to produce detailed guidelines for next year's event, and to give detailed feedback on all the packets. I know this means sinking even more time into the editing process, but I think it will wind up saving you time: I think there are plenty of teams out there that are capable of following guidelines and producing better packets, if feedback and clear guidelines are presented. There's always a disconnect between the editors' vision for the tournament set and the players' visions (and I think Andrew and others have commented on this in years past), but I think we're getting more and more competent writers on the circuit that can follow guidelines and absorb feedback to produce questions that adhere fairly closely to the editors' vision of ACF Nats, and you guys should definitely do what you can to shift more of the burden of producing the set onto the players.

Also, once Jerry's searchable question database is set up, I think that will be a useful resource for people trying to figure out whether or not their tossup on Bertel Thorvaldssn is a good idea.

On a related note:
Chris Frankel wrote:it was immensely irritating to watch the submissions trickle in and realize that anything less than a substantial razing of the submitted material would have resulted in an arts distribution that was nothing but esoteric 20th Century art and composer biography, opera, and architecture.
First off, I hope you've recovered. Second, I know this is irritating, but, as you say, it comes with the territory: there's always some small category that gets way over- or under-represented in the submissions. I don't there's anything that can be done to prevent a random fluctuation of mostly 20th century visual art in the submissions. What can be done, I think, is to give clear guidelines on target tossup and bonus difficulty; hopefully this will result in random fluctuations of accessible 20th century art, rather than esoteric 20th century art, which is somewhat better--at least you'll be able to use more of the submissions before you have to start tossing stuff out to avoid having too much 20th century stuff. It might also be possible to mitigate the effects of overabundances of composer biography and opera by putting in a subdistribution minimum of 1 non-composer biography/opera classical music. As other people have pointed out in this thread, this will probably net you some weaker questions on non-composer biography/opera classical music, but hopefully you'll be in better position to edit submissions rather than having to throw things out and write everything from scratch. I don't think there's anything that can be done about random influxes of architecture questions (assuming they came in at a maximum rate of 1 per packet; if teams submitted more than that, let them know via packet feedback and subdistribution guidelines that you don't want that next year).

A couple questions/comments about subdistribution: I was under the impression that fine arts was typically 1/1 painting/sculpture, 1/1 classical music, 1/1 whatever, and our packet had a painting tossup and a sculpture bonus (and that was it for visual arts); is that not okay? My impression was that painting and sculpture are often paired, in quizbowl and in art classes (they were in the only art history class I took), so I sent in our packet without adding a painting bonus. If that contributed to the editing headache, I apologize. Next, RMP: are we shifting to a subdistribution of 1/1 philosophy and 2/2 myth/religion? I support giving writers and editors more wiggle room to write and edit to their strengths, as long as the subdistribution doesn't get too imbalanced.

I can't think of any criticisms of the fine arts or literature right now--maybe once the set is posted. In the meantime, there were some issues with some of the science, but given the situation that's perfectly understandable and I don't think it particularly reflects on anyone's editing abilities. There were also some issues with the myth; again, given the much-less-than-ideal conditions of the editing process, I don't think this year's set necessarily reflects on Matt or anyone else that worked on the myth. It's a little hard to remember things without the set, but I know the tossups on Izanagi and Hera had lead-ins that were too well-known (actually, in Izanagi's case, I'd say the issue was more one of fraudability than widespread knowledge of the Tsukiyomi/Ukemochi episode). I think the lead-in for the Nestor tossup was probably fine, but it got to "son of Neleus" pretty quickly. Perhaps I'll have more detailed feedback once the set is posted, or perhaps I'll go post some more about the myth in the other thread.

I'll say this over in the main discussion thread as well, but this was a strong set and I really enjoyed playing it. My impression is that last year's nationals set was more even, but given the upheaval in the roster of editors close to the tournament I think the editors did a great job.

-Seth
Seth Teitler
Formerly UC Berkeley and U. Chicago
President and Chief Editor, NAQT
Emeritus member, ACF

User avatar
Mr. Kwalter
Tidus
Posts: 617
Joined: Tue Nov 18, 2003 1:48 am
Location: Houston, TX

Re: Problems with Packets

Post by Mr. Kwalter » Tue Apr 29, 2008 1:36 pm

First off, Mike, the Chicago Open comment was just an offhand remark. Obviously people shouldn't go wild in their CO packets. As we are playing together I'm sure we can come up with a packet that's perfectly serviceable and on target difficulty-wise.

On to the rest. I am unapologetic for my first two posts. I got mad because not following the subdistribution seems like a specific issue that people really haven't addressed vocally enough. If no one says anything people will just keep saying ok well I like opera so let's write 1/1 of that in our packets, even if it does mean the editor is definitely going to have to replace one of our questions. Why do you write at all if you are writing something you know will be replaced? Same thing with questions of inordinate difficulty. Occasionally someone submitting a tossup on something really hard can spark a new interesting idea for a question, but it still means that tossup or bonus is going to be replaced. If you're going to write questions that either violate the distribution or are definitely too hard, you might as well not write at all. I hope that in the future teams will be more conscious of this when writing their packets for tournaments.

As to my response to Andrew, while my tone was admittedly overly vituperative, I stand by my claims. The tone of his post was condescending and his remarks about the way we handled things were baseless and unacceptable. I don't appreciate being told that I'm being a child and should accept the fact that I (and the other editors) are responsible for the situation in which we found ourselves after other editors completely abandoning us. I'm not trying to blame every problem with the set on that; I have repeatedly encouraged people not to let the situation color their overall appraisal of nationals (except, perhaps, of science), because even if I did have to rush to finish the arts I would rather hear "ok these questions suck" than "well, you did what you could." But I don't think Andrew's post served to do anything but diminish discussion of an issue that, as I said before, really needs discussing, and I hope people will consider what they accuse others of more carefully in the future.
Eric Kwartler
Alumnus, University of Texas School of Law

User avatar
Mr. Kwalter
Tidus
Posts: 617
Joined: Tue Nov 18, 2003 1:48 am
Location: Houston, TX

Re: Problems with Packets

Post by Mr. Kwalter » Tue Apr 29, 2008 1:38 pm

Seth: 1/1 painting/sculpture has on occasion been given as a subdistribution guide, so that ambiguity is understandable. As we've said before, we're going to produce a set of guidelines for next year that exhaustively covers every detail of the subdistribution and weak distribution.
Eric Kwartler
Alumnus, University of Texas School of Law

msaifutaa
Lulu
Posts: 37
Joined: Tue Feb 12, 2008 6:40 pm

Re: Problems with Packets

Post by msaifutaa » Tue Apr 29, 2008 1:40 pm

setht wrote:There were also some issues with the myth; again, given the much-less-than-ideal conditions of the editing process, I don't think this year's set necessarily reflects on Matt or anyone else that worked on the myth. It's a little hard to remember things without the set, but I know the tossups on Izanagi and Hera had lead-ins that were too well-known (actually, in Izanagi's case, I'd say the issue was more one of fraudability than widespread knowledge of the Tsukiyomi/Ukemochi episode).
I didn't play the round with Hera--what was the lead-in?

As for Izanagi, I feel like the Ukemochi incident was a giveaway for Tsukiyomi/Susanoo (different versions have either of the brothers do the deed, but that doesn't change the answer, since they asked for the father), and that Izanagi being their father is a giveaway for him. Wikipedia, at least, seems to confirm this (if we consider it a common denominator for the most well-known clues, at least) by having no real information on Tsukiyomi other than his birth and the Ukemochi thing. That said, some people don't do Shinto at all, so in many matches, it probably didn't make too much of a difference. Maybe I just know the obscure stories and not the common ones, but are there any other famous clues for Tsukiyomi other than Ukemochi, "Shinto moon god", and "right eye of Izanagi"?
Mark
MIT

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Problems with Packets

Post by No Rules Westbrook » Tue Apr 29, 2008 2:16 pm

I mean, I guess I agree with Seth and think that people like Eric and Chris are being too dogmatic about subdistribution requirements here, but if that's what they want, that's fine and can tell people that. I was always under the impression that composer bio/opera was perfectly valid classical music and, when I write a packet, I usually submit one of those, one painting tossup, and one "other" fine arts tu like architecture/film/jazz/photography/etc. I submit that because those are the things that I feel I can write well - I'd rather not give people some sucky "pure music" tu that's unusable - if there's demand for lots of those questions from music specialists, then they should write them, since they're presumably good at writing them and that will solve the subdistro problem.

Admittedly, when it comes to subdistribution, I'm one of these people who doesn't much care and just goes by the "you know, make it close enough" philosophy. Of course, I suspect that's because I'm a sweeping generalist of sorts and don't really have a category of things that I want to see come up, so I never get too mad when one subdistro is a little weak. But, I just figure that supply and demand should work here - if people want those kind of tus more, they should submit them. Granted, there should always be some of those tus in a set and if the editors have to write them on their own, they should do that. But, beyond a certain minimum number, I don't see the need to be so dogmatic about subdistros - even within the context of ACF Nats, where admittedly you're going to be pretty rigid about making sure subdistros are reasonably fair.

Btw, with regard to CO, I think Eric just meant that it's a little more loosey-goosey in that it's not a national tourney - you can afford to be a little self-indulgent, not that much is at stake. Same thing with difficulty - like I said in the CO thread, I really have no problem with a good deal of variation - if you want to go easier than the target difficulty or harder, as long as the packet is consistent, the questions pyramidal, and somewhere within the realm of reasonable, that's fine.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
Mr. Kwalter
Tidus
Posts: 617
Joined: Tue Nov 18, 2003 1:48 am
Location: Houston, TX

Re: Problems with Packets

Post by Mr. Kwalter » Tue Apr 29, 2008 3:20 pm

Ryan, the problem with being so lax is that you end up with people complaining that there wasn't enough music or that there was too much opera or something else along those lines, and those complaints are justified. The reason we have the subdistributions is to make sure that teams with, say, good music players get the same scoring opportunities as teams with good visual art players. It's not fair to advantage one or the other in the long run over the course of a tournament. As to classical music/opera, those opera questions often end up counting as both pure music and other arts due to confusion or expediency, which can lead to a glut either of opera questions in addition to music questions or to other kinds of "other arts" questions. In order to preserve the balance it's best to ensure that we have clear boundaries between the categories. If people think the arts subdistribution is a topic worth discussing at greater length, perhaps a thread should be started to do so.
Eric Kwartler
Alumnus, University of Texas School of Law

User avatar
Matt Weiner
Sin
Posts: 8411
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Problems with Packets

Post by Matt Weiner » Tue Apr 29, 2008 3:29 pm

Something that was in motion before this tournament and is going to be completed in the next few days is the adoption of standard packet submission guidelines for all ACF events, which will incorporate a subdistribution based on the one I put out for Regionals.
Matt Weiner
Founder of hsquizbowl.org

User avatar
Sima Guang Hater
Auron
Posts: 1852
Joined: Mon Feb 05, 2007 1:43 pm
Location: Philadelphia, PA

Re: Problems with Packets

Post by Sima Guang Hater » Tue Apr 29, 2008 4:42 pm

msaifutaa wrote:As for Izanagi, I feel like the Ukemochi incident was a giveaway for Tsukiyomi/Susanoo (different versions have either of the brothers do the deed, but that doesn't change the answer, since they asked for the father), and that Izanagi being their father is a giveaway for him.
The main problem with this question was it forced me to parse out what exactly they were asking for in the first line. I'm fine with lateral thinking on middle clues, but the first clue should always be about the person or thing being asked directly rather than a second-order clue. There's scores of better leadins for Izanagi; I would say the Amenonuhoko clue is harder than the Ukemochi incident, or the clue about his headdress being turned into grapes when he threw it away while running from his woman.
msaifutaa wrote:Wikipedia, at least, seems to confirm this (if we consider it a common denominator for the most well-known clues, at least) by having no real information on Tsukiyomi other than his birth and the Ukemochi thing.That said, some people don't do Shinto at all, so in many matches, it probably didn't make too much of a difference. Maybe I just know the obscure stories and not the common ones, but are there any other famous clues for Tsukiyomi other than Ukemochi, "Shinto moon god", and "right eye of Izanagi"?
After ignoring all of the results relating to Naruto (*shakes fist*), a quick Google search tells me a tossup on Tsukiyomi could be done, especially at nationals; there's some holiday's dedicated to him, a shrine in Yamagata to him that Basho visited, you could put a clue about the heavenly cloud-man being sent to finish the job he was originally sent to do, etc.
Eric Mukherjee, MD PhD
Washburn Rural High School, 2005
Brown University, 2009
Medical Scientist Training Program, Perelman School of Medicine at the University of Pennsylvania, 2018
Intern in Internal Medicine, Yale-Waterbury, 2018-9
Dermatology Resident, Vanderbilt University Medical Center, 2019-

Member Emeritus, ACF
Member, PACE
Writer, NAQT, NHBB, IQBT

"The next generation will always surpass the previous one. It's one of the never-ending cycles in life."

msaifutaa
Lulu
Posts: 37
Joined: Tue Feb 12, 2008 6:40 pm

Re: Problems with Packets

Post by msaifutaa » Tue Apr 29, 2008 5:39 pm

ToStrikeInfinitely wrote:The main problem with this question was it forced me to parse out what exactly they were asking for in the first line. I'm fine with lateral thinking on middle clues, but the first clue should always be about the person or thing being asked directly rather than a second-order clue. There's scores of better leadins for Izanagi; I would say the Amenonuhoko clue is harder than the Ukemochi incident, or the clue about his headdress being turned into grapes when he threw it away while running from his woman.
Those would be better clues to lead with--I did not hear them because I buzzed immediately.
ToStrikeInfinitely wrote:After ignoring all of the results relating to Naruto (*shakes fist*), a quick Google search tells me a tossup on Tsukiyomi could be done, especially at nationals; there's some holiday's dedicated to him, a shrine in Yamagata to him that Basho visited, you could put a clue about the heavenly cloud-man being sent to finish the job he was originally sent to do, etc.
Ah no, no. You misunderstood me. You could easily write a Tsukiyomi toss up, and I would enjoy it--in fact, my logic only works if you agree that Tsukiyomi iss toss-upable at Nats. I didn't mean that you couldn't find more clues on him. I meant that there is no easier clue on him. As in--if this isn't a giveaway for him, nothing is.

So, in essence, if you agree with my proposition that this is a giveaway for Tsukiyomi, and if you also agree that Tsukiyomi was reasonable to toss up in ACF Nats (and third if you agree that being Tsukiyomi's father is giveaway for Izanagi), then you can logically deduce that the lead-in here was off (since it would have likely been part of the giveaway for a Tsukiyomi toss-up).
Mark
MIT

User avatar
cvdwightw
Auron
Posts: 3446
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: Problems with Packets

Post by cvdwightw » Tue Apr 29, 2008 6:54 pm

Tsukuyomi has shown up at least once as a tossup answer in the past three years with the Uke Mochi clue as a lead-in. I don't know what if anything this proves, but I thought I'd point it out.
Aztlan Cup II/BUTT 2006 Maryland Packet wrote:12. He angered his sister when he killed Uke Mochi, the goddess of food, after the former sent him to represent her at a feast in which Uke Mochi created the food in such a way that disgusted him to the point of killing her. His sister’s anger at this action resulted in their always occupying different parts of the sky. His name is a combination of the Japanese words for reading and the celestial figure he represents. The second of Izanagi’s three noble children born when he cleansed himself of his sins, name, FTP, this male moon god of Japanese mythology, brother of Amateratsu.
Answer: Tsukuyomi no Mikoto (or Tsuki no kami)

msaifutaa
Lulu
Posts: 37
Joined: Tue Feb 12, 2008 6:40 pm

Re: Problems with Packets

Post by msaifutaa » Tue Apr 29, 2008 7:15 pm

cvdwightw wrote:Tsukuyomi has shown up at least once as a tossup answer in the past three years with the Uke Mochi clue as a lead-in. I don't know what if anything this proves, but I thought I'd point it out.
Aztlan Cup II/BUTT 2006 Maryland Packet wrote:12. He angered his sister when he killed Uke Mochi, the goddess of food, after the former sent him to represent her at a feast in which Uke Mochi created the food in such a way that disgusted him to the point of killing her. His sister’s anger at this action resulted in their always occupying different parts of the sky. His name is a combination of the Japanese words for reading and the celestial figure he represents. The second of Izanagi’s three noble children born when he cleansed himself of his sins, name, FTP, this male moon god of Japanese mythology, brother of Amateratsu.
Answer: Tsukuyomi no Mikoto (or Tsuki no kami)
Hmm, that toss-up only contains the clues I mentioned above and seems strongly paraphrased (in some cases wording simply lifted) from Wikipedia. But yeah, it does indeed have the same lead-in.
Mark
MIT

User avatar
Auks Ran Ova
Forums Staff: Chief Administrator
Posts: 4047
Joined: Sun Apr 30, 2006 10:28 pm
Location: Minneapolis
Contact:

Re: Problems with Packets

Post by Auks Ran Ova » Tue Apr 29, 2008 7:45 pm

Wow, that is not a good tossup on Tsukiyomi.
Rob Carson
University of Minnesota '11, MCTC '??
Member, ACF
Member, PACE
Writer and Editor, NAQT

Locked