PARFAIT "discussion"

Old college threads.
Locked
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

PARFAIT "discussion"

Post by grapesmoker »

I put the discussion in quotes because there is still a mirror to be run, so obviously we cannot discuss specific questions. When that mirror is over, I will certainly offer more specific criticisms.

Anyway, for what it's worth, I found PARFAIT to be easily the worst tournament I've played since I sat in on 3 rounds of Moon Pie last spring. Maybe that says something about the level of tournament quality recently; most of the tournaments since have been pretty high quality, and in truth, this year's PARFAIT would not have looked out of place in 2002 or thereabouts. But there's just no way that PARFAIT was up to the standards set by good tournaments over the past 2 years.

First of all, there was the distribution. Several packets appeared to be underdistributed in some areas of the distribution as announced, and the distribution of categories within the packets was also questionable, with categories frequently bunched near the beginning or the end. Certain areas, I won't say which ones, were greatly overrepresented, and seemed directly in line with what I believe were pet topics of the various writers.

The next problem was the difficulty. I was led to believe that this tournament would be between ACF Fall and ACF Regionals in difficulty, but this was way off the mark. This tournament was entirely too easy, for the most part, and combined with the generally low quality of the questions, most tossups reduced to "what's the first thing I can think of that matches this description?"

These two complaints are of course secondary to my main point, which is that the question quality vacillated between the low end of mediocre and the abominable. Many questions began with vague clues that did not help players in any way. Many other questions began with what was essentially the giveaway, leading to buzzer races on the first or second clue. We saw the return of the 6-part bonus, which wouldn't be so bad if they hadn't been so lazily written. I have no idea what sort of "editing" was involved here, but clearly most of the packets did not receive anything more than a cursory overview. There were a few packets which were pretty decent, since they had come from more experienced writers, but one of the Berkeley packets, the Truman State packets, and maybe half of the Princeton packets were mostly awful. For a tournament that was announced months in advance, and to which supposedly the whole club contributed, there seemed to be shockingly little screening of contributions from less experienced writers.

One the whole, while the logistics portion of the tournament was pretty smooth, PARFAIT was a huge disappointment. The feeling I got was that Princeton was just riding the reputation of Chris Frankel's good work from 2005, and had expended minimal effort to really put together a good tournament.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Mike Bentley
Sin
Posts: 6466
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Post by Mike Bentley »

I think Jerry got a little carried away in his criticisms of this tournament, but there were also a number of valid points made in his post. I don't think that anyone is going to disagree that the one Berkely packet was pretty terrible. But, for the most part, the other packets were at least playable, and a few of them were not too bad.

A frequent problem was that certain subjects seemed more difficult than others. A few seemed to be written at the ACF Fall level and others seemed to be surpassing the ACF Regionals level.

Also, it would be nice if this tournament would stop using purely randomized shuffling of questions. I would wager that pretty much everyone enjoys having questions spread out through the entire packet rather than having 3 literature tossups in a row or no science questions until the second half of a packet. This can lead to big swings in games that serves to make a lot of people upset.

Unfortunately, this tournament did not seem as good as last year's tournament. It was certainly worth playing and I had a generally good time doing so, but it would be nice to see an improvement next year. I will obviously have more to say once we can actually talk about the questions.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Post by cvdwightw »

I apologize to the Stanford B players for having put up with my rather ill-natured groaning about the Truman State C packet, although I think you felt the same way and were much better about containing your emotions.

If this was a novice tournament, it would have been almost good. But it wasn't a novice tournament (the WIT field was arguably the strongest recently seen on the West Coast), and therein lie the problems. Some things - repeats, egregious factual errors - are problems regardless of field intentions. Other things, like giveaways in the first line, are also bad. But the main problem I found with the question quality is that mid-level clues repeatedly showed up in the first line, which is not necessarily a bad thing except that the field at all 3(?) sites contained a lot of players who typically buzz on mid-level clues and thus there were a lot of first-line buzzer races or people sitting going "that can't possibly be the first clue for Answer X".

In the immortal words of Willie Chen, "Who edited this? DISTRIBUTE!" Pseudorandom distributions are far preferred by, I think, just about everybody to truly random distributions that yield the possibility of, to cite an example that happened at this tournament, back-to-back American history tossups. Whole swathes of subdistributions also seemed to be missing entirely from the tournament or barely touched upon. Also, to echo Jerry, certain pet topics were overrepresented to the point where I began to call the inevitable bonus on (a certain topic) the "Daily Double", as, like its Jeopardy! namesake, it shows up once per round and no one knows who's going to get it.

This is not to say that this tournament was without redeeming features. I found several tossups to be rather well-written and would post the answers if it were not for the three-week delay between the mirrors. Also, despite all my grumbling, I did have fun and I don't believe that I detracted significantly from anyone else's enjoyment of the tournament.

I would highly advise that since the mirror is three weeks away, Princeton look heavily at, in particular, the Truman State packets and the Berkeley A packet and consider major revisions before sending those packets to Alabama. If this happens, then I would consider the set we played on to be a "partially edited beta packet set" and remain only a little bitter about having had to play on the partially-edited set (also, the set used ultimately as future practice questions would reflect much more highly upon the club). However, if Princeton feels that their packet set is perfectly acceptable to be used at Alabama, then maybe Jerry's gut feeling was right.
User avatar
ecks
Wakka
Posts: 248
Joined: Sun Feb 04, 2007 4:03 am
Location: Springfield, IL / Kirksville, MO

Re: PARFAIT "discussion"

Post by ecks »

grapesmoker wrote:There were a few packets which were pretty decent, since they had come from more experienced writers, but one of the Berkeley packets, the Truman State packets, and maybe half of the Princeton packets were mostly awful.
As you noted, we aren't particularly experienced question writers at the college level. Our club used to do it but the practice has mostly dropped off except for our HS tournaments, which are markedly easier to write for.

Nevertheless, we did put a lot of time and effort into writing and editing our questions, and it was a group effort. Kent and myself particularly spent quite a bit of time reading up on guides on writing well, trying hard to avoid easily correctable mistakes (grammatical awkwardness, pronoun confusion, 'lazy' binary PLEASE MAKE FUN OF ME BECAUSE I SPEAK NEITHER LATIN NOR ENGLISH, not having the first clue being uniquely identifying, etc.). Now, there was heavy revising of most of the questions we submitted to Princeton; other than the answer given, a lot of them were unrecognizable to me. Whether they changed for the better or the worse is certainly debatable, although some of them at least were changed for the better.

Anyway, writing good questions is something we certainly aspire to do, and I would definitely appreciate your feedback on specific questions you had issues with. You can e-mail me at cstone -at- truman -dot- edu or just wait until Alabama's mirror is done; I'd like to compare our submissions to the final product and see where things went wrong so we don't make similar mistakes in the future.

Edit: The invitation to e-mail me specific criticism is extended to anyone who wants to comment on our questions, not just Jerry.
Last edited by ecks on Sun Nov 18, 2007 8:28 pm, edited 1 time in total.
Christopher Stone
Truman State University '09
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

Sure Chris, I would be happy to do that, although I only have the packets in print form rather than electronic. Obviously, there is no way I can know what was submitted by you and what was edited at Princeton; I commented on the packets I played as I saw them. If you have the originals you sent to Princeton, I'd be glad to look at those too for comparison.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Mike Bentley
Sin
Posts: 6466
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Post by Mike Bentley »

By the way, when I said that the one Berkeley packets was "pretty terrible", I do realize that it was mostly new people writing this packet. My criticism is less about the fact that the questions produced by these newer players weren't that good (that is to be expected), but more about the fact that the questions weren't edited or that the people organizing the tournament felt that a packet from new people would be dependable.

I think maybe it would have been a good idea, especially after that Florida mirror dropped out, to have requested freelance packets or packets from teams coming to the tournament to help you guys out a bit. Perhaps next year this tournament would work better as a packet submission or packet submission optional tournament.
Mike Bentley
Treasurer, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008
User avatar
vcuEvan
Auron
Posts: 1085
Joined: Mon Feb 23, 2004 5:49 pm
Location: Richmond VA

Re: PARFAIT "discussion"

Post by vcuEvan »

grapesmoker wrote:This tournament was entirely too easy, for the most part, and combined with the generally low quality of the questions.
I wouldn't say entirely... my main problem was bonuses that wouldn't look out of place in an NAQT IS set right next to bonuses in certain subjects that even the best teams had no clue on.

Also the aforementioned tossups that seemed antipyrimidal. At least the tournament seemed logistically organized well and many of the moderators were capable.
User avatar
Sima Guang Hater
Auron
Posts: 1965
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Post by Sima Guang Hater »

Does anyone have an electronic copy of these yet? If so, please send to Eric [underscore] Mukherjee [at] brown [dot] edu.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
ezubaric
Rikku
Posts: 370
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Post by ezubaric »

ToStrikeInfinitely wrote:Does anyone have an electronic copy of these yet? If so, please send to Eric [underscore] Mukherjee [at] brown [dot] edu.
Only the hosts have electronic copies of the packets, and because it's just too easy for electronic copies to float around and compromise security, we're not sending them out until after the mirror. Everyone will get them then.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
vandyhawk
Tidus
Posts: 555
Joined: Sat Dec 13, 2003 3:42 am
Location: Seattle

Post by vandyhawk »

ezubaric wrote:Only the hosts have electronic copies of the packets, and because it's just too easy for electronic copies to float around and compromise security, we're not sending them out until after the mirror. Everyone will get them then.
There goes my plan to average 180 ppg individually in a couple weeks (assuming we can muster enough interest to go right before finals).
vandyhawk
Tidus
Posts: 555
Joined: Sat Dec 13, 2003 3:42 am
Location: Seattle

Re: PARFAIT "discussion"

Post by vandyhawk »

grapesmoker wrote:The next problem was the difficulty. I was led to believe that this tournament would be between ACF Fall and ACF Regionals in difficulty, but this was way off the mark. This tournament was entirely too easy, for the most part
Based on the posted stats, I have to say the difficulty seems like it was on target, at least taken as a whole. It seems like you're meaning the tossups were too easy too quickly, or there was really easy stuff mixed with really hard stuff? For bonuses, when full strength Chicago, Maryland, Brown, and Yaphe are all converting 21-22 ppb, Sorice and Weiner individually are around 18 ppb, and most other teams are 10-18 ppb, that seems about right for an invitational, provided the bonuses are consistent, which it sounds like they may not have been. Really, I don't see the need for any (EDIT: non-national) tournament, even ACF Regionals, to be much, if any, harder than that, but again, provided it's done in a consistent and well-written manner.
Last edited by vandyhawk on Wed Nov 21, 2007 3:14 pm, edited 1 time in total.
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Re: PARFAIT "discussion"

Post by cvdwightw »

vandyhawk wrote:Based on the posted stats, I have to say the difficulty seems like it was on target, at least taken as a whole. It seems like you're meaning the tossups were too easy too quickly, or there was really easy stuff mixed with really hard stuff?
Tossup difficulty was all over the place, many tossups had either meaningless fluff or clues that should not have been the lead-in in the lead-in, and bonus difficulty occasionally drifted down to the high school level. The issue wasn't that people weren't converting the bonuses or were converting them at too high a rate, it was more that there were quite a few gimme 30s balanced out with bonuses where it was tough to get 10.
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

It was too easy in the sense that the category into which an answer fell was immediately obvious (e.g. "this Portuguese author," obviously not a real example) and the answer was immediately deducible from those clues by guessing the most obvious thing from that category.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Birdofredum Sawin
Rikku
Posts: 400
Joined: Tue Nov 09, 2004 11:25 pm
Location: Mountain View

Post by Birdofredum Sawin »

Yeah, these questions pretty much blew. Lots of pointless, unbuzzable sentences as the "lead-ins" to tossups; lots of giveaways in the first words; lots of packets where nobody bothered to distribute the questions (e.g. four science tossups in a row would be followed by four lit tossups in a row); lots of wildly uneven bonuses (ranging from borderline ACF nats level difficulty to high school stuff). I wouldn't pile on if this was just a novice effort by some young writers, but my understanding of this tournament was that a core of experienced -- or should I say "experienced" -- editors would be overseeing the finished product. As such, a real disappointment.
mcalmvp
Lulu
Posts: 42
Joined: Mon Apr 17, 2006 12:47 am

Post by mcalmvp »

Dwight and Yaphe pretty much hit it on the head. Whoever is mirroring PARFAIT (I think it's Alabama?) would probably do better if they mirrored something else or make the tourney in house, unless the PARFAIT packets are heavily editted to take out the factual inaccuracies of certain packets, distribute the tossups more evenly, distribute bonus difficulty better, etc. (oh btw..for Dwight's "certain subject bonus"..if I recall correctly it was also mostly in the back of the packet)
User avatar
ezubaric
Rikku
Posts: 370
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: PARFAIT "discussion"

Post by ezubaric »

grapesmoker wrote:The feeling I got was that Princeton was just riding the reputation of Chris Frankel's good work from 2005, and had expended minimal effort to really put together a good tournament.
You must mean 2004. Frankel contributed some much appreciated questions to the 2005 PARFAIT set, but didn't edit 2005 or 2006.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
jhn31
Wakka
Posts: 178
Joined: Sun Sep 03, 2006 9:18 pm
Location: Starkville, MS

Post by jhn31 »

Back from the Alabama mirror, I have to say that I agree with the above sentiments.

For the first 6 rounds, the tossups were decent, but the bonuses were awful. They were so random whether they'd be easy or hard. We 0'ed so many of them, but occasionally there'd be a really easy one that everyone could 30.

Starting in round 7, the tossups were just as bad. I can't really put it into words, but it just stopped being fun.
User avatar
ezubaric
Rikku
Posts: 370
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Post by ezubaric »

Packets are now available here:

http://www.princeton.edu/~cbowl/parfait2007/packets.zip

They'll also be going up on the archive soon, hopefully.

Obviously, it's fine to talk about the questions specifically now. Many people have detailed comments they want to share, and now is the time. Some people have already shared comments with me or guy privately, and I hope you share your comments publicly, as this is very likely the last tournament that either Guy or I will edit (we're both trying to graduate next year). I'll get the ball rolling ...

I'm sorry it was received so poorly; there were a couple of times I winced while reading, and there were things that were obviously left undone. Part of the problem I think was that I tried to keep clue order and answer selection intact in the questions that I edited, and both of those things contributed to things that were noted as problems (what people have termed false lit and early giveaways, e.g. Calvino).

The bonus difficulties were not as regular as they should have been (particularly when it came to science), and this got by me. It shouldn't have; I guess it didn't come out during playtesting because we have a science heavy team, and none of the mirrors complained when we sent out the questions a week in advance for them to playtest.

There was too much bible, but we started off with a bible-heavy distribution that only got worse when our mirror questions came in. When we saw what was happening, I tried to add non-bible questions, but that's an area I'm particularly weak in, and I obviously didn't do enough.

I'm sorry that people didn't enjoy the questions, but Guy and I did try to put on a good tournament. We did put in a lot of work, and I think the questions improved during the course of the editing process. At the time, I thought we were able to handle producing more packets to make up for the loss of our third mirror, but it obviously hurt our quality and diversity of questions to an unforeseen extent.

This brings me to Jerry's absurd claim that we are being lazy and cashing in on Chris Frankel's (admittedly strong) reputation. First, although the final product might not have lived up to Jerry's exacting standards, we were not lazy. Guy, Irene, and the rest of the team put in many hours to write these questions when we could have just decided to throw an IS tournament. This is also an insult to the efforts of Dan Benediktson who put in nearly as many hours as Chris did on Chris's Kickboxer (Chris's last tournament, over three years ago) and my first two PARFAITs. Dan's hand was clearly missed here.

EDIT: Giving the all clear
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
Sima Guang Hater
Auron
Posts: 1965
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Post by Sima Guang Hater »

ezubaric wrote:Packets are now available here:

http://www.princeton.edu/~cbowl/parfait2007/packets.zip

They'll also be going up on the archive soon, hopefully.
Great. I'll save these files next to my video of Chris Ray playing CBI at Arizona state.
ezubaric wrote:Obviously, it's fine to talk about the questions specifically now. Many people have detailed comments they want to share, and now is the time.
I've kind of lost the will to do this; so many of your tossups could have been improved by doing a simple Google scholar search or looking through old packet sets for interesting leadins (Transposons come to mind).
ezubaric wrote:Part of the problem I think was that I tried to keep clue order and answer selection intact in the questions that I edited, and both of those things contributed to things that were noted as problems (what people have termed false lit and early giveaways, e.g. Calvino).
Why would you do this? That's not editing, that's just glancing!
ezubaric wrote:The bonus difficulties were not as regular as they should have been (particularly when it came to science), and this got by me. It shouldn't have; I guess it didn't come out during playtesting because we have a science heavy team, and none of the mirrors complained when we sent out the questions a week in advance for them to playtest.
The problem with your science wasn't that it was irregular; the problem was that it absolutely sucked. I can't think of a single question in my field in which the giveaway didn't come too early or that the leadin was completely uninteresting or that the answer choice was simply transparent and poor.
ezubaric wrote:There was too much bible, but we started off with a bible-heavy distribution that only got worse when our mirror questions came in. When we saw what was happening, I tried to add non-bible questions, but that's an area I'm particularly weak in, and I obviously didn't do enough.
At least it gave me the opportunity to make a sign with "You White People and Your Bible on it". Quite entertaining, at least for me. Why are you starting out with a bible-heavy distribution anyway, when you know that it wouldn't be well-received? It seems like a silly thing to do.
ezubaric wrote:I'm sorry that people didn't enjoy the questions, but Guy and I did try to put on a good tournament. We did put in a lot of work, and I think the questions improved during the course of the editing process. At the time, I thought we were able to handle producing more packets to make up for the loss of our third mirror, but it obviously hurt our quality and diversity of questions to an unforeseen extent.
Nice to hear that the questions improved, at least. And it sucks that a mirror bailed out on you; still, in the time between your deciding to run this tournament and it actually taking place, there's no reason you could have cranked out the packets yourself instead of hitting up mirrors for them.
ezubaric wrote:This brings me to Jerry's absurd claim that we are being lazy and cashing in on Chris Frankel's (admittedly strong) reputation. First, although the final product might not have lived up to Jerry's exacting standards, we were not lazy. Guy, Irene, and the rest of the team put in many hours to write these questions when we could have just decided to throw an IS tournament.
What good would throwing an IS set tournament do you? No respectable team would show up! That doesn't even matter; this is a horribly fallacious argument anyway. You're justifying the poor quality of your writing by saying that you could have been even lazier and bought the questions instead of assembling them a la Frankenstein from well-known chesnuts and horrible leadins.

Are we supposed to be grateful that instead of 50% of the questions being horribly transparent and falling off of the Gunn-Peterson difficulty cliff like so many quasar spectra lemmings, that 90% of them didn't? I spent upwards of 7 hours getting to this tournament, our club paid a fair fee to play in it, only to show up at get hosed repeatedly. Was this supposed to be an exercise in learned helplessness or something? If so, I salute you, Jordan Boyd-Graber, as the avatar of Martin Seligman that the quizbowl community so desperately needs. Give me a break.

And while you insist that you and guy weren't lazy in writing this tournament, I say that's impossible by any objective standard. This isn't some tournament assembled by a group of young players who don't know what clues are reused all the time. You and Irene have been around quizbowl for some untold number of years - Matt Weiner even claims that you played against him at high school nationals. And yet you write horribly skewed packets full of chestnuts for leadins and blatant transparency, and claim that you weren't lazy about it? I find that decidedly hard to believe.

Also, stop framing this as Jerry expecting too much out of a tournament; demonizing him isn't going to make people enjoy your set any more. While I'm usually the first to criticize him (to his face, not on here) for getting angry and being down on questions, in this case I feel that his indignation was completely justified. Even our freshman teammate, whose previous college experience had come solely from ACF Fall and EFT2, started to complain about the questions. This isn't ICT, this isn't Moon Pie, this isn't fucking VETO; this was supposed to be a decently written set by an experienced team, and everyone (not just Jerry) was disappointed.
ezubaric wrote:This is also an insult to the efforts of Dan Benediktson who put in nearly as many hours as Chris did on Chris's Kickboxer (Chris's last tournament, over three years ago) and my first two PARFAITs. Dan's hand was clearly missed here.
Not knowing who this fellow is, I'll take your word for it that he would have been able to single-handedly pull PARFAIT out of the muck. But from what I understand, you and Irene have been around since god knows when, and have had the pleasure of probably going to several well-written tournaments in that time. Appealing to the absence of some other quizbowl dinosaur seems hardly justified.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
User avatar
Skepticism and Animal Feed
Auron
Posts: 3238
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Post by Skepticism and Animal Feed »

I haven't played PARFAIT, but it seems that the Princeton A packet at BoB today suffered from many of the same problems that people are talking about in this thread.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source
User avatar
grapesmoker
Sin
Posts: 6345
Joined: Sat Oct 25, 2003 5:23 pm
Location: NYC
Contact:

Post by grapesmoker »

ezubaric wrote:stuff
I'll second mostly everything that Eric has said, and point out also that nowhere near 90% of the questions were problem-free. If that had been the case, this set would not have elicited such a negative response.

Look, you guys announced this thing well in advance and it's not like you didn't have time to think about what you were going to do or how you were going to get there. No one expects a flawless set, and I know we're all pressed for time always. However, I just don't see that as a justification, especially since you had so much time to work on this, and a fair bit of manpower. Relying on mirror packets to bail you out is a bad policy, especially if the quality of what's coming from the mirror is dubious at best.
This brings me to Jerry's absurd claim that we are being lazy and cashing in on Chris Frankel's (admittedly strong) reputation. First, although the final product might not have lived up to Jerry's exacting standards, we were not lazy.
Hey, how about we stop making this about my caaaraaazy demands and exacting standards? I'm not posting here asking for tossups on my favorite novels or anything ridiculous. I'm just asking that the questions you write conform to some sort of minimal standard of quality. Sorry if that's too much to ask of the Princeton club. I guess if that's your atttiude, we just won't be coming to PARFAIT next year; it's not like you particularly need our money anyway, so I guess there's no real incentive for you to change.
Guy, Irene, and the rest of the team put in many hours to write these questions when we could have just decided to throw an IS tournament.
I'm guessing you did this because, as Eric said, most teams would probably not travel far to play on IS questions; I wouldn't play on them if all I had to do was to roll out of bed in the morning. It looks like the result was about the same though.
This is also an insult to the efforts of Dan Benediktson who put in nearly as many hours as Chris did on Chris's Kickboxer (Chris's last tournament, over three years ago) and my first two PARFAITs. Dan's hand was clearly missed here.
Whoops, I guess you were coasting on the good work of two people, not just one. We here at the snark department apologize for the omission.

In summary, this tournament was quite poor, which is all the more surprising given that there seemed to be more than 10 years of collective experience at the club. You can be as defensive as you want about that, but that doesn't change the facts. I hope Princeton does a better job at future editions of this event, but given the general downward trend in PARFAIT quality, I'm not sure how that's going to happen.
Jerry Vinokurov
ex-LJHS, ex-Berkeley, ex-Brown, sorta-ex-CMU
presently: John Jay College Economics
code ape, loud voice, general nuissance
User avatar
Matt Weiner
Sin
Posts: 8148
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Post by Matt Weiner »

So, are the full stats from this ever going to be posted, or what?
User avatar
ezubaric
Rikku
Posts: 370
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Post by ezubaric »

ToStrikeInfinitely wrote: Why would you do this? That's not editing, that's just glancing!
I was saying that, I left the original answers when I felt they would be okay and when original clues were maintained, I left them in their original order.
ToStrikeInfinitely wrote:Why are you starting out with a bible-heavy distribution anyway, when you know that it wouldn't be well-received? It seems like a silly thing to do.
It wasn't like we said, "let's write all bible questions and let the mirrors fix it." People independently chose to write their religion questions on the bible. This was an oversight, and a problem that we noticed.
ToStrikeInfinitely wrote:there's no reason you could have cranked out the packets yourself instead of hitting up mirrors for them.
This was part of the mirroring agreement. We thought that asking for three packets over the summer was reasonable, and we got assurances that those packets were on their way up until September. And we did indeed crank those packets out, which limited the amount of time we had for editing.
ToStrikeInfinitely wrote:Also, stop framing this as Jerry expecting too much out of a tournament; demonizing him isn't going to make people enjoy your set any more. While I'm usually the first to criticize him (to his face, not on here) for getting angry and being down on questions, in this case I feel that his indignation was completely justified.
This isn't an attempt to justify the questions, which did have problems. The questions didn't get where I wanted them to be, and I am not arguing that point. My point was that Chris has not been editing Princeton's tournaments for a while (even when he was, he wasn't doing it alone), and this was not a bait and switch tournament. It didn't work out as well as it should have, and I take responsibility for that.

What I took issue with was the insinuation that we never intended to put on a good tournament. That was our goal, and it appears we didn't get there.

You do make a good, constructive point about the pyramidality falloff of the science questions. I think I can see how that came about. We got a number of very transparent questions from the very first lines. We attempted to add better leadins, and we obviously didn't integrate them well enough to avoid buzzer races.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
Sima Guang Hater
Auron
Posts: 1965
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Post by Sima Guang Hater »

ezubaric wrote:
ToStrikeInfinitely wrote:Why are you starting out with a bible-heavy distribution anyway, when you know that it wouldn't be well-received? It seems like a silly thing to do.
It wasn't like we said, "let's write all bible questions and let the mirrors fix it." People independently chose to write their religion questions on the bible. This was an oversight, and a problem that we noticed.
Ok, this is a valid point, and also the basis for a very good object lesson. Its understandable that something like this could happen, but there's a very simple way to ensure it doesn't: Google Docs. All you have to do is create an answer matrix (ie a document or spreadsheet containing a sheet for each packet, with each prospective tossup/bonus answer written in the cells) to solve this. That way, not only can you ensure there's no poor answer choices, you can also let everyone that's writing the tournament see what other people are writing about (even before the questions are written) so that distribution issues don't become a problem.


EDIT: Also, is it really necessary to describe the plot of the Ramayana in a tossup about the Mahabharata? You're not helping anyone by doing this.
Last edited by Sima Guang Hater on Sun Dec 09, 2007 6:26 pm, edited 1 time in total.
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
vandyhawk
Tidus
Posts: 555
Joined: Sat Dec 13, 2003 3:42 am
Location: Seattle

Post by vandyhawk »

This is slightly off topic, but anyone (Billy or Chris B. especially) know what happened to South Florida, specifically Ahmad I guess? Strange to just disappear like that.
wd4gdz
Tidus
Posts: 702
Joined: Mon Jan 19, 2004 5:40 pm
Location: Tallahassee

Post by wd4gdz »

I have no idea, but what do you mean by disappeared? As in, they haven't been to any tournaments this semester, or as in, they disappeared after round 3 or something?
User avatar
ValenciaQBowl
Auron
Posts: 2560
Joined: Thu Feb 05, 2004 2:25 pm
Location: Orlando, Florida

Post by ValenciaQBowl »

I assume Matt is referring to the announced-then-dropped Parfait mirror at USF. Though fortunate to be friends with the USF guys, I'm not privy to too much inside info about what happened with all that, but my sense is that non-QB issues caught up with folks. But Ahmad and Jim Baker of USF are alive and well and recently joined me at a talk by Slavoj Zizek in Orlando. They sound like they'll be playing in spring.
User avatar
cvdwightw
Auron
Posts: 3291
Joined: Tue May 13, 2003 12:46 am
Location: Southern CA
Contact:

Post by cvdwightw »

ToStrikeInfinitely wrote:
ezubaric wrote:This is also an insult to the efforts of Dan Benediktson who put in nearly as many hours as Chris did on Chris's Kickboxer (Chris's last tournament, over three years ago) and my first two PARFAITs. Dan's hand was clearly missed here.
Not knowing who this fellow is, I'll take your word for it that he would have been able to single-handedly pull PARFAIT out of the muck. But from what I understand, you and Irene have been around since god knows when, and have had the pleasure of probably going to several well-written tournaments in that time. Appealing to the absence of some other quizbowl dinosaur seems hardly justified.
Dan is not a quizbowl dinosaur, I played against him our senior years at HSNCT, so he was an undergrad when editing those tournaments. And I do believe that given his previous body of work he would have been able to pull the science out of the muck.

That said, here are five tossups which I had serious problems with. I don't know which packets, but if someone wants to find out, go ahead:
Kiev. I held off from buzzing off of "Golden Gate" thinking that maybe there was some other city that had a less-famous Golden Gate. Then my teammate, following the "CBIish" strategy that mostly worked on these packets of "buzz with the first thing that comes to mind", negged with "Budapest", whose former status as cities on two sides of a river is much better-known and given the preponderance of easy clues in the first couple of sentences wasn't an illogical guess at all.

Jay Berwanger. Based on the CBIish strategy, I really, really wanted to neg with Red Grange for about half the question, who is a much more accessible answer and known for more things. This question boiled down to "This dude played halfback in the Big Ten in the 1930s (maybe not even that, if you didn't know Ford played at Michigan in the mid-'30s). He didn't play in the NFL. FTP name this first Heisman winner."

Winfield Scott Hancock. He's not the "Old Fuss and Feathers" guy, that's Winfield Scott.

Bunker Hill. Blah Blah Blah unnecessary useless information Israel Putnam. When Israel Putnam is the first clue anyone can buzz off of, that's a bad question.

Smetana. The only reason I didn't get this after the first phrase is because I confused Smetana and Bartok. Translating things into English and assuming people either don't know the English translation or can't translate it into something they recognize is not good.
User avatar
theMoMA
Forums Staff: Administrator
Posts: 6001
Joined: Mon Oct 23, 2006 2:00 am

Post by theMoMA »

Some of the packets were decent, and I really appreciated that tossup on Gabriel Conroy, but this tournament in general stopped being fun after round three or so.

Aside from chestnut-filled questions and wildly different bonus difficulty that people have mentioned, several rounds had terrible distributional issues. The round we played Mike Sorice, I casually remarked around question eight that there hadn't been a lit tossup yet. Ten questions later, literature made its first tossup appearance. That's unfair and wrong.

Another distribution issue was the absolutely ridiculous amount of fake lit, Bible, and theoretical math/computer science. It's not fair to the teams playing to write on your pet topics over an entire tournament.
User avatar
ecks
Wakka
Posts: 248
Joined: Sun Feb 04, 2007 4:03 am
Location: Springfield, IL / Kirksville, MO

Post by ecks »

I wrote both the Kiev and Berwanger tossups, so I'd like to talk about those more in-depth to get at why they were bad questions. I'm particularly interested because Truman State's packets in particular seemed to draw a lot of the ire, whereas I haven't seen any substantial criticism of the Indie/Alt/Emo and World Lit packets we wrote for TTGT11.

For comparison, I'll post here what was submitted to Princeton and how it was edited by them.
Unedited Kiev TU wrote:As a cultural and economic center of life for many Slavic peoples, this city has had a long and tumultuous history. Straddling both sides of the river that contributed to the city’s wealth that was almost nearly destroyed by a Mongol invasion in 1240; later on, it found itself at various times as part of Lithuania, Poland, Russia, and even its own independent republic briefly in the early 20th century. As a center of East Slavic culture, it also has several famous buildings, such as the St. Sophia Cathedral. Located on the Dnieper River, name FTP, this city whose population speaks mostly Russian despite being the biggest city of the Ukraine.
ANSWER: Kiev
Edited Kiev TU wrote:Important sites in this city include the Monastery of the Caves and the Golden Gate, constructed by Yaroslav the Wise. Straddling both sides of a river which contributed to the city’s wealth, it was almost destroyed by a Mongol invasion in 1240; later on, it found itself at various times as part of Lithuania, Poland, Russia, and even its own independent republic briefly in the early 20th century. As a center of East Slavic culture, it also has several famous buildings, such as the St. Sophia Cathedral. Located on the Dnieper River, name, for ten points, this city whose population speaks mostly Russian despite being the biggest city of the Ukraine.
Answer: Kiev
Okay, so looking back, I see that I failed to make the question uniquely identifying in the first sentence, a problem Princeton attempted to correct with the inclusion of the "Monastery" and "Golden Gate" clues. Looking back, I recognize that "straddling both sdes of a river which contributed to the city's wealth" isn't a particularly good clue, as there are almost no cities that restrict themselves to one side of a river and don't benefit from being on it. I can't really see significant problems with the rest of the question, however, so am I right in thinking that this problem would have been a good one with better clues at the beginning that would distinguish it from other Eastern European cities? One thing I would like to point out is that Budapest isn't a particularly Slavic city, nor is Hungary a slavic nation; they are in Eastern Europe and were under the influence of the Soviets, but culturally and linguistically they are quite seperate from most of the Slavic world (well, their cultural seperatism is debatable...).

Now for the Berwanger TU, which was almost entirely unchanged:
Unedited Berwanger TU wrote:This former football star was born in Dubuque, Iowa and is known for multiple firsts in football. Although he played halfback, he was well-known for his versatility and had many nicknames, including “Genius of the Gridiron” and “the One-Man Team.” After graduating from his high school in Iowa, he went on to have an acclaimed career in the Big Ten, even giving eventual President Gerald Ford a scar. Despite being picked first in the NFL draft, he did not play in the league, becoming a salesman and occasional sports column writer instead. For ten points, name this man, a former star at the University of Chicago and first winner of the Heisman Trophy.
Answer: Jay Berwanger
Edited Berwanger TU wrote:This former football star was born in Dubuque, and is known for multiple firsts in football. Although he played halfback, he was well-known for his versatility and had many nicknames, including “Genius of the Gridiron” and “the One-Man Team.” After graduating from his high school in Iowa, he went on to have an acclaimed career in the Big Ten, even giving eventual President Gerald Ford a scar. Despite being picked first in the NFL draft, he did not play in the league, becoming a salesman and occasional sports column writer instead. FTP, name this man, a former star at the University of Chicago and first winner of the Heisman Trophy.
ANSWER: Jay Berwanger
I'll admit that the accesibility of this question was hampered by my spotty knowledge of college football, which mostly extends to the past four years and random things I've heard about past events, of which Berwanger was one. I had already written a bonus on modern college football (which had problems of its own), so I wanted to do something from the deep history of college football, and it seemed like Berwanger was a good candidate for that. I guess I should have guessed that there's nothing really notable about him other than he one the first Heisman, but I assumed that people would know more on him--so I apologize for that assumption.
Christopher Stone
Truman State University '09
User avatar
DumbJaques
Forums Staff: Administrator
Posts: 3109
Joined: Wed Apr 21, 2004 6:21 pm
Location: Columbus, OH

Post by DumbJaques »

This former football star was born in Dubuque, and is known for multiple firsts in football.
What? How could someone not edit this opening clue. Seriously, what?

Likewise there's way too much fluff in both questions, which is something that's fine for Truman State submitting a packet (younger, newer team) but clearly needs to be picked up on the editing side.
Chris Ray
OSU
University of Chicago, 2016
University of Maryland, 2014
ACF, PACE
User avatar
Sen. Estes Kefauver (D-TN)
Chairman of Anti-Music Mafia Committee
Posts: 5647
Joined: Wed Jul 26, 2006 11:46 pm

Post by Sen. Estes Kefauver (D-TN) »

On the Berwanger, a big problem I see with it is that it seems to fall back on the "Born in..." leadin, which is invariably a poor writing choice, and then it has lateral thinking emphasis (the "multiple firsts") without giving real clues, and a reliance on non-uniquely identifying or funn information (such as the "he gave Ford a scar").
As for your unedited Kiev tossup, honestly there's not a buzzable clue in the unedited version until St. Sophia Cathedral. That's bad. Find some locations, rulers, residents, things that happened there etc. to fill up the rest of the question, instead of non-specific Slavic clues without a real uniquely identifying thing in it.
Note - I don't think anyone's really villifying Truman State since this is their first year doing ACF-types of competitions, and I think most of us are all for fostering that. Instead, it was Princeton's responsibility to make your packets better, and browsing through the set they clearly did not turn them into usable packets. I would recommend sending in your original packets to some known ACF editors who are good and letting them rip your stuff apart so that you can learn what went wrong and how to fix this. Would any of the cabal members be willing to volunteer for this?
Charlie Dees, North Kansas City HS '08
"I won't say more because I know some of you parse everything I say." - Jeremy Gibbs

"At one TJ tournament the neg prize was the Hampshire College ultimate frisbee team (nude) calender featuring one Evan Silberman. In retrospect that could have been a disaster." - Harry White
User avatar
naturalistic phallacy
Auron
Posts: 1490
Joined: Tue May 01, 2007 12:03 am
Location: Minneapolis, MN
Contact:

Post by naturalistic phallacy »

theMoMA wrote: Another distribution issue was the absolutely ridiculous amount of fake lit, Bible, and theoretical math/computer science. It's not fair to the teams playing to write on your pet topics over an entire tournament.
I would have to agree that the religion distro in this tournament was seriously fucked up. Like Andrew pointed out, there were far too many Bible tossups, throwing focus on Judeo-Christian stuff and off of other interesting things like Buddhism and Islam. Kudos for the Shintoism bonus, though. Just because the Bible is there doesn't mean that it should be the only resource for inspiration to fill that Religion gap in a packet.

Even within the Bible category, the distribution was wacky. There were two tossups about New Testament books of the Bible (Matthew and Acts), but the rest of the questions were virtually all from the Judges time period . Aside from a bonus about Apocryphal Books, I can recall a Judges TU, a bonus about Samuel (a Judge), and side-by-side bonuses about Sampson (again, a Judge) and the Philistines (also from Judges). It seems to me that whoever wrote the questions had just studied that section of the Bible or just loved it so much that they felt compelled to write five questions about it. I know that it all seems to be just an overrepresentation of the Bible to many people, but writing five questions about Judges is akin to writing five questions about, say, the Vienna Circle or DNA replication for one tournament just because you know enough about it to do so.
Bernadette Spencer
University of Minnesota, MCTC
Member, NAQT
Member, ACF
Member Emeritus, PACE
User avatar
Sima Guang Hater
Auron
Posts: 1965
Joined: Mon Feb 05, 2007 1:43 pm
Location: Nashville, TN

Post by Sima Guang Hater »

Matt Weiner wrote:So, are the full stats from this ever going to be posted, or what?
Eric Mukherjee, MD PhD
Brown 2009, Penn Med 2018
Instructor/Attending Physician/Postdoctoral Fellow, Vanderbilt University Medical Center
Coach, University School of Nashville

“The next generation will always surpass the previous one. It’s one of the never-ending cycles in life.”
Support the Stevens-Johnson Syndrome Foundation
wd4gdz
Tidus
Posts: 702
Joined: Mon Jan 19, 2004 5:40 pm
Location: Tallahassee

Post by wd4gdz »

In addition to the aforementioned reasons, this tournament was not up to par with previous PARFAITs merely for not including any Seinfeld questions (to my knowledge).
Locked