Incentive for Improving Submissions...

Old college threads.
Locked
User avatar
SnookerUSF
Rikku
Posts: 310
Joined: Sun Oct 03, 2004 2:55 am
Location: USF-Tampa, FL
Contact:

Incentive for Improving Submissions...

Post by SnookerUSF » Sun Nov 30, 2008 5:58 pm

I hope this has not been discussed here before, if so, if you could direct me to the proper posts that would be appreciated. But, I have been thinking, given the all too witty banter going in the chat room regarding T-Party and the general consensus that packet submissions for tournaments cannot be adequately relied upon for constructing a good tournament, is there some way to continue improve submissions. To that end, things like generous packet discounts for early submissions and significant packet penalties for late submissions have been adopted.

As another possible suggestion, would it be worthwhile for Tournament Editors or even tournament participants to formally recognize the "best packet" of the tournament at the major submission events, and to offer the winning person or team a further discount or perhaps free admission at next year's event (provided they write a packet). I envision a scenario where the tournament editor could announce before the tournament began, which rounds were eligible, and then the players themselves would vote on the packet of the tournament. The results of which would be announced on the board here, much to the delight of all. Something similar to this was done nicely at CO 2008 by Mr. Westbrook. I even thought of suggesting that those members or teams who consistently produce quality packets be given a sort of pass to submitting packets closer to the deadline to allow for busy schedules, though perhaps this would be easily abused or it happens informally anyways.

I recognize the possible marginal effect such a proposal would have, but over the years as the community has devoted some effort in recognizing the best individual players, and last year's rating of the best question writers by R and E (or was it W and K) was a step in this direction. It gives a legitimate way for those members of the community who take the time to craft the most important part of this game to be recognized.

Though I recognize there is nothing revolutionary in this suggestion; it would be nice to know what some of you think.

Regards,
Ahmad Ragab, itinerant moderator at the New School for Social Research

ACF Nationals 2011:"Too real for the streets"
-Auroni Gupta

"Can 40,000 redacted topic Tossups be wrong?"

"With my gnomes I'm highlighting the danger of political opportunism and right-wing ideology. I get the feeling that this gnome has reopened an old wound."
-Ottomar Hoerl

User avatar
Matt Weiner
Sin
Posts: 8413
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Incentive for Improving Submissions...

Post by Matt Weiner » Sun Nov 30, 2008 6:01 pm

My opinion is that the general quality of unedited writing has gotten tremendously better over the past several years. Getting people to turn in their packets on time, or convincing people to attend a packet-submission tournament or submit an optional packet when finals are going on, is another issue, but I think we're doing pretty well on teaching people how to write acceptable questions.
Matt Weiner
Founder of hsquizbowl.org

User avatar
dtaylor4
Auron
Posts: 3733
Joined: Tue Nov 16, 2004 11:43 am

Re: Incentive for Improving Submissions...

Post by dtaylor4 » Sun Nov 30, 2008 6:16 pm

I'm going to merely address the issue of timing of submissions, as I am in no position to address quality.

Some time last spring, I thought about the issue of all too many submissions getting in two days before the deadline. At the time, I was taking a course on Accountancy Control Systems, and decided to ask my professor for her input. Note: this was her first semester as a professor, and knew nothing of anything related to quizbowl. Her main observation was that in her opinion, the monetary incentives are not working. Because most clubs raise their own funds and do not pay out of their own pocket, the effect of such a control is limited.

As for suggestions for controls to implement, she made two suggestions: either refuse to let delinquent teams play, or institute some sort of handicap that would affect the outcome of games themselves (i.e. a 50 pt penalty). The latter of the two I disagree with, but I'd be interested to hear opinions concerning not letting teams play. My opinion is that I think someone needs to implement such a penalty and have the balls to fully enforce it. However, this leaves an issue when this would bring the size of the tournament down drastically.

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Incentive for Improving Submissions...

Post by Mechanical Beasts » Sun Nov 30, 2008 6:26 pm

dtaylor4 wrote:As for suggestions for controls to implement, she made two suggestions: either refuse to let delinquent teams play, or institute some sort of handicap that would affect the outcome of games themselves (i.e. a 50 pt penalty). The latter of the two I disagree with, but I'd be interested to hear opinions concerning not letting teams play. My opinion is that I think someone needs to implement such a penalty and have the balls to fully enforce it. However, this leaves an issue when this would bring the size of the tournament down drastically.
We would have had Brandeis scrimmage itself on the Bentley and West Coast freelance packets, so... yeah, your last observation is correct!

Also, teams are unlikely to have that kind of balls when they're running the tournaments to make money to pay for other tournaments. If we didn't have HFT as a cash cow, we'd be about $300 in debt right now. Killing T-Party (and its mirrors in turn because they couldn't run on three total packets) would have left us not attending any tournaments until HI made us money. So... ACF Nationals, with plane tickets paid for out of pocket with the hope that they might be reimbursed? That'd be a tough situation.
Andrew Watkins

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Incentive for Improving Submissions...

Post by No Rules Westbrook » Sun Nov 30, 2008 6:37 pm

Yeah, I agree with Weiner that packet-writing has improved a whole lot recently, and that's directly attributable to the infusion of new blood into the game. There are a lot more capable people around these days who understand the basics of good question writing, even if they're relatively young - there are more people who can contribute to packets, and more people who are potential editors of tournaments in their own right.

That said, you can't rely upon packet submissions from teams or people with very little writing experience - and often, that's a majority or at least a significant portion of the teams attending a given tournament. If every tourney had the field of Chicago Open, it would be a way different story. You also can't very well just exclude teams from playing - for one thing, like Andrew said, it costs money and often deprives your tournament of a reasonable number of teams. For another thing, telling teams they can't play if they don't submit doesn't help them write better questions (if anything, it helps them rush and write worse questions just to get done and be able to play) - pretty much, you have to know you're not going to get very usable packets from inexperienced teams.

In addition to the fact that several teams often have no experience, certain tournaments seem to fall at unfortunate times of the year...so that, for whatever reason (other commitments, general languor), the writing for those tournaments just doesn't get done as well or as prompty as it does for other tournaments (or doesn't get done at all, in some cases). It looks like T-Party is suffering from this. I think rewards and recognition for good writing are great, since writing is what all good QB is built upon, but I'm not sure it will do anything to fix the above problems, except for maybe discouraging the "general languor."

Maybe it would also encourage more freelance packets and partial packets, which could make up for inexperienced teams. But, writing is a lot of work and it's exceedingly easy for good writers to decide they'd rather watch TV instead of slave away at a packet for an event they're not going to attend (read: it's easy for me to do that), so it might be tough to provide realistic incentives.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
Skepticism and Animal Feed
Auron
Posts: 3188
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Incentive for Improving Submissions...

Post by Skepticism and Animal Feed » Sun Nov 30, 2008 6:54 pm

Is "I won't let you play my tournament" really a credible threat?

Except for ICT, SCT, the vast majority of tournaments are hosted for profit, either to a club or to an individual. Thus teams have a huge incentive to let as many teams as possible play. Banning a team from a tournament hurts this in two ways: (1) one less paying team (and one less paying team with a big late penalty, which means you lose $$$ rather than $); and (2) the smaller the field, the less attractive it becomes to other teams. This is especially true when the delinquent team is a highly desirable opponent that brings prestige to a field (e.g., Sorice).

A tournament that is already swimming in cash might credibly threaten to ban a team. A tournament that already has a glorious field might credibly threaten to a ban a team. A tournament run by a known fanatic who nobody would doubt is willing to take a financial hit to promote aesthetic ideals of good quizbowl (e.g., Weiner) might credibly threaten to ban a team. But other than those three exceptions, I'm not sure a threat of "I won't let you play" is believable.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

User avatar
Skepticism and Animal Feed
Auron
Posts: 3188
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Incentive for Improving Submissions...

Post by Skepticism and Animal Feed » Sun Nov 30, 2008 6:57 pm

Also: somebody should study why teams don't subtly promote late submissions. It would seem to me that the cost of having to write more questions is outweighed by the huge extra income that comes with teams that pay late fees. It's not like most quizbowlers would be using the time saved by not writing questions to engage in profit-making activities whose payout exceeds late fees.
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

User avatar
BuzzerZen
Auron
Posts: 1517
Joined: Thu Nov 18, 2004 11:01 pm
Location: Arlington, VA/Hampshire College
Contact:

Re: Incentive for Improving Submissions...

Post by BuzzerZen » Sun Nov 30, 2008 7:58 pm

Whig's Boson wrote:Also: somebody should study why teams don't subtly promote late submissions. It would seem to me that the cost of having to write more questions is outweighed by the huge extra income that comes with teams that pay late fees. It's not like most quizbowlers would be using the time saved by not writing questions to engage in profit-making activities whose payout exceeds late fees.
I think part of the reason is that the ideal packet-submission tournament contains questions that are based on the submitted questions as much as possible without constituting a complete replacement. Players derive some utility from hearing teams buzz in on questions they wrote. I figure a borderline question submitted a week before the tournament is more likely to be replaced outright than to be edited into shape. The degenerate form of "people shouldn't worry about late fees" is that all tournaments cost $200 and are house-written.
Evan Silberman
Hampshire College 07F

How are you actually reading one of my posts?

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Incentive for Improving Submissions...

Post by No Rules Westbrook » Sun Nov 30, 2008 8:24 pm

Evan, I think Bruce means that editors should root for late questions, cause it makes them richer. Eh, this has some truth to it - but in my experience, the ideal situation for an editor is to have some questions come early, some come in the middle, and some come late (with any luck, the late ones will be really good and won't need much editing) - this way, you make money off late fees, but you still have questions to work on in the interim. You don't want all questions coming just a few weeks before the tourney (unless they're all really good), because editing takes time and will be done badly if rushed - at the same time, you can't edit everything all at once, so you don't really need to have everything early. That said, this is just fluff, of course - everyone should write packets as promptly as almighty possible; let's not give anyone the wrong idea.

And, contrary to Evan's assumption, if you submit a late question that's part of a packet that I'm depending on - I'm certainly not inclined to completely replace it, but rather hastily edit it (unless you have extra questions just sitting around to use as replacements, which is damned rare). Complete replacement takes a lot of time - editing an already-written question should be quicker, that's the whole point.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

User avatar
Captain Sinico
Auron
Posts: 2840
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: Incentive for Improving Submissions...

Post by Captain Sinico » Sun Nov 30, 2008 8:55 pm

everyday847 wrote:We would have had Brandeis scrimmage itself on the Bentley and West Coast freelance packets, so... yeah, your last observation is correct!
You've begged the question: the whole point is that the fee structure we've got now is leading to some undesirable results (like what you saw.) Donald is saying his professor has suggested that changing the structure by adding a drop-dead point might lead to better results with respect to submission timeliness/existence.
I'm honestly not sure that isn't a bad idea in theory, though I haven't seen enough data to see how it might work in practice. The data I have argue against: the time I enacted such a penalty (for MCMNT two years ago,) three teams still either submitted after the drop-dead date or did not send a submission at all and I still let the two of them that wanted to play anyway as I had enough questions at that point. However, perhaps these teams acted in this way because they are used to tournaments that would let them submit very, very late or not at all; perhaps if drop-dead times were consistently employed and enforced, results would improve.

MaS

PS:
Whig's Boson wrote:...This is especially true when the delinquent team is a highly desirable opponent that brings prestige to a field (e.g., Sorice).
Hey, what's that supposed to mean?!

PPS: I've subsequently realized it refers to the many, many times my submissions were horribly late. Sorry about that, quizbowl!
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Incentive for Improving Submissions...

Post by Mechanical Beasts » Sun Nov 30, 2008 9:08 pm

Captain Scipio wrote:
everyday847 wrote:We would have had Brandeis scrimmage itself on the Bentley and West Coast freelance packets, so... yeah, your last observation is correct!
You've begged the question: the whole point is that the fee structure we've got now is leading to some undesirable results (like what you saw.) Donald is saying his professor has proposed that changing it in the way proposed might lead to better results.
Sure; my statement was admittedly hyperbolic, too: if we said no packet by date x, no play, we'd obviously lose some teams, but it would also obviously not be our entire field. (I could only imagine that Brown and Chicago, for example, would have zero trouble writing a packet by an earlier hard cutoff, if one had been instituted.) I meant, rather, to illustrate the reason that TDs might be very reticent to make that pronouncement, because even while many teams would simply get their shit together and submit earlier, you simply have no idea how many. (Or perhaps this is a fear only held by the greenest of TDs, which explains why I was uncomfortable having one and a half freelance packets and one and a half required packets sometime in the middle of last week: I couldn't trust that the rest really, really were forthcoming.)
Andrew Watkins

User avatar
setht
Auron
Posts: 1178
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: Incentive for Improving Submissions...

Post by setht » Sun Nov 30, 2008 9:59 pm

dtaylor4 wrote:two suggestions: either refuse to let delinquent teams play, or institute some sort of handicap that would affect the outcome of games themselves (i.e. a 50 pt penalty). The latter of the two I disagree with, but I'd be interested to hear opinions concerning not letting teams play. My opinion is that I think someone needs to implement such a penalty and have the balls to fully enforce it. However, this leaves an issue when this would bring the size of the tournament down drastically.
These are both interesting suggestions. I think the second suggestion merits more consideration than it's gotten: as people have noted, most tournaments don't have such large fields that they can easily send teams packing without compromising the quality of the tournament. Also, I don't see a good way to implement the first suggestion to give people incentives to submit packets a little bit late but not super late--with the first suggestion the TD can offer financial incentives for early packets, financial disincentives for late packets (which don't appear to do much), and then a hard drop-dead deadline for really late packets. With the second suggestion the TD can offer financial incentives for early packets, then increasing game-play penalties for increasingly late packets.

I'm pretty sure almost every team would be motivated to send their packet in before they start incurring penalty points; I'm also pretty sure almost every TD would be willing to impose and carry out this sort of penalty, which makes this a more useful penalty than barring teams from tournaments. Finally, I think the points penalty has a sort of poetic justice to it: teams sending packets in late compromises the quality of a question set, which compromises the validity of matches in a tournament. This penalty shifts (some of) the "burden of invalidity" onto the offending teams.

-Seth
Seth Teitler
Formerly UC Berkeley and U. Chicago
President and Chief Editor, NAQT
Emeritus member, ACF

User avatar
Matt Weiner
Sin
Posts: 8413
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Incentive for Improving Submissions...

Post by Matt Weiner » Sun Nov 30, 2008 10:05 pm

Are you guys being sarcastic? I can't believe anyone thinks that penalizing people for late packets with in-game points deductions is a remotely good idea.
Matt Weiner
Founder of hsquizbowl.org

User avatar
Captain Sinico
Auron
Posts: 2840
Joined: Sun Sep 21, 2003 1:46 pm
Location: Champaign, Illinois

Re: Incentive for Improving Submissions...

Post by Captain Sinico » Sun Nov 30, 2008 10:38 pm

No, I don't think they're being sarcastic. The data seem to indicate a lot of teams and stuff don't care about money; rather, they care about quizbowl. So, okay, these seem like two ideas to effect better submission practices by costing offenders something they seemingly care about. Perhaps they're good ideas and perhaps not; I guess that's the issue at hand. I thus propose that we we debate these ideas on their merits rather than attempting to dismiss them as ridiculous out-of-hand.

MaS
Mike Sorice
Coach, Centennial High School of Champaign, IL (2014-) & Team Illinois (2016-2018)
Alumnus, Illinois ABT (2000-2002; 2003-2009) & Fenwick Scholastic Bowl (1999-2000)
ACF
IHSSBCA
PACE

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Incentive for Improving Submissions...

Post by Mechanical Beasts » Sun Nov 30, 2008 10:48 pm

I agree with Mike and Seth. While you can't quantify how much a late packet submission affects the validity of a round, it does. If everyone receives the same bonus against that team--and ideally, that late-submitted packet is the only one that's at all uneven in difficulty, so n points is n points on whatever packet you receive them, meaning that the only team that's penalized relative to any other should be that offending team.

The question is, of course: do our values lie such that the result of a match ought to be sacred beyond all else? Then perhaps we'd dismiss this idea out of hand--but if that's the case, then all the stronger is the submitting team's aversion to compromising their experience.
Andrew Watkins

User avatar
dtaylor4
Auron
Posts: 3733
Joined: Tue Nov 16, 2004 11:43 am

Re: Incentive for Improving Submissions...

Post by dtaylor4 » Sun Nov 30, 2008 11:05 pm

Captain Scipio wrote:The data seem to indicate a lot of teams and stuff don't care about money; rather, they care about quizbowl. So, okay, these seem like two ideas to effect better submission practices by costing offenders something they seemingly care about.
Mike does a good job of expressing what I'm trying to say. The current control system in place (i.e. solely discounts/penalties,) in my view, are passable. It is clear that teams strapped for cash are going to do their best to get packets in earlier, but as Andrew pointed out, this can lead to lower-quality submissions, but it gives editors more time to work on them. Also, it has no mechanism for punishing good writers who have a track record of writing good questions but try to be lazy and not write (or submit a packet late,) i.e. one Mike Sorice.

As I mentioned in a previous post, money as the crux behind a control system in this case has not been effective. At the same time, I think that what Sorice did in 2007 for MCMNT (late fees + exclusion deadline) could be effective if fully thought out and enforced. Note: I am not calling Sorice's judgment into question. I do not know all the details concerning what happened with MCMNT 2007, so I am in no position to criticize.

User avatar
Matt Weiner
Sin
Posts: 8413
Joined: Fri Apr 11, 2003 8:34 pm
Location: Richmond, VA

Re: Incentive for Improving Submissions...

Post by Matt Weiner » Sun Nov 30, 2008 11:07 pm

Alright, if you want me to take this seriously, the reasons this is a bad idea are legion:

1) The purpose of the game score is not to enforce penalties for things we don't like, even if we don't like them for good reason. How about we dock 50 points per game for showing up to a trash tournament unshowered, too?
2) Financial penalties can be internally distributed by a team towards those who were actually responsible for the packet being late; a points penalty affects even those who fulfilled their responsibility.
3) Something similar was once tried at a UTC tournament--teams who had submitted packets were given a free win in the standings, effectively making it the tiebreaker. No one liked it and it had no apparent effect on packet-submission behavior.
4) This will, in fact, discourage attendance. Teams will not want to show up to events that they start 50 points in the hole. You'll get no packet at all, no money at all, and no benefit to the size and diversity of your field at all, as teams drop out rather than go through the motions of such farcical games.
5) Docking points won't help anyone edit packets that come in on Thursday any more than a $150 penalty does. It's a remarkably drastic solution that does nothing to address the problem.
Matt Weiner
Founder of hsquizbowl.org

User avatar
theMoMA
Forums Staff: Administrator
Posts: 5695
Joined: Mon Oct 23, 2006 2:00 am

Re: Incentive for Improving Submissions...

Post by theMoMA » Sun Nov 30, 2008 11:09 pm

If any tournament wants to stick its neck on the line and decide games based on packet submission, we can talk about this. But it certainly seems like the legitimacy of a tourney would suffer if this happens.
Andrew Hart
Minnesota alum

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Incentive for Improving Submissions...

Post by Mechanical Beasts » Mon Dec 01, 2008 12:03 am

Matt Weiner wrote:1) The purpose of the game score is not to enforce penalties for things we don't like, even if we don't like them for good reason. How about we dock 50 points per game for showing up to a trash tournament unshowered, too?
This is a strawman; the difference between actively damaging a tournament's legitimacy through forcing packets to be edited the night before and smelling is pretty clear.
Matt Weiner wrote:2) Financial penalties can be internally distributed by a team towards those who were actually responsible for the packet being late; a points penalty affects even those who fulfilled their responsibility.
I feel like it's hard to get people who write their part of the packet much later than the rest of their team and so can be characterized as "irresponsible" would be unlikely to be easily charged money by the rest of the team. And sometimes there's reason that you won't have the ultimatum of "... or you're banned from the club," and that might not be a healthy thing at all. it's not like you don't let them answer questions and learn. On the whole, though, this is a point that bears thinking about.
Matt Weiner wrote:3) Something similar was once tried at a UTC tournament--teams who had submitted packets were given a free win in the standings, effectively making it the tiebreaker. No one liked it and it had no apparent effect on packet-submission behavior.
What were the complete packet-submission guidelines for this tournament? Sometimes they're a bit wonky (i.e. a $50 base fee, you also write high school or trash questions to make up the fee, whatever) and perhaps that could have an effect. Or maybe this tournament went by the same standards that are common elsewhere? I can't comment much otherwise, save to say that i think you're marginalizing the difference between a point penalty and a free win.
Matt Weiner wrote:4) This will, in fact, discourage attendance. Teams will not want to show up to events that they start 50 points in the hole. You'll get no packet at all, no money at all, and no benefit to the size and diversity of your field at all, as teams drop out rather than go through the motions of such farcical games.
Doesn't this presuppose 3, that packet submitting behavior is unchanged? If 3 holds, then it's trivial to see that this policy is useless, since it does no good; 4 is unnecessary. If 3 does not hold, then you will have more teams getting packets in on time, so it's not like all the monetarily penalized late submitters are spontaneously changed into points penalized late submitters.[/quote]
Matt Weiner wrote:5) Docking points won't help anyone edit packets that come in on Thursday any more than a $150 penalty does. It's a remarkably drastic solution that does nothing to address the problem.
Again, this presupposes that packet-submitting behavior is unchanged.

EDIT: complete sentences.
Andrew Watkins

User avatar
Theory Of The Leisure Flask
Yuna
Posts: 843
Joined: Fri Aug 29, 2003 11:04 am
Location: Brooklyn
Contact:

Re: Incentive for Improving Submissions...

Post by Theory Of The Leisure Flask » Mon Dec 01, 2008 12:23 am

So, like, you realize what's bound to happen sooner rather than later if this practice of docking people points for turning packets in late actually catches on?

One team is going to answer as many or more tossups than the other team, and not neg more, and have at least as high a bonus conversion, and still lose. That's right, we'll have ourselves a Colvin Unfair Result. Remember those?

I never played CBI, and I would likewise never play in a tournament that assigned in-game bonuses or penalties for packet timeliness.
Chris White
Bloomfield HS (New Jersey) '01, Swarthmore College '05, University of Pennsylvania '10. Still writes questions occasionally.

User avatar
Mechanical Beasts
Banned Cheater
Posts: 5673
Joined: Thu Jun 08, 2006 10:50 pm

Re: Incentive for Improving Submissions...

Post by Mechanical Beasts » Mon Dec 01, 2008 12:55 am

Theory Of The Leisure Flask wrote:So, like, you realize what's bound to happen sooner rather than later if this practice of docking people points for turning packets in late actually catches on?

One team is going to answer as many or more tossups than the other team, and not neg more, and have at least as high a bonus conversion, and still lose. That's right, we'll have ourselves a Colvin Unfair Result. Remember those?

I never played CBI, and I would likewise never play in a tournament that assigned in-game bonuses or penalties for packet timeliness.
Sure, but in this case you could at least have avoided said unfair result if you did what the TD asked you to.

In recalling what a Colvin Unfair Result is, I came across this:
http://www.ocf.berkeley.edu/~quizbowl/n ... er_1-8.txt
Andrew Watkins

User avatar
setht
Auron
Posts: 1178
Joined: Mon Oct 18, 2004 2:41 pm
Location: Columbus, Ohio

Re: Incentive for Improving Submissions...

Post by setht » Mon Dec 01, 2008 1:03 am

I probably should have noted in my previous post that I'm not convinced the current system is so broken that we need to start looking into new sorts of penalties. My impression is that there are more high-quality tournaments than ever, and that the ludicrous amount of time editors have to put in to produce a high-quality set hasn't changed much over the past few years--it sounds like submissions are as ridiculously tardy as ever, but higher in quality; on the other hand, I think standards have risen enough to off-set the improvement in submissions. More importantly, I think there may now be a large enough community of good-hearted, quality freelance writers out there to make a significant difference if a TD sends out a distress call. The main point I wanted to make in my previous post was that if other people do feel the current system is broken and want to look into other penalty systems, the points penalty seems more attractive to me than the barring teams penalty. Continuing on that basis--arguing only over whether a points penalty or a barring teams penalty is a better option, should it ever come to that--I'll respond to Matt's points.
Matt Weiner wrote:Alright, if you want me to take this seriously, the reasons this is a bad idea are legion:

1) The purpose of the game score is not to enforce penalties for things we don't like, even if we don't like them for good reason. How about we dock 50 points per game for showing up to a trash tournament unshowered, too?
2) Financial penalties can be internally distributed by a team towards those who were actually responsible for the packet being late; a points penalty affects even those who fulfilled their responsibility.
3) Something similar was once tried at a UTC tournament--teams who had submitted packets were given a free win in the standings, effectively making it the tiebreaker. No one liked it and it had no apparent effect on packet-submission behavior.
4) This will, in fact, discourage attendance. Teams will not want to show up to events that they start 50 points in the hole. You'll get no packet at all, no money at all, and no benefit to the size and diversity of your field at all, as teams drop out rather than go through the motions of such farcical games.
5) Docking points won't help anyone edit packets that come in on Thursday any more than a $150 penalty does. It's a remarkably drastic solution that does nothing to address the problem.
I agree that the game score's purpose is not penalizing people for things we don't like, but if we ever get to the point of implementing a points penalty, it'll be because "people submitting packets late" has become a much bigger issue than "something we don't like." The second point seems to me like the strongest objection to the points penalty; presumably the "barring teams" penalty can be changed to a "barring the people who didn't write their questions in time" penalty, while I don't see a good way of individualizing the points penalty without making it completely anemic. This may be a big enough objection to switch me over to the "barring teams" crowd; thank goodness I don't actually feel like instituting either of these penalties! Third point: I am shocked, shocked to see Matt using a UTC tournament as evidence for what will happen at any non-UTC tournament; more seriously, it's an interesting story but I'm not sure how much faith I have in that data point as a predictor: it's not quite the same thing as a points penalty, which I suspect people would care about more (hopefully in that "oh man, better get this packet in on time to avoid a points penalty" way). I'm not sure I agree with point 4--I think teams will still show up and play under a points penalty. I could be wrong (Chris White's post suggests I am), and if I am I'd be even less inclined to consider instituting such a penalty. However, in debating the relative merits of the points penalty and the barring teams penalty, clearly this one goes to the points penalty--it can't possibly discourage attendance more than the barring teams penalty. I would also ask how many teams were discouraged from attending that UTC tournament. Matt's fifth point raises the tantalizing specter of a solution that actually addresses the problem of editing packets that come in a day or two before a tournament. I can think of no such solution (as in, I'm editing a tournament, a packet comes in two days before the tournament; what possible penalty system could I have in place that alleviates me of the burden of editing the packet up to snuff on such short notice?). Can anyone else? Because that would be totally sweet.

Look, any TD who puts up a large financial penalty for late packets, or threatens to bar teams, or says they'll dock points or collect a pound of flesh per day past the drop-dead deadline or whatever--they really don't want to assess that penalty. They really want all the teams to submit decent packets in a timely fashion. Actually, I guess Ryan and Bruce might prefer getting some late submissions to get more money out of running a tournament, but I think what I said holds for most TDs. The trick is to find a penalty that is a sufficient deterrent to people who would otherwise submit late packets, without doing more harm to other important things. Game validity certainly is one of those important things. Since a points penalty almost certainly isn't a perfect disincentive (gets everyone to submit packets on time, doesn't discourage anyone from attending), I would not advocate implementing it under the current conditions--as I said above, I think the current situation packet submission isn't great, but it's not so bad that I'd be comfortable as a TD with barring teams from play or docking points as a penalty for late submissions.

-Seth
Seth Teitler
Formerly UC Berkeley and U. Chicago
President and Chief Editor, NAQT
Emeritus member, ACF

User avatar
No Rules Westbrook
Auron
Posts: 1223
Joined: Mon Nov 22, 2004 1:04 pm
Contact:

Re: Incentive for Improving Submissions...

Post by No Rules Westbrook » Mon Dec 01, 2008 12:03 pm

Let's be clear, I was just indulging Bruce's cost-benefit analysis - the next time I edit something, don't submit late on the theory that you're doing me a favor!

Okay, anyway, I disagree with Seth that teams would still show up if we (one crazy day, heaven forbid) decided to implement a points penalty. Telling teams they're going to start each match 50 points behind seems...I don't know, insulting?, is that the right word?...I just predict that teams would say "ah, screw it, we want to play this game, but we're not gonna play an unfair version of it." It seems equivalent to me to telling teams they have to play with their dominant hand tied behind their back (Seth might call this poetic justice par excellence), or they all have to wear big signs that read "I am a horrible person - I submit packets late!"

Anyway, I want to make one important point. There is only one class of writers/teams who really matter in this analysis - "writers and teams who are competent and know how to write okay packets, but still packets that need a little bit of work to make them really good." Completely inexperienced people don't matter, because you basically know you're going to have to replace their stuff anyway - you may as well just start writing your own packet to replace theirs. Really good writers don't matter that much, because you can just make minor changes to their stuff and it's good to go - unless there are things like repeat issues, but that's fairly minor. My conclusion being: the only people editors should be really interested in deterring are the "okay but not great writers." Of course, everyone should act in good faith and submit things on time, but I'm being pragmatic about what matters most.
Ryan Westbrook, no affiliation whatsoever.

I am pure energy...and as ancient as the cosmos. Feeble creatures, GO!

Left here since birth...forgotten in the river of time...I've had an eternity to...ponder the meaning of things...and now I have an answer!

Locked