What protests are for & why you should make less of them

[Matt Weiner]

Hey folks. This is a big area in my current project to propose rule revisions to PACE (and to ACF and HSAPQ which use slightly modified forms of the PACE rules) to deal with edge cases and other unforseen situations that have demonstrated gaps in the current rules since their adoption in 2008. It's also something that's increasingly been an issue in quizbowl over the past few years.

What do people want to see done about answer acceptability in light of the tension between quizbowl as a game and the principle of rewarding knowledge? In the distant past, the game aspect dominated--if you buzzed in and said "The Man Who Corrup--err, I mean, The Man That Corrupted Hadleyburg" you were absolutely wrong. You also couldn't buzz in on the clue "this man was king when the Jacobites rebelled--" and say George II and then protest that the clue you buzzed on could apply to him, when previous clues made only George I acceptable. The responsibility was on the writer to make a reasonable effort to avoid punishing imperfect knowledge, and on the player to take calculated risks. Questions that are not factually inaccurate and answers that don't fit all of the clues are not protestable.

My view is that the onus is on the question writer and the players to find a reasonable balance between not trapping people by having ambiguous clues late in a question that are wrong due to prior clues, and using common sense to allow for questions to be structured in a pyramidal way and flow properly. The writer does not have to say "it's not Louis XVI, but…" eight clues into a tossup on "Charles I" that uses the clue "he was beheaded by…". It's your responsibility as a player to recognize from context, other clues, and the structure of English sentences that you cannot buzz there with "Louis XVI" and then protest based on "but Louis XVI *was* beheaded!"

The purpose of protests is not to find tendentious reasons to make wrong answers correct. The purpose of protests is not to remove "bad questions" from the game at the discretion of players or the TD. The purpose of protests is solely to allow for players to have the opportunity to correctly answer the questions presented to them when errors in the question prevent this. Thus, an overly restrictive answer line, a clue that does not refer to the answer at all, or a fully formed leadin that point to multiple possible answers are protestable, as is a question with contradictory information. "That leadin is too easy" or "technically the eleventh clue in this fourteen-clue tossup could apply to my answer even though the other thirteen clues do not" are not protestable. They may be signs of a bad question, but that is not what protests are for.

In the other sense, protests based on a misunderstanding of language or facts that try to take points away from opposing teams are also a problem. I don't care that your definition of historical "correctness" says that "Russia" is not ever a promptable name for "the USSR"—the fact that people at the time and history books now often use this synecdoche means that it's acceptable, though it should be clarified.

There has also been a failure to clearly announce ahead of tournaments what rules will be used, and then correctly apply that ruleset. Too many people want to throw their hands up and go with the old "well, just throw the question out and read a new one" evasion instead of ruling on a protest. This just means that you have 50/50 chance of screwing someone, instead of doing the work to resolve the protest correctly and then having to deal with the people who were ruled against getting pissy. Don't do this if you claim to be using the ACF or NAQT rules and you are ruling on a simple issue of correctness or "should have been prompted"--the answer is either right or wrong, and you award points on the original question accordingly.

Though I don't have statistics to point to, I get the sense just from being a player and TD of various events since the start of the century that protests in both directions have gone up. Clarifying the rules can only help, and will certainly remind people ruling on protests that a right answer is a right answer, not a reason to replay the question. But there also has to be a good-faith effort from teams to stop trying to game the system. Sometimes you lose games. Sometimes questions are bad. Sometimes questions are not bad but you think they are. Find a way to deal with these situations instead of abusing the protest rule, and certainly stop getting so upset when your protests are ruled on correctly because you wished the TDs would have invented new rules out of thin air or relied on "facts" provided by you that aren't true.

And definitely start learning that English sentences have meaning and quizbowl exists in a larger context of objective factual relationships. I almost hesitate to open this can of worms, but I'm pretty sure the "just buzz in and say something kind of related to the question topic, and expect to be right" mentality is a product of the "canon"/packet-memorizing approach to quizbowl. If you don't actually understand what these words refer to in the external world and are just Chinese-rooming a big stew of one-to-one buzz/answer relationships, then it's no surprise that you can't distinguish two things that are "associated with" each other in your head from the actual specific relationship between the clue and the answer specified in the question. If this describes you, then it's your problem with the shortcuts you have taken to improve, not the writer or the TD's burden to accommodate you.

[grapesmoker]

Lots to think about here, but pertinent to the example of "The Man that Corrupted Hadleyburg," I would like to see a rule change which states that a player is allowed to modify their answer mid-way. In other words, the moderator should wait until the player has finished delivering their answer before ruling; if a player realizes half-way through that they're saying something incorrect and go back to reverse themselves, that should be allowed. We all sometimes have moments when mouths run ahead of brains or people just stumble over words, and as long as you correct yourself and deliver the proper information, I think that should be allowed and more in the spirit of rewarding knowledge.

[1992 in spaceflight]

One thing I'd like to maybe see is a "How to Resolve a Protest" guide published by some ACF editors or community members. This would be really useful to point first-time hosts to.

EDIT: Like "how to determine if a protest will matter if a tossup is protested," how to determine if you need to throw out and replay a question, etc. That's the kind of guide I was thinking of.

[ThisIsMyUsername]

Here are two cases, each of which I have seen resolved three different ways over the course of my quizbowl career. I've also seen them argued from all sides on the forums. I'm not, for the moment, trying to re-open debate, but I would like to know if ACF, PACE, and NAQT have official policies on what the correct thing to do is in both of these situations, so I know for future. If they do not, I would like to see provisions for these situations explicitly spelled out. (I agree with Matt Weiner that all tournaments should, in advance of the tournament, announce which protest rules they are going by. I would add: moderators also need to be instructed in correct procedures.):

Case 1: Player buzzes in on the second line of a tossup and says an answer that is a subset of the written correct answer (e.g. "eagles" instead of "birds"). This more specific answer is correct for the clue buzzed in on, but does not apply to all of the other clues earlier in the tossup (e.g. some of them are about non-eagle birds). Is the correct procedure:

A. To anti-prompt: Player must eventually give the written answer to be deemed correct.
B. To rule the player incorrect: A correct answer must apply to all previous clues
C. To rule the player correct: Player should not be penalized for not knowing earlier clues. If he had known the earlier clues, he would have buzzed earlier!

If the correct procedure is A or C, then writers/editors should make it a matter of course to provide a list of the more specific answers that the clues might trigger. (Some writers/editors have already begun doing this.)

Case 2: The tossup points to two mutually exclusive answers (i.e. there are clues that unambiguously refer only to Answer X and also clues that unambiguously refer only to Answer Y). Player buzzes in after a clue referring to Answer Y and says "Answer Y", but the tossup's stated answer is Answer X. Is the correct ruling:

A. To rule the player correct: Player should not be penalized for not knowing that earlier clues applied to Answer X. If he had known that the clues apply to Answer X, he would have buzzed earlier to say Answer X.
B. To throw out the tossup entirely and read a new one to both teams: The questions is deemed entirely illegitimate because "there is no right answer" and needs to be replaced.

Corollary to Case 2: If 2B is the correct protest ruling, then a corollary situation arises. If Player says Answer X and is ruled "correct", the other team should be able to protest this too to have the tossup thrown out. I've noticed that there seems to be a stigma around protesting the opposing team's correct answers in these situations. However, the whole point of claiming the tossup to be illegitimate is to say that both Answer X and Answer Y are incorrect. If protesting against the other team's "correct" answer is not allowable, then this creates a situation where the only way to get an ambiguous tossup thrown out is to neg it oneself, which is utterly preposterous.

[Matt Weiner]

In the second case, the ACF rules are pretty clear that they get the question (e.g. the "Black Sea" example). The first one, and anti-prompting in general, are one of the phenomena that has emerged since the last time the rules were seriously revised and should be addressed under the current project.

[The King's Flight to the Scots]

After talking about this on the IRC a bit, I think a major problem is that H.7 (where protest resolutions are laid out) is dangerously ambiguous about when you should throw a question out and when you should award points to the team that buzzed in - specifically, it says the following:

7.1. For a team claiming that the answer they gave to a tossup was correct but ruled incorrect by the packet or moderator under appropriate sections in rule H.4: If the other team subsequently converted the tossup, their tossup points, and their bonus points earned pursuant to that tossup, will be removed. The tossup points will be given to the team whose protest was upheld and their neg removed. The protesting team will be reseated and play a bonus, if necessary.

and

7.3. For a team seeking the replacement of a tossup under rule H.4: If the other team subsequently converted the tossup, their tossup points, and their bonus points earned pursuant to that tossup, will be removed. The teams will be reseated and a new tossup will be read. If the other team gets the tossup right, they will get the tossup points as well as the points earned on the original bonus. If the protesting team gets the tossup right, they will get the tossup points and be read a new bonus.

However, it doesn't really specify what situations fall under these headings. We should definitely remedy that before the next ACF event, if at all possible.

[Cheynem]

I was under the impression that you could change your answer midway through if you corrected yourself without finishing.

[Cody]

Under NAQT rules, yes.

Under the current ACF rules, once you have completed one incorrect word, you are wrong and cannot correct yourself. If the answer is "The Celebrated Jumping Frog of Calaveras County", you cannot say "The Celebrated Leaping Fro---no wait, The Celebrated Jumping Frog of Calaveras County" and be correct. You can say "The Celebrated Leap---no wait, The Celebrated Jumping Frog of Calaveras County" and be correct.

[Cheynem]

What bullshit.

[grapesmoker]

What bullshit.

I don't know the exact origins of this rule; whether it's something carried over from olden days (like not reading answers to bonuses used to be) or whether it's a more recent innovation, but as long as we're contemplating rule changes, I want to propose this for elimination. I don't really see what purpose it serves.

[Matt Weiner]

I can speak to that:

I wrote out a laboriously detailed explanation of how to deal with "immediate corrections of answers" for the PACE NSC because PACE decided a long time ago that we wanted to allow this. It's extremely complex to specify what is allowed and seems to satisfy almost no one, so it was omitted from the ACF rules when they were formalized in 2008. Translating it to ACF (that is, keeping it in the NSC rules that are being used as the base of the current revision) is certainly possible, but it won't be pretty.

[grapesmoker]

I can speak to that:

I wrote out a laboriously detailed explanation of how to deal with "immediate corrections of answers" for the PACE NSC because PACE decided a long time ago that we wanted to allow this. It's extremely complex to specify what is allowed and seems to satisfy almost no one, so it was omitted from the ACF rules when they were formalized in 2008. Translating it to ACF (that is, keeping it in the NSC rules that are being used as the base of the current revision) is certainly possible, but it won't be pretty.

Is the difficulty in specifying what constitutes a complete answer? I would think that as long as the person is talking, they're still answering. As soon as they stop talking, the moderator can make the ruling.

[vinteuil]

I'm guessing that what's being referred to is

H. Acceptable Answers

1. Players may correct their own pronunciation of answers in the following cases:
1.1. If a player makes an immediate correction of a clearly misspoken answer (e.g., “Shattle of Biloh, I mean Battle of Shiloh”) and the latter is correct, the moderator will accept the answer unless the correction began after the moderator began to verbally or physically rule the answer incorrect, in which case the correction will not be accepted and the answer will be ruled incorrect. To avoid giving extra information to the other team, the moderator will not comment as to the reason for ruling the answer incorrect until the entire play of the question by both teams is complete.
1.2. If the player “corrects” to a wrong answer (e.g., “John McCain, I mean Ron McCain”), the moderator should rule the answer incorrect. To avoid giving extra information to the other team, the moderator will not comment as to the reason for ruling the answer incorrect until the entire play of the question by both teams is complete.
1.3. If the moderator believes that the player is, intentionally or innocently, giving two plausible answers (e.g., “Monet, I mean Manet”), the moderator will immediately ask the player to clarify what his answer is and rule on the subsequent answer as correct or incorrect. If the answer given on the prompt does not resemble one of the answers given initially, the moderator will rule the answer incorrect regardless of whether an answer given at some point was correct. To avoid giving extra information to the other team, the moderator will not comment as to the reason for ruling the answer incorrect until the entire play of the question by both teams is complete.
1.4. In the event of two or more answers that cannot reasonably be confused with each other in pronunciation (e.g., “Botticelli, I mean Peyton Manning”), the entire answer will be ruled incorrect regardless of whether one answer given was correct. To avoid giving extra information to the other team, the moderator will not comment as to the reason for ruling the answer incorrect until the entire play of the question by both teams is complete.
1.5. When ruling an answer incorrect, if the moderator gives any indication of the reason for ruling incorrect (e.g., “I can’t accept that pronunciation”, “Sorry, too late”, “You gave too many answers”) that may indicate to the other team that one of the answers was correct, the incorrectly buzzing team may request that the tossup be thrown out and replaced with a substitute question to be read as normal for the opposing team only. If said offense by the moderator occurs on a bonus part, the team may request that the bonus for the bounceback opportunity be replaced under the procedure described in rule G.5. However, if the moderator does not commit such an error, the other team may answer the question as normal.
1.5.1. Conversely, teams may not protest that they were ‘misled’ by a moderator's disallowal of an answer they presumed to be correct.
These rules are intended to allow for the correction of an obviously mispronounced answer only; players are never allowed to receive credit for a correct answer after substantially changing the content of their answer.

[ThisIsMyUsername]

In the second case, the ACF rules are pretty clear that they get the question (e.g. the "Black Sea" example).

I've now been reading the ACF and PACE rules and I don't understand how you came to this conclusion.

The ACF Rules give two possible options for how this protest can be resolved:

7.1. For a team claiming that the answer they gave to a tossup was correct but ruled incorrect by the packet or moderator under appropriate sections in rule H.4: If the other team subsequently converted the tossup, their tossup points, and their bonus points earned pursuant to that tossup, will be removed. The tossup points will be given to the team whose protest was upheld and their neg removed. The protesting team will be reseated and play a bonus, if necessary.

7.3. For a team seeking the replacement of a tossup under rule H.4: If the other team subsequently converted the tossup, their tossup points, and their bonus points earned pursuant to that tossup, will be removed. The teams will be reseated and a new tossup will be read. If the other team gets the tossup right, they will get the tossup points as well as the points earned on the original bonus. If the protesting team gets the tossup right, they will get the tossup points and be read a new bonus.

The hypothetical I gave for Case 2 is that the tossup has already unambiguously pointed to two different, mutually exclusive answers. The Player has said the one of the two answers that is not listed. So, to use your Black Sea example: the situation would be I have buzzed in on a tossup whose written answer is Black Sea. The clues have been about both the Black Sea and Aegean Sea, and I buzz in and say "Aegean Sea". I am deemed incorrect, and so I protest, on the grounds that one of the clues I buzzed in on pointed unambiguously to the Aegean Sea.

You seem to be saying that ACF declares I should get the tossup, according to rule H.7.1, because Aegean Sea is a "correct answer" to the tossup, since some of the clues apply to it. But on what grounds in the current ACF rules is it clear that this case should be covered by H.7.1 rather than H.7.3? As first glance, the current language makes it sound like it's determined by what I "seek": if I demand that my answer be correct, it will be deemed correct; but if I demand that the tossup be thrown out, it will be thrown out. But surely, given the fact that current ACF rules seem to demand that a correct answer "fits all the clues given in the entire question text" (H.4.2.), a case could be made for why the correct protest resolution should be according to rule H.7.3? Also, the portion of the ACF rules covering the situation I described declares: "hence, there is no correct answer to the question" (H.4.5). Doesn't this preclude applying H.7.1, because no answer to the tossup can be deemed correct?

[Matt Weiner]

You seem to be saying that ACF declares I should get the tossup, according to rule H.7.1, because Aegean Sea is a "correct answer" to the tossup, since some of the clues apply to it.

Pretty much this. And my habit is to rely on what there exists of quizbowl "common law" and go with precedent when there are gaps in the rules.

Of course, I agree with your implied premise that we should work to eliminate such gaps and codify the rules objectively when possible, and thus that this section of the rules needs work.

[Cody]

I think H7 could be made a lot more clear, but I also don't think it's that ambiguous currently:

Protests that fall under H4.1-4.4 should be resolved using H7.1-7.2 because H4.1-4.4 are about giving a correct answer that is ruled incorrect.
Protests that fall under H4.5 should be resolved using H7.3-7.4 because the situation described in H4.5 states there is "no correct answer to the question" (thus, it would be impossible for your answer to be ruled correct). (well, also, there would be no reason to have H7.3-7.4 if it didn't apply to one of H4.x since it was clearly written with a certain situation in mind)
Protests that fall under H4.6 should be resolved using H7.5-7.6 because H4.6 is about accepting an incorrect answer from an opposing team.

[ThisIsMyUsername]

I think H7 could be made a lot more clear, but I also don't think it's that ambiguous currently:

Protests that fall under H4.1-4.4 should be resolved using H7.1-7.2 because H4.1-4.4 are about giving a correct answer that is ruled incorrect.
Protests that fall under H4.5 should be resolved using H7.3-7.4 because the situation described in H4.5 states there is "no correct answer to the question" (thus, it would be impossible for your answer to be ruled correct). (well, also, there would be no reason to have H7.3-7.4 if it didn't apply to one of H4.x since it was clearly written with a certain situation in mind)
Protests that fall under H4.6 should be resolved using H7.5-7.6 because H4.6 is about accepting an incorrect answer from an opposing team.

Yeah, what Cody says here strikes me as the most sensible reading of the current text, and this also matches up with what I've seen put into practice in most situations.

[Sam]

One of the "potential errors" in the list talks about answers that apply to "all the clues up to a significantly deep point in the question." What does "significantly deep" in this case mean? Is it an instance of discretion ("anyone who knows more about Louis XVI than a list of buzzwords would have realized long ago some earlier clues don't apply") or is there a hard and fast rule? If there is a rule, is it based either on basic knowledge of English (maybe the question repeatedly said "she" and the player answered with a man) or question structure (e.g., once the first clue after the lead-in is read)?