Should distributions have some round-to-round leeway?

[setht]

There's been some talk recently about how different topics are treated in distributions, e.g.

I firmly believe that allowing the distribution to change within a tournament subverts the purpose of having a distribution at all.

I disagree with Matt's sentiments (as I understand them), and I think this is a potentially worthwhile topic for discussion, since it might affect how geography's place in the ACF Nationals distribution is codified, or more generally how any topic's place in the distribution is set.

Matt, could you say a little bit more about what forms of distributional leeway seem acceptable to you, and which do not? Assuming the distribution listed here is one that you approve (which seems safe since you wrote it)--perhaps not in every detail but at least in form--then it would seem that you're okay with having rounds that vary somewhat in the number of religion or myth or philosophy questions, that you're okay with having your choice questions, etc. I'm fine with all of those things too, but I think that being fine with that means we should also be fine with allowing the number of fine arts or geography or social science questions to vary (within some reasonable bounds) from round to round as well, and I get the impression that you disagree.

I'm also interested in hearing what other people have to say, of course, but Matt is one of the few people who have expressed or hinted at an opinion on this matter.

-Seth

[Pilgrim]

I don't think having a definite distribution for every round is either feasible or desirable.

Assume for the sake of argument that there is some "ideal distribution" that every round should aspire to (I'm really not sure if I agree with this or not, but it's irrelevant). Now, with splitting every part of the distribution up into its smallest subcategories, we have way more than 20 or 40 categories - thus, we are guaranteed to have parts of the distribution that deserve questions in the tournament, but can't have a guaranteed number of questions in each round.

Similarly, what if in this ideal distribution, we have decided that fine arts should make up exactly 13/80 of the distribution? Then it obviously makes more sense to have 3/3 required FA and allow people to write an extra FA question if they like, because then we get closer to the ideal distribution than if we required either 3/3 FA or 7 total FA questions.

EDIT: I guess this doesn't necessarily rule out a policy of have no more than one question worth of wiggle room for any category in any given round. I don't have a strong opinion on this.

[DumbJaques]

I think if you're going to argue that slight variations in the distribution (and I think we can all agree that what we're talking about here is what to do with that extra 1/1 or 2/2 that is filled with extra fine arts, geography, social science, trash, RMP, etc) are incompatible with fair play, then theoretically you've got to submit that every packet must meet identical sub-sub-distributions in terms of world area, time period, etc. Such things are often much more consequential than the strict distributional area to a particular team or play having an advantage - I'd imagine that Paul Gauthier, for example, is far more likely to get a social science question on linear b and a history question on Suppiluliumash the Great (may he live forever!) than he is to get a social science question on Syntactic Structures and a history question on the Young Turks. So if the stated purpose of fixing the distribution without any fluctuation is to avoid game results being invalid as a result of questions hitting a certain teams way - well, that already happens (and would probably happen even if we fixed every conceivable aspect of the distribution). Obviously you have to draw the line somewhere, but I don't think you can say that having an extra fine arts bonus in one packet and an extra social science bonus in another is in any way more consequential than, say, what century your European history submissions tend to be on

[grapesmoker]

I'm gonna jump on this bandwagon. The distribution already has a lot of variation in it that we're all actually fine with. To wit, even in relatively rigid subdistributions like history and literature (which leave very little room for variation), the constraints are mostly negative; they ensure that you vary your questions by time period and country but most of them don't require you to write on anything specific. I think in situations where it's possible to fix the distribution definitively (my ideal candidate for this is an ACF Nationals with no trash: 4/4 each of the big 3, 3/3 each of RMP and arts, 2/2 social science, 1 of the big 3 as a tiebreaker) it probably ought to be done, but many tournaments leave some wiggle room with the 1/1 your choice option. I think that's fine too and I've never noticed that it has impacted the quality of any tournament that I've attended.

[Mechanical Beasts]

In some sense, you're always allowing round-to-round leeway. After all, we're only accepting that a distribution like the one you offered, Jerry, doesn't have round to round leeway in the sense that it has the same number of top-level categories per round. But we're content to vary on social science subcategories because we see the distinctions between different social science categories as sufficiently negligible. If we didn't (and I could imagine a mathematical take on econ being construed as far more different from sociology than sociology is from some of what quizbowl categorizes as philosophy) then we'd need two hundred question rounds to be "fair" per se.

[aestheteboy]

Having a whole number distribution is simply a matter of convenience. There is no reason to say that to have decimal distribution (that is to say to allow slight variations in per packet distribution) is unfair.

It becomes unfair only when the deviation of per packet distribution from per tournament distribution becomes anything larger than the minimum variation necessary (i.e. one question in any category, subcategory, subsubcategory etc.) For example, if we want 2.6/2.6 of SS on average for the entire tournament, the only combination of SS questions that should occur in any packet is 2/3, 3/2, or 3/3, and they should be in 2:2:1 ratio.
I personally prefer this more nuanced distribution, but obviously it requires a lot more attention and technical work, and it's almost impossible to implement with packet-submission tournaments. Given that there is no consensus of the "ideal" tournament distribution, it makes sense to just do the convenient whole number distribution unless there is a compelling reason not to. I'm not sure if geography is one of them or not.

[bmcke]

I've always wondered if the sub-distributions could be mapped out for a quota over whole tournaments.

1/1 Russian History
2/2 economics
1/0 or 0/1 Shakespeare
1/1 geology
4/4 Greek myth
1/1 World War Two
1/0 or 0/1 jazz
3/2 or 2/3 current events
1/1 anthropology
1/0 or 0/1 astronomy

Like that, except for 240/240.

For any tournament, the expected overall distribution is much more specific than the distribution written into the rules of packet writing. I don't know if there are any written guidelines to fit these expectations, or if editors can just see what "feels right". How much would it guide/restrict writing to have a super-specific distribution model, spanning a whole tournament (or else a batch of 100 questions)? Have other people used this kind of system?

[AlphaQuizBowler]

I mean, in the Fall Novice set, we mapped out everything by category and subcategory fairly thoroughly, though things like Shakespeare and World War I probably don't need to be planned out.

[Matt Weiner]

Obviously there's going to be things in some rounds but not others; every tournament will have more then zero Latin American history questions, while no tournament will have one in every round. To reject making rounds as consistent as possible because it's not realistic to make them perfectly consistent is Westbrookian in its use of idealism discovered five minutes ago as grounds for refusal to address the actual argument.

I think there are more things we can do, including making sure that social science within a round breaks down in some sort of logical way (perhaps among laboratory vs. speculative disciplines, just to throw out an idea).

[grapesmoker]

Obviously there's going to be things in some rounds but not others; every tournament will have more then zero Latin American history questions, while no tournament will have one in every round. To reject making rounds as consistent as possible because it's not realistic to make them perfectly consistent is Westbrookian in its use of idealism discovered five minutes ago as grounds for refusal to address the actual argument.

I think if people argue this it's because it's not particularly clear what the middle ground entails. We already have top-level requirements that seem pretty straightforward, and even the subdistributions have requirements as well. What more do we need?

I think there are more things we can do, including making sure that social science within a round breaks down in some sort of logical way (perhaps among laboratory vs. speculative disciplines, just to throw out an idea).

See, I don't know that this is an important thing to observe, or a good idea. I'm fine with telling people "your social science distribution must include no more than one question from any social science discipline," (for example), but I don't see much point in demanding a specific breakdown in lab vs. speculative disciplines. That's akin to me saying, "you must write one of your physics questions on topics of theoretical importance and another on a topic of experimental importance," and I don't see that this is important to do.

[Magister Ludi]

Obviously there's going to be things in some rounds but not others; every tournament will have more then zero Latin American history questions, while no tournament will have one in every round. To reject making rounds as consistent as possible because it's not realistic to make them perfectly consistent is Westbrookian in its use of idealism discovered five minutes ago as grounds for refusal to address the actual argument.

I think there are more things we can do, including making sure that social science within a round breaks down in some sort of logical way (perhaps among laboratory vs. speculative disciplines, just to throw out an idea).

I agree with Matt here. I understand that when you're editing a packet submission tournament it is unrealistic to try to guarantee a perfect distribution, but I think it is admirable to try to get as close to this ideal as possible. Personally I prefer to write full tournaments because I have complete control over the sub-distribution. Before writing HI, I mapped out how many questions I wanted in each sub-area and made a list of how many easy, normal, and hard tossup answers I wanted. I agree with the sentiment Mike Sorice expressed in some other thread where he said there should at least be a certain number of Civil War questions in every tournament. I feel like it is the responsibility of the head editor to ensure that there is a bare minimum of certain sub-categories over the course of the tournament. I dont mean to suggest that every tournament needs to have as strict a distribution as I use when I write, but I think it is foolish to malign this ideal if someone is willing to put in the work to make it happen.

[No Rules Westbrook]

Unsurprisingly, I hate laboriously mapping out subdistributions. My general rule is just to be reasonable and fair, and use the best questions where possible. There shouldn't be any unconscionable deviations from the norm - i.e. no opera questions the whole tournament, no Norse mythology, one geography tossup the whole tourney if you advertised around a 1/0 per packet distro, etc.

But outside of those general constraints, I think meticulous subdistros are pretty much the last thing we should be worrying about. If there's a comparative dearth of Middle Eastern history in one tournament, maybe there will be a wealth in the next tournament - as long as there's no bad faith (keeping out topics by spite, lets say), I don't think you have any grounds to complain. Take a more long term view - it all evens out in the end...if people aren't writing enough questions on X subject that merits more attention, people will realize it and it'll be corrected - plus the law of averages will just have its way.

[Skepticism and Animal Feed]

Round-to-round leeway that is built into a distribution (for instance, a "Your Choice" distro or a 2.5/2.5 scheme for RMP/FA) does not seem inherently unfair. Rather it seems like a tool that can be used for good, for evil, or for something between good and evil.

[setht]

Obviously there's going to be things in some rounds but not others; every tournament will have more then zero Latin American history questions, while no tournament will have one in every round. To reject making rounds as consistent as possible because it's not realistic to make them perfectly consistent is Westbrookian in its use of idealism discovered five minutes ago as grounds for refusal to address the actual argument.

I think there are more things we can do, including making sure that social science within a round breaks down in some sort of logical way (perhaps among laboratory vs. speculative disciplines, just to throw out an idea).

Okay, so we all agree there are some coherent categories ("Latin American history" or "earth science" seem like fine examples) that should get some questions per tournament, but generally less than one question per packet. In the case of things like Latin American history and earth science we've packaged them with some other things that should get less than one question per packet (like "Asian history" and "computer science") and with things that should consistently get more than one question per packet (like "American history" and "physics"). Most of our packaging choices make sense from an academic standpoint, but if (as I believe) the primary purpose of the packaging is to help deal with cases like Latin American history and earth science where treating these topics as top-level categories in the distribution would lead to systematic misrepresentation in tournament sets, then what's the objection to something like "2/2 social science + geography, with a maximum of 1/1 geography in that 2/2"? Yes, it means that some rounds have more geography than other rounds. Is there some reason to think that that's particularly worse than having some rounds with more earth science than other rounds?

I don't think "making rounds as consistent as possible" is the primary goal we should be setting for ourselves. For a packet submission tournament, I think it makes more sense to start by setting a fair amount of the 20/20 distribution (e.g. start with at least 4/4 each of literature, history, science, 2/2 each of fine arts and RMP, etc.), then leave some wiggle room--which we currently do by leaving space for some "your choice" in the first 20/20--which the editors can fill based on various criteria (which remaining questions in the submissions seem strongest, which questions cover topics that seem under-represented in the rest of the set, etc.). If that means that some rounds have 8 literature questions and others have 9, or that some rounds have 6 fine arts and others have 5, I think that's fine. As long as the amount of leeway doesn't creep too high and people are given fair warning on how much leeway to expect I don't see a problem. Producing stronger packet-submission sets by giving editors a little more choice on which questions to use seems more worthwhile to me than setting things up so that teams can regularly predict the topic of tossup 20 after hearing tossup 19.

To be clear: I think the ACF distribution (as it's listed here) gives some leeway per round, but I would like to see it move towards giving a little more leeway in the first 20/20.

I agree with Matt here. I understand that when you're editing a packet submission tournament it is unrealistic to try to guarantee a perfect distribution, but I think it is admirable to try to get as close to this ideal as possible... I feel like it is the responsibility of the head editor to ensure that there is a bare minimum of certain sub-categories over the course of the tournament.

I'm not sure you do agree with Matt, but perhaps I'm misreading Matt's post or your post (or both). From what you wrote, I would guess that you're fine with some leeway in the round-to-round distribution if it helps move a set closer to the hypothetical perfect distribution. My impression is that Matt finds at least some forms of leeway unacceptable--for instance, if the ideal number of geography questions per tournament averages out to 1.75 questions per packet, my impression is that Matt would rather see 1/1 geography in every round than have a set with 1/1 geography in 3/4 of the rounds and 1 geography in the other 1/4, and my impression is that you would go the other way.

-Seth

[Dresden_The_BIG_JERK]

It seems like there's an issue of category versus subcategory. I think it would be completely illegitimate to have a standard subdistro (Oh, geez, no Shakespeare question yet, I better be ready!), and even a completely round-to-round, rigid category structure seems unnecessary, although it has been done. The IHSA (yes, high school) mandates a certain number of each category per round, and it is rigid. It got to the point where second half and timeout substitutions would be made based on what TU categories still had to show up...a little silly, frankly.

If one round has 5 lit and 3 history, and the next round the reverse, that seems okay to me.

[grapesmoker]

If one round has 5 lit and 3 history, and the next round the reverse, that seems okay to me.

Virtually no round in collegiate qb has such variation. What variation there is is usually in some tradeoff between arts and RMP and social science/geography.

[Dresden_The_BIG_JERK]

If one round has 5 lit and 3 history, and the next round the reverse, that seems okay to me.

Virtually no round in collegiate qb has such variation. What variation there is is usually in some tradeoff between arts and RMP and social science/geography.

Just an example...

[grapesmoker]

Just an example...

Sure, I'm just pointing it out because I think there's pretty good agreement that every round should have 4/4 each of the big 3, and even perhaps some tentative agreement that arts and RMP should be another 3/3 each (although that's varied in the past). I think the main source of variation is the necessity of squeezing things like your choice and trash into academic distributions, which results in something else being left out. Owing to the general acceptance of the subdistribution of history, literature, and science, that something usually ends up being social science, arts, or RMP.

[Captain Sinico]

If one round has 5 lit and 3 history, and the next round the reverse, that seems okay to me.

Virtually no round in collegiate qb has such variation. What variation there is is usually in some tradeoff between arts and RMP and social science/geography.

I don't think it's true that virtually no round has such variation. Of course the conventional wisdom is that it's fairer to trade among arts, RMP, and social science than it is to trade between history and literature, say, but a practice conventionality's fails to make any argument for its rectitude.

MaS