The Quizbowl Resource Center

The question management system I'm working on, Packetizor, currently does not randomize packets. That is by design.

In my early days of tournament production, I used randomization software to place questions under certain constraints (major categories balanced by halves, consecutive questions not in same category, minimizing tossup/bonus pairings within category, etc.). People didn't seem to like the results. There was always some kind of feng shui violation. So, I stopped trying to randomize.

But some people still seem to advocate for randomization, so I wonder if I should include it somehow in Packetizor. Do you like randomization? Is it actually better than editors deciding where to put their questions? Does it save much effort? What are the common feng shui pitfalls with it that we might try to avoid?

Thanks in advance for the input.

A classic complaint from back in the day was that close games on packets were the final question was trash were derided for being "decided by trash", despite of course the game actually being determined by the previous 19 tossups just as much as by tossup 20. I don't know if there's even trash in quizbowl anymore, but don't put any potentially illegitimate questions as TU 20. No trash, no experimental tossups that might not go over well, etc.

what I did for Math Monstrosity was randomize it and then make manual adjustments for the remaining feng shui violations.

I think randomization is possibly like kerning; you can get most of the way there with a simple algorithm but at some point you just have to look at it and adjust by hand.

When I was brainstorming my own Packetizor, I had a design to just have a "Randomization" options page with common "fengshui" violations:

- No trash tossup in first/last three questions.
- No back to back questions of same category.
- No tossup/bonus of same category on the same "number"

And other the heading of randomizing questions _across_ rounds:
- No same round, or back-to-back rounds, of questions with the same or very similar answerlines

And then you could toggle whatever options you'd like, and the algorithm would either logically attempt to satisfy everything -- or more decidably, just run for a minute (or some time limit) attempting to find a solution, then give up and give you the ordering with the least amount of violations.

And then beside that options page, there would be the ability to drag/drop questions around across rounds / within a round after randomization had occurred.

These are all fengshui things that would be really annoying to keep track of mentally by hand, but would be really easy for a machine to do.

You should probably talk to Cody to get him to tell you exactly how he automated packetizing for HSAPQ. For PADAWAN, he gave me HSAPQ's standard rules, I tweaked them, and he executed accordingly. Once editors actually tagged their questions by category, it took (according to him) about an hour to packetize the entire set. His management made the process very smooth. Of course, we checked it afterwards and made further adjustments.

All of the rules that Raynor mentioned were in force, plus some more. These were the rules:

1 tossup in intervals 1-5, 6-10, 11-15, 16-20 - Literature, History, Science
1 tossup in intervals 1-6, 7-14, 15-20 - RMP, Fine Arts
1 tossup from each in interval 1-10, 11-20 - Social Science, Geo/CE/Trash
No consecutive tossups - Literature, History, Science, Fine Arts
Not tossup 19 or 20 - Geo/CE/Trash

One of the things we checked by hand is that there were a maximum of two tossups on Classics per round (only viable for a housewrite, probably). And, of course, all of this took place after we decide what questions went in which packets to begin with.

I also mark which questions need to come before or after each other in the set. Max, I believe, still has nightmares about executing these requirements for CRR, because I was very fastidious about these, and we were randomizing entirely by hand at that point.

ThisIsMyUsername wrote:Once editors actually tagged their questions by category, it took (according to him) about an hour to packetize the entire set.

This strikes me as a long time for an automated system. That's starting to approach the territory where there's no real time savings over question-by-question placement, especially when you consider then manually checking and adjusting.

Having manually created sets, a hour would be very short indeed.

On the other hand, in my experience using HSAPQ's randomization, a hour is a very generous approximation. You have to do a lot of grunt work both before and after randomizing (in particular, I found HSAPQ's randomizer LOVED doing things like placing 3 tossups on "Japan" from 3 different categories in the same packet).

Cheynem wrote:Having manually created sets, a hour would be very short indeed.

Agreed. But I feel like the last set I packetized (in Packetizor) took 3-4 hours for 12.5 packets. If there is some precursory work leading up to an hour of computation and followed up by the manual checks and fixes, I'd guess that the time is in the 3 hour neighborhood again.