I'm not sure if I posted along these lines already in some other thread, so sorry if this is redundant, but Will's last point about "for everyone" reminds me of a pretty interesting chat room conversation I had about "regular difficulty" with some folks a few months ago, right after I announced ACF Regionals (I remember Mike Cheyne, Cody Voight, and Marshall Steinbaum were the main interlocutors but there were almost certainly others; all of the following will be pretty broad-stroke paraphrase since I didn't log it). The main issue was whether it would be inherently disingenuous for me to bring out a tournament of past Regionals' historical difficulty and claim that it is of an appropriate difficulty 'for all teams to play,' from the very best to the very worst. The rough conclusion we converged on was that "regular difficulty" as it currently stands is generally good for distinguishing a regional sample from the top 50, or maybe 60, or maybe 80 teams, but there really is a broad class of teams below that threshold for which it is not meaningfully rewarding knowledge. And that betrays something about the way regular tournaments are written, in part because the best editors are usually the best players: Most writers of regular sets nowadays see "regular" as a way to distinguish the best teams down to and just below some national median point, rather than working "bottom-up" to see what would work best for teams on the margins. There's almost always a bias towards over-ensuring that matches between team #1 and team #2, or team #10 and team #20, shake out well whenever there'd be a tradeoff with distinguishing team #100 from team #200.
Marshall came up with a pretty useful definition of regular difficulty [as practiced by, say, ACF Regionals 2014 or 2015] that looks sort of like this:
Regular difficulty is the minimum difficulty level at which the very best team can still be bothered to bring its full lineup against another similarly-good team. (Perhaps we can call this a "minimax" conception of regular difficulty.)
It seems to me like MUT-like events are, perhaps unconsciously, written using a reverse of that principle:
This tournament is at the maximum difficulty level at which the very worst team can still be bothered to show up and get points against another similarly-ungood team. (a "maximin" conception).
These two constraints don't seem to yield the same conversion statistics.
Will wrote: A tournament like MUT or MFT (as written in the 2013-2014 season) is probably much the same - I wouldn't be too shocked if a team like Jacob/Grace Yale or MIT took a game off Stanford, Michigan, or even UMD on one of those two sets (which were highly enjoyable to play).
I think this point may be a bit overstated -- upsets of the "team #11ish beats team #5ish" sort happen even at current Regionals difficulty (and Nationals difficulty!) and don't necessarily indicate a failure of the question set to be properly hard. That said, it is definitely the case that as you get lower in difficulty, you lose the ability to see how people "scale" to harder/newer/less-recycled material, and it becomes EASIER for high school generalists to coast on their bank of memorized clues, not harder (contra Mike Bentley's worries above). There have also been years where three of the top 5 teams, or so, have all been in the same region (e.g. Chicago/Illinois/Minnesota 2008-10, UVA/Penn/Maryland literally right now), so distinction at the top does actually have to matter no matter what regular difficulty is finally picked, unless one believes that it doesn't matter who among those three teams wins locally.
Sorry to be so waffly about taking a stance here; I'm earnestly not sure what I think and working out the pros and cons seems to be a constructive way for me to engage while unsure.