Reviving the collegiate poll

[Matt Weiner]

I'd like to have someone take charge of conducting a collegiate poll four times a year, doing all of the attendant work, and posting the results on the front page. This will be officially endorsed as a "Quizbowl Resource Center" project. Please post or email me if you are interested.

[Cheynem]

I'm interested.

[Adventure Temple Trail]

Is it worth considering maybe asking people to create a poll for a number of teams that is smaller than 25 (such as 20 or 15)? It seems like there are few enough college teams that attempting to accurately rank 23 versus 24, or 19 versus 21, can bog people down in writing their polls and produce few interesting results.

[merv1618]

Is it worth considering maybe asking people to create a poll for a number of teams that is smaller than 25 (such as 20 or 15)? It seems like there are few enough college teams that attempting to accurately rank 23 versus 24, or 19 versus 21, can bog people down in writing their polls and produce few interesting results.

Agreed

[Cheynem]

I think a top 15 poll would be fine. The "also receiving votes" section would probably be used to create an unofficial top 25.

[mhayes]

I think a top 15 poll would be fine. The "also receiving votes" section would probably be used to create an unofficial top 25.

Maybe, but for the most part, the community seems to agree on the best 15 teams. There may not be much diversity among top-15 ballots.

[AKKOLADE]

Keep it at 25. The last spots are more interesting.

[Skepticism and Animal Feed]

I think a top 15 poll would be fine. The "also receiving votes" section would probably be used to create an unofficial top 25.

Maybe, but for the most part, the community seems to agree on the best 15 teams. There may not be much diversity among top-15 ballots.

I think most people agree that "these teams, in some order, are the best 15 teams in the country". I don't follow the day-to-day results of quizbowl tournaments anymore, but if they are anything like the day-to-day results of tournaments from 2004-2010 (the time I was paying attention), then there is probably disagreement, even vicious disagreement, about the exact order. Perhaps even within the top 6 or so!

I think that Mike Cheyne's suggestion about using the "also recieving votes" section to construct the 15-25 ranking is excellent and should be adopted.

[Matt Weiner]

I'm going to request that this be a traditional Top 25 poll, because my goal is to use it for some tournament marketing purposes. Those who do not feel capable of ranking 25 teams are asked to refrain from voting.

[Rococo A Go Go]

Any ranking done on HSQB wouldn't be all that informative for the quizbowlers who post on here, but rankings do help people make connections that expand and advance individual teams and/or the community at large. In addition to tournament marketing, quizbowl teams have used rankings (or success at tournaments with ranked teams present) to further their fundraising and external support. This is fairly common at the high school level, but college teams have been known to use rankings for similar purposes.

[Church51907]

Would doing regional rankings be an option as well as a supplement to the top 15
Matt Church
U of l

[Adventure Temple Trail]

I'm going to request that this be a traditional Top 25 poll, because my goal is to use it for some tournament marketing purposes. Those who do not feel capable of ranking 25 teams are asked to refrain from voting.

Fair enough.

[AKKOLADE]

Would doing regional rankings be an option as well as a supplement to the top 15
Matt Church
U of l

I think the issue here is that some regions might not even have 15 or 10 teams to make these rankings worthwhile (specifically, the west coast). And then you have typical "is Kentucky in the south or the midwest?" type questions.

[Cheynem]

Was anyone else interested in running this? If I get the position, I think post-SCT, maaybe post-Regionals would be an appropriate time for a poll.

What were the logical times to run the four polls? My feelings are:

-preseason, September
-midseason, either January or after SCT and Regionals. Doing a poll in December and January seems dicey because easier tournaments tend to dominate the docket until then and I think waiting until SCT and Regionals could reveal more interesting results.
-right before Nationals, early April. Gets in a few more tournaments and lets us see how people are predicting the big ICT and Nats push.
-postseason, May

I could see flipping it so that you do a December or January poll to reflect the first "semester," and then having your pre Nats poll be like in March that would also reflect SCT and Regionals. What do people think?

[Adventure Temple Trail]

Was anyone else interested in running this? If I get the position, I think post-SCT, maaybe post-Regionals would be an appropriate time for a poll.

What were the logical times to run the four polls? My feelings are:

-preseason, September
-midseason, either January or after SCT and Regionals. Doing a poll in December and January seems dicey because easier tournaments tend to dominate the docket until then and I think waiting until SCT and Regionals could reveal more interesting results.
-right before Nationals, early April. Gets in a few more tournaments and lets us see how people are predicting the big ICT and Nats push.
-postseason, May

I could see flipping it so that you do a December or January poll to reflect the first "semester," and then having your pre Nats poll be like in March that would also reflect SCT and Regionals. What do people think?

Personally, I like "preseason, after first semester, before Nats, post-Nats" best as a set. A lot depends on where tournaments end up in the schedule, though - if we get a first semester where there are only really two competition opportunities for full teams but there's a third competition in January, for example, doing the second poll after the January competition would be better to get more data points.

[Matt Weiner]

Mike's going to be running this henceforth. It will run four times a year: preseason, after the last fall semester tournament, after whichever of SCT and Regionals is later in a given year, and after whichever of ICT and Nationals is later in a given year. The first iteration will be after this year's ACF Regionals.

The rules I've suggested are as follows, though from after this post onwards, Mike Cheyne will handle any requests for clarification and make whatever policy changes are needed based on how the poll process actually turns out:

*The old "just use whatever criteria you want" standard is no more. Voters are asked to rank teams based on your opinion of what their average finish would be at DI ICT and Nationals under the assumption that teams are bringing their best possible lineups to both tournaments. You will assume this even if you know for a fact that it is not true; i.e., if a team isn't going to ICT this year, or if you know they will be missing their science player at Nationals, or they are playing their strongest player in DII, you should still rank the hypothetical best lineup. In the post-Nationals poll, you should still be compensating for things like "I think that 245-235 upset was a fluke" or "this team would have played better if John Q. showed up" but you should not do things like vote a team that finished 3rd at both tournaments in 14th place because you thought they underperformed at Pennance. The standard of what a "best possible team" would be is subject to common sense--if someone is enrolled at a school but hasn't played a tournament for them in two years, they aren't part of their "possible team" for purposes of this poll, but someone plays 6 tournaments a year for a school even though they are ineligible for NAQT due to being a subject editor, they are part of the possible and hypothetical best ICT team.

You may use other tournament results, your gut, margin of victory, your personal opinion of whether ICT or Nationals is more valid, or, in summary, the old "any criteria you want" to break ties and resolve cases where a numerical average is insufficient. E.g., if you think Colorado State is going to beat Farleigh-Dickinson in the finals of ICT, but Farleigh-Dickinson is going to beat Colorado State in the finals of ACF Nationals, then you may rank them 1-2 based on who will have the larger margin of victory, which tournament is more important to you, which team just feels like the better one to you, who did better at Regionals, or so on. You may not rank them both "1" or "1.5."

*All voters must vote for exactly 25 teams, ranked in the whole numbers 1 to 25 without ties, in order for your ballot to be counted.

*B teams (and C and beyond teams) and high school teams are eligible to be ranked as Top 25 college teams if you feel that they are. SEE BELOW FOR A RULING FROM POLLMEISTER CHEYNE

*In order to encourage more and honest voting, individual ballots for the poll will NOT be publicly released, though a list of the names of all voters might be. Mike has the unlimited right to reject or question a ballot that seems wildly misinformed.

I'll be helping with some e-mail awareness campaigns for this poll once it's announced so that we get a more diverse voter base (in terms of overall size and in types of regions and skill levels of teams represented) than some recent polls.

[theMoMA]

Considering that high school teams can't play ICT, how should that affect the rankings?

[Frater Taciturnus]

Presumably along some hypothetical standard of "if this team WERE to play ICT, how would that team finish?"

[1992 in spaceflight]

Hey everyone, I took over the collecting stats project that Eric Mukherjee did last year. Stats for most of this year's tournaments (except for Minnesota Open, which will be added soon, and Regionals, which hasn't happened yet) can be found here.

[Cheynem]

I agree with what Matt Weiner has proposed in terms of criteria and will be posting an official explanatory thread as Regionals comes closer.

Here's some ideas I have:

-Do not publicly post your ballots. I feel like while potentially fine discussion starters, people end up voting on a skewed or compensatory level based on publicly posted ballots.

-I agree that you must rank 25 teams. If a vote seems particularly odd, I will ask for clarification, although I also understand that in most cases the bottom of the Top 25 gets a little murky.

-With some teams, I think there is some wiggle room in what the "Best possible lineup" is (Chicago seems the clear example here, as Chicago B is always a threat), so in some cases, different voters will have different ideas of what team they are voting for in regards to "Chicago A" and "Chicago B." I don't see this as a big problem though (and if you asked teams to jot down the lineups they were envisioning when voting for an A and B team, that would get chaotic).

[theMoMA]

Presumably along some hypothetical standard of "if this team WERE to play ICT, how would that team finish?"

That works, I guess, but it seems weird. While some college teams don't play both nationals in a given year (or don't run out their full lineups at both because of conflicts or attempting to win DII, etc.), at least all college teams can play both nationals. For a poll that measures ability at only two tournaments, it seems strange to rank teams that can never play one of those tournaments. If this poll were to measure regular-season performances, I'd agree that high school teams should be ranked. But given the current criteria, it seems strange to include them.

[Cheynem]

I guess you would be ranking HS teams purely on their proposed Nats performances and then guessing how they would do at ICT.

[AKKOLADE]

Presumably along some hypothetical standard of "if this team WERE to play ICT, how would that team finish?"

That works, I guess, but it seems weird. While some college teams don't play both nationals in a given year (or don't run out their full lineups at both because of conflicts or attempting to win DII, etc.), at least all college teams can play both nationals. For a poll that measures ability at only two tournaments, it seems strange to rank teams that can never play one of those tournaments. If this poll were to measure regular-season performances, I'd agree that high school teams should be ranked. But given the current criteria, it seems strange to include them.

It's the same thought experiment as teams that only go to one national, though. I mean, it's the exact same scenario, just a different reason to not go to nationals.

[theMoMA]

It's the same thought experiment, sure, but why would you make it in the case of the high school team? The rules say they can't play ICT. It's not like a situation in which circumstance or lack of funds precluded a team from playing a tournament, in which case speculating about how that team might have done actually references something possible. High schools can't play ICT, by rule, ever. At the very least, it's just weird to have half of a particular school's ranking dependent on speculation about an impossibility.

Here's what I'm saying: The poll is set up so that ability on ICT and Nationals-level competition is the relevant inquiry. A set of presently eligible teams can never play one of those tournaments, so their ability to play it is not only completely speculative, but completely irrelevant to how good those teams are at quizbowl. If the poll's criteria stay the same, those teams shouldn't be included, because they can never play one of the two tournaments that the poll cares about. If people feel that ranking high school teams on the collegiate poll is important, the poll's criteria could also be expanded to consider more of the tournaments that high school teams can play on the collegiate circuit. (If the latter, those tournaments should, of course, be considered for all teams, not just HSers.)

[Matt Weiner]

Well, people have ranked high school teams in the past. Not ranking them is a legitimate option, but if they are going to be ranked, I think it makes sense to project their hypothetical ICT finish, because the old standard where everyone put in a ballot without being told what they were ranking teams on didn't make any sense. Mike, a decision?

[marnold]

Why even include high school teams? It's the college poll, and if this is for a tournament marketing purposes it seems to only add confusion.

[theMoMA]

Well, people have ranked high school teams in the past. Not ranking them is a legitimate option, but if they are going to be ranked, I think it makes sense to project their hypothetical ICT finish, because the old standard where everyone put in a ballot without being told what they were ranking teams on didn't make any sense. Mike, a decision?

It makes sense if the standard is only based on Nationals/ICT finish, but that seems to be putting the cart before the horse. You could also come up with a standard that incorporates regular-season results.

In the end, it might make the most sense to simply preclude high schools from being ranked. This would be the most logical to people unfamiliar with quizbowl, and it would help the teams who might have otherwise gotten bumped from the top 25 to promote themselves to their administrations for funding. I think this also might make sense in the context of this discussion: there are two national tournaments, and high schools can only play one of them, so (aside from the whole part where they're high schools) they're not really capable of being full collegiate teams for the purpose of the poll.

[Cheynem]

Here are my thoughts:

1. There is already a poll for high school teams, as well as various rankings.

2. High school teams cannot play ICT and actually only rarely play ACF Nationals. Some play college tournaments (and do well), but the majority of collegiate tournaments do not see organized intersection between college teams and high school teams.

3. I think this poll has strong potential for marketing and funding purposes for collegiate teams, and thus having as many successful collegiate teams ranked instead of HS teams that hypothetically may play one of the two nationals they are allowed to is a good thing.

4. I am thus going to not allow votes for high school teams in this poll. This could lead to odd things like a State College type team that actually wins a trophy and title at ACF Nationals not being ranked, but again, I agree with some of the people in this thread that having every voter do a thought experiment with every high school team is difficult. Furthermore, I do not anticipate an influx of high school teams to be playing ACF Nationals.

5. I encourage anyone who has strong feelings on certain high school teams to vote for them in the high school polls. If a high school team does play Nationals and do well, they will be rewarded with their placement in the standings.

In short: Please do not vote for high school teams. Any ballots submitted with HS teams will be marked as incomplete.

[ryanrosenberg]

Why even include high school teams? It's the college poll, and if this is for a tournament marketing purposes it seems to only add confusion.

Exactly. Huntington Prep (the team with the #1 HS basketball player in the nation) runs scrimmages against college teams, but they aren't going to be in the NCAA tournament any time soon. Obviously, this isn't how the intersection of high school/college quiz bowl works, but if it helps outside people understand it more easily, then so be it.

[ValenciaQBowl]

It figures that the UNC guy would drop an Andrew Wiggins reference. Feelin' optimistic?

Anyway, though I imagine it will be somewhat rare, I wanted to remind folks to not forget about the top CC team(s). This is a year in which Chipola seems to be worthy of consideration.

[1992 in spaceflight]

Hey everyone, I took over the collecting stats project that Eric Mukherjee did last year. Stats for most of this year's tournaments (except for Minnesota Open, which will be added soon, and Regionals, which hasn't happened yet) can be found here.

This has been updated to include Minnesota Open Stats and all of the SCT sites.

[Nine-Tenths Ideas]

Why no B teams?

[AKKOLADE]

Why no B teams?

They are allowed, aren't they?

[Cheynem]

You can vote for a B team (or a C team or what have you), sure. I only noted that it could get sticky if different voters have different conceptions of what a "B" team is (or if voters don't take sophisticated approaches to looking at stats).

Any community college and B/C/D/what have you team is acceptable for votes.