The Quizbowl Resource Center

Hey, I wonder if we could discuss the fact that NAQT, now alone among legitimate quizbowl providers, charges for its old packets. I've brought this up in private with Jeff before and would like to hear what others think.

First let it be said that we must all acknowledge that NAQT is the owner of these packets and has the legal and moral right to do what they want with them. They are not "evil" for choosing not to release them, and this thread is not the place to debate your probably ill-informed views about the legal and moral issues surrounding copyrights and the restriction of access to information.

However, I think it would be a good idea for quizbowl if NAQT were to post its archive for free. Basically, it seems unfair that teams with long histories who managed to get paper copies of packets at tournaments, and/or with deep pockets who can order the practice material under the current arrangement, have access to more NAQT practice sets than newer/less wealthy teams. We talk a lot about getting new teams or teams from less traditionally powerful academic schools into quizbowl and up to a competitive level, and charging for old packets seems really counterproductive to that set of goals. Furthermore, it's an open secret that there is a huge underground packet-trading network which already has access to nearly every NAQT set; I think it would be nice if having the level of access and trust needed to gain entry to that network was no longer a huge advantage in gaining practice material. Thirdly, since NAQT obviously cannot stop the underground trading of packets, as much as they may have the right to do so, it would probably be the least-bad solution to stop forcing people to participate in shady activities in order to collect tournament sets. Lastly, by making it known that packets will be available for free at the end of the competition year, NAQT will no longer have to deal with as many of the integrity issues that arise when people send still-in-use tournament sets around the country.

The only real objection I've heard from NAQT members is that NAQT may have to offset the financial loss of no longer charging for packets in some fashion (eg: raising HSNCT entry fees, raising licensing costs, lowering contributor payments). I think that most teams would be happy to pay another $2-3 per tournament attended in exchange for limitless access to several hundred NAQT sets. It's a great bargain that would cost no team more than about $100 (if they go to every IS tournament a year with multiple teams) and cost most teams far, far less. If my numbers are way off, please let me know.

What do people, NAQT-affiliated or otherwise, think about this issue and my proposed solution?

From a purely practical standpoint, I question whether NAQT has the infrastructure for that. Already their site goes down a lot, particularly when many people are trying to access it (e.g. HSNCT); I imagine the problem would get much worse when everyone is trying to get to files that are greater in both number and size than the pages that seem to already cause problems.

If the practice question market for recent NAQT tournament sets is too lucrative, I wonder if it would be possible to release at least the collegiate sets and older high school sets. I don't know this for a fact, but I'd imagine that SCT, ICT, and IS sets of older vintage aren't big sellers, compared to recent IS sets.

Yeah, I agree with Andrew--I can't imagine anyone clamoring for 2002 ICT DII (which at one point, I had a legitimate copy of), but sure, HSNCT's, ICT's, SCT's of the last...four years or so would be a great and pretty simple start.

jonah wrote:From a purely practical standpoint, I question whether NAQT has the infrastructure for that. Already their site goes down a lot, particularly when many people are trying to access it (e.g. HSNCT); I imagine the problem would get much worse when everyone is trying to get to files that are greater in both number and size than the pages that seem to already cause problems.

I can't possibly imagine that they can't get a server that can handle hosting maybe 100 MB worth of old IS sets. Quizbowlpackets.com and the Stanford Archive seem to have no problem doing this.

Anyway, one idea to offset some of the costs from lost packet sale income could come from offering schools electronic copies of sets to be used at tournaments. If a school is able to run the tournament on computers rather than making copies of the questions, they'll in general save maybe $5 per set of packets they would have had to print.

Bentley Like Beckham wrote:

jonah wrote:From a purely practical standpoint, I question whether NAQT has the infrastructure for that. Already their site goes down a lot, particularly when many people are trying to access it (e.g. HSNCT); I imagine the problem would get much worse when everyone is trying to get to files that are greater in both number and size than the pages that seem to already cause problems.

I can't possibly imagine that they can't get a server that can handle hosting maybe 100 MB worth of old IS sets. Quizbowlpackets.com and the Stanford Archive seem to have no problem doing this.

I have no doubt that it would be easy to get a host that can handle the load NAQT.com experiences; I'm just pointing out that they have not done so for years despite it being woefully apparent that they should, and this proposal might exacerbate that problem.

Right, Jonah, but if the packets were available for free and freely redistributable, they'd appear on the archives more or less immediately, rendering the NAQT site issues moot.

M

Captain Sinico wrote:Right, Jonah, but if the packets were available for free and freely redistributable, they'd appear on the archives more or less immediately, rendering the NAQT site issues moot.

Or, NAQT might choose to send them directly to George and the Stanford Archive, thereby circumventing the issue.

Matt's first post is a good summary of our thoughts on this. The basic tradeoff is this: the revenue from selling packets would need to be replaced, and the only way to do that would be an across-the-board increase in tournament entry fees. When I've talked about this with R., we concluded that resistance to paying more money to play would probably outweigh the benefits of a free packet archive.

jonah wrote:

Bentley Like Beckham wrote:

jonah wrote:From a purely practical standpoint, I question whether NAQT has the infrastructure for that. Already their site goes down a lot, particularly when many people are trying to access it (e.g. HSNCT); I imagine the problem would get much worse when everyone is trying to get to files that are greater in both number and size than the pages that seem to already cause problems.

I can't possibly imagine that they can't get a server that can handle hosting maybe 100 MB worth of old IS sets. Quizbowlpackets.com and the Stanford Archive seem to have no problem doing this.

I have no doubt that it would be easy to get a host that can handle the load NAQT.com experiences; I'm just pointing out that they have not done so for years despite it being woefully apparent that they should, and this proposal might exacerbate that problem.

I think they've hosted podcasts, which are a lot bigger than packets, on their site for years without problems.

One of the advantages of charging for packets is that coaches have a source questions that they know are blind to their players. With that said, the claim about unfairness created by the underground-trading circuit is legitimate, and the practice seems counterproductive to the goal of promoting good quizbowl.

As people suggested earlier, it might be a good compromise to at least offer HSNCT and ICT sets for free, so that all teams can prepare for the national championship equally, regardless of their access in the trading circuit or the depth of their team funds.

bt_green_warbler wrote:Matt's first post is a good summary of our thoughts on this. The basic tradeoff is this: the revenue from selling packets would need to be replaced, and the only way to do that would be an across-the-board increase in tournament entry fees. When I've talked about this with R., we concluded that resistance to paying more money to play would probably outweigh the benefits of a free packet archive.

Jeff, am I incorrect to assume that 90% of NAQT's practice question sales come from high school sets (the bulk of those being of recent vintage)? Would the benefit outweigh the revenue loss should NAQT release SCT, ICT, and sets written over five years ago?

Now, I would certainly appreciate NAQT releasing their questions, and can certainly imagine that college sets aren't very big sellers and (5+ year) old IS sets aren't very big sellers. However, would releasing old IS sets benefit anyone? I'm pretty sure that modern NAQT sets are much, much better written than they were in the past. I doubt that college teams ever go and practice on, say, 1999 ACF Nationals, even though it is available online. Would old NAQT IS sets be that much different? Are people really running out of material for practice that badly despite the tournaments posted on archives now?

From NAQT's website, it seems that a dozen or so tournaments every year use sets from previous years at locations that havn't yet heard them. I suppose this option wouldn't be available if all questions prior to the current year were available online. Just another thing to think about.

I spoke to a couple of NAQT folks about a possible solution to this, particularly Chad Kubicek. I personally would be very interested in seeing NAQT make their questions available in digital format, particularly with the ability to cull questions by distribution category, and would be willing to pay, say, an annual subscription fee for the ability to access such a database. I understand the reluctance to lose income by way of back-room copying, but the flexibility of a database which can be searched strategically would make it far more valuable to me than a bulky roomfull of photocopied rounds, and I think many coaches would agree. Imagine being able to create "quiz" rounds for practice that just work the areas your team is weakest, using the huge NAQT archive as a starting point? It would be a huge help for time-pressed coaches. The database would also solve the copying problem, as it couldn't just be downloaded (although security would obviously be an issue), and it could produce a limited number of results per run to keep from people just dumping huge lists until they had the whole database on paper.

Charge $50 a year for a subscription with limits, and restrict contents to sets two years out of date or more. You'll still get your cash from packet purchases and practice pack purchases from recent sets, but will have a more consistent stream as well as people get used to having a valuable resource around.


Thoughts?

Joshua Rutsky wrote:I spoke to a couple of NAQT folks about a possible solution to this, particularly Chad Kubicek. I personally would be very interested in seeing NAQT make their questions available in digital format, particularly with the ability to cull questions by distribution category, and would be willing to pay, say, an annual subscription fee for the ability to access such a database. I understand the reluctance to lose income by way of back-room copying, but the flexibility of a database which can be searched strategically would make it far more valuable to me than a bulky roomfull of photocopied rounds, and I think many coaches would agree. Imagine being able to create "quiz" rounds for practice that just work the areas your team is weakest, using the huge NAQT archive as a starting point? It would be a huge help for time-pressed coaches. The database would also solve the copying problem, as it couldn't just be downloaded (although security would obviously be an issue), and it could produce a limited number of results per run to keep from people just dumping huge lists until they had the whole database on paper.

Charge $50 a year for a subscription with limits, and restrict contents to sets two years out of date or more. You'll still get your cash from packet purchases and practice pack purchases from recent sets, but will have a more consistent stream as well as people get used to having a valuable resource around.


Thoughts?

I have always hoped that one day NAQT questions would be digitized and also have some sort of database like the ACF database. I know that we would be very interested in paying some fee to subscribe, as I am sure our players (and as a coach) would find such a database incredibly helpful.

Keep in mind that ACF did not make the database--they just posted the packets, and somebody else set up a program to scan them in and allow people to sort them by category. If NAQT posted their old packets, this community probably would do the same thing. (I wouldn't, because I'm too old to understand how it works, but I would be happy to help with the sorting.)

As someone who has spent a significant amount of money over the past decade purchasing NAQT practice questions, I would be strongly opposed to any system that would make that investment moot by making those questions freely available, especially if the cost of this was passed along to me as a tournament director/participant.

Matt Weiner wrote:The only real objection I've heard from NAQT members is that NAQT may have to offset the financial loss of no longer charging for packets in some fashion (eg: raising HSNCT entry fees, raising licensing costs, lowering contributor payments). I think that most teams would be happy to pay another $2-3 per tournament attended in exchange for limitless access to several hundred NAQT sets. It's a great bargain that would cost no team more than about $100 (if they go to every IS tournament a year with multiple teams) and cost most teams far, far less. If my numbers are way off, please let me know.

Just reporting in to let everyone know that we've run some numbers on this.

Matt's figure ($2-3 per tournament) is a substantial underestimate. The actual increase would be $6.40 per team per IS set (this would be a 40% hike from our existing per-team fee of $16). This further assumes that tournament attendance would remain constant in the face of higher fees- which is fairly clearly false; incorporating this effect would drive the fees even higher.

To sum up: NAQT believes that making the packet archive free would increase the cost to play our tournaments, and thus likely hurt more new teams than it would help. Thus we have no plans to change our policy re: selling practice questions at the present time.

bt_green_warbler wrote:

Matt Weiner wrote:The only real objection I've heard from NAQT members is that NAQT may have to offset the financial loss of no longer charging for packets in some fashion (eg: raising HSNCT entry fees, raising licensing costs, lowering contributor payments). I think that most teams would be happy to pay another $2-3 per tournament attended in exchange for limitless access to several hundred NAQT sets. It's a great bargain that would cost no team more than about $100 (if they go to every IS tournament a year with multiple teams) and cost most teams far, far less. If my numbers are way off, please let me know.

Just reporting in to let everyone know that we've run some numbers on this.

Matt's figure ($2-3 per tournament) is a substantial underestimate. The actual increase would be $6.40 per team per IS set (this would be a 40% hike from our existing per-team fee of $16). This further assumes that tournament attendance would remain constant in the face of higher fees- which is fairly clearly false; incorporating this effect would drive the fees even higher.

To sum up: NAQT believes that making the packet archive free would increase the cost to play our tournaments, and thus likely hurt more new teams than it would help. Thus we have no plans to change our policy re: selling practice questions at the present time.

Who is still buying all these 2002 (old year chosen at random) sets and why are they operating under the assumption this is anything resembling a good investment?

Frater Taciturnus wrote:Who is still buying all these 2002 (old year chosen at random) sets and why are they operating under the assumption this is anything resembling a good investment?

Obviously a large majority of our sales are the more recent (say, last three years) sets. We thought that if we instituted a policy of "old questions available for free", it would just reduce the sales of the newer sets anyway and we might as well make the whole archive free. (That is: teams would think "let's just download the 2006 HSNCT for free instead of paying $49 for the 2010 edition.") Maybe the quality of our sets has been increasing fast enough that that's a bad assumption?

Well, it's not a bad assumption considering how asymmetrical the information is. Teams that have been following quizbowl intently know that NAQT has had a vast improvement in quality over time, but I doubt that group of teams covers more than 100 programs, and I would say that's an incredibly high estimate. The majority of NAQT's customer base is probably not fully aware of quizbowl history, or even what makes for the best questions, which means it's pretty easy to see where they might think buying all the HSNCT sets plus a bunch of old school IS sets wouldn't be a bad purchase. I personally think the solution would be for NAQT to publicly release really old sets and then post a caveat saying something to the effect of "we are releasing these because we do not think they are going to be as good a help for your teams as these more recent sets, which you can buy here."

Frater Taciturnus wrote:

bt_green_warbler wrote:

Matt Weiner wrote:The only real objection I've heard from NAQT members is that NAQT may have to offset the financial loss of no longer charging for packets in some fashion (eg: raising HSNCT entry fees, raising licensing costs, lowering contributor payments). I think that most teams would be happy to pay another $2-3 per tournament attended in exchange for limitless access to several hundred NAQT sets. It's a great bargain that would cost no team more than about $100 (if they go to every IS tournament a year with multiple teams) and cost most teams far, far less. If my numbers are way off, please let me know.

Just reporting in to let everyone know that we've run some numbers on this.

Matt's figure ($2-3 per tournament) is a substantial underestimate. The actual increase would be $6.40 per team per IS set (this would be a 40% hike from our existing per-team fee of $16). This further assumes that tournament attendance would remain constant in the face of higher fees- which is fairly clearly false; incorporating this effect would drive the fees even higher.

To sum up: NAQT believes that making the packet archive free would increase the cost to play our tournaments, and thus likely hurt more new teams than it would help. Thus we have no plans to change our policy re: selling practice questions at the present time.

Who is still buying all these 2002 (old year chosen at random) sets and why are they operating under the assumption this is anything resembling a good investment?

My guess is people that buy old IS sets are unaware of the quality and/or quantity of sets that are available for free online.

bt_green_warbler wrote:

Frater Taciturnus wrote:Who is still buying all these 2002 (old year chosen at random) sets and why are they operating under the assumption this is anything resembling a good investment?

Obviously a large majority of our sales are the more recent (say, last three years) sets. We thought that if we instituted a policy of "old questions available for free", it would just reduce the sales of the newer sets anyway and we might as well make the whole archive free. (That is: teams would think "let's just download the 2006 HSNCT for free instead of paying $49 for the 2010 edition.") Maybe the quality of our sets has been increasing fast enough that that's a bad assumption?

If this is true, how about releasing just college sets?

Joshua Rutsky wrote:I spoke to a couple of NAQT folks about a possible solution to this, particularly Chad Kubicek. I personally would be very interested in seeing NAQT make their questions available in digital format, particularly with the ability to cull questions by distribution category, and would be willing to pay, say, an annual subscription fee for the ability to access such a database. I understand the reluctance to lose income by way of back-room copying, but the flexibility of a database which can be searched strategically would make it far more valuable to me than a bulky roomfull of photocopied rounds, and I think many coaches would agree. Imagine being able to create "quiz" rounds for practice that just work the areas your team is weakest, using the huge NAQT archive as a starting point? It would be a huge help for time-pressed coaches. The database would also solve the copying problem, as it couldn't just be downloaded (although security would obviously be an issue), and it could produce a limited number of results per run to keep from people just dumping huge lists until they had the whole database on paper.

Charge $50 a year for a subscription with limits, and restrict contents to sets two years out of date or more. You'll still get your cash from packet purchases and practice pack purchases from recent sets, but will have a more consistent stream as well as people get used to having a valuable resource around.


Thoughts?

Well it is obvious that it is not practical for NAQT to just offer free sets online (or anything similar), is the above a possibility in any way?

I don't see why not, you'd combine a PHP front end that passes arguments to a Perl program that would then slice and dice based on the numeric code at the bottom. I'm more sure of the Perl than the PHP, I don't deal much with front-end stuff.

STPickrell wrote:I don't see why not, you'd combine a PHP front end that passes arguments to a Perl program that would then slice and dice based on the numeric code at the bottom. I'm more sure of the Perl than the PHP, I don't deal much with front-end stuff.

All their stuff is written in Java, which R. wants to stick with. No big deal, though, since Java could still do this. It's certainly possible, though whether it's a good investment of NAQT's time is another question; I for one would love it, and I think lots of other people would too.

Given that NAQT has a database of questions that have category codes etc. from which packets are built in the first place, screen-scraping PDFs and reassembling that database isn't a problem.

Crazy Andy Watkins wrote:Given that NAQT has a database of questions that have category codes etc. from which packets are built in the first place, screen-scraping PDFs and reassembling that database isn't a problem.

If you have access to the full database, sure, but most people don't and it's still not a convenient way to do things.

jonah wrote:

Crazy Andy Watkins wrote:Given that NAQT has a database of questions that have category codes etc. from which packets are built in the first place, screen-scraping PDFs and reassembling that database isn't a problem.

If you have access to the full database, sure, but most people don't and it's still not a convenient way to do things.

Right, but presumably NAQT, who would be making this subscription-accessible database, does. In fact, I know they do. And in fact, I'm pretty sure their database has at least the capabilities of all the screen-scraped questions, which would not be sorted into categories. And it has more.

Crazy Andy Watkins wrote:

jonah wrote:

Crazy Andy Watkins wrote:Given that NAQT has a database of questions that have category codes etc. from which packets are built in the first place, screen-scraping PDFs and reassembling that database isn't a problem.

If you have access to the full database, sure, but most people don't and it's still not a convenient way to do things.

Right, but presumably NAQT, who would be making this subscription-accessible database, does. In fact, I know they do. And in fact, I'm pretty sure their database has at least the capabilities of all the screen-scraped questions, which would not be sorted into categories. And it has more.

Yes. I thought you were talking about someone outside of NAQT doing it themselves, because—unless NAQT's database is set up in a very strange way—what you described isn't a good way to do it from their end.

Well, everyone seemed to be saying "here's this elaborate screen-scraping project so that we could create a subscription database that maybe NAQT would be okay with." I'm saying that NAQT could do the same project in thirty seconds.

I'm pretty sure that NAQT creating and marketing said database was the suggestion I posted upthread.

Joshua Rutsky wrote:I'm pretty sure that NAQT creating and marketing said database was the suggestion I posted upthread.

Yeah, _you_ made sense.