The Quizbowl Resource Center

http://hdwhite.org/qb/stats/
GitHub project page
Support me on Patreon!

I made a script that allows one to search the HSQB Tournament Database, NAQT's tournament results, and Neg5 for all results for a given team, ordered by date. At the moment, it's fairly bare-bones, but as far as I can tell, it works perfectly fine. It's going to be limited to what is actually listed on the stats page itself, so searching for "Virginia" will return results for "Virginia A", but not "UVA" (and it'll also show results for "Virginia Tech").

Let me know if you have any suggestions or find any errors -- using Regex to parse a webpage isn't exactly the most precise of methods.

Very nice! I hope that a feature like this can eventually be implemented into hsquizbowl.org's own search tool.

I am glad such a thing exists now. For better or for worse, the primary feature of the site up until now has been to input stats for tournaments that have happened, not to post anything about tournaments that have not yet happened. But finding stats after a few days have passed and the "Recent posted" list has cycled is a bit frustrating, especially if the person who submitted the stats didn't fully fill out the tournament information (location, set used, etc.).

So me forgetting to close a tag made this site not work for a day. Sorry about that; it has been fixed.

This has been super useful for me; thanks for doing this, Harry.

I've made a few updates to the search. Using apostrophes should no longer break the thing (and naming your team "Ladue'); DROP TABLE teams; --" should no longer make me cry). I've also implemented searching with OR (so you don't have to search for TJ and Thomas Jefferson in two different searches). Let me know if you guys would like me to add in certain features.

Also, is there a way to tell the last time a tournament database entry has been updated?

huff paste wrote:Also, is there a way to tell the last time a tournament database entry has been updated?

No, I think the only timestamps I keep track of at this point are when a stat report was first uploaded (or created, I'm not sure) and when a question set had packets posted. These are currently just used for the query to populate the "recently posted" sections of the main page - I don't believe they are ever provided in the HTML output.

I do eventually want to integrate team search functionality into the database itself, but in the meantime, is there anything simple I could do to help make your search tool more useful/efficient?

ScoBo wrote:I do eventually want to integrate team search functionality into the database itself, but in the meantime, is there anything simple I could do to help make your search tool more useful/efficient?

I think the main help would be being able to know when entries have been updated. Right now I have to rebuild the database when I do an update, and so knowing which tournaments have actually changed would save a lot of time on my part.

Anyway you could implement a search for individual players? That might help with eligibility rules, like with Collegiate Novice.

Grams's Go-Go Boots wrote:Anyway you could implement a search for individual players? That might help with eligibility rules, like with Collegiate Novice.

Done. It might take a bit for all the results to show up, though. It's taking me a while for me to build the database.

EDIT: All 144,449 player entries and 36,797 team entries are now in there.

huff paste wrote:I've also implemented searching with OR

This should actually work now.

My hosting provider is having problems, so my site is down for the time being. Unfortunately I have no idea when it will be up again.

EDIT: It's up again.

It's now on Github! You may now feel free to make fun of my awful programming abilities.

I have now added naqt.com's tournament results to the database. You might notice some duplicates in the search results now, due to NAQT's different method of handling tournament, team, and player names.

I'm getting a bad link when I try to click on the NAQT link for stats for individual players. However, it seems to work fine for teams, so I'm not really sure what the issue is there. Great new function though, it makes it a lot more convenient to look for stats at one centralized location now!

Einhard wrote:I'm getting a bad link when I try to click on the NAQT link for stats for individual players.

Thanks for catching that. It should be working as desired now.

I just wanted to thank you for adding that feature. It really is making the season preview work easier.

It appears that the site may not have updated over the past month plus?

The script was running into some unexplained errors, but I think I have it fixed now. I'm hoping I'll have things updated tonight.

EDIT: It has been updated!

I think you're hitting some errors again - I'm having a hard time finding stats that should be popping up from the HSQB DB.

It looks like NAQT changed their page format slightly which caused some trouble. It's fixed now.

In other news, we have passed 5,000 tournaments, 100,000 teams, and 300,000 player stats indexed.

Public safety diving wrote:It looks like NAQT changed their page format slightly which caused some trouble. It's fixed now.

In other news, we have passed 5,000 tournaments, 100,000 teams, and 300,000 player stats indexed.

This is a great tool; thanks for making it.

The following ACF Fall mirror doesn't appear to be in your database: http://www.hsquizbowl.org/db/tournaments/2335/

cwRsync wrote:The following ACF Fall mirror doesn't appear to be in your database: http://www.hsquizbowl.org/db/tournaments/2335/

It appears to be there for me.

I assume you're still working on this, but it keeps timing out for me, just so you're aware.

Aaron's Rod wrote:I assume you're still working on this, but it keeps timing out for me, just so you're aware.

It's an issue with my hosting provider. It should hopefully be fixed soon.

EDIT: Things are back to normal.

I'm in the process of transferring hosting. If things decide to not work all of a sudden, please let me know.

The main page now shows a list of all tournaments that have uploaded stats since the last database update (which now happens regularly and automatically early on Tuesday mornings). It won't register stat corrections, but it will pick up adding in things like combined stats a week after prelims and playoffs have been uploaded.

I fixed a bug that was preventing the most recent NAQT tournaments from appearing and fixed the issue where the New Tournaments Added section wasn't properly working.

I just noticed that NAQT results aren't showing up since they did their redesign. Hopefully I can get that fixed shortly.

Everything should be working again. Let me know if there's any further hiccups.

It works if you click on the name of the tournament, but if you click on NAQT, which should go to the team's stats for the tournament, it always goes to https://www.naqt.com/stats/tournament/t ... ?team_id=1, which does not seem to have any information.

Big Y wrote:It works if you click on the name of the tournament, but if you click on NAQT, which should go to the team's stats for the tournament, it always goes to https://www.naqt.com/stats/tournament/t ... ?team_id=1, which does not seem to have any information.

I'll have to rerun the parser, but once that's done that should be fixed.

I have included an option to search for exact strings. As usual, please let me know if you find any bugs.

It's been long overdue, but Neg5 results are (finally) being indexed. As usual, let me know if there are any issues that arise.

Unfortunately I have been informed that this site violates the terms of use of naqt.com, and so therefore it will no longer be including NAQT results for the time being. I would encourage you to use NAQT's player/team search in conjunction with my site.

Haaaaaaaarry Whiiiiiiiiiite wrote: ↑Thu Mar 21, 2019 12:23 pm Unfortunately I have been informed that this site violates the terms of use of naqt.com, and so therefore it will no longer be including NAQT results for the time being. I would encourage you to use NAQT's player/team search in conjunction with my site.

Thanks for making my life harder, NAQT!

We're going to work with Harry on a better approach (that is in keeping with our terms of service and also much, much more efficient than scraping our entire tournament history every week).

you've scraped my heart, jonah

The database once again contains results from NAQT tournaments! Some of the division labels might look a bit funky, but that should hopefully be resolved soonish.

I think I have it working such that results from recent Yellowfruit tournaments no longer have odd links in there that shouldn't be there. It's fairly likely that new releases of Yellowfruit might inadvertently mess up the HTML parsing that I've been doing, so if there's a result that should be there but isn't, please let me know.

hsquizbowl tournament links seem to be broken (specifically, I think there's a missing slash between hsquizbowl.org and db in the links directed; otherwise they work, so it's only a minor inconvenience)

Thanks for the great tool!

EDIT: apologies - it seems that it's a redirect issue on hsquizbowl.org's end (https works, http doesn't)

Hey guys! I now have a Patreon set up for my site and my various Quizbowl projects at https://www.patreon.com/hdwhite. I'm mostly doing this to cover hosting costs and to help motivate me to continue working on projects.

It looks like a Yellowfruit build changed the way stat reports are generated, which ended up causing the parser to not run properly on those tournaments. They should be fixed now, though if anyone notices any incorrect reports please let me know.

Also, I've noticed that if a name is changed in NAQT's database, it won't necessarily be reflected in my database. If you notice that your name is consistently listed incorrectly, please let me know and I can update it.

I think the NAQT stats are broken again on here. When I search my team, the only tournaments that show up are non-NAQT.

Halinaxus wrote: ↑Fri Oct 15, 2021 3:20 pm I think the NAQT stats are broken again on here. When I search my team, the only tournaments that show up are non-NAQT.

It should be fixed. Unfortunately there's an issue with getting Neg5 results, so they won't show up for the time being.

Whiter Hydra wrote: ↑Fri Oct 15, 2021 11:18 pm It should be fixed. Unfortunately there's an issue with getting Neg5 results, so they won't show up for the time being.

Great, thank you!