Tournament Database Search by Team

Whiter Hydra · Post by **Whiter Hydra** » Fri Mar 29, 2013 1:23 pm

http://hdwhite.org/qb/stats/
GitHub project page
Support me on Patreon!

I made a script that allows one to search the HSQB Tournament Database, NAQT's tournament results, and Neg5 for all results for a given team, ordered by date. At the moment, it's fairly bare-bones, but as far as I can tell, it works perfectly fine. It's going to be limited to what is actually listed on the stats page itself, so searching for "Virginia" will return results for "Virginia A", but not "UVA" (and it'll also show results for "Virginia Tech").

Let me know if you have any suggestions or find any errors -- using Regex to parse a webpage isn't exactly the most precise of methods.

Corry · Post by **Corry** » Fri Apr 05, 2013 12:19 pm

Very nice! I hope that a feature like this can eventually be implemented into hsquizbowl.org's own search tool.

jonpin · Post by **jonpin** » Fri Apr 05, 2013 2:59 pm

I am glad such a thing exists now. For better or for worse, the primary feature of the site up until now has been to input stats for tournaments that have happened, not to post anything about tournaments that have not yet happened. But finding stats after a few days have passed and the "Recent posted" list has cycled is a bit frustrating, especially if the person who submitted the stats didn't fully fill out the tournament information (location, set used, etc.).

Whiter Hydra · Post by **Whiter Hydra** » Thu Apr 11, 2013 10:23 pm

So me forgetting to close a tag made this site not work for a day. Sorry about that; it has been fixed.

AKKOLADE · Post by **AKKOLADE** » Wed May 22, 2013 9:29 am

This has been super useful for me; thanks for doing this, Harry.

Whiter Hydra · Post by **Whiter Hydra** » Fri Aug 23, 2013 4:44 pm

I've made a few updates to the search. Using apostrophes should no longer break the thing (and naming your team "Ladue'); DROP TABLE teams; --" should no longer make me cry). I've also implemented searching with OR (so you don't have to search for TJ and Thomas Jefferson in two different searches). Let me know if you guys would like me to add in certain features.

Also, is there a way to tell the last time a tournament database entry has been updated?

ScoBo · Post by **ScoBo** » Fri Aug 23, 2013 5:10 pm

huff paste wrote:Also, is there a way to tell the last time a tournament database entry has been updated?

No, I think the only timestamps I keep track of at this point are when a stat report was first uploaded (or created, I'm not sure) and when a question set had packets posted. These are currently just used for the query to populate the "recently posted" sections of the main page - I don't believe they are ever provided in the HTML output.

I do eventually want to integrate team search functionality into the database itself, but in the meantime, is there anything simple I could do to help make your search tool more useful/efficient?

Whiter Hydra · Post by **Whiter Hydra** » Fri Aug 23, 2013 5:34 pm

ScoBo wrote:I do eventually want to integrate team search functionality into the database itself, but in the meantime, is there anything simple I could do to help make your search tool more useful/efficient?

I think the main help would be being able to know when entries have been updated. Right now I have to rebuild the database when I do an update, and so knowing which tournaments have actually changed would save a lot of time on my part.

AKKOLADE · Post by **AKKOLADE** » Fri Aug 23, 2013 5:37 pm

Anyway you could implement a search for individual players? That might help with eligibility rules, like with Collegiate Novice.

Whiter Hydra · Post by **Whiter Hydra** » Fri Aug 23, 2013 6:54 pm

Grams's Go-Go Boots wrote:Anyway you could implement a search for individual players? That might help with eligibility rules, like with Collegiate Novice.

Done. It might take a bit for all the results to show up, though. It's taking me a while for me to build the database.

EDIT: All 144,449 player entries and 36,797 team entries are now in there.

Whiter Hydra · Post by **Whiter Hydra** » Fri Aug 23, 2013 8:49 pm

huff paste wrote:I've also implemented searching with OR

This should actually work now.

Whiter Hydra · Post by **Whiter Hydra** » Sat Mar 29, 2014 10:06 pm

My hosting provider is having problems, so my site is down for the time being. Unfortunately I have no idea when it will be up again.

EDIT: It's up again.

Whiter Hydra · Post by **Whiter Hydra** » Sun Apr 27, 2014 11:35 pm

It's now on Github! You may now feel free to make fun of my awful programming abilities.

Whiter Hydra · Post by **Whiter Hydra** » Thu Jun 05, 2014 10:12 am

I have now added naqt.com's tournament results to the database. You might notice some duplicates in the search results now, due to NAQT's different method of handling tournament, team, and player names.

Beevor Feevor · Post by **Beevor Feevor** » Tue Jun 10, 2014 11:26 am

I'm getting a bad link when I try to click on the NAQT link for stats for individual players. However, it seems to work fine for teams, so I'm not really sure what the issue is there. Great new function though, it makes it a lot more convenient to look for stats at one centralized location now!

Whiter Hydra · Post by **Whiter Hydra** » Tue Jun 10, 2014 6:36 pm

Einhard wrote:I'm getting a bad link when I try to click on the NAQT link for stats for individual players.

Thanks for catching that. It should be working as desired now.

AKKOLADE · Post by **AKKOLADE** » Fri Jun 20, 2014 10:49 am

I just wanted to thank you for adding that feature. It really is making the season preview work easier.

AKKOLADE · Post by **AKKOLADE** » Mon Nov 03, 2014 3:54 pm

It appears that the site may not have updated over the past month plus?

Whiter Hydra · Post by **Whiter Hydra** » Mon Nov 03, 2014 7:53 pm

The script was running into some unexplained errors, but I think I have it fixed now. I'm hoping I'll have things updated tonight.

EDIT: It has been updated!

AKKOLADE · Post by **AKKOLADE** » Fri Feb 13, 2015 8:10 am

I think you're hitting some errors again - I'm having a hard time finding stats that should be popping up from the HSQB DB.

Whiter Hydra · Post by **Whiter Hydra** » Mon Feb 16, 2015 3:27 am

It looks like NAQT changed their page format slightly which caused some trouble. It's fixed now.

In other news, we have passed 5,000 tournaments, 100,000 teams, and 300,000 player stats indexed.

Charbroil · Post by **Charbroil** » Mon Feb 16, 2015 4:03 am

Public safety diving wrote:It looks like NAQT changed their page format slightly which caused some trouble. It's fixed now.

In other news, we have passed 5,000 tournaments, 100,000 teams, and 300,000 player stats indexed.

This is a great tool; thanks for making it.

dxdtdemon · Post by **dxdtdemon** » Wed Feb 18, 2015 1:40 am

The following ACF Fall mirror doesn't appear to be in your database: http://www.hsquizbowl.org/db/tournaments/2335/

Whiter Hydra · Post by **Whiter Hydra** » Wed Feb 18, 2015 1:51 am

cwRsync wrote:The following ACF Fall mirror doesn't appear to be in your database: http://www.hsquizbowl.org/db/tournaments/2335/

It appears to be there for me.

Aaron's Rod · Post by **Aaron's Rod** » Sun Apr 05, 2015 1:55 pm

I assume you're still working on this, but it keeps timing out for me, just so you're aware.

Whiter Hydra · Post by **Whiter Hydra** » Sun Apr 05, 2015 3:10 pm

Aaron's Rod wrote:I assume you're still working on this, but it keeps timing out for me, just so you're aware.

It's an issue with my hosting provider. It should hopefully be fixed soon.

EDIT: Things are back to normal.

Whiter Hydra · Post by **Whiter Hydra** » Mon Aug 22, 2016 3:55 pm

I'm in the process of transferring hosting. If things decide to not work all of a sudden, please let me know.

Whiter Hydra · Post by **Whiter Hydra** » Tue Aug 30, 2016 12:35 pm

The main page now shows a list of all tournaments that have uploaded stats since the last database update (which now happens regularly and automatically early on Tuesday mornings). It won't register stat corrections, but it will pick up adding in things like combined stats a week after prelims and playoffs have been uploaded.

Whiter Hydra · Post by **Whiter Hydra** » Wed Feb 15, 2017 1:47 pm

I fixed a bug that was preventing the most recent NAQT tournaments from appearing and fixed the issue where the New Tournaments Added section wasn't properly working.

Whiter Hydra · Post by **Whiter Hydra** » Tue Feb 06, 2018 6:54 pm

I just noticed that NAQT results aren't showing up since they did their redesign. Hopefully I can get that fixed shortly.

Whiter Hydra · Post by **Whiter Hydra** » Fri Feb 09, 2018 4:40 pm

Everything should be working again. Let me know if there's any further hiccups.

Stained Diviner · Post by **Stained Diviner** » Fri Feb 09, 2018 8:22 pm

It works if you click on the name of the tournament, but if you click on NAQT, which should go to the team's stats for the tournament, it always goes to https://www.naqt.com/stats/tournament/t ... ?team_id=1, which does not seem to have any information.

Whiter Hydra · Post by **Whiter Hydra** » Sat Feb 10, 2018 12:28 am

Big Y wrote:It works if you click on the name of the tournament, but if you click on NAQT, which should go to the team's stats for the tournament, it always goes to https://www.naqt.com/stats/tournament/t ... ?team_id=1, which does not seem to have any information.

I'll have to rerun the parser, but once that's done that should be fixed.

Whiter Hydra · Post by **Whiter Hydra** » Mon Apr 30, 2018 11:41 pm

I have included an option to search for exact strings. As usual, please let me know if you find any bugs.

Whiter Hydra · Post by **Whiter Hydra** » Mon Mar 04, 2019 5:09 pm

It's been long overdue, but Neg5 results are (finally) being indexed. As usual, let me know if there are any issues that arise.

Whiter Hydra · Post by **Whiter Hydra** » Thu Mar 21, 2019 12:23 pm

Unfortunately I have been informed that this site violates the terms of use of naqt.com, and so therefore it will no longer be including NAQT results for the time being. I would encourage you to use NAQT's player/team search in conjunction with my site.

AKKOLADE · Post by **AKKOLADE** » Thu Mar 21, 2019 1:18 pm

Haaaaaaaarry Whiiiiiiiiiite wrote: ↑Thu Mar 21, 2019 12:23 pm Unfortunately I have been informed that this site violates the terms of use of naqt.com, and so therefore it will no longer be including NAQT results for the time being. I would encourage you to use NAQT's player/team search in conjunction with my site.

Thanks for making my life harder, NAQT!

jonah · Post by **jonah** » Thu Mar 21, 2019 2:33 pm

We're going to work with Harry on a better approach (that is in keeping with our terms of service and also much, much more efficient than scraping our entire tournament history every week).

AKKOLADE · Post by **AKKOLADE** » Thu Mar 21, 2019 4:13 pm

you've scraped my heart, jonah

Whiter Hydra · Post by **Whiter Hydra** » Wed Jun 26, 2019 1:00 pm

The database once again contains results from NAQT tournaments! Some of the division labels might look a bit funky, but that should hopefully be resolved soonish.

Whiter Hydra · Post by **Whiter Hydra** » Fri Nov 01, 2019 12:12 pm

I think I have it working such that results from recent Yellowfruit tournaments no longer have odd links in there that shouldn't be there. It's fairly likely that new releases of Yellowfruit might inadvertently mess up the HTML parsing that I've been doing, so if there's a result that should be there but isn't, please let me know.

Wei la la. After this it gets deep. · Mon Nov 11, 2019 1:54 am

hsquizbowl tournament links seem to be broken (specifically, I think there's a missing slash between hsquizbowl.org and db in the links directed; otherwise they work, so it's only a minor inconvenience)

Thanks for the great tool!

EDIT: apologies - it seems that it's a redirect issue on hsquizbowl.org's end (https works, http doesn't)

Whiter Hydra · Post by **Whiter Hydra** » Tue Jan 14, 2020 3:01 pm

Hey guys! I now have a Patreon set up for my site and my various Quizbowl projects at https://www.patreon.com/hdwhite. I'm mostly doing this to cover hosting costs and to help motivate me to continue working on projects.

Whiter Hydra · Post by **Whiter Hydra** » Wed Feb 24, 2021 12:59 pm

It looks like a Yellowfruit build changed the way stat reports are generated, which ended up causing the parser to not run properly on those tournaments. They should be fixed now, though if anyone notices any incorrect reports please let me know.

Also, I've noticed that if a name is changed in NAQT's database, it won't necessarily be reflected in my database. If you notice that your name is consistently listed incorrectly, please let me know and I can update it.

Halinaxus · Post by **Halinaxus** » Fri Oct 15, 2021 3:20 pm

I think the NAQT stats are broken again on here. When I search my team, the only tournaments that show up are non-NAQT.

Whiter Hydra · Post by **Whiter Hydra** » Fri Oct 15, 2021 11:18 pm

Halinaxus wrote: ↑Fri Oct 15, 2021 3:20 pm I think the NAQT stats are broken again on here. When I search my team, the only tournaments that show up are non-NAQT.

It should be fixed. Unfortunately there's an issue with getting Neg5 results, so they won't show up for the time being.

Halinaxus · Post by **Halinaxus** » Sat Oct 16, 2021 11:19 am

Whiter Hydra wrote: ↑Fri Oct 15, 2021 11:18 pm It should be fixed. Unfortunately there's an issue with getting Neg5 results, so they won't show up for the time being.

Great, thank you!