Human-computer quiz bowl match at the 2015 HSNCT

Dormant threads from the high school sections are preserved here.
Locked
User avatar
Important Bird Area
Forums Staff: Administrator
Posts: 5499
Joined: Thu Aug 28, 2003 3:33 pm
Location: San Francisco Bay Area
Contact:

Human-computer quiz bowl match at the 2015 HSNCT

Post by Important Bird Area » Thu May 21, 2015 8:45 pm

naqt.com wrote:NAQT is pleased that its 2015 High School National Championship Tournament will showcase the first quiz bowl match in which a computer goes thumb-to-thumb against a team of humans.

On Saturday evening, a team of researchers will debut its computerized question-answering system (QANTA) in a tossup-only competition against four NAQT staffers. Unlike other question answering systems that are allowed to see the entire question at once (e.g., IBM Watson of Jeopardy! fame), this system decides when it has enough information to answer a question and is thus able to handle quiz bowl-style tossup questions.

Prior to the exhibition game, Jordan Boyd-Graber and his colleagues from the University of Colorado and the University of Maryland will describe the state-of-the-art machine learning research that went into the system and how interested students can get involved by using and improving the open-source system. Jordan is himself a former quiz bowl player for Caltech and Princeton.

The talk will start at 8:00 p.m. in the Grand Ballroom with the exhibition game to follow.
full details
Jeff Hoppes
President, Northern California Quiz Bowl Alliance
former HSQB Chief Admin (2012-13)
VP for Communication and history subject editor, NAQT
Editor emeritus, ACF

"I wish to make some kind of joke about Jeff's love of birds, but I always fear he'll turn them on me Hitchcock-style." -Fred

User avatar
UlyssesInvictus
Yuna
Posts: 813
Joined: Thu Feb 10, 2011 7:38 pm

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by UlyssesInvictus » Thu May 21, 2015 8:51 pm

Ken Jennings wrote:I, for one, welcome our new computer overlords.
Raynor Kuang
quizdb.org
Harvard 2017, TJHSST 2013
I wrote GRAPHIC and FILM

User avatar
jonpin
Forums Staff: Moderator
Posts: 2031
Joined: Wed Feb 04, 2004 6:45 pm
Location: BCA NJ / WUSTL MO / Hackensack NJ

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by jonpin » Thu May 21, 2015 9:14 pm

Is there any chance the talk and/or the match will be streamed online or recorded for later broadcast?
Jon Pinyan
Coach, Bergen County Academies (NJ); former player for BCA (2000-03) and WUSTL (2003-07)
HSQB forum mod, PACE member
Stat director for: NSC '13-'15, '17; ACF '14, '17, '19; NHBB '13-'15; NASAT '11

"A [...] wizard who controls the weather" - Jerry Vinokurov

User avatar
Skepticism and Animal Feed
Auron
Posts: 3175
Joined: Sat Oct 30, 2004 11:47 pm
Location: Arlington, VA

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Skepticism and Animal Feed » Thu May 21, 2015 9:58 pm

Hooray! One step closer to "Quiz Bowl Mogul"
Bruce
Harvard '10 / UChicago '07 / Roycemore School '04
ACF Member emeritus
My guide to using Wikipedia as a question source

jonah
Auron
Posts: 2300
Joined: Thu Jul 20, 2006 5:51 pm
Location: Chicago

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by jonah » Thu May 21, 2015 10:44 pm

jonpin wrote:Is there any chance the talk and/or the match will be streamed online or recorded for later broadcast?
I will record it in the same fashion as all the podcasts (i.e., audio only), but I don't know enough about what to expect from the QANTA match to have any idea whether it will be interesting to listen to (e.g. how QANTA will be signaling or giving its answers). If the recording has any value and all participants approve, it will be posted.

I have heard something about Jordan Boyd-Graber et al planning to video record the match, but I don't know if that's for their own purposes, or whether it will be posted publicly, etc.
Jonah Greenthal
National Academic Quiz Tournaments

User avatar
Mike Bentley
Auron
Posts: 5794
Joined: Fri Mar 31, 2006 11:03 pm
Location: Bellevue, WA
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Mike Bentley » Fri May 22, 2015 6:44 pm

Back when the Watson Jeopardy matches aired, IBM hosted a Q&A with some people who worked on it at the University of Washington. I asked them if they had ever considered trying Watson out on the quizbowl format (explaining a bit some of the ways it differed from Jeopardy). Sadly they had not. Glad to see someone's trying.
Mike Bentley
VP of Editing, Partnership for Academic Competition Excellence
Adviser, Quizbowl Team at University of Washington
University of Maryland, Class of 2008

User avatar
Benin Rebirth Party
Yuna
Posts: 782
Joined: Sat Jun 12, 2010 8:46 pm
Location: Farhaven, Ontario

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Benin Rebirth Party » Sat May 23, 2015 6:27 pm

It would be cool to see, post finals, like the NSC All-Star game, where the HSNCT All-Stars (however defined) play against the robot.

Also, the robot should sign up for a consolation mini-tournament.
Joe Su
Lisgar 2012, McGill 2015, McGill 20--

FINALIST -- 2017 ILQBM MEME OF THE YEAR

User avatar
dwd500
Lulu
Posts: 83
Joined: Sat Apr 21, 2012 8:33 am

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by dwd500 » Sat May 23, 2015 7:16 pm

The Last 21 Stanley Cup Winners wrote:Also, the robot should sign up for a consolation mini-tournament.
"QANTA would like to fit in one more game before flight leaves."
David Dennis
Middle School Choir Director
District Scholar Bowl Coach
Washington, MO

Murray State University, 2001
Breckinridge County High School, KY 1996

User avatar
Haaaaaaaarry Whiiiiiiiiiite
Auron
Posts: 1123
Joined: Tue Dec 04, 2007 8:46 pm
Location: Fairfax, VA
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Haaaaaaaarry Whiiiiiiiiiite » Sat May 30, 2015 9:09 pm

I will be liveblogging the match at http://tinyurl.com/2015hsnctprelims.
Harry White
TJHSST '09, Virginia Tech '13
Member, PACE
Tournament Database Search by Team
Will run stats for food

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Sun May 31, 2015 12:30 am

And the end result after 40 tossups is 200 vs. 200. In a sign of poor planning, there was no contingency for a tie. I was expecting the humans to trounce QANTA.

We hope to make more appearances and to compete in a full tournament next year.

We'll be posting the full video on YouTube soon.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
the return of AHAN
Auron
Posts: 1919
Joined: Thu Feb 01, 2007 10:40 pm

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by the return of AHAN » Sun May 31, 2015 1:23 am

ezubaric wrote: We hope to make more appearances and to compete in a full tournament next year.
future NAQT website release wrote: Congratulations to Kealing Middle School of Austin, TX for winning the 2016 MSNCT championship in a one-game final over QANTA, 450-315. This marks the 4th time a quizbowl machine has won the MSNCT in its 6-year history, and, coincidentally, the first time an actual computer has played in the final.
Jeff Price
Barrington High School Coach
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Sun May 31, 2015 3:47 pm

I created a new post about the potential rule sets for human / computer quiz bowl matches here:

http://www.hsquizbowl.org/forums/viewto ... 91#p303372
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
Deviant Insider
Auron
Posts: 4603
Joined: Sun Jun 13, 2004 6:08 am
Location: Chicagoland
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Deviant Insider » Sun May 31, 2015 4:54 pm

A robot cheering section would be pretty cool.
David Reinstein
PACE VP of Outreach, Head Writer and Editor for Scobol Solo and Masonics (Illinois), TD for New Trier Scobol Solo and New Trier Varsity, Writer for NAQT (2011-2017), IHSSBCA Board Member, IHSSBCA Chair (2004-2014), PACE Member, PACE President (2016-2018), New Trier Coach (1994-2011)

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Tue Jun 02, 2015 6:42 pm

bird bird bird bird bird wrote:
naqt.com wrote:NAQT is pleased that its 2015 High School National Championship Tournament will showcase the first quiz bowl match in which a computer goes thumb-to-thumb against a team of humans.
full details
Video available here:

https://www.youtube.com/watch?v=LqsUaprYMOw
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
Benin Rebirth Party
Yuna
Posts: 782
Joined: Sat Jun 12, 2010 8:46 pm
Location: Farhaven, Ontario

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Benin Rebirth Party » Tue Jun 02, 2015 11:34 pm

ezubaric wrote:
What we did: During our demo, we were showing what the computer was thinking; i.e., the the output from the guesser and the associated scores.
Is there video of this?
Joe Su
Lisgar 2012, McGill 2015, McGill 20--

FINALIST -- 2017 ILQBM MEME OF THE YEAR

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Wed Jun 03, 2015 9:37 am

The Last 21 Stanley Cup Winners wrote:
ezubaric wrote:
What we did: During our demo, we were showing what the computer was thinking; i.e., the the output from the guesser and the associated scores.
Is there video of this?
We tried to use Quicktime to get a screen capture, but it flaked out while we were still giving our introduction. This is what it looked like on the final question.

Q: This national park is home to Artist's Drive, Zabriskie Point, and Dante's View,
from which one can see Telescope Peak. Its Racetrack Playa contains mysterious
"sailing stones," which can travel hundreds of

Vancouver -3.940807
Central Park -4.234285
Zion National Park -4.406702
Rio de Janeiro -4.421752
Mount Kosciuszko -4.737045
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

Edmund
Wakka
Posts: 175
Joined: Sun Feb 07, 2010 8:25 pm

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by Edmund » Wed Jun 03, 2015 7:18 pm

Hugely impressive. I enjoyed reading about the science that went into this; great to see the video too.

Question: How much time did you invest in getting QANTA to sound more like Sinistar before committing to a live game?
Edmund Dickinson
BuzzerQuiz
University of Oxford '11

User avatar
AKKOLADE
Sin
Posts: 15261
Joined: Thu Apr 24, 2003 8:08 am

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by AKKOLADE » Wed Jun 03, 2015 7:22 pm

Not that this really matters, but I'm just curious: why were Jeopardy players chosen rather than top quiz bowl players?
Fred Morlan
PACE President, 2018-19
International Quiz Bowl Tournaments, co-owner
University of Kentucky CoP, 2017
hsqbrank manager, NAQT writer (former subject editor), former hsqb Administrator/Chief Administrator, 2012 NASAT TD

User avatar
the return of AHAN
Auron
Posts: 1919
Joined: Thu Feb 01, 2007 10:40 pm

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by the return of AHAN » Thu Jun 04, 2015 12:06 am

Stefan HSQBRankovich wrote:Not that this really matters, but I'm just curious: why were Jeopardy players chosen rather than top quiz bowl players?
I'd guess that having Jeopardy champions gave the match a little more cachet than just having an All-Star team selected from LASA, DCC, Arcadia, etc, never mind the potential for jealousy over this guy being selected, but not that one.
To be sure, it was confided to me, 2nd hand, that at least one player was a bit perplexed about being selected, too, professing to be a poor quizbowler.
Jeff Price
Barrington High School Coach
Barrington Station Middle School Coach (2013 MSNCT Champions, 2013 & 2017 Illinois Class AA State Champions)
------------------------------------------------------------------------------

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Thu Jun 04, 2015 8:33 am

Edmund wrote:Question: How much time did you invest in getting QANTA to sound more like Sinistar before committing to a live game?
We chose a built in TTS system from OS X. This is "Tom".
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Thu Jun 04, 2015 8:40 am

Stefan HSQBRankovich wrote:Not that this really matters, but I'm just curious: why were Jeopardy players chosen rather than top quiz bowl players?
NAQT did the choosing for us, and I think they optimized for something that would look good in a press release. I think they did a great job; it was a really fun match and neither side was dominant.

It would be great to play a team of Yaphe, Hoppes, Teitler, and Weiner, but what I've been telling people is that our system needs to be able to beat humans before it can take on super humans.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

jekbradbury
Lulu
Posts: 22
Joined: Fri Jul 09, 2010 9:41 pm

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by jekbradbury » Thu Jun 04, 2015 1:17 pm

A group presenting at the cs224d.stanford.edu poster session yesterday (the class is taught by Mohit's former boss Richard Socher, who mentioned QANTA in class) said they got 5% better performance than QANTA on both history and lit using Tree-LSTMs. So the race is on :smile: and the question dataset you guys released could even become a major test set for academic NLP/question answering...
James Bradbury, Stanford '16, TJHSST '12

User avatar
mtn335
Lulu
Posts: 24
Joined: Wed Jul 28, 2010 4:38 pm
Location: (north of) Seattle, WA

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by mtn335 » Thu Jun 04, 2015 2:21 pm

At least part of the reason we didn't have the computer play against our own members (Yaphe, Hoppes, Teitler, Hart? *shudders*) was question selection. We would have had to source some questions that those guys hadn't seen; it was a lot easier to use two rounds of the actual HSNCT set, which rules out anybody with a writing or editorial role. That also rules out a goodly number of the very good quiz bowl players who were on our staff, because we weren't going to tinker with set construction to make this work.

At some point one of us noticed that we had four recent, recognizable Jeopardy superchamps all on staff, and that none of them had a writing role, so that seemed the obvious thing to do for simplicity and also just for fun.
Nathan Murphy
NAQT Member
HSNCT/ICT Logistics Director
Gonzaga University 2007

User avatar
AKKOLADE
Sin
Posts: 15261
Joined: Thu Apr 24, 2003 8:08 am

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by AKKOLADE » Thu Jun 04, 2015 4:55 pm

I get it. It would be interesting to see how it does against a top qb team when it's more developed.
Fred Morlan
PACE President, 2018-19
International Quiz Bowl Tournaments, co-owner
University of Kentucky CoP, 2017
hsqbrank manager, NAQT writer (former subject editor), former hsqb Administrator/Chief Administrator, 2012 NASAT TD

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Fri Jun 05, 2015 2:32 pm

Stefan HSQBRankovich wrote:I get it. It would be interesting to see how it does against a top qb team when it's more developed.
Agreed. We do intend to do more exhibition matches. We're planning one on the West Coast in early Fall (Bay Area or Seattle), Montreal in December, DC in early spring, and then a national tournament in late Spring.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

User avatar
ezubaric
Rikku
Posts: 366
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Human-computer quiz bowl match at the 2015 HSNCT

Post by ezubaric » Fri Jun 05, 2015 2:34 pm

jekbradbury wrote:A group presenting at the cs224d.stanford.edu poster session yesterday (the class is taught by Mohit's former boss Richard Socher, who mentioned QANTA in class) said they got 5% better performance than QANTA on both history and lit using Tree-LSTMs.
Richard was also on the original QANTA paper. We also have improved quite a bit over the original QANTA in the leadup to the match, we need to recover from the final push and writeup the results.
jekbradbury wrote: So the race is on :smile: and the question dataset you guys released could even become a major test set for academic NLP/question answering...
We sure hope so! More citations, the better.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/

Locked