Question Writing Comp. / Human vs. Computer Expo (@PACE NSC)

Dormant threads from the high school sections are preserved here.
Locked
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Question Writing Comp. / Human vs. Computer Expo (@PACE NSC)

Post by ezubaric »

We'll be running an exhibition game between a quiz bowl-playing artificial intelligence and a human team at PACE NSC 2018 in Reston, Virginia on June 2 (more details on venue and time as we get closer to the actual event). Unlike previous matches that used stock quiz bowl questions, we need your help to write questions that are difficult for computers.

Our research group has been working on building computers that can answer quiz bowl questions well. We’ve beat Ken Jennings, we narrowly beat a team of quiz bowl all-stars at HSNCT 2017, and Studio OUSIA soundly defeated the best possible quiz bowl team we could assemble at NIPS last year. You might think that computers are pretty smart, right?

You’d be wrong. Quiz bowl questions are supposed to be written pyramidally: harder clues at the start, easier clues at the end. The problem is that question writers don't write questions pyramidally for computers: they write questions for humans. If humans could write questions pyramidally for computers, computers would be in a whole heap of trouble, and look pretty dumb. Our goal is to offer a true test for how smart computers are.

Visit http://write.qanta.org to write questions that are difficult for computers and submit your questions to the competition!

Introduction video:
https://www.youtube.com/watch?v=DENelwdcqhk
Last edited by ezubaric on Mon Apr 30, 2018 2:41 pm, edited 1 time in total.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Best Question Winners, Week 1

Post by ezubaric »

Winners for Week #1 (04/23 - 04/30):

Here our the two winners for the first week! Both questions were able to completely trick QANTA. They use similar methods, both describe a real life action that is difficult for the model to solve by matching keywords from past questions. It is especially difficult to recognize phrases like "self-inflicted method of death", where suicide has only been previously described as "ending one's life". It is also difficult to know Cannibalism is taboo in modern society. Looking forward to more great questions in the future!

(We'll post all of the winners at http://qanta.org)

James V. McConnell encouraged flatworms to engage in this behavior in order to test whether doing this affected memory. An atypical morph of the tiger salamander is distinguished at the age of four weeks by physical changes to their jaws which increase their efficiency in this activity. The common name of the Latrodectus genus is a reference to their participation in this behavior as a mating practice, which is speculated as a way for female specimens to acquire additional nutrition. If you do this in Papua New Guinea, you acquire an elevated risk of contracting the prion disease kuru, which is spread through infected brain tissue. For 10 points, name this practice which is taboo in humans, the consumption of the flesh of one's own species.

Answer: Cannibalism

Lily Bart performs this action in The House of Mirth by Edith Wharton and Martha performs this action in The Children’s Hour by Lillian Hellman. The protagonist performs this action at the end of “A Perfect Day for Bananafish,” and Joe Keller and Willy Loman carry out this action in two separate Arthur Miller plays. Two members of the Monzano family perform this action with “ice-nine” in Cat’s Cradle, and Quentin Compson does this in the Charles River after fastening tailor’s weights to his feet. In real life, Sylvia Plath performed this action by placing her head in an oven. For the points, name this self-inflicted method of death.

Answer: Suicide
Last edited by ezubaric on Sat May 05, 2018 2:36 pm, edited 1 time in total.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

This week's winner uses several phrases to evoke racism that computer systems cannot link to major US cities. While a human might be able to connect Gone with the Wind to Atlanta, this is much more difficult for a computer to figure out.
Opponents of this city’s public transit system have called it a scheme for “moving Africans rapidly through” this city. Oscar winner Hattie McDaniel was unable to attend the gala for the premiere of Gone with the Wind due to racial segregation laws in this city. Booker T. Washington proposed this city’s (*) namesake compromise in a speech given at the Cotton States and International Expo in, for ten points, what home of the world’s busiest airport and capital of Georgia?
Atlanta
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

We've been able to increase the cash value of our prizes. Please keep submitting your questions!

Prizes:
$10 for the best questions submitted every week
$300 for the person with the most selected questions
$200 for the person with the second most selected questions
$100 for the person with the third most selected questions
$10 for every question used in the exhibition match (only for those not eligible for the grand prize winners above)
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

Congrats to last week's best question:
An aircraft carrier bearing this man's name hosted the "Mission Accomplished" speech. A city named after this man is the site of the only unicameral state legislature. A tunnel named after this man was the second tunnel built to cross the Hudson River. An organization named for this man fought against Francisco Franco's forces. A building dedicated to him was the site of the "I have a dream" speech. For 10 points, name this subject of the portrait on the penny.
Abraham Lincoln
For this week, we're raising the best question prize to $100. So get your question in before Saturday 16:00 UTC for your chance to win $100 (while still being eligible for the $500 grand prize). Tell your friends!

Note that questions don't have to be blind. You can rewrite existing questions that might have appeared elsewhere.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

We had some brief downtime this morning, but the site is back up. Chance for $100 prize for best question and $500 for best set ends in about 24 hours!
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

We're looking for players for the team to take on OUSIA on questions that are designed to be difficult for the computer:

https://docs.google.com/forms/d/e/1FAIp ... sp=sf_link
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

We have a strong human team assembled, we have an outstanding set of computer-stumping questions, and we have our uplink with the computer system with the Japanese quiz bowl robot completed. Looking forward to seeing everyone in Regency Ballroom tonight at 8pm in Reston!
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

We have a strong human team assembled, we have an outstanding set of computer-stumping questions, and we have our uplink with the computer system with the Japanese quiz bowl robot completed. Looking forward to seeing everyone in Regency Ballroom tonight at 8pm in Reston!
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
User avatar
Blackboard Monitor Vimes
Auron
Posts: 2362
Joined: Sat Aug 18, 2007 5:40 pm
Location: Richmond, VA

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by Blackboard Monitor Vimes »

This event is being moved to Grand Ballroom D. It will still take place at 8 PM.
Sam L,
Maggie L. Walker Governor's School 2010 / UVA 2014 / VCU School of Education 2016
PACE
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

Thanks for everyone who helped make tonights event a lot of fun. Not spoiling any results, as I'm hoping to upload video tomorrow morning.
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
A Dim-Witted Saboteur
Yuna
Posts: 973
Joined: Tue Aug 02, 2016 12:31 pm
Location: Indiana

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by A Dim-Witted Saboteur »

Was video for this ever uploaded? If so, could you post a link?
Jakob M. (they/them)
Michigan State '21, Indiana '2?
"No one has ever organized a greater effort to get people interested in pretending to play quiz bowl"
-Ankit Aggarwal
User avatar
34 + P.J. Dozier
Wakka
Posts: 197
Joined: Sun Oct 15, 2017 10:01 pm
Location: Chicago, IL

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by 34 + P.J. Dozier »

An Economic Ignoramus wrote:Was video for this ever uploaded? If so, could you post a link?
This should be it, I think.

https://www.youtube.com/watch?v=oNXN1SlVevc
Wonyoung Jang
Belmont '18 // UChicago '22
ACF; NAQT; PACE
User avatar
ezubaric
Rikku
Posts: 369
Joined: Mon Feb 09, 2004 8:02 pm
Location: College Park, MD
Contact:

Re: Question Writing Comp. / Human vs. Computer Expo (@PACE

Post by ezubaric »

We now have a new video (better editing) here:
https://www.youtube.com/watch?v=wf5GHwDb6Ig
Jordan Boyd-Graber
UMD (College Park, MD), Faculty Advisor 2018-present
UC Boulder, Founder / Faculty Advisor 2014-2017
UMD (College Park, MD), Faculty Advisor 2010-2014
Princeton, Player 2004-2009
Caltech (Pasadena, CA), Player / President 2000-2004
Ark Math & Science (Hot Springs, AR), Player 1998-2000
Monticello High School, Player 1997-1998

Human-Computer Question Answering:
http://qanta.org/
Locked