Annotate QB Questions for Fun, Practice, Profit, and Science

[ezubaric]

Some of you might remember that a few years ago some students and I put out a web app that folks could use to practice questions. It proved very popular and helped spawn fuller-featured platforms like Protobowl.

We used that information for a research paper that you can read here:
http://www.umiacs.umd.edu/~jbg/docs/qb_emnlp_2012.pdf

We’re still working on helping computers understand quiz bowl questions, and we’re launching a new web app that will help us gather important data and will help *you* study quiz bowl questions by annotating when characters, authors, and works appear in quiz bowl questions:

http://qb.umiacs.umd.edu/tutorial.php

Our goal is to get annotations of when people, works, and characters are mentioned in quiz bowl questions. As before, we’re offering prizes to the people who participate the most. However, we hope that everyone will be able to benefit from reading existing quiz bowl questions carefully.

We’ve been testing it out here, and it’s helped me to remember plot details in works that I never knew or had long forgotten. Annotating these questions, I’ve found, is much more effective than simply reading the question, as you need to reconstruct how the author wrote the question.

I hope that you’ll enjoy it, especially as the annotations that you provide will help us create automated techniques to help computers understand natural language, in particular, quiz bowl questions.

Please try it out; we're offering Amazon gift cards to the three most prolific users as well as for additional random users.

http://qb.umiacs.umd.edu/tutorial.php

[Gautam]

Awesome! Thanks for linking us to your paper.

[Wynaut]

I really like this idea, but I have a quick question. What happens if, say, something like Three Sisters is mentioned in a Chekhov tossup or, in my case, Three Soldiers is mentioned in a Dos Passos tossup, and the question actually references part of a group of title characters? Does the rule in the FAQ about "Tinker, Tailor, Soldier, Spy" apply to these situations as well?

[ezubaric]

What happens if, say, something like Three Sisters is mentioned in a Chekhov tossup? ... Does the rule in the FAQ about "Tinker, Tailor, Soldier, Spy" apply to these situations as well?

Yes, the rule in the FAQ also applies there (including the caveat about how it's tricky). In general, we only want exact (or nearly exact) matches. So if "Olga, Maria, and Irina" appear in the question, you can mark those as a coreference, but if only one of the sisters appears, they should not be marked as a coreference.

Ideally, we'd like to do something better with this (e.g., to encode group membership), but it's very complicated, so we're just taking this approach for now.

[Santa Claus]

I think it would be a pretty cool thing if you could rearrange annotated terms in the side menu, so that if you wanted to change which group something/multiple somethings were in, you wouldn't have to reselect everyone of them; you could just drag them around.