About a year and a half ago, I started working on this whole idea of question asking online. It has led me to what I believe is my larger project, namely online communication. But it started here, and it started with me reading up on the whole idea of Q&A online. Which started with two articles: High Performance Question/Answering and Evaluation in Nautral Language Generation: The Question Generation Task. I’m not sure I’d say these are foundational texts for the field, but they made up my foundation, in that they were the first ones I read.To start with, let’s look at High Performance Question/Answering by Marius A Pasca and Sanda M Harabagui. They’re both computer science people, which is why I went to them; it seemed like computer science and psychology were the most likely places to find the answers I was looking for.
While the paper is a report on a study implimenting some methods of Q/A, there were bits and pieces that really fit what I was looking for. First and foremost, they talked about a taxonomy of answer types. They write that “A text passage containing a candidate answer constains not only some of the question keywords, but necessarily one concept of the same semantic category as the concept inquired by the natural language question, be it a persons name, a number, a date, a measure, a location or an organization” (367). Here I see that there are tags for questions and for answers, which helps me to develop a taxonomy. I also see the different types of answers, and what they could be addressing; both of which helped me in my task.
The different semantic categories of answers led to an Answer Type Taxonomy for Pasca and Harabgiu (367), which in turn led them to realize that “we need to identify the question word(s) that determine the expected answer type” (368, italics in original). So there are question words (like who, what, when, where, how and why) that lead to certain kinds of answers. After all, a question that asks Who is going to get a different answer than one that asks Why. And then there are the additional words, like Which, Should, Do, Must, and Have; these also clue us in on what kind of answer we’re looking for.
While I took this to help categorize questions, they continued along with answers. They said that “Finding the answer to a natural language question involves not only knowing what to look for (i.e. the expected answer type) but also where to look for the answer (369, italics in original). The different sources for answers will provide different kinds of answers. You can go to wikipedia to find out who the 23rd President was, but if you want dating advice, you should probably go somewhere else. Makes sense.
What the authors discovered was that the answer type taxonomy they developed was very effective. They also discovered that feedback was “significantly more efficient than multi-word indexing or annotations of large corpora with predicate-argument information” (374). This tells me that looking just for the words isn’t enough. While key words might help to taxonomize answers (and thus questions), it it better to see the intention of the question (or answer) and use that feedback to determine the answer/question type.
My second exploration, Evaluation in Natural Language Generation: The Question Generation Task was written by Vasile Rus, Zhiquiang Cal, and Arthur C. Graesser. Vasile is another computer scientist, but the other two are both from the department of psychology
While a short work, there are a few important things present. They talk about Natural Language Generation (NLG), an important task for artificial intelligence. They write that “A robust NLG system requires the modeling of speaker’s intentions, discourse planning, micro-planning, surface realization, and lexical choices” (20). So part of the problem of creating a program that generates language in a natural way is to understand and model all the little thing we do when we talk, both on the surface and when planning for how to acheive our goals in a conversation.
Once modeled and created, we’d need to be able to test such a system. According to Vasile et. al, “The evaluation of any NLG system includes multiple criteria, such as user satisfiability, linguistic well-foundedness, maintainability, cost efficiency, output quality, and variability” (20). While pretty straight forward for their own purposes, this told me something very interesting for my own goals. Namely, it told me that part of improving questions would involve not only clear communication and satisfied users, but also the quality and variety of answers provided.
So these two works set me on my way through online communication. I was still panning for gold at this point, but there were nuggets here that led me to other work. Which I will discuss another time.
Cogito, Cogitas, Cogitate 