Motivation for Question Answering System
History of Question Answering Systems
Agenda of Question Answering
- Question Classes
- Question Processing
- Context and QA
- Data Sources and QA
- Answer Extraction
- Answer Formulation
- Real time question answering
- Cross lingual or Multilingual QA
- Interactive QA
- Advanced reasoning For QA
- Information Clustering for QA
- User Profiling for QA
Roadmap ........................................... Error! Bookmark not defined
Bloom’s Classification of Learning Domain
In the later sections, we would like to categorize the questions according to how difficult it is to answer them.
Bloom’s Taxonomy for Affective Domain
Taxonomy for Psychomotor Domain
Perception is the ability to use sensory units to guide motor activities, such as judging the position of a cricket ball in motion in the air in order to catch it with the hands. Set is the level of readiness to act, it involves not being physically ready, but mentally and emotionally. The directed response level is the level at which the student learns about a complex skill by imitation, trial and error.
Learning to solve a complex math equation using the instructor's help is considered a skill at the directed response level. It is learning a skill at an intermediate level, which may include performing operations on a personal computer or turning off a leaking faucet. The level at which these skills operate is the Complex Open Response Level which involves parking a car in a narrow parallel parking space.
Origin is the final level which involves creating new movement patterns for performing certain actions.
Question Classification under Bloom’s Taxonomy
- Knowledge questions
- Comprehension questions
- Application questions
- Analysis questions
- Synthesis questions
- Evaluation questions
- Summary
One of the issues identified is the introduction chapter of the Question Answering System was the data sources for the QA System, in relation to Watson we will address the "Acquisition and engineering of textual resources". Although most of the resources in Watson come from the unstructured text corpus, there are components. In 2011, an open domain question answering system was able to beat two of the top Jeopardy players.
Most sources of information are unstructured for humans, understanding the unstructured information is relatively easy. It should be mentioned that each of the steps has many alternatives that will look into details throughout the chapter. It was discovered that using the strategies of the Watson system was able to provide candidate answers, precisely the correct answer pool to 87.17% of the blind questions put to the system (J. Chu -Carroll, J. Fan, B. K. Boguraev, D. Carmel, D. Sheinwald, C. Welty., 2012).
The reliability of the answers will be based on the amount of evidence collected for each of the answers. In most question answering system, we give a big penalty to elicited answers that are present in the question. Question: This person born in 1869 is also known as the father of the nation.
Each of the implemented algorithms was fully optimized to meet the latency requirements. Question: The syndrome characterized by narrowing of the extrahepatic bile duct from mechanical compression of a gallstone impacted in the cystic duct. We follow the roadmap presented at the beginning of the chapter and conclude the discussion of Watson beyond Jeopardy, where Watson is used in the medical domain.
34; Experiments with Open Domain Text Question Answering." Proceedings of the 18th Conference on Computational Linguistics-Volume 1. 34; Question Classification Using Head Words and Their Hypernyms." Proceedings of the Empirical Methods in Natural Language Processing conference. 34; Improved inference of answer type from questions using sequential models.” Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing.
Overview of Watson
- What it takes to win at Jeopardy
- Inside DeepQA: The architecture underlying Watson
Answering questions is a way to explore this field where we want to infer answers that process unstructured information. Natural language processing techniques, also known as computational linguistics or text analytics, tend to explore the meaning of textual data by analyzing the syntax, semantics, and context of the phrases in the data. A lot of research has also been done in question answering system, but building an end-to-end question answering system is difficult.
In the time period 2001-06, researchers at IBM built the Unstructured Information Management Architecture (UIMA) a framework to integrate different computer software, each different and independent to work together. UIMA provided an architecture for the feasibility of building a real-time and scalable query answering system. In May 1997, as one of IBM's ambitious projects, the Deep Blue computer defeated Gary Kasparov, who launched IBM Research.
One might ask a question about whether it is difficult to win a game of chess or to build a question answering system. In 2006, the head of IBM Watson challenged researchers to build a question-answering system that would be able to compete on Jeopardy. A notable mention is the PIQUANT question answering system which was developed by IBM dating back to 1999.
To compete with the contestants on Jeopardy, the system is self-contained and must not be able to connect to Internet search. It has to analyze the questions, look for the answers in the resources it has and then be able to answer the questions. It turned out that on average a player tries 40% to 50% of the questions and gets between 85% and 95% of the answers correct.
Based on this, a performance goal was set, aiming for Watson to buzz 70% of the time and get 85% of them correct. To win at Jeopardy, the system must achieve such accuracy and must be able to do so in seconds. Watson uses hundreds of algorithms in the pipeline to get answers to the question posed to the system.
Question analysis: How Watson reads a clue
- Foundations of Question Analysis
- Focus and LAT Detection
- Extracting LATs from the category
- Question Classification and QSection issues
Classification of questions: The class of questions here is factual questions, however there may be other categories like Definition, Multiple Choice, Puzzle, Common Connections, Fill in the Blanks and Abbreviation. Special treatment: Here the case Question Section or Q Section is not visible, however another example when Q Section requires special treatment is if the question has section like "3 letter word" or "4 letter word". Query analysis is based on the analytics and semantic analysis capabilities of IBM Watson.
It consists of the English slot grammar parser, ESG with a Predicate Argument Structure (PAS). The analyst will be discussed in a later section, however to give an idea of what it can do, we provide a simple illustration. POETS AND POETRY: He was a bank clerk in the Yukon before publishing "Songs of a Sourdough" in 1907.
THEATRE: A new play based on this canine classic by Sir Arthur Conan Doyle opened on the London stage in 2007. In one of the plays Watson played, for the "Celebrations of the Month" category, which featured the following questions, misidentified LAT as "day". The question has been classified into QClasses so that later stages are easier for Watson.
Watson analyzes the questions and finds the QClass, and depending on the QClass, it tries to answer the questions. That's the question in Jeopardy where we're required to fill in the blanks. Answers have semantic relation to the question, where the category will specify the relation between them.
The detection in Watson is mostly done rule-based which includes regular expression patterns to detect QClass. To this classifier, the binary features corresponding to issuing rules are fed and trained against previously annotated QClass as the label. A special mention is the QSection, which is important for identifying the particular handling of questions.
Deep parsing in Watson
Passage scoring that aligns the evidence in the passage depends on Watson's Deep Parsing components. Type Coercion system uses PAS to compare the requested type with that in the natural language text.
Textual resource acquisition and engineering
Automatic knowledge extraction from documents
When a question is asked of Watson, the search component searches what is available to it. Candidate generation component uses the above steps and does further analysis on the retrieved text. By using different strategies which include using metadata of the fetched content like document title, anchor text of the hyperlink, it is able to hypothesize the answers.
Typing candidate answers using type coercion
Dbpedia: It will try to find what dbpedia entry "AR Rahman" refers to and it will find that it is indeed musician.
Textual evidence gathering and analysis
The evidence gathering system has a great impact on the question-answering system as it can now give confidence to the answers, which is useful in deciding whether or not to buzz during the Jeopardy games (J. W. Murdock, J. Fan, A . Lally , H. Shima, B.K. Boguraev., 2012).
Relation extraction and scoring in DeepQA
Structured data and inference in DeepQA
Special Questions and techniques
Q: Who was that first Prime Minister of India, Jawaharlal Nehru or Lal Bahadur Shastri.
Identifying implicit relationships
Fact-based question decomposition in DeepQA
The two components can be analyzed independently and we can state that both give "Mahatma Gandhi". So the question answering system will be sure to answer that answer is "Mahatma Gandhi".
A framework for merging and ranking of answers in DeepQA
Making Watson fast
In the game: The interface between Watson and Jeopardy
IBM Watson is written in JAVA and C++, comprising 100 analytical components and approximately 1 million lines of code.
Watson: Beyond jeopardy
The speech component will convert the text response into speech that will be presented to the audience. It relied on the change in results to verify that the given answer was indeed correct. On the other hand, DeepQA can be directly applied to readily accessible natural language text and queries can be used to retrieve relevant information.
In a clinical setting, it can be used to develop a physician assistant, where input to the system consisting of cases and patient health status would be provided. The DeepQA engine would then do all the reasoning and provide assistance to the physician on the appropriate actions to take. The questions to be answered are almost as threatened as in the medical field.
With few modifications to the DeepQA engine that worked at Jeopardy, we can have a robust medical diagnosis system (David Ferrucci, Anthony Levas, Suato Bagchi, David Gondek nad Eric Mueller, 2011).
Summary
34;Tregex and Tsurgeon: Tools for querying and manipulating tree data structures." Proceedings of the Fifth International Conference on Language Resources and Evaluation.
