Tesis doctoral de Cesar De Pablo Sanchez
Question answering (qa) systems add new capabilities to traditional search engines with the ability to find precise answers to user questions. Its main objective is to enable easier information access by reducing the time and effort that the user requires to find a concrete information among a list of relevant documents. In this thesis we have carried out two works related with qa systems. the first part introduces an architecture for qa systems for spanish which is based on the combination and adaptation of different techniques from information retrieval (ir) and information extraction (ie). This architecture is composed by three modules that include question analysis, relevant passage retrieval and answer extraction and selection. The appropriate processing of named entities have received special attention because of their importance as question themes and candidate answers. the proposed architecture has been implemented as part of the miracle qa system. This system has taken part in independent evaluations like the clef@qa track in the cross-language evaluation forum (clef). Results from 2004 to 2007 campaigns as well as the details and the evolution of the system have been described in deep. The miracle qa system has obtained moderate performance with a first answer accuracy ranging between 20% and 30%. Nevertheless, it is important to highlight results obtained in the 2005 main qa task and the realtimeqa pilot task in 2006. The last one included response time as an important additional factor of the evaluation. These results back the proposed architecture as an option for question answering from textual collection and confirm the results obtained for english and other languages. on the other hand, the analysis of the results along evaluation campaigns and the comparison with other qa systems point problems with current systems and new challenges. According to our experience, it is more difficult to tailor qa systems to different domains and languages than ir systems. The problem is inherited by the use of complex language analysis tools like pos taggers, parsers and other semantic analyzers, like named entity recognition and classification (nerc) and relation detection and characterization (rdc) tools. the second part of this thesis tackles this problem and proposes a different approach for qa systems for different languages and collections. The proposal focuses on acquiring knowledge for the semantic analyzers in qa systems based on lightly supervised approaches. The goal is to obtain useful resources that help to perform nerc or rdc using as few annotated resources as possible. Besides, we try to avoid dependencies from other language analysis tools with the purpose that these methods apply to different languages and domains. First of all, we have study previous work on building nerc and rdc modules with few supervision, particularly bootstrapping methods. We propose a common framework for different bootstrapping systems that help to unify different evaluation functions for intermediate results. The main proposal is a new algorithm that is able to simultaneously acquire instances and patterns associated to a relation of interest. It also uses mutual exclusion among relations to reduce concept drift and achieve better results. A distinctive characteristic is that it uses a query based exploration strategy of the text collection which enables their use for larger collections. Candidate selection and evaluation is based on incrementally building a graph of instances and patterns which also justifies our evaluation function. The discovery approach is analogous to the front of exploration in a web crawler and it is able to find the most similar instances to the available seeds. this algorithm has been implemented in the spindel system. We have selected for evaluation the task of acquiring resources for the most common named entity classes, person, location and organization. The objective is to acquire name instances that belong to any of the classes as well as contextual patterns that help to detect mentions of ne that belong to that class. We present results for the acquisition of resources from raw text from two different languages, spanish and english. We also performed experiments for spanish in two different collections, news and texts from a collaborative encyclopedia, wikipedia. Both cases are tackled with limited language analysis tools and resources. with an initial list of 40 instance seeds, the bootstrapping process is able to acquire large name lists containing up to 30.000 instances with a variable quality. Besides, large lists of indicative patterns are obtained too. Our indirect evaluation confirms the utility of both resources to classify ne using a simple dictionary recognition approach. Best results for spanish obtained a f-score of 67,17 and for english this value is 55,99. The module requires much less development effort than annotation for supervised algorithms although the performance is not in pair yet. This research is a first step towards the development of semantic applications like qa for a new language or domain with no annotated corpora that requires less adaptation effort.
Datos académicos de la tesis doctoral «Bootstrapping named entity resources for adaptive question answering systems«
- Título de la tesis: Bootstrapping named entity resources for adaptive question answering systems
- Autor: Cesar De Pablo Sanchez
- Universidad: Carlos III de Madrid
- Fecha de lectura de la tesis: 22/10/2010
Dirección y tribunal
- Director de la tesis
- Paloma Martinez Fernandez
- Tribunal
- Presidente del tribunal: Juan Llorens morillo
- Antonio Toral ruiz (vocal)
- José Carlos González cristobal (vocal)
- José Luis Vicedo gonzález (vocal)