Information extraction is the analysis, most often shallow, of texts in order to retrieve specific information for a given task. RALI has been working in this area for many years on different aspects:
- microblog analysis
- syntactic and semantic annotation of actants
- opinion and sentiment analysis
- conversation analysis in a search and rescue context
- automatic e-mail answering
Syntactic and semantic annotation of actants
Opinion and sentiment analysis
Narjès Boufaden has worked on information extraction starting from the manual transcription of telephone conversations gathered in the context of search and rescue operations. Her thesis (March 2005) under the supervision of Guy Lapalme and Yoshua Bengio, dealt with such aspects as:
- automatic segmentation into themes
- generalized named entity recognition.
Automatic e-mail processing
Merkure was a project dealing with the automatic processing of e-mail sent to a corporation. It has been funded from 1999 to 2003 by Bell University Laboratories (BEUL) and a NSERC Collaborative Research and Development (CRD) Grant.
E-mail is one of the modes of communication whose use is most rapidly increasing recently. Unfortunately, few companies have fully integrated this medium of communication within their customer relationship management. This is why it is important to study the use of natural language technology for automatic e-mail processing and follow-up.
We have studied three complementary approaches to this problem in the context of the investors relationship service of Bell Canada Enterprises (BCE):
- text classification which groups messages according to common characteristics. This study was the subject of a Master's thesis (in French) by Julien Dubois who tried many different approaches.
- Case Based Reasoning is an approach which is quite natural for this problem because it corresponds to answering an e-mail by modifying a previously sent message to a similar question. This work was the subject of Ph.D. thesis (in French) by Luc Lamontagne. He has developed a methodology and a prototype which computes the similarity between a new message and the messages in a case base composed of previous e-mail-answer pairs. The system then adapts the answer according to the new information.
- specialized question-answering for this domain in which questions are short and ask for factual answers that can be found in a textual database (e.g. How many branches do you have in North America?). The analysis of e-mails sent to the investors relationship service of BCE have demonstrated the interest of using this approach. The thesis (in French) of Luc Bélanger developed an architecture for automatic processing of e-mail and made links with question-answering systems, such as Quantum described below.
Quantum was a question-answering system developed by Luc Plamondon between 2000 and 2003 in the context of Merkure. Quantum participated in the TREC-QA evaluation between 2000 and 2002. The goal of question-answering is to identify in a textual database a factual answer to short and explicit question (e.g. What is the maiden name of Hillary Clinton ?). This work was the subject of Master's thesis (in French) by Luc Plamondon who continued to work on the system as a researcher at RALI.
Quantum performs an analysis of the question in order to determine its focus and then conducts an information retrieval search using the terms of the question to identify passages in the textual database that have the best chance of containing the answer. These passages are then further analyzed to extract particular named entities (person names, dates, ...). Depending on the type of answer that is asked for, the most probable answer is returned to the user.
In collaboration with National Research Council (NRC), Quantum has also been used for crosslingual question-answering (questions in French on an English corpus) for the CLEF 2003 competition. Questions and answers have been translated by a statistical machine translation engine developed by George Foster.
To help develop the system, NRC had the TREC questions translated into French. This bilingual corpus of almost 2000 questions in both French and English is a unique resource for the development of not only multilingual question-answering systems, but also for more general translation systems, since questions have a different structure from those found in usual corpora.