Séminaires RALI-OLST

Attending Knowledge Facts With BERT-Like Models In Question-Answering : Disappointing Results And Some Explanations

Guillaume Le Berre (guillaume (point) le_berre <at> depinfonancy (point) net)


Wednesday 18 November 2020 at 11:30 AM

Réunion Zoom (below)

Since the first appearance of BERT, pretrained BERT inspired models (XLNet, Roberta, ...) have delivered state-of-the-art results in a large number of Natural Language Processing tasks. This in- cludes question-answering where previous models performed relatively poorly particularly on datasets with a limited amount of data. In this paper we perform experiments with BERT on two such datasets that are OpenBookQA and ARC. Our aim is to understand why, in our experi- ments, using BERT sentence representations inside an attention mecha- nism on a set of facts tends to give poor results. We demonstrate that in some cases, the sentence representations proposed by BERT are limited in terms of semantic and that BERT often answers the questions in a meaningless way.

Presentation recording (in French): https://drive.google.com/file/d/17nc5yu4r15xt8fA3d-63nUF0W-AOc6EW/view?usp=sharing

Join with Zoom at this url.
Meeting ID: 916 9097 5818, 𝑷𝒂𝒔𝒔𝒄𝒐𝒅𝒆: 343273.
Phone numbers: https://umontreal.zoom.us/u/abitNZzLg.
One tap mobile: +14388097799,,91690975818#,,,,,,0#,,343273#

To receive weekly talk announcements, please send an e-mail to majordomo@iro.umontreal.ca. Simply write a message containing the single line 'subscribe ralli' (without the quotes).

See all the weekly talks for the year:

1991 1992 1993 1994 1995 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021