Séminaires RALI-OLST

Efficiently Collecting Relevance Information from Clickthroughs for Web Retrieval System Evaluation

Jing He (peaceful (point) he <at> gmail (point) com)

RALI, DIRO

Le mercredi 7 décembre 2011 à 11 h 30

Salle 3195, Pavillon André-Aisenstadt


Various click models have been recently proposed as a principled approach to infer the relevance of documents from the clickthrough data. The inferred document relevance is potentially useful in evaluating the Web retrieval systems. In practice, it generally requires to acquire the accurate evaluation results within minimal users' query submissions. This problem is important for speeding up search engine development and evaluation cycle and acquiring reliable evaluation results on tail queries. In this talk, I present a reordering framework for efficient evaluation problem in the context of clickthrough based Web retrieval evaluation. The main idea is to move up the documents that contribute more for the evaluation task. Both user study and TREC data based simulation experiments validate that the reordering framework results in much fewer query submissions to get accurate evaluation results with only a little harm to the users' utility.


Pour recevoir les annonces hebdomadaires par courriel, envoyez un message à l'adresse majordomo@iro.umontreal.ca. Il suffit d'envoyer un message ne contenant que la ligne 'subscribe ralli' (sans les apostrophes).

Liste de tous les séminaires pour l'année :

1991 1992 1993 1994 1995 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019