Using Context to Improve the Evaluation of Information Retrieval Systems

Abstract

The crucial role of the evaluation in the development of the information retrieval tools is useful evidence to improve the performance of these tools and the quality of results that they return. However, the classic evaluation approaches have limitations and shortcomings especially regarding to the user consideration, the measure of the adequacy between the query and the returned documents and the consideration of characteristics, specifications and behaviors of the search tool. Therefore, we believe that the exploitation of contextual elements could be a very good way to evaluate the search tools. So, this paper presents a new approach that takes into account the context during the evaluation process at three complementary levels. The experiments gives at the end of this article has shown the applicability of the proposed approach to real research tools. The tests were performed with the most popular searching engine (i.e. Google, Bing and Yahoo) selected in particular for their high selectivity. The obtained results revealed that the ability of these engines to rejecting dead links, redundant results and parasites pages depends strongly to how queries are formulated, and to the political of sites offering this information to present their content. The relevance evaluation of results provided by these engines, using the user's judgments, then using an automatic manner to take into account the query context has also shown a general decline in the perceived relevance according to the number of the considered results.