Domain: Diverse Ranking

Overview

Most users leverage Web search engine as a predominant tool to fulfill their information needs. Users' information needs, typically described by keyword based queries, are often ambiguous or multi-faceted. On the one hand, for some ambiguous queries, there are multiple interpretations of the underlying needs (e.g., query "band" may refer to the rock band, frequency band or rubber band). One the other hand, queries even with clear definition might still be multi-faceted (e.g., "britney spears"), in the sense that there are many aspects of the information needs (e.g., news, videos, photos of Britney Spears). Therefore, search result diversification has attracted considerable attention as a means to tackle the above problem. The key idea is to provide a diversified result list, in the hope that different users will find some results that can cover their information needs.

Dataset List

Welcome to the to the TREC 2009 Web Track. Our goal is to explore and evaluate Web retrieval technologies over the new billion-page ClueWeb09 Dataset. The dataset was crawled from the Web during January and February 2009 and 50 topics will be used. For the purposes of the diversity track, each topi...
BASELINE ALPHA_NDCG@5 ALPHA_NDCG@10 ALPHA_NDCG@20 ERR_NDCG@5 Evaluation
PAMM(α-NDCG) 0.6370 0.4080 0.4270 0.5640 Detail
PAMM(ERR-IA) 0.5210 0.4250 0.4220 0.5670 Detail
BASELINE ERR_NDCG@10 ERR_NDCG@20 Evaluation
PAMM(α-NDCG) 0.2910 0.2840 Detail
PAMM(ERR-IA) 0.3400 0.2940 Detail
Welcome to the to the TREC 2010 Web Track. Our goal is to explore and evaluate Web retrieval technologies over the billion-page ClueWeb09 Dataset.. The dataset was crawled from the Web during January and February 2009 and 48 topics will be used. For the purposes of the diversity track, each topic w...
BASELINE ALPHA_NDCG@5 ALPHA_NDCG@10 ALPHA_NDCG@20 ERR_NDCG@5 Evaluation
PAMM(α-NDCG) 0.7290 0.6640 0.5250 0.4130 Detail
PAMM(ERR-IA) 0.6330 0.6250 0.5100 0.4190 Detail
BASELINE ERR_NDCG@10 ERR_NDCG@20 Evaluation
PAMM(α-NDCG) 0.3810 0.3820 Detail
PAMM(ERR-IA) 0.4090 0.3860 Detail
Welcome to the to the TREC 2011 Web Track. Our goal is to explore and evaluate Web retrieval technologies over the new billion-page ClueWeb09 Dataset. The dataset was crawled from the Web during January and February 2009 and 50 topics will be used. For the purposes of the diversity track, each topi...
BASELINE ALPHA_NDCG@5 ALPHA_NDCG@10 ALPHA_NDCG@20 ERR_NDCG@5 Evaluation
PAMM(α-NDCG) 0.8290 0.6790 0.6460 0.5610 Detail
PAMM(ERR-IA) 0.6870 0.6510 0.6360 0.5970 Detail
BASELINE ERR_NDCG@10 ERR_NDCG@20 Evaluation
PAMM(α-NDCG) 0.5710 0.5380 Detail
PAMM(ERR-IA) 0.5950 0.5470 Detail