With popular queries, it possible for the search engine to learn from the past. It can judge the effectiveness of search results and advertisements by counting how many people clicked on them. But for queries that seldom repeat, there is no real history from which to learn.
So how can a search engine ever understand these more obscure queries? Broder and a team of Yahoo! Researchers including Marcus Fontoura, Evgeniy Gabrilovich, Amruta Joshi, Vanja Josifovski, and Tong Zhang, set out to tackle this problem. Their work is outlined in a paper called Robust Classification of Rare Queries Using Web Knowledge, that appeared in SIGIR 2007
To address the problem, the Yahoo! team proposed a methodology for using search results, as well as information available on the Web, as a source of external knowledge. To this end, they sent rare queries to a search engine and assumed that a majority of the highest-ranking search results were relevant to the query. Categorizing these results allowed the team to classify the original query with high accuracy.
Direct to: Robust Classification of Rare Queries Using Web Knowledge
Source: Yahoo Research
