Online Searching
NOTE: After You Read the Background Article, Visit This New Page from Daypop, Where You Can See Word Bursts From Weblogs.
–
“Word scans indicate new ways of searching the Web”
When Dr. John Kleinberg of Cornell Univesity talks/writes about searching, it’s more than worth listening. In a recent presentation to the American Association for the Advancement of Science, Dr. Kleinberg discussed research on how “burstiness” might be a new tool for incorporation into relevancy measures. From the announcement, “[Kleinberg] devised a search algorithm that looks for “burstiness,” measuring not just the number of times words appear, but the rate of increase in those numbers over time. Programs based on his algorithm can scan text that varies with time and flag the most “bursty” words. “The method is motivated by probability models used to analyze the behavior of communication networks, where burstiness occurs in the traffic due to congestion and hot spots,” he explains…For searching the Web, Kleinberg suggests, such a technique could help zero in on what a searcher wants by recognizing the time context of such material as news stories. For instance, he says, a person searching for the word “sniper” today is likely to be looking for information about the recent attacks around the nation’s capital — but the same search nearly four decades ago might have come from someone interested in the Kennedy assassination.” Dr. Kleinberg was a researcher on IBM’s often discussed but never publicly released Clever project. Many of the underlying concepts from Clever are being utlilized by Teoma.
See Also: “Word ‘Bursts’ Could Help Refine Web Searches” (via Scientific American 2.19.03)
See Also: “Hypersearching the Web” (via Scientific American 6.1999)
Co-authored by Kleinberg, this is one of the better papers on web search aimed at a non-technical audience I’ve read. Again, many of the concepts discussed re: Clever are being used at Teoma.
See Also: “Clever New Way to Search?” (via Wired 11.27.98)

This entry was posted on Monday, February 24th, 2003 and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.