—word burstiness n.
—Pamela LiCalzi O'Connell, "Online diary," The New York Times, March 13, 2003
Jon Kleinberg, at Cornell University in New York, has developed computer algorithms that identify bursts of word use in documents.
While other popular search techniques simply count the number of words or phrases in documents, Kleinberg's approach also takes into account the rate at which the word usage increases.
Kleinberg suggests that the method could be applied to weblogs to track new social trends. For example, identifying word bursts in the hundreds of thousands of personal diaries now on the web could help advertisers quickly spot an emerging craze.
—Will Knight, "Word 'bursts' may reveal online trends," New Scientist, February 19, 2003
Also:
He devised a search algorithm that looks for "burstiness," measuring not just the number of times words appear, but the rate of increase in those numbers over time. Programs based on his algorithm can scan text that varies with time and flag the most "bursty" words.
—"Buzzwords of history, revealed by computer scans, indicate new ways of searching the Web," Cornell News, February 18, 2003
Professor Kleinberg demonstrated his technique by applying it to all of the U.S. Presidential State of the Union Addresses, from 1790 to 2002. He found that the bursty words reflected the concerns and events of the times. For example, in the early 1800s, among the words exhibiting the highest burstiness were militia, British, enemy, and Spain. In the 1930s, the bursty words were depression, banks, relief, and recovery, while the 40s coughed up wartime, democracy, fighting, Japanese, and production (but, surprisingly, not German or Germany). Among the many bursty words in the 1990s are families, crime, medicare, challenge, 21st, and century. You can see the full list here:
