Friday, September 24, 2004

Google News Bias

Very interesting story at OJR on bias in Google News and Yahoo News story rankings, relevant to my post on blog pancakes:

Bharat said Google News uses a mix of techniques to ensure that users are presented a diverse range of perspectives. The ranking and prominence of stories are based on several factors: How many publications are writing about a topic; how recent the articles are; the size of the story, with substantive pieces ranking higher than short items; and the frequency of the search term within the article. The computer algorithms, he said, "are trying to understand how hot and how big the story is."

Every 15 minutes a new edition of Google News is generated and the ranking changes. The formula rearranges the headline blurbs in each story cluster based on the freshness of each article and the importance of the source. "The algorithms do not understand which sources are right-leaning or left-leaning," Bharat said. "They're apolitical, which is good."

Google News does not use the same formula as Google's general search engine, which ranks results based on how many people are linking to a site or article. (While "John Kerry" results in 100,000 results on Google News, the same term draws 4.3 million results on Google.) Special interest groups use a linking technique known as "Google bombing" to skew Google's general search engine results to their liking. For example, the first result for a search on "Dan Rather" is not the CBS News site but Bharat points out, however, that link popularity plays no result in Google News' rankings. "Our mission is to be all-inclusive," Bharat said. "We want breadth and variety. I would like Republicans and Democrats alike to read pro Kerry and anti-Kerry articles, but it's not our job to change the natural range of opinions that you see in the press. We're showing you the world the way it is." But are they? Why does clicking on a "John Kerry" link in Google News turn up so many second-tier conservative sites but so few liberal sites?

Automated systems are subject to tampering and manipulation, that's just the way it is. Yahoo News uses a combination of automated processes and editorial staff:

Like Google News, Yahoo won't disclose how a term like John Kerry or George Bush makes it to the front page of its search results, but Birkeland said the factors include the source, the freshness of the story, and a method of determining relevance. Yahoo achieves balance in political coverage by using a wide variety of news partners and an editorial staff that pulls together "a very wide cut at what the news is on a given day," Birkeland said.

"We use actual humans," he added. "News is far too human of an endeavor to rely 100 percent on automation."

Humans editors and reporters can also be manipulated, as we've seen with the Dan Rather goof, but less easily. The question is whether bloggers themselves can present a balanced view of the world..can a few raise themselves to the level of public trust required for consideration from other than the fringes? Can technology be developed to provide an alternative to the "he who shouts loudest is amplified further" function of the blog pancake?