Friday, December 10, 2004
Inside the Echo Chamber: Measure of Media Bias
Update, Dec. 10
Per Kevin Drum, throw one more caveat on the fire: the number of times a network fails to properly identify a typical man on the street as the National Taxpayers Union shill he really is.
"A Measure of Media Bias"
, a report first released in September, 2003, is re-circulating among our friends on the right, starting with fawning coverage in The Weekly Standard
Bias Beyond a Reasonable Doubt
The question is, Who is right? Is there a left- or right-wing bias, or have the media actually managed to be objective? A serious assessment requires quantification of the output put forth by the media. The best analysis I know along these lines is the ongoing study "A Measure of Media Bias," by professors Tim Groseclose of UCLA and Jeffrey Milyo of the University of Missouri.
These researchers use a clever statistical technique to construct an objective measure of conservative or liberal bias in the news coverage of major U.S. television and radio stations, newspapers, magazines, and the Internet. Their main finding is that the liberal inclination of the mainstream media is clear. Among 20 major outlets, Fox News and the Washington Times emerge as conservative, but the other 18 range from slightly to substantially left of center.
The story has been reported in Capitol Hill Blue, Instapundit,
Business Week, National Review, and Andrew Sullivan.
Most recently, the report was profiled at The Key Monk:
Beyond a reasonable doubt - the liberal media bias
The Weekly Standard has a good piece by Harvard professor Robert Barro who explains an innovative study done by Tim Groseclose (UCLA/Stanford) and Jeff Milyo (Chicago) called "A Measure of Media Bias".
The study compares how often--in news articles or reports--each outlet cites certain think tanks vs. how often members of Congress cite them. Using the ADA (Americans for Democratic Action) ratings for each member of Congress a theoretical ADA rating is computed for each outlet.
(The ADA, an ultra-liberal organization, takes 20 votes a year in both the House and Senate and gives a legislator 5 points if they vote the the (sic) way the ADA wants. For example, Ted Kennedy has about a 90 lifetime rating and Orrin Hatch, about 5)
What is excellent about this study is it is pretty objective - conservatives cite certain think tanks much more than liberals and vice versa
The Freepers always have something to say:
I'm sure someone else has already put this up, but I wanted to get it out there again. Drudge had it up on his page, of course, and The Annotated Media had it too, but no one else seemed to picked it up (shocking!). (link)
MSNBC.COM is the most biased, liberal POS on the web masquerading as a mainstream news site. They are currently in orgasmic throes over the Toon's book. Other breathless headlines. "UN says US Stealing Iraq money" "Bush losing support, now even with Kerry on the war issue" Blah blah blah blah. Straight from Seattle ultra ultra liberal land to you. What a crock. (link)
Most welcome :-) I thought it was significant in that it uses mathematical evidence rather than the traditional anecdotal. (link)
Good points. This study, like any good academic study, is limited and focused. All it sets out to do is examine the rate at which media outlets cite various think tanks in pure news stories and rate political orientation of the outlets based on the think tanks used. The political orientation of a think tank is determined from how often it is cited by Congressman and Senators with known political biases.
The result is limited in scope and far from proving a liberal media bias. The conclusion is simply that most media sources cite the same think tanks as a moderately left of center Congressman in news stories. Implicit is the assumption that elected Democrats and Republicans are accurate barometers of liberalism and conservatism as a whole, which is perhaps the weakest link in the study. There also does not seem to be any discrimination in what types of stories are reported because that may well affect the think tanks consulted. And, like you pointed out, editorial content, in particular editorial comments by anchors during a broadcast, is an important factor in media bias.
In sum, the result is something most people accept: the news is reported from a moderately liberal perspective. Claims that conservatives dominate the news or that the mainstream media is far-left are not supported by this study. Given that reporters describe themselves as socially liberal but economically conservative, this is not that surprising.
Now, it'd be instructive to see a similar study on editorial content in news broadcasts with the same level of objective rigour or a study looking at which Congressman and Senators are cited in reports since that is even more direct than the methods used in this study. (link).
Outside the Echo Chamber
You can find an in-depth discussion at Language Log (with follow up from the report's authors here). What does the report actually prove? Well, if you:
Begin with an arbitrary score, here a rating assigned by Americans for Democratic Action (one could just as easily begin with a rating from any interest group), then
non-transparently adjust the arbitrary score, then
assume the average non-transparently adjusted arbitrary score for a Republican-controlled Congress is the definition of the political center, then
Claim that you pick the 20 most cited think thanks in Congress, but find that the Alexis de Tocqueville Institute (sic) is cited more often than, say, Planned Parenthood, the National Association of Manufacturers, or the NRA, then
Pretend that the Children's Defense Fund is a "think tank" in the same sense as the Heritage Foundation, the American Enterprise Institute, and the Cato Institute, then
Count all the sentences that appear in certain media outlets over six months in 2002, six months in 2003, or over 13 years, except the sentences that you choose to exclude, then
use 2,000 sentences as a representative sample of an evening news report's slant in coverage over 13 years,
Release the result before you complete the work, "A large number of economic studies give theoretical reasons that bolster the view
that the media does not have a systematic bias. (See xx, xx, xx and xx)." (from page 4 of the report), and, finally,
Compare the ratio obtained by looking at 2,000 sentences referencing one of 20 arbitrarily chosen, ill-defined "think tanks" to the median ratio of similar citations made in the Congressional Record and compare the ratios to derive a ranking for the news organization based on your non-transparently adjusted arbitrarily chosen ratings for Congress
...well, you can "prove" whatever you want to prove. In this case, that Fox News is centrist and that the CBS Evening News is somewhere between mild sedition and Trotsky. And if you hide your shenanigans behind "exp(aj + bj yi ) / ∑k=1
J exp(ak + bk yi )", people will call it "research".