The Annual Misuse of Hate Crime Statistics

NOTE: SIGNIFICANT UPDATES BELOW THE JUMP.

Every year around this time, the FBI publishes its statutorily-mandated annual report on hate crime statistics.  Like clockwork, every year that report gets misused no matter what the FBI does to discourage that misuse (previous examples of misuse here).  This year is no exception, as several prominent liberal sites have picked up on the lede that this year’s report shows a “sharp increase” in anti-gay hate crime while also noting that race-based hate crimes barely decreased at all

The problem is that these particular FBI statistics are virtually useless for evaluating year-to-year trends – always have been, always will be.  This year, the FBI itself went out of its way to warn against such readings, stating “our Uniform Crime Reporting (UCR) Program doesn’t report trends in hate crime stats—yearly increases or decreases often occur because the number of agencies who report to us varies from year to year.”

Yet in reporting an 11 percent increase in hate crimes against gays while lamenting a mere 1% decrease in race-based hate crimes, Think Progress and Feministing ignore this important disclaimer.  This failure is significant because several hundred more law enforcement agencies participated in this year’s survey than last year’s survey: last year’s survey had the participation of 13,241 agencies, this year’s of 13,690.  Of those agencies, 2025 in 2007 and 2145 in 2008 actually reported any hate crimes.  This discrepancy in reporting agencies alone makes a worthwhile one-to-one, year-to-year comparison very difficult to make.  At a minimum, for the purposes of this year’s numbers, the discrepancy in reporting agencies accounts for somewhere between 1/3 and 1/2 of the apparent increase in hate crimes against gays.

Moreover, the composition of the agencies participating in the report changes from year to year, and varies greatly from state to state.  For instance, in 2007, only 18 law enforcement agencies in the state of Georgia, covering a population of 680,591, participated in the report; in 2008, this number actually decreased to 6 law enforcement agencies covering 554,193 people.  Conversely, in 2007, North Carolina had 369 reporting agencies covering 7,421,293 people; in 2008, it had 495 reporting agencies covering 9,128,679 people.  The statistics, however, do not account for these variations in state by state, year-to-year reporting making year-to-year comparisons (in any direction) highly suspect.

Another reason why these statistics are particularly meaningless (for purposes of a year-to-year comparison at least) is that the statistics do not account for differences in state laws or prosecutorial attitudes and priorities.  This is especially important when discussing anti-gay hate crime because of the fact that sexual orientation is still a relatively new addition to hate crimes laws such that the willingness (and, for that matter, ability) of prosecutors to charge a defendant with an anti-gay hate crime – or of law enforcement officers to report an anti-gay hate crime – is likely to be increasing significantly every year.  There may, therefore, be an increase in the number of reported anti-gay hate crimes even as there may (or may not) be a decrease in the number of actual anti-gay hate crimes.  Indeed, as this map shows, 19 states still do not recognize anti-gay hate crimes as hate crimes at all – even for reporting purposes.  This means that even small local changes in state-level prosecutorial priorities are likely to have an outsized effect on the national statistics for anti-gay hate crimes. 

This discrepancy in law enforcement priorities is readily apparent when you compare the statistics for a state like socially liberal Vermont with a state with a history of open racism like Mississippi.  In 2007, Vermont had reports from 80 agencies representing 610,907 people, with a total of 21 hate crimes reported by those agencies.  Mississippi, by contrast, had reports from 58 agencies representing 704,703 people, with a total of exactly zero – yes, zero – hate crimes reported.  This year, Missippi had reports from two more agencies than last year, resulting in one agency reporting 4 hate crimes; Vermont, with two additional agencies reporting but with slightly fewer people represented in the report, still reported 20 hate crimes from 12 different agencies.  It seems implausible, to say the least, that hippie-dippie Vermont’s actual hate crime rate was more than 200% higher than Mississippi’s in 2008 – and infinitely higher in 2007.  Far more likely is that Vermont has tougher hate crimes laws than Mississippi and is thus significantly more likely to report hate crimes than Mississippi.

Brian Levin at Huffington Post does a good job explaining why such caution must drawn about drawing inferences from these statistics – although he still notes the overall slight decline in hate crimes, writing:

Three of the five states with the highest proportion of African-Americans, who account for 35% of all reported hate crimes nationally in 2007, barely reported hate crime at all that year:

1. Mississippi, 38% African-American, 0 hate crimes

2. Louisiana, 32% African-American, 31 hate crimes

3. Georgia, 31% African-American, 13 hate crimes 

Contrast these totals with that of South Carolina, a state in the same region with similar demographics and a total population almost the same as Louisiana and half that of Georgia’s. South Carolina has over one million African-American residents comprising 29% of the state’s total population. The state reported 127 hate crimes in 2007. In 2007 Boston, a city with a dedicated police hate crime unit and a population of 500,000, counted more hate crime cases than Georgia, Alabama, Arkansas, Louisiana, and Mississippi combined with a total population of about 24.4 million.

The point here is that it is extraordinarily difficult – if not impossible – to draw any kind of conclusions about trends in hate crime on the basis of the FBI’s annual reports, unless there are truly massive swings in a data set or unless a trend continues for a number of years without any plausible explanation related to changes in law enforcement priorities or state-level hate crimes legislation. 

Because of the severe limitations of this annual report, I am indeed skeptical whether it has any value whatsoever.  At a minimum, however, interpretations of the data in these reports need to be limited to assessments of where law enforcement priorities lie and of what forms of hate crimes are, relatively speaking, the most common.

UPDATE: Also spotted misusing the statistics: Raw Story and, to a lesser extent, NPR.  All via memeorandum.

UPDATE II: Via greginak in comments, Think Progress has updated the previously mentioned post to make note of the FBI’s disclaimer against using these statistics to make year-to-year comparisons.  Think Progress’ update still suggests that the data may be evidence of an increase in hate crimes against gays or just  result of the difference in reporting agencies, however, rather than noting that there is no basis for a statistical comparison at all.  I’m not saying that there hasn’t been an increase in anti-gay hate crime, by the way – just that the hate crimes statistics provide no evidence that there has been such an increase and, to my knowledge, there is not currently any conceivably reliable statistical evidence to suggest either an increase or a decrease in anti-gay hate crime.

UPDATE III: Crikey.  It looks like there’s been a host of misreporting on this topic by old media outlets, including the Associated Press (though it at least bothered to include a warning about the data quality from the aforementioned Brian Levin).  The Detroit Free Press uses the deeply misleading headline “Michigan 4th in nation in hate crimes.”  The Jewish Telegraphic Agency reports that “Hate crimes hit seven-year high.”  USA Today reports that “Hate crimes against blacks, religious groups rise.” CBS News reports “Hate Crimes Up in 2008” – and doesn’t even bother to mention the FBI’s disclaimer or Levin’s warning, as do some of the other stories.  The list could go on.  Of the old media outlets I’ve seen report on this, only CNN used a headline that was not absurdly misleading, proclaiming (accurately and unsensationally) “FBI cites thousands of hate crimes in ’08.”

Please do be so kind as to share this post.
TwitterFacebookRedditEmailPrintFriendlyMore options

21 thoughts on “The Annual Misuse of Hate Crime Statistics

  1. everybody should take at least half a year of stats in HS and read the book How to Lie with Statistics.

    good stuff mark. i’m surprised Think Progress whiffed on this,they are usually good.

    Report

    • Thanks, Greg! I don’t know what it is about this particular report, but it seems to invite this kind of a misread every single year. The previous example I cited was even worse in some ways, because the headline was disseminated by CNN. I suspect it has something to do with the fact that the report is statutorily mandated, ostensibly for the purposes of keeping track of annual trends, so it’s very easy to fall into the trap of thinking that it’s a meaningful report.

      The positive is that this year, at least, it looks like CNN reported the data appropriately:
      http://www.cnn.com/2009/CRIME/11/23/hate.crimes.fbi.report/index.html

      Report

  2. Sometimes stuff likes this makes me think there are Americans who hate America(ns) and they aren’t just from make believe stories Republicans tell their children to scare them.

    What really bothers me here is the all too common way in which people combine good intentions with an irresponsible lack of thought or competence.

    In this case, the goal is to highlight crimes against gays, women, blacks, etc…, unaffectionately called “hate crimes,” in order to advance awareness and preventative action. Yet, instead of advancing that cause, they’ve given legitimate fodder for critics to harp on – the accuracy of their claims and a tendency to distort data to serve their purpose.

    Which, in turn, leads to an argument between the good people who “get it” and “care” versus the bad people who don’t. When, in fact, the good people are just really bad at crafting their message and rather than own that cast aspersions. Rinse and repeat.

    Report

  3. FYI- think progress has posted this update “The FBI adds a disclaimer that it “doesn’t report trends in hate crime stats.” So increases or decreases may be due to changes in the number of actual hate crimes, or changes in “the number of agencies who report to us varies from year to year.”

    Whatever the faults of web news and opinion, many of them are willing to update and admit mistakes far more then the MSM

    Report

    • I don’t know I find print outlets are good at admitting mistakes, they just tend not to put them front and center in a way that is structurally more likely to happen with a blog.

      Though, it’d be nice if – aware of the irony of not doing this myself – blogs did more ombudsman work. Sully sort of does it with his dissents but otherwise, not so much.

      Report

  4. For a long time government has kept statistics on various types of crimes. Statistics are not the real topic here. What is the real meaning of hate crimes/hate speech? Who wields this term the most often and to what purpose? It seems to me, more and more, “hate” is a nasty epithet hurled by the Left against a broad swath of opponents of its various programs. It is a short step from hate speech to thought crimes. If you are interested, you can read more in my blog entry “Hate Speech” at hudowen@wordpress.com. Link: http://hudowen.wordpress.com/2009/06/15/hate-speech/?preview=true&preview_id=340&preview_nonce=6c041beaeb

    Report

  5. So the report is more a measure of how agencies report crimes, and it shows an increase in police-agency reporting hate crimes against gays.

    Though it’s counterintuitive, perhaps it’s good news in the struggle for equal rights.

    Report

  6. I think that we can draw a clear conclusion from this data.

    Our data gathering for these statistics is deeply screwed. It would be nice if we could at least get all agencies reporting.

    Report

    • That’s certainly part of it. But a good chunk of this is also that each state has wildly varying hate crimes laws. So even if every agency had a formal mechanism for reporting hate crimes, you’d still get impossible-to-interpret data simply because of the different definitions of what constitutes a hate crime, to say nothing of variations in local law enforcement priorities.

      Report

      • I’d guess there’s general under-reporting to the authorities, as well. I’m sure many crimes committed simply because a victim is gay, like many sexual assaults against women, aren’t reported because of the stigma involved; particularly for people who haven’t come out. I’m sure other crimes are reported as assault, vandalism, etc. though there was a ‘hate’ aspect to the crime.

        Report

  7. If I may ask a wicked question, did the run of the mill, just a garden variety assult and battery, this was only a random guy who happened to be Muslim who shot people while yelling “Allahu Akbar” kinda crimes go up or down?

    I ask because, if *I* were a reporter whose favorite headline was “WE ARE ALL GOING TO DIE” and the crimes that consisted of little more than random people killing other random people for reasons we can’t really determine went *DOWN*, I might be tempted to read these numbers and say “Hey, I might be able to use my favorite headline after all…”

    Report

  8. Discussing the uselessness of the data as a measure of trends seems rather, well, distracting. I, for one, would like to know how the seven bias-based killings last year break down by bias. Oddly, the stats don’t seem to offer those details…

    Report

    • It’s not distracting if the primary way in which the data is being used is as a measurement of trends, which is in fact how the data is being touted. It is thus important to point out that the data does not, in fact, show any trends.

      As for how various crimes break down by bias, table 4 of the report provides that information. It shows 1 race-based killing, 3 anti-gay killings, 2 anti-bisexual killings, and 1 anti-Hispanic killing. However, it’s difficult to read anything into those numbers given not only the very small sample size but also, again, the fact that state hate crime laws vary so wildly.

      Report

  9. Pingback: Box Turtle Bulletin » Sexual Orientation Hate Crimes: Did They Really Go Up?

  10. Pingback: More on Hate Crimes Statistics | The League of Ordinary Gentlemen

Comments are closed.