Richard Florida's Numbers: Shaky Under Scrutiny?
We need more math nerds in the non-profit arts world. And by "math nerd" I'm not talking about experts in multivariate calculus or linear algebra. I just mean people who aren't afraid of numbers and are willing to poke them with a stick occasionally.
Last week Ian Moss provided one of the most thorough and detailed critiques of Richard Florida's "creative class" theories I've ever read. Anyone involved in urban planning and cultural policy work (which arguably I am) is familiar with certain qualitative criticisms of Florida's (admittedly hugely influential) work. Perhaps the most common of these is that he pays insufficient attention to the negative consequences of gentrification, especially displacement. But Ian's critique is the first I've seen to really take a look at Florida's methodology and numbers. Frankly I'm ashamed that I never thought to do this myself, since only the barest scrutiny reveals them to be flimsy at best, dishonest at worst.
If Florida and his team ran a regression on [their stated primary criteria for creativity-driven growth] and job creation or per-capita income, controlling for other factors, we are not shown the results. In fact, the notes to chapter 13 document a correlation between the Creative Class concentration and employment growth that, while statistically significant, is only 0.03!...
The final insult, however, is Florida’s vaunted Creativity Index, the results of which seemed so strange to me when I first laid eyes on them. After some investigation of the methodology, I soon realized why. The Creativity Index in the original edition of the book is based on four equally weighted factors: the concentration of Creative Class workers in the area, a “High Tech” index measuring a region’s share of national tech industry output as well as the concentration of tech industries within the region, the number of patents filed per capita, and the concentration of same-sex domestic partners within the region. No explanation or evidence whatsoever is given to support the idea that these factors should be equally weighted. Instead, each of 268 metropolitan areas is ranked on each of the four factors, and the Creativity Index is calculated simply by subtracting the region’s rank order in each category from 1076, which, oddly, is four times 269. For example, Portland, ME ranks 28th on the Creative Class metric, 89th on the High-Tech Index, 134th on the Innovation (patent) Index, and 12th on the Gay Index. Its Creativity Index is 1076 – (28 + 89 + 134 + 12) = 813. No attention is paid to the distribution of the actual values within those ranks, which is not very useful if the distribution is anything other than linear, or differs between the four factors. Say there’s a giant cluster of cities in the Creative Class index that are almost tied from #140 to #157, but the city at #157 in the patent index is a huge drop from #156; this metric wouldn’t pick such common subtleties up. Rigorous scientific inquiry this is not.
Ian actually has some very positive things to say about Florida as well, so I encourage you to read the whole piece. But it sure is refreshing to see someone in this field whip out a calculator once in a while!