By Michael Johnston
We thank Matthew Stephenson for his sincere response to Michael Johnston’s post “Breaking Out of the Methodological Cage”, in “The Level-of-Aggregation Question in Corruption Measurement” on the Global Anticorruption Blog. Professor Johnston keeps the conversation going with his response below.
There is a lot to like in Matthew Stephenson’s blog post: the more we debate the issue of measuring and comparing corruption the better off we all are. And I’m not even sure that our disagreements run very deep.
In fact, toward the end of his critique he summarizes my fundamental position quite well: “…for the research questions we do or should care about regarding the causes and consequences of corruption, identifying correlates of these summary measures is not useful.” That’s a major aspect of that methodological “cage” I hoped to highlight and criticize.
I do not in any way object to quantitative methods as a way to study corruption issues; indeed, as I noted I have used corruption indices myself. Rather, my problem is with what I see as a tendency to reify many of the measures we use, to overgeneralize from statistical results (which after all are usually probabilistic inferences based in part on proxy measures), and – by extension – to chase what ought to be big and important questions down the rabbit hole of substantively insignificant methodological elaborations (again, I could tell stories about manuscripts I regularly review).
Follow the conversation between Michael Johnston and Matthew Stephenson! Start with “Breaking out of the Methodological Cage” >“The Level-of-Aggregation Question in Corruption Measurement” > “1.39 Cheers for Quantitative Analysis” > and read next > “Are Aggregate Corruption Indicators Coherent and/or Useful?: Further Reflections“
Understanding the origins, consequences, and reform challenges of corruption problems
Corruption matters, among other reasons, because it raises major issues of justice, and questions about whether and how people can govern themselves, and govern themselves well. I doubt we would differ greatly at that level.
With respect to corruption indices themselves, I do maintain that they flatten out critical variations among and within societies. Some kinds of corruption involve abuses of wealth in pursuit of power, as Huntington pointed out many years ago, while others involve abuses of power in pursuit of wealth. In some settings corruption serves as an alternative to violence, while in others corruption and violence feed upon each other. At times corruption capitalizes upon the insecurity of citizens and business people, while in others it underwrites a degree of de facto stability.
There are cases in which much of what might be termed corrupt is legal, or at least unclassified by the law, others in which corrupt lawbreaking makes up the bulk of the problem, and still others in which powerful individuals or factions enjoy a degree of impunity that makes legality irrelevant.
We can and should think carefully what such patterns have in common that we can call corrupt.
But particularly in light of our longstanding inability to come to a working consensus over how to define corruption, to summarize the differences among such cases with just a more-versus-less number is to obscure contrasts critical to understanding the origins, consequences, and reform challenges of corruption problems in contrasting settings.
In my previous work I have argued that four qualitatively different syndromes of corruption can be identified in various parts of the world; while that number of syndromes and my understanding of them may well be in error, I do believe that understanding qualitative variations in corruption is at least as important as the sometimes dubious precision we impute to scores on one-dimensional indices.
Levels of aggregation
The second issue has to do with levels of aggregation. In no way do I object to aggregating evidence at the national level; most of the statistics we routinely employ about whole countries are aggregations of one sort or another.
But what is being aggregated? GDP per capita statistics are indeed aggregations, but they bring together diverse activities that can generally still be assessed and added up on a common underlying dimension – money – and whose estimated totals do tell us something of importance (though far from everything, it is true) about a national economy. But what do corruption indices aggregate?
Overwhelmingly they aggregate perceptions of a problem whose full scope is unknown, that are gathered from different groups of respondents (some within a society, some not) who are asked at different times to make contrasting kinds of judgments, and whose relationships with that being judged (as international experts, small business owners, extortion victims) can differ starkly. There are few clear-cut ways to weigh cases of perceived corruption in terms of their significance. GDP figures reflect corrections for domestic versus cross-border economic activities, but corruption indices generally do not: indeed, corrupt schemes orchestrated from abroad often end up depressing index scores in struggling societies.
The core concerns that corruption perceptions may share – or are assumed to share – point us toward fascinating and important questions. More innovative proxy variables not tied to perceptions are emerging, at times with highly suggestive results; but it is telling that many of them focus upon detailed sorts of data about specific interactions.
Still, the major international country-level indices, judged in terms of understanding and comparing the corruption problems of real people in actual societies (not to mention, generalizing about “cultures of corruption” or the overall quality of institutions and leadership, which are among the uses to which such data sometimes are put) seem to me to obscure what is most interesting and important about corruption for the sake of producing interval-level data.
I would be the last to argue that such indices should be abandoned, but I am not as ready as some analysts seem to be to treat them as literal truth.
About the Article
Photo: By uwdigitalcollections – Early women cheerleaders at UW Madison Uploaded by nusumareta, CC BY 2.0
About the Author
Michael Johnston is Charles A. Dana Professor of Political Science Emeritus at Colgate University. He lives, works, and overindulges in enchiladas in Austin, Texas.