I've got a fever and the only prescription is more Peter Turchin.
A follow up to my review of Turchin's End Times
When I wrote my review of Peter Turchin’s End Times back in April, I said Turchin
seems like a clear-sighted guy who has acquired a fair amount of knowledge about history since getting bored with animal population studies and embarking on an autodidactic adventure in the social sciences. Where I get stuck is trying to figure out his claims for a great leap in scientific method and precision, which are not really supported by the information included in End Times. Turchin’s big database, like a kind of social science MacGuffin, is mysteriously missing from many of the segments in End Times that contain interesting insights.
In my summation, I said I would “consider putting some of his more technical books into the reading queue to gain a better understanding his methods.” This post summarizes those efforts, starting with three of Turchin’s other books. Then I’ll try to use what I found to address some of the questions that End Times did not answer.
War and Peace and War: the Rise and Fall of Empires (Penguin 2006)
This is perhaps the most enjoyable of the Turchin books, if what you enjoy is a thesis-driven romp through several thousand years of mostly Euro-mediterranean history. A basic premise of War and Peace and War is that modern social scientists, especially economists, have erred in focusing on self-interest when trying to explain human behavior. All the interesting things in human history resulted from humans figuring out how to overcome selfishness and cooperate on a large scale. With that in mind, War and Peace and War has three basic ideas relevant to understanding Turchin’s predictions in End Times:
1. Asabiya. Turchin says that populations living in conflict zones between clashing civilizations develop an exceptional ability to cooperate for self-protection. Turchin calls this ability to cooperate asabiya, borrowing the term from Ibn Khaldun, a 14th century Islamic scholar. Asabiya includes both fighting spirit in war and the ability to govern effectively in peace. Asabiya can be roughly quantified, mostly by looking at results. A population with lots of asabiya can conquer an empire and then govern it effectively, at least for a while. A population with middling asabiya can govern and defend a middling country with middling effectiveness. When a population has low asabiya, the elites steal everything and the poor suffer what they must.
Turchin does not express an opinion as to whether high asabiya emerges through a learning process or a darwinian selection process, but it takes a long time and historically it only seemed to happen in extreme conflict zones. Keep in mind that Turchin is making an observation about the rise and fall of empires historically. Turchin does not express an opinion about whether we should want to develop imperial levels of asabiya in modern societies. Perhaps we would be better off living in middling countries with middling levels of asabiya and not attacking each other. But two thousand years ago if you were surrounded by Gauls and Huns and Visigoths, a high level of asabiya was pretty useful.
Interestingly, asabiya and the thousand-year asabiya cycle were the leading themes in War and Peace and War, but they are totally absent from Turchin’s later books about the United States. I don’t know whether this is because the thousand-year cycle is too long to be relevant in discussing U.S. history, or because the subject is too depressing. If humans only learn to cooperate on a large scale is by experiencing genocide, then there is not much we can do about our current situation except continue to bicker until things get genocidal and we can start generating asabiya again. Turchin is clearly trying to persuade Americans to break from the ancient pattern, or at least to use whatever asabiya they still have to avoid disaster.
2. Cliodynamic cycles. Turchin proposes that a substantial part of all history since the adoption of agriculture can be explained by nested cyclical tendencies operating on three different time scales. This theory of nested cycles is the basis for Turchin’s “cliodynamic” approach to history, mentioned but not thoroughly explained in End Times. Turchin does not consider the cycles to be deterministic because outcomes depend on the choices made by the people and the cycles can vary quite a bit in length, but the cyclical tendencies continue to operate in the background and they seem to be pretty powerful.
The longest cycle is the amount of time it takes to develop enough asabiya to launch an empire, and then the time it takes to dissipate the asabiya once the empire starts to become decadent. This takes at least several hundred years, and often more like a thousand years. War and Peace and War is mainly about Eurasia, but Turchin notes that the United States achieved imperial (or near imperial) levels of asabiya because of the lengthy and brutal wars between settlers and native Americans. Turchin doesn’t say it in so many words, but because the settler wars are over, the United States no longer qualifies as a place to build asabiya by traditional means; it would follow logically that the United States is either at a plateau or declining in its traditional asabiya and ability to project imperial power.
The middle cycle is called a secular cycle, consisting of an integrative phase when a society is building itself up and a disintegrative phase when the society experiences internal conflict. This cycle is what End Times is mostly about. The secular cycle is based on structural dynamic factors including the tendency to overpopulation when things are going well, and the tendency of elites to take advantage when they can. Whether a society is integrating or disintegrating, it will continue to do so until the structural-dynamic fundamentals change direction. Secular cycles can range from about 100 to 300 years, and it usually takes multiple secular cycles to complete each leg of an asabiya cycle.
The shortest cycle is what Turchin calls the “fathers and sons” cycle, consisting of a generation that engages in a lot of conflict, followed by a generation that avoids conflict. This cycle lasts about 40-60 years, and is more prominent in disintegrative phases because that’s when conflict is more of an issue. Violence is likely at the high point of conflict, and this may present an opportunity for a directional change in the secular cycle, but it often takes more than one cycle of violence to achieve a change of direction and bring an end to a disintegrative phase.
3. Identification of variables. Turchin’s approach is supposed to be based on data and modeling, so we need to identify variables and collect data, right? War and Peace and War talks about this, but the book is mainly about fitting the big-picture theories to historical examples. Sooo many examples, but not a lot of detail about how the variables are defined or how they interact.
The variables driving the asabiya cycle seem to be relatively simple. Historically, the main way to create an imperial level of asabiya was for a cultural group to survive through long periods of horrible genocidal warfare on a frontier between clashing civilizations. The main way of dissipating asabiya was by developing extreme levels of inequality once you had an empire with excess wealth. Turchin is clear on that last point – in the long run, extreme inequality destroys the ability of a society to cooperate effectively.
Other variables affecting the development of imperial states include the distribution of wealth and power, ethnicities encompassed in the group, availability of discipline from moralists, and the development of symbolic signaling. These variables seem to be of lesser interest to Turchin because they do not have the large-scale, consistent impact of asabiya, so he has not figured out how to model them. This is a theme in Turchin’s work – he does not pretend to be able to model everything. If you ask “what about X?” the answer is often “we treat X as an exogenous variable and attribute random impacts when we run multiple iterations of the model.”
Turchin describes a secular cycle thusly:
Stability and internal peace bring prosperity; prosperity causes population increase. Demographic growth leads to overpopulation; overpopulation causes lower wages, higher land rents, and falling per-capita incomes for the commoners. At first, low wages and high rents bring unparalleled wealth to the upper classes, but as their numbers and appetites grow, they too begin to suffer from falling incomes. Declining standards of life breed discontent and strife. . . .
The collapse of order brings in its wake the four horsemen of the apocalypse—famine, war, pestilence, and death. Population declines, and wages increase, while rents decrease. As incomes of the commoners recover, the fortunes of the upper classes hit bottom.
War and Peace and War at 257. A disintegrative phase generally only ends when the excess elites have been eliminated by factional fighting, foreign wars, or plain old downward mobility. At that point the share of wealth that the elites siphon from the populace returns to a more sustainable level. The key variables for diagnosing a disintegrative period are popular immiseration, too many elites competing for the available spoils, and increasing governmental dysfunction. Of these, the excess elites seem to be the most predictive factor, probably because elites have most of the power in a society by definition.
The fathers and sons cycle does not really have any interesting variables. Violence ebbs when people get tired of it, and then makes a comeback when people have forgotten how unpleasant it is. This takes about 50 years on average.
In the concluding chapters of War and Peace and War, Turchin admits many critics do not believe ancient history is useful for analyzing the world today. Turchin expresses hope that under modern conditions, people will be able to improve their large-scale cooperation skills without going through the thousand-year asabiya cycle and without being caught up in genocidal violence.
Ages of Discord: A Structural-Demographic Analysis of American History (Beresta Books 2016)
Ages of Discord picks up where War and Peace and War left off, testing the frameworks based on ancient history to see if they hold up when applied to the United States. But Ages of Discord is much more technical than either War and Peace and War or End Times. It explains how Turchin’s models are constructed, which variables he treats as exogenous, and how he decided to focus on certain data sources. Ages of Discord has as many equations and graphs as War and Peace and War has obscure historical anecdotes. If you’re the analytical type, I suggest buying Ages of Discord and keeping it handy while you’re reading End Times. Ages of Discord will answer many of your questions about how Turchin reached his conclusions in End Times, and it is organized in a way that makes it relatively easy to find an explanation on a specific subject when you want it. It is recent enough that Turchin’s methods and conclusions have not changed all that much.
The gist of Ages of Discord is that the United States is in the disintegrative phase of its second secular cycle. First there was an integrative phase from the 1780s to about 1830 when things seemed to be going well. Then the trend changed. The poor started getting poorer, the rich started getting richer, and political consensus disintegrated. Extreme levels of instability led to the Civil War, but that didn’t actually change the factors that control the secular cycle. After the Civil War things quieted down a little, but throughout Reconstruction and the Gilded Age in the late 19th century, the rich kept getting richer and politics kept being turbulent. Finally, beginning in the Progressive Era and culminating with the New Deal in the 1930s, a political consensus developed that led to reforms. This changed the trend and ended the first secular cycle.
The second integrative phase in the United States lasted from about 1930 to 1980. Wealth and income inequality went down and politics was a lot calmer. Turchin recognizes that this benefited white people the most, but the overall trend was toward less political violence and instability. The trend changed again around 1980, when elites got tired of paying high taxes and started agitating for business-friendly policies. Since then we have been in our second disintegrative phase, and measures of political instability are once again reaching extreme levels.
One point in Ages of Discord that helps to explain the reasoning of End Times is that Turchin recognizes four types of ruling elites – military, ideological/religious, bureaucratic (based on centralized state power), and plutocratic (based on wealth). Different countries are dominated by different types of elites. Iran is an example of a country primarily ruled by ideological/religious elites; Egypt is a country that has consistently been ruled by the military; Russia is an example of a country where the elites are heavily bureaucratic. The United States is the preeminent example of a country that is mainly ruled by wealth, so wealth and income are the main ways elite status is measured.
The starting point for the analysis in Ages of Discord is the database that was behind the pre-modern theories described in War and Peace and War, but many of the categories that were significant in pre-modern societies do not apply to the United States. That means Turchin had to go looking for other things to measure that would fit within the structural dynamic theory of secular cycles. Some of these are more persuasive than others. One of the primary measures of intra-elite competition discussed in Ages of Discord is the number and cost of law degrees, medical degrees, and recently MBAs. These seem like decent measures of how badly people want to break into the elite ranks, but I was somewhat surprised to see these standing nearly alone as proxies for the most important variable driving the secular cycle.
As another example, an increase in the measure of political violence during the current disintegrative phase is mostly based on classifying mass shootings as a type of political violence, even though the connection of these shootings to political motives is far from clear. Mass shootings have gone up substantially in the past 50 years, but without the mass shootings political violence would have stayed roughly the same or even gone down from the levels measured during the integrative period of 1930-1980. Including mass shootings as episodes of political violence is consistent with Turchin’s overall thesis that a disintegrative phase of the secular cycle is characterized by generalized instability and not necessarily by purposeful political activity, but at the same time Turchin says that the general homicide rate does not correlate very well with other measures of instability. Classifying mass shootings as political violence rather than ordinary homicide is a judgment call, and illustrates that the devil is in the details of any attempt to quantify social science.
One example of a model that was of particular interest to me: the fifty-year fathers and sons cycle is based on the concept of successive generations having different predilections for violence. A common critique of theories based on generations is that people cannot be arbitrarily divided into discrete generational cohorts because people are born continuously. Turchin builds a social contagion model to show that you can have a cultural phenomenon oscillating at a generational rate even though individuals are entering and exiting continuously and cannot be divided into discrete cohorts. The point of the model is not to describe reality accurately – the model is far too simple for that. The point is to show that cultural waves on a generational scale are plausible, so we should not necessarily reject generational explanations for data. Turchin notes that the fathers and sons cycle does not seem to apply in the same way to data from China, but since Ages of Discord is about the United States Turchin does not follow up on that observation.
Assuming a fifty-year cycle exists, we are approaching the culmination of it. But as with the Civil War, an acute crisis does not guarantee that the trend will change and we will enter another integrative phase of peace and prosperity. Unless we change the fundamentals of labor demand and overproduction of elites, we’ll be facing another 50 years of disintegration when the acute crisis is over.
Despite some quibbles, the overall patterns in Turchin’s observations seem plausible, if not robust. There certainly appear to be recurring multi-decade periods in history when the incomes of wage-earners stagnate or go down while the incomes of elites go up, as described by the secular cycle theory. In the United States, these periods for wage earners correlate with large waves of immigration, so the fact that many wage-earners today focus on immigration as a problem is not entirely without basis.
Figuring Out the Past: the 3,495 Vital Statistics that Explain World History (2020)
This book is only mentioned because I bought it for this review, but it is not particularly useful for any purpose I can discern. It does not explain world history. It has lists of data about arbitrarily chosen state level societies in a format that was apparently modeled after the Pocket World in Figures published by The Economist. It gives you a sketchy idea of the types of data that are included in Turchin’s SESHAT database, but detailed information about the SESHAT database is available online for free.
This brings us back to the premise of this post: I read End Times before I read Turchin’s other books, and I found its explanations of Turchin’s prior work to be a little skimpy, leaving me with more questions than answers. I’ll conclude with a few questions that are likely to arise from reading End Times by itself, and try to answer those questions so you can enjoy End Times more if you decide to read it.
Are Turchin’s theories well supported?
As noted in the introduction, when I first read End Times I subjectively agreed with a lot of what Turchin was saying, but I wasn’t too convinced that he had made big advances in data science to support his theories. For example, the concept of “elite overproduction” is based on an analogy between two wildly different phenomena: elite birth rates in medieval times and college education in the late twentieth century. I feel confident that a computer algorithm did not come up with this analogy. Turchin or one of his collaborators came up with the analogy and gave it the name “elite overproduction.” Giving an analogy a name turns it into a category, and that makes it part of our reality (and part of the database). Putting all the different versions of elite overproduction into a single category and coming up with a catchy name for it is inherently more of a qualitative achievement than a quantitative process.
In the End Times appendices describing (very vaguely) how his model works, Turchin admits that knowledge of the past “needs to be translated into a form amenable to analysis with the tools of cliodynamics.” End Times at 274. A cynic might suspect that translating data into “a form amenable to analysis” means subjectively pounding square pegs into round holes until they give you the answer your thesis demands, which is not much different from what historians have always done. But Turchin gives a somewhat persuasive account of how the work is actually done by a hierarchy of research assistants and experts. After reading Ages of Discord I am more convinced that Turchin’s iterative approach to fitting proxies with variables is probably necessary to make progress. Nevertheless, the subjective aspects must be kept in mind when Turchin claims that his approach is wholly scientific, and that all prior theories and approaches are therefore inherently inferior.
In the end, it is likely that massaging the data is what yields the insights. The insights manifest themselves as categories amenable to analysis. The categories may also be amenable to incorporation into ideologies and political debates. If Turchin’s work gives rise to an ideology that make more sense than what the red team and the blue team are currently saying, then maybe people will figure out ways to get the country back on an integrative, upward trend.
Is Turchin’s theory too deterministic?
Any historical theory that contains a cyclical element is always accused of being too deterministic and giving too little credit to human agency. This review of End Times on the website of the American Institute of Economic Research says:
Although Turchin does not appear to be a fan of Marx, his approach shares the Marxian problem of envisioning the sweep of history in a very mechanistic way. His human agents are materialistic and, like the rabbits and wolves, act in (obviously shortsighted) ways to maximize their take in the great tug-of-war for social resources between the classes. Human agency has little place in his theory. As he says, “The great-man theory is the most ‘anti-cliodynamic’ theory of history I can think of,” (where “the great-man theory” is one that makes the agency of individuals pivotal to the flow of history).
This critique is a little strange, because I don’t think that failing to attribute enough agency to “great men” has ever been a problem in any civilization we know about. The hot topic among social scientists for the last fifty years or so has been to remember that the little people have agency too. Figuring out how the collective agency of the little people works on a large scale is the problem Turchin is trying to solve.
Turchin addresses this directly in War and Peace and War. Turchin cites Tolstoy’s call to develop a calculus to “integrate” and solve for how the wills and actions of everyone create history. The chaotic nature and sensitive dependencies in complex systems mean that small inputs by individuals can have big effects, but it is hard to predict exactly when or how. The overall trend of Turchin’s books is to note troubling historical parallels, but to emphasize that we could choose differently today, especially in light of our advantages in productivity, technology, and communications.
Why not just use economic frameworks?
The same AIER review also says:
What exactly do elites do, for example? Are they simply leeches on society, or do they serve some productive function? [followed by multiple additional rhetorical questions]
As economists, we think Turchin fails to account for the moderating role of prices. In a market society, a surplus of elites should change the relative wages of elite versus non-elite jobs. Indeed, the federal government’s persistent subsidization of college education has created a surplus of low-skill college graduates and a shortage of technical workers. The typical salary for a grievance studies graduate is a pittance of what an arc welder makes. Not surprisingly, many people are rationally forgoing college and opting, instead, for trade careers.
Turchin explains in War and Peace and War that neoclassical economics and rational choice theory can explain some things such as prices, but they simply do not explain where values come from and why they differ between individuals and social groups. In one time and place, hordes of people will volunteer to fight on the front lines of a bloody war, while in a different situation they will all run away. That is the type of problem Turchin is trying to get at, and it is not explained by prices.
More troubling is that these economists seem to have completely missed the point of End Times, which is that the most important variable leading to disintegration and crisis is overproduction of elites. Turchin very clearly understands and illustrates that when too many people go to college, the wages of college graduates are likely to go down. But the problem is not solved when wages for college graduates reach a market-clearing level. The college graduates are unhappy because they took out huge loans and did not get the elite jobs they expected, and they don’t necessarily get a do over. They become frustrated elite aspirants. When the number of frustrated elite aspirants gets too high, history shows you got trouble, you got terrible terrible trouble, with a capital T and that rhymes with E and that stands for Elite Overproduction.
What about the war in Ukraine?
Turchin says in End Times that wars are particularly suited to quantitative modeling and prediction. I was therefore very disappointed when Turchin made no attempt to model or predict the outcome of the Ukraine war, which started a year before End Times came out. Seems like low hanging fruit, a good test of the methodology.
Turchin explained in a series of posts on his blog beginning in July of 2023 that his final manuscript deadline was August 2022, so he did not have the time or the data to work up a model for the Ukraine war. Turchin said that by 2023 Ukraine had become a war of attrition, and he compared two competing theories of who would win. Paul Krugman stood with the U.S. government, noting that the combined economic power of the Ukraine, the United States, and Europe, measured by GDP, was more than 20 times the economic power of Russia, and therefore Ukraine had the advantage. A bunch of bloggers on the internet took a contrary view based on availability of replacement soldiers and armaments. Turchin built a model and projected that the motley crew of bloggers was correct – Russia will win the Ukraine war. The basic problem with the economic model was that people, armaments, and dollars are not actually fungible in the real world. As the great military expert Gomer Pyle used to say, “surprise, surprise, surprise.”
This sequence of events raises the question of whether Turchin’s models can really predict much of anything until the answer has become fairly obvious by ordinary observation. Turchin’s discussion of the history of military prediction in End Times and in his recent blog posts convinced me that anyone who had decent data, such as the United States Government intelligence agencies tasked with analyzing these things, should have been able to predict that Ukraine had a low probability of success before the war actually began, and should have known that Ukraine was definitely going to lose by late 2022. Anything to the contrary stated by any high U.S. Government official is an intentional falsehood.
What next?
You may have guessed that if Turchin made a specific prediction such as a civil war in the United States, I would have mentioned it by now. Turchin is pretty clear in End Times that the path of disintegration is predictable, but the end result is not. A group of researchers recently applied Turchin’s methods to evaluate whether the outcomes of crises could be predicted. They concluded:
Our analyses show that the consequences experienced by each crisis is highly variable. The outcomes themselves are uncorrelated with one another and, overall, the set of consequences is largely unpredictable when compared to other large-scale properties of society . . . . We conclude that there is no ‘typical’ societal crisis of the past, but crisis situations can take a variety of different directions.
The main thing you can predict for the United States right now is that the disintegrative trend will keep going until something happens to reverse it. That could be a civil war, a foreign war, or a plague that wipes out enough people to reset labor/elite dynamics. It could also be a political compromise that slows down the wealth pump enough to calm the masses. Given that no political party is arguing for such a compromise, I would say that continuing warfare and plagues are far more likely.
A much better writeup than your original review of just "End Times".
One note: "Turchin’s discussion of the history of military prediction in End Times and in his recent blog posts convinced me that anyone who had decent data, such as the United States Government intelligence agencies tasked with analyzing these things, should have been able to predict that Ukraine had a low probability of success before the war actually began, and should have known that Ukraine was definitely going to lose by late 2022. Anything to the contrary stated by any high U.S. Government official is an intentional falsehood. "
This is much too generic and broad a statement. Among other things: an assumption is made that the US Government, its intel agencies, the analysts within these agencies, the appointed executives overseeing them, etc etc are actually interested in truth or good analysis. GIGO works for a computer but does not work for people much less organizations.
In fact, I would posit the opposite: it has been stated by credible people such as Ray McGovern, that the CIA under Brennan morphed from a results based focus to a politics based focus - specifically that Brennan was far more interested in pandering to American politicians' interests than presenting truth and good analysis.
To me, this is simply another form of consequence from elite overproduction: when any one or group of elites is eminently replaceable due to massive oversupply, the benefits of demonstrating integrity are dramatically eroded.