Chapter

The Future of Social Science & National Security Project

Big Data Seeks Context, for Long-Term Relationship: Reflections on the Empirical Studies of Conflict Project (ESOC)

Flexible funding, practitioners trained in social science, and a bit of luck fostered a scholarly community that applies data-driven research to a broad range of national security challenges.

By Eli Berman • Jacob N. Shapiro

Grand Strategy

August 17, 2023

For the last 15 years the Empirical Studies of Conflict Project (ESOC) has supported a global community of scholars who bring tools from cutting-edge social science to bear on an expanding range of national security questions. During its early growth three factors were critical: flexible funding helped ESOC seed novel data collection and build partnerships; practitioners trained in social science and committed to evidence-based policy provided necessary context for analyzing complex, high-fidelity data on conflict events, development programs, and public opinion; and the fortuitous applicability of social scientific tools to operational challenges created rich opportunities for policy engagement. Developing funding, pipelines, and institutional structures that facilitate similar conditions in the future will help ensure that data-driven social science can inform and improve national security policy at all levels.

For the last 15 years we have been privileged as Directors of the Empirical Studies of Conflict Project (ESOC) to support a global community of scholars who bring tools from cutting-edge social science to bear on critical national security questions. ESOC succeeded because of some luck, some well-timed funding, and a group of determined practitioners, scholars, and support staff who believed that social science had a key role to play in the future national security environment. We hope this story provides some useful insights as the United States reconceptualizes its scientific investments in national security.

The Infeasible Project

In the fall of 2004, Jake Shapiro, one of the authors of this chapter, was a hard-working but frustrated PhD student in Political Science at Stanford University. Over at the Econ Department he could learn the flashy tools of causal inference, which were gaining influence in economics at a rate proportional to Moore’s law as the tech giants down the 101 in Silicon Valley built bigger and better data science tools to play with. In his own department, however, conflict studies was full of arguments based on untested, and even untestable, theories.

Shapiro had chosen to study conflict for good reason: policy questions were screaming for answers, terrorism was running rampant, and a nasty insurgency had broken out in Iraq. From his own experience as a Naval officer and reservist, he knew that policy discussions among practitioners often lacked evidence-based discipline. The logic of anecdotes from cable news jumped in where wisdom could not yet tread: the talking heads said that suicide attacks were the work of terrorists motivated by rewards of virgins in heaven, and terrorism and civil war sprang from poverty and grievance.¹ Those assertions, which could in principle be tested with existing data, went largely unchallenged.

To be clear, compelling scholarship using conventional research methods did exist: examples included Martha Crenshaw’s work on terrorism;² Nathan Leites and Charles Wolf’s analytical framework proposed in 1970 (and other Vietnam War historians and practitioners since);³ Todd Sandler’s empirical work on terrorism;⁴ and Richard English’s magisterial history of the IRA.⁵

Yet little of that research was fully exploiting available data.⁶ A set of methods that had emerged (more or less) from the University of Chicago in the 1970s conquered applied economics in the 1980s and was making inroads into political science by the early 1990s: simple rational-choice based theories, rigorously grounded in individual optimization, expressed through game-theoretic models, combined with econometrics focused on causal inference, with minimal respect for traditional sub-field boundaries. With Gary Becker and his students as standard bearers, education, crime, demography, management, health, and other subjects had been colonized as fields in economics, yielding policy-relevant inferences which turned out to be quite influential. In parallel, some traditional fields of economics were embracing even more exacting standards of empirical proof: development economics and research on domestic policy were establishing the randomized controlled trial as the benchmark for inferring causal relationships.

By 2004 some movement in this direction was starting in political science, primarily employing careful structured comparisons in research on civil war. Roger Peterson at MIT sought to understand the rational calculations driving participation in rebel organizations⁷ and Dan Posner at UCLA investigated why certain ethnic groups fought under some circumstance but not others.⁸ Jim Fearon and David Laitin at Stanford were using cross-national data to understand civil war dynamics in ways that were more robust than prior work had been.⁹ Stathis Kalyvas at Yale and Jeremy Weinstein at Stanford were combining micro-level data with fieldwork to revisit the received wisdom on why civilians are targeted in civil war (Kalyvas 2006).¹⁰ And a younger generation of scholars was beginning to design surveys to understand why people fight both for and against the government in civil wars.¹¹ This last community was growing fastest in Europe where Tilman Brück, Patricia Justino, and Philip Verwimp established the Households in Conflict Network to address a lack of quantitative research on the micro-level effects of violent conflict.

Yet these new empirical methods were not achieving the success in conflict studies that they had in economics. The problem could not just be a lack of data or access. Conflict generates grim harvests of administrative incident data on casualties, perpetrators, munitions, and methods; and anthropologists, ethnographers, and journalists conducted research in conflict zones all the time. So why wasn’t all this potential getting expressed as inference on pressing national security topics?

In the meantime, 9/11 had created a sense of urgency among economists, just as it had among other social scientists, and Eli Berman, a veteran of the Israel Defense Forces, was frustrated by how poorly theorized the policy response to terrorism seemed, compared with what he had seen in Israel. He also wondered if the rapidly emerging scholarship on the economics of religious sects—based on the seminal work of Laurence Iannaccone on economic clubs¹² —could yield insights into the organizational vulnerabilities of terrorist organizations, which were mostly built on Islamist clubs. He would develop that point empirically in research with David Laitin,¹³ but, like Shapiro, Berman and Laitin lacked a path to bring their insights to policymakers, or to further develop them with other data collected by the national security community.

Why Infeasible?

From the perspective of a scholar of non-state conflict in 2004, that path would appear to be obscured by a lot of obstacles. Consider a few we both heard at the time.¹⁴

Data Exist But They Are Secret

Practitioners don’t see the value in data sharing, and nobody forces them to do so. Defense programs are under constant pressure to perform, but they are spared the external scrutiny faced by domestic programs. And domestic economic policy had a teaching moment at the tail end of the Johnson administration’s War on Poverty; expensive welfare programs were claimed as successes using observational methods, but poverty remained. As a result, since the 1960s some social welfare programs have required evaluation using randomization, like drug trials do.¹⁵ International development agencies had begun to accept the same level of scrutiny in their programs; submitting to randomized controlled trials involving external academics.¹⁶ Yet the Pentagon does not, perhaps because experiments on coercive and violent methods are immoral, and violence indeed makes up much of what the Pentagon trains to do. Moreover, practitioners in national security often require very quick answers, whereas empirical research cooks slowly.

No One Will Talk to You

Even if you had administrative data or intelligence collection from the military, understanding the context of the information and how it was collected is critical, and that, in turn, requires being embedded with practitioners, or at least in constant conversation with them. Embedding raises ethical conflicts that your research colleagues will cringe at—for instance, you might influence coercive action taken on human subjects. Doing work that is too close to the military will be anathema at many universities who severed ties to the defense community during the Vietnam era.¹⁷

Additionally, researchers on base require protection, food, and space to work, taking up scarce resources, not to mention patience.

Even if a researcher was appropriately embedded, deployment cycles leave incoming units confused about context themselves. So even if practitioners wanted to share why certain things were done in some places but not others, or how a variable gets coded—both prerequisites for good causal inference—they simply won’t know much of the time.

If practitioners wanted to host researchers, they would start with the Federally Funded Research and Development Centers (FFRDC), government-scientist partnerships that date back to the Manhattan Project. Their researchers have security clearances, the Department of Defense has earmarked budgets to spend on their work, and their tight relationship with the military makes them less likely to draw embarrassing conclusions. And so, we would often hear, anyone who you do get interested in serious empirical work will just end up going to RAND, CNA, or IDA.

Generating New Data is Dangerous

Civilian attitudes toward combatants are key to understanding civil wars (according to some theories), but how could civilians share those with researchers without endangering human subjects (i.e., themselves)? Obtaining those data in a way that allowed researchers to understand the nuances would require physical presence in a conflict zone. Which is not going to happen. Researchers lack security clearances and training. Burnout is bad enough among development researchers in non-conflict settings—now you think we should go to even harder places?

Even if you could travel to those places, where would you even recruit researchers with a baseline understanding of conflict? Many social science PhDs and their students have worked in areas like health, development, and elections—or they have neighbors or relatives who do. By contrast, very few have served in the military or know veterans, generating what we came to call the “tiny demographic overlap” problem. At the same time, the PhD social scientists employed in the professional military education (PME) system are not incentivized to execute the style of empirical social scientific research that top journals require.

The Unlikely Achievement

And yet…starting around 2005, a few of us were able to chisel out a space to address these constraints and support rigorous study of conflict. We were motivated by a core idea: there is a scientific agenda spanning security and development economics that coincides with policy challenges facing the international community. If we could traverse the gap between those two worlds, and maybe bring them a bit closer, we could make better progress in both. Once we started to develop trust and build relationships across the gap, the bridge-building became a self-reinforcing process, and our Empirical Studies of Conflict Project (ESOC) was the result.

By the summer of 2010, the ESOC had held two academic conferences, was working with the U.S. government and aid contractors to empirically evaluate the impact of interventions in Afghanistan, Colombia, and Pakistan, and ESOC member Rahda Iyengar—then Assistant Professor of Economics at London School of Economics (now Deputy Undersecretary of Defense for Acquisitions and Logistics)—was in Kabul briefing the International Security Assistance Force (ISAF) leadership on the results of a data-intensive study on the effect of civilian casualties on attacks sustained by coalition forces.¹⁸

As of this writing, ESOC exists and thrives, with an annual conference, tenured alumni at two dozen universities worldwide, more than 100 supported publications in peer-reviewed journals, over 15,000 citations, partnerships with the Carnegie Endowment for International Peace, International Crisis Group, and the Modern War Institute at the U.S. Military Academy, and the affiliated Irregular Warfare Initiative whose podcast has thousands of followers. The group has helped implement a huge range of research projects, from studies on mining-related conflict in Africa and India, to how agents of online disinformation react to deplatforming.

The model of leveraging ongoing contact with practitioners to inform careful quantitative empirics is now being used by others investigating conflict in political science and economics, from research on the impact of interstate wars, to the full range of non-state and sub-state uses of violence and coercion.

Wait, How Did That Happen?

In retrospect, those obstacles were not exaggerated. ESOC’s unlikely productivity required negotiating them, with the help of a confluence of changes in the research and policy environment, and some luck.¹⁹

The Teaching Moment

In the years following 9/11, academic researchers found themselves in a “teaching moment” with the national security communities (in multiple countries). Subnational conflicts involved an intersection of three huge, unsolved strategic problems: the Iraq War had become a raging insurgency; reconstruction in Afghanistan had stalled, and the Global War on Terrorism was not reducing global terrorism.

Of direct significance to ESOC is that US Central Command, with responsibility for both Iraq and Afghanistan campaigns, was so desperate for analytical assistance that they reached out to economists for help. LTC Pat Buckley, an MIT Economics PhD teaching at West Point joined the staff of CENTCOM commander John Abizaid. Buckley had heard about some of our research through academic friends, and called Berman with an invitation to come give a talk on terrorism and discuss how one might design research that could answer questions about counterinsurgency. On that visit we discovered, in a windowless basement full of servers at MacDill Air Force Base in Tampa, the awe-inspiring quantity of data that CENTCOM collects, including detail on incidents that one might find in a police report: geolocation, timestamp, perpetrator, victim, and the like.

The Unicorns

If one looked around closely, there were already individual scholars clearing some of those obstacles. The single best example of these was Joe Felter, who was a year ahead of Shaprio in the Stanford Political Science PhD program and lived with a foot in academia and another in special forces. Before rotating to Stanford, Felter had spent much of the previous decade training local forces in counterinsurgency as a US Army Special Forces A-Team Leader and Company Commander deployed across southeast Asia and most recently as military attaché in the Philippines. The Armed Forces of the Philippines that Felter worked closely with during his three years in this restless archipelago was actively engaged fighting a long-running Muslim separatists insurgency in the southern island of Mindanao, neo-Marxist New People’s Army rebels operating in rural areas across the country, and also Islamist terrorists concentrated in the far south provinces of Sulu and in southwestern Mindanao.

When then-Major Felter arrived at Stanford to complete a PhD, he brought with him three critical research assets: hypotheses based on his field experience, a rich network of contacts to provide context, and access to a trove of data on operational incidents in the Philippine insurgency. The result was an empirical thesis rigorously testing those hypotheses using incident data: “Taking Guns to a Knife Fight: A Case for Empirical Study of Counterinsurgency.”²⁰ The subtitle speaks volumes about the ambitious research agenda Shapiro and Felter were plotting out, and also hints at the intellectual support they were getting for that agenda, from Felter’s advisors David Laitin and James Fearon. Ironically, one of Felter’s major findings echoes our theme about method-context complementarity: the context provided by troops with experience in a locality is a crucial complement to the tactical skills embodied in special forces.

In retrospect, Felter’s thesis demonstrated a clever solution to the obstacles of acquiring data in context. Radha Iyengar needed to be on the ground in Afghanistan to brief ISAF leadership on the ramifications of civilian casualties, yet being embedded with troops is not always necessary. Formerly embedded may be enough, if you maintain—as Felter did—a strong reputation with your contacts.

Felter’s next military posting was at West Point, where he became a military-academic unicorn pointed in the other direction, directing the recently established Combatting Terrorism Center at West Point, a research/policy/teaching unit. Among his lasting contributions there is the Harmony Program, which directly addresses data access obstacles by declassifying documents from Al Qaeda and the Islamic State, archiving them, and making them available to researchers.²¹

For Shapiro, Felter’s thesis was an inspiring example, but Harmony was a live opportunity: he jumped on it, using Harmony documents on the inner workings of Al Qaeda to empirically test some hypotheses he had developed from agency theory about the organizational vulnerabilities of terrorists. The fact that terrorist organizations suffered from the same kinds of internal disputes and HR problems as other groups—well documented in Al Qaeda’s internal correspondence—helped demystify the group. Almost a decade later US forces in Iraq and Syria would play on such internal disputes in their fight against the Islamic State.

In the meantime, just as Felter was arguing for the value of research to policy, West Point had become a friendlier environment for his research agenda and talents, as demand from the Iraq war meant that faculty with Social Science PhDs were in senior positions at Central Command and in Iraq. The Head of the Department of Social Sciences, Michael Meese, a political scientist and Army colonel, deployed four times to Iraq (and later to Afghanistan). Jeff Peterson, an economist, had commanded a battalion in Baghdad during “the surge”. Pat Buckley we mentioned already. Meese not only supported the Combatting Terrorism Center, but would also lend his experience to ESOC. Faculty at other PME institutions also began providing guidance, such as Kalev Sepp at the Naval Postgraduate School, a retired Special Forces officer (who served on the ESOC founding advisory board). It’s important to stress the importance of collaboration between professional military and civilian academic institutions for enabling basic research where context is critical to inference. The PMEs often include the best minds with the most military knowledge—as well as a stake in the game—which can well complement civilian universities with their multidisciplinary knowledge, cutting edge methods, and large cohorts of exquisitely trained scholars.

By then a plan had almost gelled for an external evaluation of development assistance projects in Iraq and Afghanistan in the style of current best practice in development economics. It was catalyzed by three events.

First, in San Diego, Berman convened a small interdisciplinary conference of analytically oriented researchers on terrorism and insurgency, to check if there was enough potential to support a research program.²² That conference not only demonstrated a critical mass of scholarship which survived preliminary peer review, but introduced him to Felter and Shapiro. Berman was fortunate enough to be affiliated with the Institute on Global Conflict and Cooperation (IGCC), which has a proud history of innovative research on security, dating back to the cold war. Policy schools such as the School of Global Policy and Strategy at UC San Diego—where IGCC is housed—are often well placed to encourage connections between research and active policymaking in challenging contexts: many of their students are experienced practitioners—often from military, government, and international NGOs—which both motivates faculty to focus on policy connections and generates research opportunities.²³

The second and third events are both due to Felter’s managerial talents. He got Iraqi incident data (SIGACTS) released for use by researchers during a tour of duty in Iraq.²⁴ Additionally, he successfully argued that vetting and coding those data into a releasable form was an appropriate learning activity for West Point cadets to do at the Combatting Terrorism Center, effectively conjuring up the resources to get that task done.

Even with those pieces in place, understanding how development assistance fit in a stabilization policy proved to be a context-intensive analytical challenge because the doctrine as written was not precise enough for testing as theory. The important implication for our research program was that we absolutely needed ongoing contact with practitioners, including a specially designed survey of personnel implementing the Commander’s Emergency Response Program (CERP) to develop a testable set of hypotheses. Not all projects are as desperate for context, but that one was.

Capacity Building

By mid-2008 we had context through connections in both national security and development sectors, and thanks to Felter’s work and that of others (discussed below), there were data available. But a project this big needs funding.

The Minerva Research Initiative (MRI) proved to be an ideal sponsor, along with parallel efforts on Human Social Culture Behavior modeling at the Office of Naval Research (ONR). MRI Director Erin Fitzgerald and ONR program manager Ivy Estabrooke both encouraged large-scale grants that could build capacity, with adequate staffing, the ability to flexibly chase transient research opportunities, postdoctoral fellows, and actively recruit enthusiastic PhD students, following a physical science model. They allowed us to launch with a teaching conference in summer 2009 to sketch existing theories, and it was attended by young scholars—many of whom would develop and staff their own projects over the next decade.

The single most important resource that the post 9/11 environment provided to research was young scholars and practitioners seeking solutions. In PhD programs, military, civilian government agencies, NGOs, and international organizations cohorts of young, talented scholars and practitioners found common purpose, particularly in the idea that military intervention and development assistance could provide security while improving wellbeing in fragile states. When combined with the inflow of young scholars from countries with more demographic overlap with practitioners, like Israel, and countries with strong universities that suffer political violence, like Colombia, it was possible for ESOC to staff the capacity that MRI funding allowed with talented young scholars.

Capacity allowed us to commit to risky projects which failed before they succeeded. War often fosters innovation through risky research ventures because circumstance forces leaders and funders to reach quick consensus.²⁵ In peacetime, however, risk requires resources. One of our most important papers on how development assistance works in conflict settings was first presented in seminar as preliminary results on the impact of building streetlights—results which were overturned when more data became available. A less patient funder than MRI might have aborted. Mike Callen and James Long’s excellent paper on election monitoring began with a tragically failed attempt to evaluate conventional monitoring by international observers.²⁶ It would turn out to be one of several papers ESOC supported which helped establish micro-level studies of conflict as an important topic in economics (which change has brought significant new talent to the field). Callen and Long’s ability to stay in the field to work out an ingenious backup plan (with the local partners), on the eve of an election, was enabled by ESOC’s MRI-augmented capacity.

Relationship Building

MRI support made it easier for ESOC faculty and graduate students to develop relationships of trust with experienced practitioners. We began working with Tjip Walker, then at USAID’s Office of Conflict Management and Mitigation, to establish connections with the missions in Afghanistan, Colombia, and Pakistan, as well as helping USAID connect with researchers for an internal “Evidence Summit” on the use of development to counter insurgency.²⁷ MRI-supported researchers began field projects in Afghanistan, Colombia, Pakistan, and the Philippines, all engaging NGOs on the ground. MRI funding also allowed our annual conference to add immediately policy-relevant talks to the traditional academic seminars, creating rich opportunities for early-career scholars to build relationships with practitioners, design research projects, and access data.

Mindful of the obstacles (listed above), in our engagements with practitioners we strove to make researchers immediately helpful with analysis. In Afghanistan, for example, Shapiro presented early analysis of links between cellular coverage and violence that could inform decisions ISAF was making about subsidizing local mobile phone companies, which, through a roundabout chain of connections, led to deeper data sharing in Iraq.²⁸ Being immediately helpful requires a sense of current strategic questions. Which, in turn, requires ongoing presence in the country of focus and continuous dialogue with stakeholders. And that kind of presence and commitment to an issue is enabled by long-term funding. That’s rare in social sciences outside of a few long-term projects on domestic policy, but it happened in Afghanistan and Iraq because of MRI commitment.²⁹

Access to administrative data through relationship-building led to successes in many places. Seed funding from ESOC enabled Beatriz Magaloni at Stanford and coauthors to develop rich new data on criminal violence in Mexico.³⁰ Field experiments seeded with MRI funding have studied diverse topics; from ways to reduce ethnic conflict,³¹ to financial decision-making under risk.³² ESOC’s engagement in Afghanistan eventually led to the declassification of data which supported both our work,³³ and a proliferation of top-tier academic publications by others.³⁴

Having capacity in place also allowed us to progress from answers in one area to the next set of questions. The positive correlation between increased employment and increased violence in Iraq prompted us to go replicate that finding in Afghanistan and Philippines.³⁵ Once we understood the role of development assistance in counterinsurgency, the contrast between local successes and national failure prompted a new project on “Proxy Wars.”³⁶ With further Minerva support we used game theory and a large number of case studies to examine how outside powers could incentivize national leaders, and statistically analyzed the causes of persistent subnational conflict in India, among other topics.³⁷

The Data Zealots

None of this would have been possible, though, were it not for the broader community committed to enumerating what was happening in conflict zones. Hamit Dardagan and John Sloboda’s Iraq Body Count pioneered the systematic use of press reporting to document the civilian costs of civil wars. Their work has been critical to studies of both what happened in Iraq, and of basic issues in the epidemiology of conflict.³⁸ The Gulf Region Division of the Army Corps of Engineers kept remarkable project-level records on all US-government aid spending in Iraq. Kyle Pizzey and his team at ISAF Joint Command Assessment Cell dedicated years of their lives to documenting the war in Afghanistan incident by incident. Additionally, a group of brave political scientists proved that high-quality survey research in conflict zones was feasible—while protecting human subjects—including Jason Lyall in Afghanistan,³⁹ Christine Fair in Pakistan,⁴⁰ and Nicolai Lidow in Somalia, to name a few.⁴¹ Advances in passive data collection in conflict zones, using satellite imaging and cellular networks, also enabled research projects without endangering researchers or subjects.⁴²

So What Happens Next, to Empirical Research on Conflict?

Empirical research on conflict has grown dramatically in quantity and quality over the last decade. Top journals in Economics and Political Science now routinely publish high-quality studies drawing on deep local expertise and impressive on-the-ground data collection. Yet many of the old obstacles remain. The field’s survival has depended so far on the keen interest of young scholars in these topics, on funders to pay for projects (data and context—not cheap dates), and on ad hoc efforts like the ESOC project which connect researchers to data and practitioners. Empirical conflict research lacks some critical components that would guarantee its continuation.

More Teaching Moments?

Michael Desch points out that, historically, collaboration of social scientists and the national security establishment improves during emergencies, and declines otherwise.⁴³ Sadly, that is probably bad news for research on sub-national conflict.

Unfortunately, withdrawal from Afghanistan and the shift to integrated deterrence as a focus of the National Defense Strategy did not eliminate strategic threats from sub-national conflict. Consider the last era of great power competition. From 1975-1991 there were 211 distinct conflicts around the world, with China, Russia, or the United States supporting at least one side in 116 of these (of which 102 were internal conflicts).⁴⁴ There is no strong reason to expect the current era of strategic competition will be any different. Beyond involvement in such conflicts, nobody can promise hermetic protection from terrorism sourced in Afghanistan, Pakistan, Syria, or Yemen, for instance. The same is true for refugee flows, which have arguably triggered waves of xenophobia upending European politics. New security threats loom closer to home (even as they worsen abroad), disinformation and domestic political violence most obviously,⁴⁵ and provide new techniques by which foreign powers can threaten US security.

We might better think of relief from our Afghan commitment to subnational conflict as a pause, and build analytical capacity for the next challenge. After Vietnam, a rising superpower with a declining rival could afford to forget lessons learned and move on. That’s not a luxury the U.S. can count on today in an increasingly multilateral world, with many rival powers –Russia and Iran in particular, specializing in subnational interventions.

A Better Wheel, or a Better Ride?

Rather than reinvent the wheel each time national security threats raise unsolved puzzles in social science, our experience with MRI and ESOC suggests building out a fully sustainable vehicle riding on four separate wheels:

First, support research teams (as MRI has historically done), but make more daring bets, structured so that grantees can invest in capacity-building and take risks through regranting to seize emergent research opportunities.⁴⁶
Second, support database creation and curation, to enable research by others not directly involved. Right now, the costs of making high-resolution data collected for specific studies more broadly accessible (e.g. by documenting and putting into well-structured databases) are not covered by grants. Yet we know from the history of organizations such as AidData and the Armed Conflict Location & Event Data Project (ACLED) that maintaining data for the broad scholarly community can enable large numbers of high-quality policy-relevant studies. Investing in database maintenance would dramatically speed basic research on conflict.⁴⁷
Third, hire more program officers with PhD training in a variety of social sciences disciplines across the defense research enterprise. As social science disciplines become increasingly technical, the number of scholars who can connect social science to traditional technical fields is growing. Hiring more of them to complement the expertise of the current corps of program officers would enable more innovative grants in emerging fields.
Fourth, help research teams build area expertise and contact with practitioners. Supporting practitioner groups, generously funding conferences and travel, and creating incentives for connections between faculty at PMEs and civilian institutions (as the DECUR Partnership does) are good starts. But fostering the deep connections that were vital in our experience requires funding mechanisms which support sustained engagement by research teams with civilian and military operational elements.

That vehicle might easily pick up some willing passengers, through practitioner outreach to the research community. In late-2019, two Army Special Forces officers, Kyle Atwell and Nick Lopez, excited by the material they were studying at Princeton, mused that they would like to find a way to communicate research findings to military personnel and other practitioners in an easily digestible format. Shapiro connected them with Shawna Sinnott, a Marine Corps Intelligence officer studying for her PhD at Stanford, and the three launched the Irregular Warfare Podcast with support from ESOC and the Modern War Institute at the United States Military Academy—again highlighting the catalytic role PME institutions can play. The project’s accessible conversations between practitioners and scholars captured the attention of a generation of servicemembers, aid workers, and diplomatic personnel who did not want to see the hard-earned lessons of the last 20 years fade away. The podcast grew into the Irregular Warfare Initiative (IWI), which today is churning out new knowledge in many forms and serving as a forum connecting researchers with practitioners across many disciplines. At this writing IWI podcasts generate over tens of thousands of downloads per bi-monthly episode, and have been cited by the Joint Chiefs, remarkable levels of attention by practitioners at the highest level. IWI works because it leverages the talent and energy of active duty and civilian volunteers, many in civilian graduate programs who believe policy-relevant social science research conducted at leading universities should inform the operators.

The Ride Ahead

We started this story with Shapiro as a frustrated PhD student looking to bring new data and empirical methods to conflict studies. We end in a much better place, though only in the subfields of conflict research that ESOC has specialized in, and even that may not sustain.

A young PhD student looking for research topics today would find a rich empirical literature on political violence and conflict, hundreds of essays linking that literature to on-the-ground conditions around the world at the IWI website or Political Violence at a Glance, a well-prepared thesis advisor at a dozen top campuses, and a mentor at ESOC as a postdoctoral scholar—even if her PhD was not from those dozen schools.

That’s a vast improvement over 20 years ago, but the system is fragile. Because a mid-career social scientist looking to direct a team at other challenges in security-related research would struggle to find the capacity-building options that MRI allowed us early on. She might opt to switch fields or scale down her ambition, causing a loss in policy-relevant research.

We can ill afford that loss of researcher attention. National security threats will likely tie up a large portion of our national economy, labor force, and attention for the next generation. That increased level of threat is not only due to great power competition – though managing relations with China and Russia will surely require substantial investments. It extends to reversing recent democratic backsliding around the world, addressing new security challenges created by rapidly-advancing AI capabilities, and dealing with the consequences of climate change and the massive migrations it will cause.

These current and future threats will predictably involve just as much social science as understanding counterinsurgency in Iraq and Afghanistan did. Defining and establishing integrated deterrence, deterring cyber threats and disinformation, building alliances on shared values, isolating the causes of the global democratic recession, managing political, economic, and demographic upheavals due to climate change – all contain unsolved problems requiring focused basic research in social science. Current doctrine on these topics could be better informed by research, just as the “civilian surge” could have been in Iraq and Afghanistan. In fact, because new flavors of threat are arriving so fast, relevant social science research needs to accelerate just to keep up.

Unfortunately, the current architecture of social science research is not built to make the most of the US and its allies’ massive qualitative advantage in social science. For the most part the PMEs and FFRDCs closest to US defense policy are not doing basic research in social science (though there are exceptions). DoD funding of social science (which goes to R1 institutions) is high return, but very small compared to other areas, and is not channeling enough PhDs, Mas, or BAs for the national security community. Critically, security is not even a course typically offered in social science fields – outside of Political Science – and so emerging talent in Anthropology, Economics, Psychology, and Sociology gets less exposure to these topics than we think ideal.⁴⁸

A thoughtfully redesigned architecture to produce more dual use social science research and education should have at least four components:

Reformed research funding, to provide flexibility, scale, data curation, and outreach, as outlined above. While granting flexibility and scale to research teams increases risk for program officers, it enables significantly greater returns when projects work well.
Support for data access to enable research with valuable data created for other purposes. ESOC’s early success was critically dependent on a range of detailed data made available by the creativity and devotion of Felter and the data zealots. Future basic and applied research would benefit massively from a dedicated effort to help identify, negotiate access, and assemble data explicitly for research purposes (e.g. project reports from USAID, identified social media data collected for information operations purposes, or historic satellite images). Those necessary steps would benefit operators seeking studies relevant to their immediate challenges as well as scholars working on longer term projects. The necessary manpower cost might be nontrivial, but would be a huge step toward becoming a learning organization which can adapt to meet a rapidly changing collection of threats.
Funding mechanisms for cooperative research to spark collaboration between civilian institutions and operational organizations. Our focused research on how to effectively use development funds in conflict zones was enabled by a combination of a commander’s need (which brought the intuition and the data) and the broad flexibility that a large Minerva grant provided. That was a lucky break – for both researchers and practitioners. In the future, a smart funding architecture could make the necessary data and R1 researchers available to a future commander with the wisdom to recognize that tackling an unsolved puzzle in social science could help inform day-to-day operations.
More officer education at R1 schools to build ties between future senior military leaders and future leaders in the civilian research community. Such ties create flexible networks that can be mobilized to generate a response to changing threats, just as Felter’s connections in military and academic circles facilitated research on CENTCOM’s priority social science challenges in Afghanistan and Iraq. Unfortunately, the demographic overlap of military personnel and PhDs in R1 institutions is currently very small: today’s graduate students and young faculty often have no friends and family in service. Sending more mid-career officers into R1 university graduate programs will create more opportunities for flexible, dual use social science – and more expertise in applying it.

An architecture built out along those lines, we believe, would enable a stable and productive relationship between the growing data collection enterprise of the national security community and the unrivaled research capacity of our R1 universities. It would leverage a strategic asset that our great power competitors cannot match, at least not without embracing an open academic culture, merit-based funding, transparency, and constructive criticism through peer review – all difficult tricks for autocracies. In retrospect, ESOC and CENTCOM were a good match focused on two very specific conflicts. We believe that they reveal the potential for a broader architecture that supports America’s security by generating lasting, long-term relationships between two remarkable communities: R1 social scientists and national security practitioners.

About the Authors

Eli Berman is Research Director for International Security Studies at the UC Institute on Conflict and Cooperation and professor of economics at UC San Diego. He co-directs the Economics of National Security meetings at the National Bureau of Economic Research and is a member of the Empirical Studies of Conflict Project. He is president of the Economics of National Security Association. Publications include Proxy Wars (with David Lake, 2019), Small Wars, Big Data: The Information Revolution in Modern Conflict (with Jacob N. Shapiro and Joseph H. Felter, 2018) and Radical, Religious and Violent: The New Economics of Terrorism (2009). Recent grants supporting his research have come from the Minerva Research Initiative and the National Science Foundation. Berman received his PhD in economics from Harvard University.

Jacob N. Shapiro is Professor of Politics and International Affairs at Princeton and Managing Director of the Empirical Studies of Conflict Project (ESOC). Shapiro has published in wide range of academic journals across fields, as well as more than 100 policy articles, reports, and book chapters. He is author of The Terrorist’s Dilemma: Managing Violent Covert Organizations and co-author of Small Wars, Big Data: The Information Revolution in Modern Conflict (with Eli Berman and Joseph H. Felter, 2018). Shapiro received the 2016 Karl Deutsch Award from the International Studies Association, given to a scholar younger than 40, or within 10 years of earning a Ph.D., who has made the most significant contribution to the study of international relations.

Notes

1
Note: For other common misconceptions, see chapter two of Berman, Eli. Radical, Religious and Violent. Cambridge, MA: MIT Press, 2009.
2
Note: Crenshaw, Martha. “The Causes of Terrorism.” Comparative Politics 13, no. 4 (1981): 379–99. https://doi.org/10.2307/421717; Martha Crenshaw, “The Logic of Terrorism: Terrorist Behavior as a Product of Strategic Choice,” Terrorism in Perspective. Ed. Sue Mahan and Pamala L Griset. India: Sage Publications, 2008: 24-33.
3
Note: Leites, Nathan, and Charles Wolf. Rebellion and Authority: An Analytic Essay On Insurgent Conflicts. Chicago: Markham Pub. Co, 1970.
4
Note: Enders, Walter, and Sandler, Todd. The Political Economy of Terrorism. United States: Cambridge University Press, 2011.; Enders, Walter, and Todd Sandler. “The Effectiveness of Antiterrorism Policies: A Vector-Autoregression-Intervention Analysis.” American Political Science Review 87, no. 4 (1993): 829–44. doi:10.2307/2938817.; Sandler, Todd, and Walter Enders. “An Economic Perspective on Transnational Terrorism.” The Economic Consequences of Terror 20, no. 2 (2004): 301–16. https://doi.org/10.1016/j.ejpoleco.2003.12.007.
5
Note: English, Richard. Armed Struggle: The History of the IRA. London: Macmillan, 2003.
6
Note: For exceptions, see the survey by Blattman, Christopher, and Edward Miguel. “Civil War.” Journal of Economic Literature. 48, no. 1 (2010): 3–57.
7
Note: Petersen, Roger Dale. Resistance and Rebellion: Lessons from Eastern Europe. New York, NY: Cambridge University Press, 2001.
8
Note: Posner, Daniel N. “The Political Salience of Cultural Difference: Why Chewas and Tumbukas Are Allies in Zambia and Adversaries in Malawi.” The American Political Science Review 98, no. 4 (2004): 529–45.
9
Note: Fearon, James D., and David D. Laitin. “Ethnicity, Insurgency, and Civil War.” American Political Science Review 97, no. 1 (2003): 75–90. https://doi.org/10.1017/S0003055403000534.
10
Note: Kalyvas, Stathis N. The Logic of Violence in Civil War. Cambridge, MA: Cambridge University Press, 2006. Jeremy Weinstein. Inside Rebellion: The Politics of Insurgent Violence. Cambridge, MA: Cambridge University Press, 2007.
11
Note: Humphreys, Macartan and Jeremy Weinstein, “Who Fights? The Determinants of Participation in Civil War.” American Journal of Political Science, 52 (2008):436-455. https://doi-org.ezproxy.princeton.edu/10.1111/j.1540-5907.2008.00322.x
12
Note: Iannaccone, Laurence R. “Sacrifice and Stigma: Reducing Free-Riding in Cults, Communes, and Other Collectives.” Journal of Political Economy, 100, no. 2, (1992): 271–91.
13
Note: Eli Berman, David D. Laitin, Religion, terrorism and public goods: Testing the club model, Journal of Public Economics, 92, Issues 10–11, (2008): 1942-1967.
14
Note: Many of these objections are, of course, flat out wrong. Our goal here is to share some of the objections we heard early on in our work together.
15
Note: Moffitt, Robert A. “The Role of Randomized Field Trials in Social Science Research: A Perspective from Evaluations of Reforms of Social Welfare Programs.” American Behavioral Scientist 47, no. 5 (January 2004): 506–40 Widerquist, Karl, “The Negative Income Tax Experiments of the 1970s,” in: Torry M. (eds) The Palgrave International Handbook of Basic Income. Exploring the Basic Income Guarantee. (Palgrave Macmillan, 2019), 202-218 https://doi.org/10.1007/978-3-030-23614-4_15. “National Academies of Sciences, Engineering, and Medicine, Division on Earth and Life Studies, Board on Environmental Studies and Toxicology, & Committee on the Review of Environmental Protection Agency’s Science to Achieve Results Research Grants Program.” A Review of the Environmental Protection Agency’s Science to Achieve Results Research Program. (National Academies Press, 2017). Chetty, Raj, Nathaniel Hendren, and Lawrence F. Katz. “The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment.” American Economic Review, 106 (2016): 855-902.
16
Note: Deaton, Angus. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48, no. 2 (2010): 424–55. http://www.jstor.org/stable/20778731.
17
Note: Desch, Michael. Cult of the Irrelevant: The Waning Influence of Social Science on National Security. Princeton: Princeton University Press, 2019. muse.jhu.edu/book/62654.
18
Note: Shapiro, Jacob N., Felter, Joseph H., Berman, Eli. Small Wars, Big Data: The Information Revolution in Modern Conflict. United States: Princeton University Press, 2018. Chapter 2.
19
Note: Stated differently, Minerva Research Initiative support enabled us to rapidly build on nascent movements in Economics and Political Science.
20
Note: Joseph H. Felter, “Taking guns to a knife fight: a case for empirical study of counterinsurgency,” PhD diss. (Stanford University, 2005).
21
Note: Joseph Felter, et al.. Harmony and Disharmony: Exploiting Al-Qa`ida’s Organizational Vulnerabilities. Combatting Terrorism Center at West Point, 2006. Accessed October 4, 2021. Clint Watts, Jacob N. Shapiro, and Vahid Brown. Al-Qaida’s (Mis)Adventures in the Horn of Africa. Combatting Terrorism Center at West Point, 2007. Brian Fishman, Peter Bergen, Joseph Felter, Vahid Brown, and Jacob N. Shapiro. Bombers, Bank Accounts, and Bleedout: Al-Qa-ida’s Road in and Out of Iraq. Combatting Terrorism Center at West Point, 2008.
22
Note: Then IGCC Director Susan Shirk should be credited with actively encouraging and mentoring that initial conference.
23
Note: According to survey of their deans, Association of Professional Schools of International Affairs (APSIA) schools “simultaneously embrace the disciplinary criteria for excellence and still try to maintain a close policy focus as they seek to bridge the gap between these two worlds.” See Desch, Michael C. , James Goldgeier, Ana Petrova, and Zi En Kimberley Peh “Deans Want It All: Results of a Survey of APSIA Deans and Top-50 Political Science Department Chairs on Hiring and Promotion,” International Studies Perspectives 23, No. 1, ( 2022) 41–70 https://doi.org/10.1093/isp/ekaa022.
24
Note: This is described in Small Wars, Big Data, chapter 2,”ESOC’s Motivation and Approach.”
25
Note: The Covid-19 pandemic did serve up examples of funders departing from their standard “consensus approach” – perhaps the climate crisis will do the same.
26
Note: Callen, Michael, and James D. Long. 2015. “Institutional Corruption and Election Fraud: Evidence from a Field Experiment in Afghanistan.” American Economic Review 105 (2015): 354–381. For a paper on Taliban efforts to suppress democracy that used data available due to ESOC efforts and MRI support, see: Condra, Luke N., James D. Long, Andrew C. Shaver, and Austin L. Wright. “The logic of insurgent electoral violence.” American Economic Review 108, no. 11 (2018): 3199-3231.
27
Note: It no doubt helped that our work on how to use aid in conflict zones coincided with renewed interest at USAID in supporting the war effort in Afghanistan, as part of the Obama Administration “civilian surge” in 2009.
28
Note: See Small Wars, Big Data, chapter 4.
29
Note: The ethics of research on high-stakes policy-relevant questions among vulnerable populations in conflict zones is of constant concern. All university researchers working with human subject go through extensive clearance processes with our institutional review boards (IRBs). But we also weigh the risks posed by sharing our findings against the specter of inaction. In the case of Shapiro’s research on cellular coverage in Afghanistan, the prospect of the Taliban regaining control presented such a hazard for so many, that sharing results which could help ISAF support the government was deemed the moral thing to do. In other situations, the choices are less clear. To help ESOC is currently supporting work on principle-setting for ethics in conflict research.
30
Note: Calderón, Gabriela, Gustavo Robles, Alberto Díaz-Cayeros, and Beatriz Magaloni. “The Beheading of Criminal Organizations and the Dynamics of Violence in Mexico.” Journal of Conflict Resolution 59, no. 8 (December 2015): 1455–85. https://doi.org/10.1177/0022002715587053.
31
Note: Condra, Luke N., and Sera Linardi. “Casual Contact and Ethnic Bias: Experimental Evidence from Afghanistan.” The Journal of Politics 81, no. 3 (July 1, 2019): 1028–42. https://doi.org/10.1086/703380.
32
Note: Blumenstock, Joshua, Michael Callen, and Tarek Ghani. “Why Do Defaults Affect Behavior? Experimental Evidence from Afghanistan.” NBER Working Paper Series, no. w23590. Cambridge, Mass: National Bureau of Economic Research, 2017.
33
Note: Berman, Eli, Michael Callen, Joseph H. Felter, and Jacob N. Shapiro. “Do Working Men Rebel? Insurgency and Unemployment in Afghanistan, Iraq, and the Philippines.” Journal of Conflict Resolution 55, no. 4 (August 2011): 496–528. https://doi.org/10.1177/0022002710393920; Berman, Eli, Joseph H. Felter, Jacob N. Shapiro, and Erin Troland. “Modest, Secure, and Informed: Successful Development in Conflict Zones.” The American Economic Review 103, no. 3 (2013): 512–17. http://www.jstor.org/stable/23469785.
34
Note: Condra, Luke N., James D. Long, Andrew C. Shaver, and Austin L. Wright. “The Logic of Insurgent Electoral Violence.” American Economic Review, 108 (2018): 3199-3231.; Fetzer, Thiemo, Pedro C. L. Souza, Oliver Vanden Eynde, and Austin L. Wright. “Security Transitions.” American Economic Review, 111, 7 (2021): 2275-2308.; Sonin, Konstantin, and Wright, Austin L., “Information Operations Increase Civilian Security Cooperation.” The Economic Journal 132 (2022): 1179-1199.
35
Note: Berman, Eli, Michael Callen, Joseph H. Felter, and Jacob N. Shapiro. “Do Working Men Rebel? Insurgency and Unemployment in Afghanistan, Iraq, and the Philippines.” Journal of Conflict Resolution 55, no. 4 (2011): 496–528. https://doi.org/10.1177/0022002710393920
36
Note: Eli Berman and David A. Lake, eds. Proxy Wars: Suppressing Violence through Local Agents. Ithaca, Cornell University Press, 2019.
37
Note: Shapiro, Jacob, Oliver Vanden Eynde. “Fiscal Incentives for Conflict: Evidence from India’s Red Corridor.” The Review of Economics and Statistics (2021); doi: https://doi.org/10.1162/rest_a_01039. Eynde, Oliver B. “Targets of Violence: Evidence From India’s Naxalite Conflict.” The Economic Journal, 128: 609, (2018) 887–916, https://doi.org/10.1111/ecoj.12438
38
Note: Johnson, Neil F., Michael Spagat, Sean Gourley, Jukka-Pekka Onnela, and Gesine Reinert. “Bias in Epidemiological Studies of Conflict Mortality.” Journal of Peace Research 45, no. 5 (2008): 653–63. http://www.jstor.org/stable/27640738. Condra, Luke, and Jacob Shapiro. “Who Takes the Blame? The Strategic Effects of Collateral Damage.” American Journal of Political Science, 56 (2012): 167-187. https://doi.org/10.1111/j.1540-5907.2011.00542.x
39
Note: Lyall, Jason, Graeme Blair, and Kosuke Imai. “Explaining Support for Combatants during Wartime: A Survey Experiment in Afghanistan.” American Political Science Review 107, no. 4 (2013): 679–705. https://doi.org/10.1017/S0003055413000403.
40
Note: Fair, C. Christine, Rebecca Littman, Neil Malhotra, and Jacob N. Shapiro. “Relative Poverty, Perceived Violence, and Support for Militant Politics: Evidence from Pakistan.” Political Science Research and Methods 6, no. 1 (2018): 57–81. doi:10.1017/psrm.2016.6.
41
Note: Driscoll, Jesse and Nicholai Lidow. “Representative Surveys in Insecure Environments: A Case Study of Mogadishu, Somalia.” Journal of Survey Statistics and Methodology. 2, no. 1 (2014): 78–95.
42
Note: Mueller, Hannes, Andre Groeger, Jonathan Hersh, Andrea Matranga and Joan Serrat. “Monitoring war destruction from space using machine learning.” Proc Natl Acad Sci USA, 11:23 e2025400118. (2021) doi:10.1073/pnas.2025400118; Blumenstock, Joshua, Tarek Ghani, Sylvan Herskowitz, Ethan B. Kapstein, Thomas Scherer, and Ott Toomet. Insecurity and Industrial Organization: Evidence from Afghanistan. Policy Research Working Papers. The World Bank, 2018. https://doi.org/10.1596/1813-9450-8301.
43
Note: Desch, Michael C. Cult of the Irrelevant: The Waning Influence of Social Science on National Security. United States: Princeton University Press, 2019, 122. See also most of Chapter 5.
44
Note: All statistics in this section are from the Uppsala Conflict Data Program which collects a range of information on all conflicts over government and/or territory that had at least 25 battle deaths in one or more calendar years.
45
Note: Disinformation and domestic political violence are highly amenable to data-driven, policy-relevant research, such as the ESOC-supported Bridging Divides Initiative which seeks to reduce the risk of political violence in America.
46
Note: For example, the MRI-supported Climate Change and African Political Stability (CCAPS) program, developed innovative data on forms of violence not previously tracked in order to understand how climate change is driving instability. CCAPS is no longer active, but its approach was taken up by the Social Conflict Analysis Database (SCAD) which provide a vital scientific resource by tracking protests, riots, strikes, inter-communal conflict, government violence against civilians, and other forms of social conflict across Africa, Mexico, Central America, and the Caribbean. And the Technology and National Security in China initiative at the University of California’s Institute on Global Conflict and Cooperation (IGCC), nurtured a generation of scholars on China’s defense industry and led to an ongoing project at IGCC.
47
Note: We know of only one effort to archive sub-national conflict data created for multiple projects in one place, the xSub project supported by the University of Michigan.
48
Note: For example, we lack updated undergraduate materials in Economics to transfer lessons from the last decade of research to tomorrow’s undergraduates on track to careers in military, intelligence, law enforcement, or diplomacy. A textbook would enable undergraduate courses in large public universities, which in turn could create faculty positions, building research capacity. We foresee channels between faculty at PME institutions, public policy schools, major universities, and practitioners as being essential to writing that book.

Recent & Related

Policy Memo

Whither Globalization: Retreat, Industrial Policy, Unintended Consequences

Mathew Burrows • Robert A. Manning

Field Note

Is France an Ally in the Struggle for an Equitable International Financial System?

Aude Darnal • Nicole Gilbert

Policy Memo

The Empty Promises of the US-EU Lobito Infrastructure Project

Chris O. Ògúnmọ́dẹdé

Big Data Seeks Context, for Long-Term Relationship: Reflections on the Empirical Studies of Conflict Project (ESOC)

By Eli Berman • Jacob N. Shapiro

Grand Strategy