David MONTGOMERY: What we’re trying to do in the Future of Social Science and National Security project is to give a sense of the broad ecosystem around social science and national security. As a former Deputy Assistant Secretary of Defense for Research and Engineering, we are hoping you can begin by describing the Department of Defense’s (DoD) investment in science broadly, and social science research more specifically. Related, we’d like you to help us think through what things should look like, to help us understand if the current model is the right model.
Melissa FLAGG: The first thing to consider before I answer that question is what is it that we’re talking about when we talk about research? When you talk about research in the DoD specifically, it is a much broader frame than if, say, you talk about it within the National Science Foundation (NSF). The NSF has a very different mission, they exist to support the discovery of new knowledge and to sustain our pipeline of scientists and engineers—and to do this they anchor their work in basic science. But when you shift to an organization like the DoD, which is a mission agency, research takes on a very different meaning. And that can be very uncomfortable for people who believe that research should be led solely by the spirit of discovery, that creativity should guide the work not a mission or an application directed by a government agency.
What’s unique about the DoD, and what Congress has imposed on the Department, is that they have these rigid, siloed “colors of money.” So, 6.1 research is basic research, and if Congress gives you money to put into 6.1 research, it is supposed to be basic research. By definition, it has to be foundational. It cannot have a specific application in mind. And that is very much what we think of when we think of NSF-type basic research. 6.2 is applied, so it’s still pretty basic but you’re starting to think about real-world problems that relate to the DoD mission.
This funding construct creates what I would consider a false linearity of the system, and we’re chopping it up into little silos of money. We’re walling off money to protect basic research, because we know that in a mission agency, you’ll always be pulled toward the defense problems of today, toward the crisis of the moment. Certainly, post-9/11, when the young men and women who signed up to serve were dying, we needed to focus on the problem of the day. But you also want to make sure that you plan for a different future that is certain to come, that you are not eating your seed corn so to speak.
The challenge is that because we only protect 6.1 research, basic research, we make it hard to use that research because we’ve built this wall around it. When you just say the role of research in the DoD, the big role of research is actually to ensure that for the crises of the future, we have options. We have developed relationships with a broad group of scientists and engineers who keep us aware of the art of the possible, that allow us to access new concepts and begin to experiment with those concepts within defense problem sets. A big part of DoD research is to develop relationships with people who understand us and understand our problems so that when a crisis happens, we are able to draw ideas out of that pipeline and respond very quickly.
In a perfect system, where we’ve planted a thousand seeds of basic research, as needs arise we start to harvest them and see what works and what fails. We try to throw failures out quickly and mature the things that work. And as the adversary adapts, we continue to bring in new ideas and apply new research. We really saw this post-9/11 with the improvised explosive device (IED) problem where we had a tremendous amount of research that had just been sitting on shelves in labs, and we had strong relationships that had been built in academia. When the moment came when we really needed new solutions, we were able to draw from those backlogs very quickly.
Research is both a solution set, but it is also a set of relationships that are a risk mitigator. In other words, you’re buying down risk by developing these relationships over time.
MONTGOMERY: Could you talk a little bit about different agencies, the ways in which they think about research, how they use it differently, and how there might be at times competing interests in what research should look like? Does this impact the ability to coordinate research in a more collaborative way?
FLAGG: I will push back slightly on whether our obsession with coordination is actually always good in a mission agency. Efficiency is something we like in peacetime. It’s very tidy. We can have organizational charts that we believe mean things. We have spreadsheets and proxy measures that we lie to ourselves and say mean things, and we can measure those and check boxes. And Congress and everyone else is happy. Or they’re mad, but they’re still giving us money so who cares?
But in wartime, when people are dying, you don’t care about efficiency; you care about the mission. You care about completing the mission and getting those men and women home alive. That is all you care about so you will spend the money. Given that, there are two different approaches that I believe lend themselves to success in these two very different environments. Centralized, tidy coordination that’s pre-planned well into the future, and says we’re going to have a breakthrough in physics on a Wednesday at 2:00 p.m. in five years is very nice in peacetime.
However, in war, the adversary gets a vote. War is a deeply human endeavor, so it is irrational and chaotic. Given that, often a decentralized approach that has allowed 1,000 different ideas to proliferate through different ideals, values, pressures, and supports gives you a higher chance of being resilient to shocks to the system. I would argue that the U.S. is fundamentally a decentralized system both within DoD and also that mirrors the reality of America as an R&D system. Because of that, it is unpleasant to compare it with the system in peace time because you cannot control it. It feels chaotic. You want to tell people what to do. You want to fit them into a spreadsheet, you want nice, easy-to-measure performance metrics that make investment decisions obvious. But this decentralized system is actually our biggest asset when the worst-case scenario is actually at our door.
And so, I want us to think about that as we think about coordination and remember that at times the lack of coordination may actually be a strength. That doesn’t mean that we’re doing it right, but it does mean that we may want to think about how we organize it through a slightly different lens. So, with that, there are some significant differences that people don’t appreciate between the Office of the Secretary of Defense (OSD) sitting at the Pentagon and what the Services do, which is really through a lens and a mission of train and equip the force.
OSD is looking at the world through a lens of prioritization, gap-filling, and mapping to the strategic vision of the political and military leadership. In contrast, the Services must deliver capability and trained warfighters. Those are two fundamentally different goals. They’re both mission-oriented, but one is much more strategic while the other is much more tactical. You cannot execute an effective national security strategy without both. And so, we need to have this tension, but we tend to want to resolve it instead of balance it. Part of the danger of focusing too much on the OSD level is that political and military leadership rotates a lot, so their vision and priorities are more changeable, but the Services must train, man, and equip regardless of the changing landscape of priorities.
Science and research require incredible patience. In order to truly build a deep bench of creative researchers that you can draw from in times of need you can’t fund research for a year, stop funding it for a year, fund it for a year, stop funding it for a year. That is dangerous no matter where you are in the ecosystem, whether it’s academia, internal to government, or in industry. If you have that kind of inconsistency in vision and funding or in the rules of the road of that funding, it is very difficult to establish the relationships that are critical in the scientific in community to actually get to more mature ideas, which is what we need to develop over time.
Michael DESCH: I see four possible tensions: The basic tension at the beginning of the Cold War was Vannevar Bush’s notion of science being controlled by the scientists and expertise as opposed to being subject to the political process. That’s probably an enduring tension that has never really gone away, even though it’s been decided in one way or another — NSF in favor of science, maybe some of these other agencies not so much. So that would be one tension.
The second tension, which you’ve already talked about, is basic versus applied research. I don’t know if you ever read Donald Stoke’s book, Pasteur’s Quadrant, but it’s a great discussion of that but also a critique in practice of the utility of that distinction. The third tension is between strategic versus tactical research. Strategic is the long-term. You want to have the capability to meet longer-term challenges. Whereas the tactical is we’ve got a problem today, for example, IEDs. How are we going to respond to them with the extant capabilities that are out there?
And then finally, efficiency versus flexibility. It makes perfect sense the way you’re laying it out. It’d be nice if you could spreadsheet it and say if we invest in A, B, and C, we’re going to get the highest return. And the idea of spending a lot of money on different things doesn’t seem very cost-effective. But to go to the strategic versus tactical, you can’t really know down the road what’s going to matter.
FLAGG: All of these tensions are real and we’ve talked about aspects of them, and they aren’t static. We have to remember that as soon as we determine a strategic priority, that is a leverage point of weakness for the adversary. The most important thing that I want from my adversary is to know what they are relying on in the future because theoretically, that is how I develop a strategy that plays to my strengths and leverages their weakness. So, one of the most important things you can do at a strategic level is to consider the adversary’s priorities as targets, potential weaknesses, and vulnerabilities. But because of that, if you decide you’re going to pre-plan everything strategically, which is very nice and tidy in peacetime, what you’ve done is lay out a plan for your competitor, you’ve outlined potential leverage points of weakness, and you will have to adapt. The only guarantee you have in the future is that you will need to adapt. And so, if you focus so much on efficiency that you give up resilience — because I would call it resilience rather than flexibility– you lose your options. So an intentional diversity of investment can offer up resilience, it basically is a hedge of optionality for when the adversary decides to cast their vote.
The other thing I find interesting when you talk about strategic vs tactical is that it can challenge common sense in reality. Your definition of strategic and tactical is correct, and if I were writing down an explanation of what those two words mean, I would characterize it as you have. However, the weird thing is that in the DoD, as a practitioner in the real world, strategy which is supposed to be the long-term vision — because it is developed by political appointees and Senate-confirmed military officers who serve at the pleasure of the President, is actually more changeable than your POM (Program Objective Memorandum) budgets and technology investments at the Service level. So, at the end of the day, you often have a more enduring, long-term focus at the Service level because you’ve locked in more investments than you do at the strategic level. So, you’ve almost reversed the world in reality. It’s definitely not how it should be, and it’s not how we describe it on paper…but it is how it actually works much of the time. And so, the tension is even more interesting.
I would also just go back and say I quite like this idea of tensions. We have a fascinating desire as humans to want to resolve the tension. But it’s unrealistic. We don’t resolve tension in a biological system, we’ve learned that pressure is what keeps you from just proliferating cells like crazy. The cells have to push each other at their boundaries to stay healthy and contained. If you don’t have that physical pressure where they’re pushing against each other in the right amount, then actually you get cancer. You have a proliferation of cells.
And so, what we’ve tried to do is drive to consensus and tidiness and resolution, but the structure that we need is one that harvests from that creative tension. And respects the tension and somehow organizes for the tension, not trying to drive out the tension. Otherwise, the utility of the tensions is not being leveraged.
MONTGOMERY: One of the things that I wonder about in terms of the tension is that OSD in the research space it is very much devolved; the bulk of research is done by the Services, which have their own priorities, which can be framed as tactical, but yet there is also a collective priority—a whole-of-DoD problem—that may not be getting properly addressed?
FLAGG: One of the primary roles of OSD is setting priorities, but it’s also gap-filling. Okay, we’ve devolved all of this to the Services. That is the appropriate place for funding to reside. OSD is not supposed to be a competing program management shop. It is supposed to be oversight, priorities, et cetera. But when OSD does oversight and sees a gap the Services are unable or unwilling to fill, where they do not want to cover the funding for another Service to derive the benefit, then it is appropriate for OSD to run a program until the roles for each Service become clear or a new organization is stood up.
What you tend to get when you devolve a program to the Services and you say go execute this program is that it turns into a process development thing; they develop the process. They put somebody in charge such as a contracting officer. They have a list of how they do things and it becomes a process. They will recycle their existing process to the maximum extent possible, which always affects the program. That process often means that the color of the money exerts a huge effect on what they are willing and able to do. If you tell them, I want you to make this science relevant to a specific problem, they may respond that they cannot do that because it’s 6.1 basic research money and it’s against policy to use that funding for applied work.
For them, they’re doing the right thing. And for you, you are doing the right thing. Tension. And Congress doesn’t leave us very much flexibility. There’s a lot of discussion of colorless money in different processes right now around reform for the Pentagon. Mostly they talk about colorless money at the acquisition and procurement level. But there’s also an unintended consequence of all of our protection of basic research that has actually made basic research less useful. We have pushed our protections so far for basic research that we’ve stripped the department of more flexible money at the 6.2 level, at the 6.3 level, where some of these relevant questions reside. Conversely, universities don’t want to take more applied money because it often comes with restrictions that aren’t allowed at the 6.1 level. You find yourself in a self-inflicted process trap.
DESCH: To go back to the Vannevar Bush issue, the concern at the beginning of the big federal expansion into basic research funding was if you get Congress involved deeply, what it’s going to do is reflect congressional interests, i.e., pork. And that will come at the expense of basic research. What it seems like you’re describing is a little bit different. That Congress, for whatever reason, has written in regulations that limit how basic research money — 6.1 money — can be spent that have the unintended consequence of making it harder to do what Congress presumably wants, which is to ensure that this money, at the end of the day, has some direct application.
FLAGG: But let’s also be clear, that we didn’t implement Vannever Bush’s vision. He said to create one agency, the National Science Foundation, and just have it focus on patient funding at universities. If you really read Science, the Endless Frontier, the entire document, not just the commonly cited excerpts, he clearly says, we have to solve actual problems — national security, cure disease, the public welfare. You must solve these problems while dealing with a huge bolus of unemployed men returning from the war. We had devastated the global scientific leaders—Germany, Britain, and France were in ruins. There was no traditional scientific leader left to turn to for assistance. Bush was focused on our need to be self-reliant in an uncertain and greatly altered future. We also had this incredibly rare opportunity to rise in a vacuum of competition and simultaneously employ the people coming home. He wasn’t describing a full solution; he assessed the landscape, assessed both the problem and the current status quo, and then he targeted what he believed was one feasible government intervention that would positively affect all the problems he saw. He chose the need to build a federal funding system for patient funding that leveraged our growing academic institutions. He clearly believed industry should be funding science and that the government should still be supporting mission agencies focusing on applied work, but that patient funding was a missing part of the ecosystem at that time.
The Office of Naval Research (ONR) was the first of these offices created in the post-World War II era in 1946. The Department of Defense pushed hard to have its own version of this vision and created ONR shortly before NSF went live in 1950. ONR was born out of a wartime agency, the Office of Scientific Research and Development (OSRD) as well as the Navy’s Office of Research and Inventions created in 1945. There is a short little paragraph tucked away in Science, the Endless Frontier, that says we basically need scientists stationed overseas to ensure we understand the global state of the art, a sort of technical intelligence capability. It clearly stated that the U.S. needed to host conferences overseas and fund foreign scientists to come to the United States and engage in these programs.
So shortly after ONR was established, the Navy took over the OSRD liaison office in London and then later went on to open an office in Tokyo and eventually Singapore and Chile. So before NSF was created, ONR was stood up out of the remnants of the scientific endeavors of WWII. We did not do what Bush recommended. He specifically said do not proliferate beyond a single organization. But we commenced to proliferate and to somehow say he would like it. Then we went on to gloss over his focus on mission, and this becomes stronger as we begin to protect basic science as it is challenged in the 1980s and 90s. And we pretend it was what Vannevar Bush told us to do. We attribute a lot of things to Vannevar Bush that are taken out of context.
What I believe he would be expecting of us now is not to continue doing what he said to do, abiding by a prescription for a 1945 problem, but actually figure out the context of the contemporary problem which he does really nicely at the beginning of his book. There he notes that our population had increased from 75 million to 130 million between 1900 and 1940. The nations that led global science were diminished. We had this host of serious problems—infectious disease was a huge problem at the time, people were dying, people were unemployed, and we would have to provide our own world-class military in the future. We had to actually tackle these things. He also talked about how the trend lines of philanthropic funding were downward. The trend lines of industrial research funding were downward. The war had ended and people were in bad financial shape. And the U.S. government provided almost zero funding for long-term research to universities.
Today, we are in a very different context. We implemented huge changes in the 50s and this federalization of science peaked in the late 60s, culminating in the U.S. Federal Government providing roughly 70 percent of American research and development (R&D) funding and a similar proportion of global funding. Now the landscape is radically different, the federal government makes up only about 21 percent of American R&D largely because industrial investment in R&D has skyrocketed, especially since 2000. The U.S. and China are both roughly a quarter of global R&D, which is now at over $2.2 to $2.4 trillion annually.
The rest of the world is contributing $1.1 trillion. That has tripled since 2000. In 2000 the total global R&D investment was roughly $890 billion, it has more than tripled in just the last two decades. There is so much science going on in the world. We are graduating over 44,000 technical Ph.Ds every year in the U.S. alone. And policymakers seem to believe that we need more because China’s producing even more. The world is not this unipolar or bilateral world where science and science funding and scientists are scarce. This is now something that is viewed through nearly every national strategy on earth as an innovation activity, not a national security issue.
The only nations that explicitly consider national security in their national science and technology strategies tend to be the United States, China, and Russia. Most nations prioritize science and innovation funding and priorities through an economic lens. They do it through their counterparts to the Department of Commerce or other technical agencies. And so, we have these mismatched expectations of how science happens in the world, but the vast majority of the world is playing a fundamentally different game. Yet here in the U.S., we are still talking about Vannevar Bush and his solutions to a 1945 set of problems.
MONTGOMERY: So the United States has a unique structure for investing in research compared to other countries, especially its strengths and weaknesses?
FLAGG: This is a sacred cow, and so it’s hard to lay it on the table and look at it objectively. But it’s not clear to me that the way we are approaching the protection of basic science is really a solution to all of our woes in America. We seem to believe that simply by pumping more and more money into scientific funding it will magically produce solutions to real-world problems. There is a great value to basic science that, in my opinion, is more about the type of person that you cultivate in terms of creativity, patience, and especially the intellectual bravery to say, “you know what, the science has changed what it’s telling us, and this is inconvenient for you guys who have decided this other thing was true because it’s not true anymore.”
That takes a kind of bravery that you have to give people space to feel and to own and to be confident in and the patience to have intellectual rigor behind that bravery for it to be real and to be executed through humility. I believe in basic science. I just don’t believe that we’re cultivating the true benefits of basic science. We want to cultivate solutions for Americans, for the military, for human disease, whatever; right? We want to initiate solutions, and we’re doing that through basic science only, which is unrealistic.
DESCH: So Stokes’ argument is that the distinction between basic and applied is artificial.
FLAGG: Absolutely.
DESCH: And in Pasteur’s Quadrant, he said a lot of the stuff in terms of basic research, that it originally came out of quite applied research, et cetera. Is that your position, or do you have a different sort of argument here about the fetishization, as you nicely called it, of basic research?
FLAGG: David made a comment earlier where he said structures matter. And I really believe this. If you go back to the Bell Lab days as an example, everyone genuflects to it, you would think it was perfect. I’m sure it was very good. I’m equally sure it probably wasn’t as good as we say it was. That’s just how nostalgia works, right? But one of the things people talk about that I think is so fascinating and probably deeply helpful for leveraging science to solve actual problems is they tried to put the theorists and the experimentalists and the engineers and the operators together.
And they created real feedback loops because we tried this thing that the engineers told us that we could do. And the engineers gave it to us because the experimentalists said, you can probably do this. And the experimentalists tried it because the theorists said it doesn’t break a law of physics. And they try it. But that doesn’t work and so they throw it back over the transom. And the theorists note that of course that didn’t work because you should have thought about it this way. And they run away and do some things and hand it to the experimentalists. And they try again, and they’re like oh, we didn’t realize you were going to do that with it; you should try this.
You have this exquisite feedback loop of a nonlinear process. That is how great science happens. If you structure your entire ecosystem to break it into parts that disincentivize those feedback loops, and in fact make them compete against one another for funding, then you lose some great potential value of your system. And in my opinion, what we’ve done by over-protecting basic science — because it is valuable but because we have so hyper-prioritized it at the expense of everything else — it no longer resides in a healthy, balanced ecosystem where you’re shifting the weight up and shifting it back as the actual question requires because sometimes you need to pull your resources to do the big experiment. And sometimes you need to shunt them back to say it didn’t work; help us figure out another path.
Let’s also be clear: the moonshot was not basic science. The moonshot was the opposite of basic science. It had the most tightly scripted goalposts. You will send a soft, squishy human body in some kind of structure through an atmosphere that wants to burn it to pieces, into a vacuum that wants to destroy it, land it on a rock, and then bring it back alive. Those are very specific applied scientific questions. It was an engineering challenge. We got great basic science from it because we had feedback loops and unimaginable funding. The dollar amounts, if you look at them, there are some estimates of what we spent to go to the moon that were basically like the Department of Defense budget annually in adjusted dollars if you were to go back in time. But we were essentially spending what we spend on the entire Defense Department now per year.
I’m not saying we don’t want to solve problems. I’m just saying these things have costs. And if what we want is to solve problems, maybe we have more of a structural problem than just a “more money for basic science” problem.
DESCH: If we were going to design a structure today or point to a structure today that you think is most advantageous, what would you point to? Or, if you can’t point to anything, how would you redesign it in a different way?
FLAGG: I’m sure there are efforts that, if I were more current in government that I could say this is actually an effort where they’re trying it. But one doesn’t pop to mind. So, I don’t have a great example. What I would say is that if we actually decided to prioritize, not just say we want to be the world leader in everything, everywhere, all of the time, then we would likely create a host of approaches, and our beautiful chaotic decentralized system would give us amazing results. At the end of the day, we knew what we wanted to do in the moonshot, and every government leader was on board, at least publicly. We knew exactly what we wanted to do: Beat the Russians. There was no lack of clarity. We had urgency and a clear timeline. We had all the money we wanted, and we had a very tightly scripted goal. And we did not waiver from it. We did not move the goalpost. We said, get a person on the moon, preferably bring them back alive.
The first thing you have to do is know what you want. You can do this in much smaller ways. Know what you want and then bring a group of people together and say you have seven years to demonstrate a working thing that does this. And we’re going to bring theorists and experimentalists and engineers, and we don’t care where they come from. We’re going to bring people in from industry, from academia, from within government. We’re going to co-locate them physically. We are going to provide them with infrastructure and patient funding. We are not going to have them writing PowerPoint slides once a month for obligations and commitments and execution rates on their dollars. And perhaps most importantly, you measure every single participant against the same goal. You don’t measure contracting officers on how many dollars they put on contract, you measure them on whether they helped the organization get closer to the goal post as quickly (and legally) as possible. There is no other incentive. And there are no supporting roles, every team member is critical. You cannot solve these problems with science alone, we must also prioritize contracting officers, lawyers, HR, program managers, operators, etc.
 To me this type of structure is feasible and it doesn’t require big changes in law or authorities. But in order to do any of this, you first have to agree on what problem you actually want to spend real money to solve, not to play with, not to futz with, not to DARPA-ize and demonstrate in Wired Magazine, but what problem would you pay money to solve?
Let’s also just start with building those feedback loops into our current structures. It’s not efficient. It doesn’t allow for the same type of daily oversight that we’ve become accustomed to because we’ve all turned into accountants—because it’s peacetime. Accepting the “risk” that comes from trusting your people to do their jobs without daily oversight is the type of thing that happens in wartime. When a crisis plays out, we bring everybody together. We get the scientists and the engineers out there in the field with the operators to see what’s happening, to understand the problem, iterate on new options, get it back out there and see how it works – and repeat. We did this through the whole 2000s in Iraq. So it does happen.
MONTGOMERY: In terms of NSF, for example, there has been a recent shift, at least within the Social and Behavioral Sciences group, to try to make it more problem-centric. Within the structure that you’re talking about, it requires NSF to be more grounded in broader issues. But DoD has problem-based types of investments and basic, fundamental research. If we look at some of our allies, their defense research agencies don’t necessarily have a 6.1 equivalent space. Does one model make more sense than another?
FLAGG: Personally, I worry that if you make NSF more problem-focused, you’ve actually lost the biggest unique thing about NSF. All other agencies have a domain-based mission. Every other science funding agency has a mission, that comes with constraints. NSF has a mission, but because it is harder to defend in Congress, they are always tempted to become more applied. If we lose NSF in focusing on truly teaching young people how to be critical thinkers, how to develop a question that’s never been answered, how to have the bravery to try to answer a question that’s never been asked, the patience to tackle that, the beauty and the true difficulty of that is profound. We’re just throwing it away. I truly believe that NSF should be allowed to focus on basic science.
Mission agencies use basic science very differently than NSF. For DoD, basic science is building a community, a set of relationships, so that you can draw on people. It’s a hiring and recruitment mechanism so that you can pull people into the labs to do classified work later. It’s allowing them to stay engaged in their communities so that even when they’re doing applied work part of the time inside, they can still publish and have a peer group and be inspired by this larger body of open work. Basic science plays a different role in DoD, in my opinion. That is the root of my frustration that we’ve treated it exactly like basic science at NSF, in the desire to protect it. But in doing that we’ve minimized its value and utility within DoD. We’ve colored what they expect of NSF in a negative way. Policymakers in trying to help have possibly harmed both institutions unintentionally.
MONTGOMERY: What’s unique about the structure within DoD, how the research evolves, and where social behavioral sciences fit? Does SBS face unique challenges among different components of the Department?
FLAGG: Social science has a challenge that is perhaps most directly tied to Vannevar Bush in the era of physics. All of our agencies came out of this era of physics and big infrastructure top-down approaches, universities developed their entire research departments around the federal government because they’d never had research funding before from anyone other than patrons. We created this enormous and powerful research endeavor in an era of big physics and we lifted up the quantifiable; we love things we can measure.
You begin to assemble a DoD research leadership that comes out of institutions like MIT. The thought leadership that goes into how we structure, value, and measure science in the Department of Defense is built on this post-World War II reality of an era of physics. And when you look at the leadership in DoD over the last 30 years, there is a preponderance of electrical engineers. When you look at the White House Office of Science and Technology Policy (OSTP), it is historically led almost exclusively by physicists. They are quantitative. Our national scientific leadership and our Congress that allocates funding tend to like hard physical things you can see, feel, touch, break, blow up, and put back together.Â
And so, one of the biggest challenges that social and behavioral sciences have is this difficult conundrum that war is literally one of the most fundamentally human endeavors of anything we engage in. It is so tragically, viscerally human. We are willing to slaughter one another against all biology. We are willing to die for things. And yet, there is a technocracy of electrical engineers and physicists that really dominates research and engineering. And so, when you try to talk about something social, or even biology, they immediately regard it as medicine and tell you to go talk to that medical agency that cares about people.Â
DoD tends to segregate anything to do with a human, and we put it in medical. We box it off over there. We make it a different funding line and we don’t really talk about it as “real” science. This division of the more quantitative physical sciences being considered real or hard science while the more qualitative sciences are considered soft or lesser somehow is quietly pervasive. However, at the leadership levels they understand that in the complexity and uncertainty of war, we won’t always be able to wait for the measurable certainty. We need the ability to take action in the face of uncertainty and complexity. The social sciences can be powerful in these spaces.
It is also important for social scientists to be aware that it’s difficult for many of us who were raised in other disciplines to accept that you have a sample size of 10 and you want me to believe something. It’s intellectually hard if I’m accustomed to large data sets and just much more concrete approaches to understanding. I still personally don’t truly understand the value of case studies. I understand them as a way to convince someone of a narrative that you like, but I don’t understand a case study as a way of giving me any kind of objective information because it feels anecdotal. I just don’t know what to do with that because I was raised as a physical scientist.
But I can accept that I have a lack of knowledge of what SBS can contribute and therefore try to surround myself with people who have more knowledge. Because I understand that war is so deeply human, I can tell myself, I know that this thinking matters, that I require diversity in my intellectual toolkit. Therefore, I must find a way to get past my discomfort and leverage it. But that kind of self-awareness isn’t always on display. Â
It’s also very hard to have time to embrace that kind of reflection with the chaos of the Pentagon and the daily pace of working there. We’ve driven nuance out and we default to our automatic settings which is often a narrow definition of what’s important.
DESCH: One small part of the Pentagon that does some of the more human-focused research and strategy in war is the Office of Net Assessment (ONA). Did you have much interaction with them or much experience with their products? How do they fit in the larger DoD research ecology?
FLAGG: The Office of Net Assessment plays a critical role. But there was the Andy Marshall Office of Net Assessment and then there’s the Office of Net Assessment now. And they are perhaps two different things because Andy Marshall was just a sheer force of nature and an unusual mind. Because of his past, he was able to command a level of respect and autonomy that is difficult to replicate. The amount of patience that was afforded to Andy Marshall by DoD leadership was astounding. He could refuse to report to anyone other than the Secretary. As I hear it, he did in fact just refuse at times if he did not see the value. What a spectacular power to wield in such a positive way, to really focus on getting the questions right, finding creative minds, nurturing them, and giving them time and space to think well beyond the horizon in service of our nation’s long-term security.Â
He could almost write his own job description because he had written the Cold War strategy that worked. He had given so much to the nation that was concrete and unassailable that, okay, fine write your own ticket. You’ve earned it. Today, the Office of Net Assessment has a much more difficult path. They must live up to that legacy but without the same unquestioned support. Its current director, in my opinion, has a harder job of having to prove the continued relevance of ONA while still giving people the time and space to truly do the critical thinking that the nation needs.Â
MONTGOMERY: One of the things that ONA claims is that it’s unique; it’s different from most of DoD. But I want to return to some ideas touched on earlier. One was about folks who are cultivated to think a particular way about DoD problems – such as through a physics or engineering frame – and another about the epistemological humility needed to question one’s own biases and assumptions. One of the things that I’ve always thought interesting is that so much of DoD’s decision-making is aimed at trying to get rid of any sense of ambiguity or uncertainty, even though futures by definition are filled with unknown; that is the life we live. Structures, of course, aim to mitigate uncertainty, but they are not always adept at accommodating the flexibility needed for living with uncertainty.Â
If we’re thinking about structures that facilitate unique perspectives—and ONA has been one example—and if we’re talking about integrating different scientific fields in new, useful ways to address current and future problems, what do you see as the most productive way forward?
FLAGG: I have actually thought quite a lot about this. The first thing I would say is we’ve become too obsessed with the federal government doing everything. We do not live in a centralized system. The United States has never truly been a top-down system. We’re a decentralized system.
You don’t go to war with 21 percent of your army. Why would we try to only leverage the 21% of the scientific ecosystem that the federal government funds? It seems crazy. So, first of all, let’s just say there are approaches to solving these problems that may be partnerships, that may be quasi-governmental organizations, that may be nonprofits, that may be foundations. Let’s be thoughtful that we don’t have to be the author of every question or the owner of every solution in order to leverage it.
Second, I would say that the one thing we don’t seem to fund or support or value in the system that ONA has always valued is the importance of defining the problem. Get the question right first. I always hear the story that Andy would say he would rather pay someone to use 90 percent of their time and money getting the question right and 10 percent of it writing something up, even if it never gets published because with a great question, he could lead other people to do great work, great thinking. But with a poor question, he would get a lot of bad actions based on the wrong question. DoD is so focused on taking action quickly, that they often don’t spend the requisite time reflecting on the question. Did we actually get the question right?
I’ll give you an example of this. During a talk at the Naval Postgraduate School, I was on a panel with a general and an AI scientist. A member of the audience asked a question about whether AI is going to take over the battlefield and stated that China is leading in it.  He asked what that meant for us and for the type of people we need to recruit. Do we need to hire AI specialists to be in the army?  I said, “You’ve allowed the Chinese to construct a narrative and offer you a choice to run down a path that they have made. Why not change the question?”
In my view, an actual strategy means I know my strengths. In the U.S. military, I would suggest that my strength is my people. We train incredibly well. We respect our people. They’re flexible. We believe our people are the best in the world. We believe that, in fact, is our strength. There is a story that the Chinese love AI because it gives them some ability to understand their people and to get feedback loops on the state of the country and also to keep them from having to trust their people. So, they believe AI helps them minimize the need for a creative human. Are we going to try to out-China China using AI in the way China would use it?Â
Why don’t we just say, okay, they need AI to be reliable and predictable? We believe our people can operate in a more unpredictable environment. So, let’s make AI unpredictable. Let’s create a totally different path of AI development and ignore what the Chinese are doing and develop what we believe is a strategy that plays to our strengths and leverages their approach as a vulnerability.Â
Organizations like ONA can help you get to better questions. Creating a structure that nurtures and supports question development is the key. ONA matters because it does that. So how do we do more of that?
DESCH: I was just going to flag another tension that’s been implicit in a lot of your discussion, but your discussion here has really crystalized it. And that’s between markets and hierarchies as organizing principles. Everything you say about China is a concrete example of a general class of hierarchical theories of organization and their advantages. And what you’ve been saying in a lot of the discussion is that there’s also the non-hierarchical or market-oriented way of thinking about these processes. I just wanted to put that on the table because it’s clearly, especially in the American case, a very stark tension because our society in general is very market-oriented in a lot of respects.
But the government, especially the federal government, and especially the DoD, also has a strong hierarchical bent. And then the fact that we’re dealing with peer competitors who are doubling down on this, reinforces that tendency. Your question of whether we ought to be letting the enemy set the terms is right on target. But it does expose this tension for me.
FLAGG: I agree. This goes back to where David started on ambiguity and why we’re so put off by it. We have jobs that are defined by uncertainty and yet we’re trying to drive the uncertainty out of everything. Maybe we have too many physicists running the place and electrical engineers and systems engineers who are taught to drive the risk to zero. People die if the risk isn’t driven out. Drive the risk to zero. It’s also unique to peacetime, I think. When the stakes are high and our people are under threat, we will take risks. Once the risk of not taking action is higher than the risk of simply trying, we do amazing things—and sometimes they even work.
If you go back to World War II and the Cold War, we tested bombs in places that would now be considered too close to populations. We took what now would seem like extraordinary risks, but few complained because the alternative of losing that war was far worse. There is one aspect that’s really changed post-Cold War that makes risk-taking even harder which is that we broke many of the feedback mechanisms from civilians to the DoD when we ended the draft. The draft ensured that it could be anyone serving so every family was invested in U.S. military actions. If your child may be sent to war, you may be more likely to educate yourself on the conflict and more likely to make your opinions on the conflict heard.
And then war taxes. As I understand it, we had never gone to war without levying a tax on the people of America to pay for that war until Iraq. We broke that feedback loop too. Now we have a military that is perpetually at war but with an American population and Congress that believe we are at peace. That has never existed in the history of our country. That tension creates a much greater problem since we are in a period where we in the Department of Defense feel we are at war, but the rest of the country thinks we are at peace. I don’t know how to solve that.
MONTGOMERY: As you noted, ONA is a great example of something that was unencumbered by these “general accounting” structures. It was special because someone was allowed to structure it that way. Everything else in the department exists within different types of structures. Is something lost by that? How should we be thinking about the tension between structure and metrics?
FLAGG: This obsession with metrics and efficiency drives us to these false structures, and you get what you measure. If what you’re measuring is execution rates or obligations and expenditures or whatever, then what you get is people who know how to get money out the door. But if you’re not measuring better questions or knowledge transfer of some kind, then you’re not going to get it. So, the great thing about ONA is they fund a lot of great ideas and a lot of people who would not fit into the normal structure. And they are able to support people with patience and with a lot of independence.
The downside of it, however, is there was very little knowledge transfer. Very few people know what happened in ONA. And that’s okay. Andy Marshall’s thing was to find and nurture smart people and then those people will go off and do other things that matter. It didn’t have to be that they did that important thing in that moment for ONA in a specific way, but we don’t have the patience or mechanisms to measure those successes. The reality is that the lack of ability to transfer that knowledge in a visible, measurable way is challenging.Â
Because of the environment where we are in a perpetual state of urgency and feeling like we are at war within the Department of Defense but we are managed by a peacetime oversight structure, there are few ways to do patient things in that structure that don’t have easy quantitative metrics because the urgency means the department isn’t patient, and the oversight approach means they need quantitative metrics for every single thing done. And so, you have a double whammy against anything that requires thought and patience.
Unfortunately, if we’re not careful, we’re going to create a generation of leaders who don’t know how to think. We can’t and shouldn’t tolerate this. We need to create quasi-governmental institutions that sit outside the department but that have a relationship with the department, that have a tight mission alignment with the department, but that have different approaches to funding.
The unpredictability of the future of work for humans in general in this evolving environment is really painful and challenging. There are lots of ways to create institutions that serve both the American people in some ways and also serve very specifically some needs from the Department. You could have a relationship where there are some people from DoD that sit on the board, but as a member not the chair. It’s the government placing itself as a seat at the table in a decentralized system, not in a lead position in a command-and-control structure.
That being said, what is missing specifically from social and behavioral sciences in the research part of DoD is we have to start figuring out how to support more qualitative work. We need leaders who can think about complexity and uncertainty, and who can understand that mixed quantitative and qualitative analysis is powerful. They need to understand why it’s powerful, what it does tell you, what it doesn’t tell you.  Â
This idea that it is basic science but it’s policy-relevant is an anathema to the system.  It’s swimming upstream against everything. And so specifically Minerva has a real challenge where policy wants it to be relevant, and the research folks believe you are just raiding the 6.1 account to do 6.8 study work. Neither of them is right, and neither of them is wrong. It’s just that we need social and behavioral science that is useful. Much like we say about cyber or a lot of software-related stuff, you go from basic to applied really fast in social and behavioral science. You can say it is 6.1 research when we start. This is a foundational question we’ve never answered. This distinction between basic research being tightly protected and most of these things not existing anywhere else in the system is just keeping us from doing the right thing at times. And we’re all so focused on the process and the structure that we’re not losing focus on the outcomes we want.
MONTGOMERY: When things didn’t go well in Afghanistan in the exit, you had people who were surprised by it. But colleagues who work on Central Asia or Afghanistan weren’t surprised. It struck me that we were largely funding work that was measuring—it had to be measurable, to meet certain types of metrics—our interventions. Questions about how effective we were in doing certain things, which is a very different question than asking what the Afghans wanted. In terms of our efforts to support a particular type of government that wasn’t seen as locally legitimate was disconnected from the types of research that we were supporting. And, of course, insights from research that focus on the “wrong” questions—perhaps not wrong in the scientific sense but maybe in the operational sense—can have a life of their own that perpetuates a myth.
FLAGG: Getting the question wrong has serious implications. A friend of mine and I are talking about a need for a nonprofit that funds people just to do problem definition. Giving people space and tools to work with would provide us with the real ability to think about community empowerment plans. Are you solving the right problem, is it relevant? Whose problem is it? Who are the losers and the winners of the solution? What do the various stakeholders see as the right end-state of any solution? You need to understand that whole space and make sure you’re getting the actual problem defined correctly. And then your path to a useful solution can include the community in that path. The same thing is relevant whether it’s domestic, global, or at DoD.
MONTGOMERY: One of the last questions, which is really related to where we ended up anyhow, on the broader policy relevance of research: How do we manage the range of spaces in research funding and evidence-based application of social science?
FLAGG: We rarely ask program managers and research organizations to tell us, “What do you know now that you didn’t before?”  There should be annual — or every three years – “this is what we learned.” We learned these new things. Few people (if anyone) have the time and breadth of expertise to read and understand every single paper published on every single one of the thousands of grants funded each year at DoD alone. We don’t know what we are actually learning. How do we roll up the value and the relevance of the knowledge we are ostensibly creating? Having a program manager focus more on the value they are creating, articulating the new knowledge that was generated, is far more important than reporting out that their program produced 10 peer-reviewed papers and 3 prizes. Articulating this type of knowledge may cause both security concerns and new opportunities that you could consider, new questions that might be better or more important to ask and answer. The lack of focus on the outcome of what we fund and the hyper-focus on the execution of the funding means we often don’t know the relevance after a program manager departs. We have a few people that we fund who are very good at getting their work out into the public space. And so, we hear about that work. But we rarely know in aggregate what we have learned that has moved us or should be moving us in a certain direction. Knowing the scope of evidence-based research in social and behavioral science is more policy-relevant than an individual study.
About the Authors and Interviewee
Dr. Melissa Flagg served as the Deputy Assistant Secretary of Defense for Research during the Obama Administration where she was responsible for policy and oversight of the Department of Defense (DoD) Science and Technology (S&T) programs from Basic Research through Advanced Technology Development. She was also responsible for oversight of DoD Laboratories, ensuring the long-term strategic direction of the Department’s S&T programs, and for developing those technologies needed for continued technological superiority of U.S. forces. After leaving the Pentagon in 2017, she founded Flagg Consulting LLC and served as a research fellow at the Acquisition Innovation Research Center and a senior advisor to the Center for Security and Emerging Technology at Georgetown University. Prior to her Government service, she was a senior program officer at the John D. and Catherine T. MacArthur Foundation Fellows program with a primary focus on science, technology, engineering and mathematics. She holds a PhD in pharmaceutical chemistry and a BS in pharmacy.
Michael C. Desch and David Montgomery, along with Mathew Burrows, co-direct the Future of Social Science & National Security Project at the Stimson Center.
