Defining scientific progress in terms of the cumulation of knowledge, predictive power, and an “approach-to-consensus” regarding the best explanation when intellectual disputes arise, Fred Chernoff raises the critically important questions of why is there relatively little progress in the field of security studies as compared to the natural sciences, and why is there more progress in some areas of security studies than in others. He argues that one important answer to these questions is that scholars in security studies, unlike those in the natural sciences, use different philosophy of science criteria of evaluation and are rarely explicit about what those criteria are. Chernoff finds support for his argument in an empirical examination of how security studies scholars make judgments about the quality of competing explanations regarding three important research questions in the field—nuclear proliferation, balance of power and alliance formation, and the democratic peace. With respect to the latter, he argues that scholars have explicitly stated their criteria, reached agreement about the appropriate criteria, and moved towards consensus on the validity of a liberal explanation (though which particular liberal explanation is still contested). Chernoff includes a discussion of alternative explanations for the lack of scientific progress in security studies, including the fact that some scholars are answering different questions rather than providing different answers to the same question. He concludes with some useful reflections on the role of metatheory in international relations research programs.
H-Diplo | ISSF Roundtable, Volume IX, No. 11 (2017)
issforum.org
H-Diplo/ISSF Editors: Thomas Maddux and Diane Labrosse
H-Diplo/ISSF Roundtable and Web/Production Editor: George Fujii
Fred Chernoff. Explanation and Progress in Security Studies: Bridging Theoretical Divides in International Relations. Stanford: Stanford University Press, 2014. ISBN: 9780804790956 (cloth, $90.00); 9780804792264 (paper, $27.95); 9780804792295 (e-book, $27.95).
Published on 20 February 2017
Shortlink: tiny.cc/ISSF-Roundtable-9-11
Permalink: https://issforum.org/roundtables/9-11-chernoff
PDF URL: https://issforum.org/ISSF/PDF/ISSF-Roundtable-9-11.pdf
Contents
-
Introduction by Jack S. Levy, Rutgers University. 2
-
Review by Jérémie Cornut, Simon Fraser University. 9
-
Review by Tuomas Forsberg, University of Tampere. 13
-
Review by Ewan Harrison, Rutgers University. 16
-
Review by James J. Wirtz, Naval Postgraduate School 19
-
Author’s Response by Fred Chernoff, Colgate University. 21
Introduction by Jack S. Levy, Rutgers University
Defining scientific progress in terms of the cumulation of knowledge, predictive power, and an “approach-to-consensus” regarding the best explanation when intellectual disputes arise, Fred Chernoff raises the critically important questions of why is there relatively little progress in the field of security studies as compared to the natural sciences, and why is there more progress in some areas of security studies than in others. He argues that one important answer to these questions is that scholars in security studies, unlike those in the natural sciences, use different philosophy of science criteria of evaluation and are rarely explicit about what those criteria are. Chernoff finds support for his argument in an empirical examination of how security studies scholars make judgments about the quality of competing explanations regarding three important research questions in the field—nuclear proliferation, balance of power and alliance formation, and the democratic peace. With respect to the latter, he argues that scholars have explicitly stated their criteria, reached agreement about the appropriate criteria, and moved towards consensus on the validity of a liberal explanation (though which particular liberal explanation is still contested). Chernoff includes a discussion of alternative explanations for the lack of scientific progress in security studies, including the fact that some scholars are answering different questions rather than providing different answers to the same question. He concludes with some useful reflections on the role of metatheory in international relations research programs.
The reviewers—each of whom has made important individual contributions at the intersection of philosophy of science and international relations—all emphasize the importance of Chernoff’s research goals of identifying the impediments to the cumulation of knowledge in international relations and security studies, and of suggesting the most promising ways of overcoming those impediments. They each acknowledge the difficulty of the task that Chernoff undertakes—both in terms of the complexity of the philosophy of science issues involved and the severity of the theoretical, methodological, political, and ideological divisions in the international relations field.[1] Each of the reviewers praises Chernoff’s earlier work in this area, and each argues that Explanation and Progress in Security Studies is an important new addition to the literature on evaluating progress in the field.[2]
Ewan Harrison begins his review by claiming that Explanation and Progress in Security Studies is “the most important book in the positivist tradition written on the philosophy of science and International Relations (IR) to date.” Harrison praises Chernoff’s depth of knowledge of philosophy of science debates, his engagement with methodological pluralism, and the tenacity with which he conducts his research. However, Harrison questions Chernoff’s conclusions about the progress of democratic peace research. He argues that after many years of generating important findings and insights, democratic peace researchers have begun to produce diminishing marginal returns, in part because of their primarily dyadic focus. Harrison suggests that a more systemic orientation would be a more fruitful path for future research. He concludes by emphasizing that the application of criteria of evaluation from the philosophy of science is important not only for assessing the past progress of a research program, but also for providing a guide for productive directions for future research.
Tuomas Forsberg shares the other contributors’ praise for Explanation and Progress in Security Studies. He raises additional questions about some standard criteria of evaluation. Noting that empirical adequacy is the most widely shared criterion among contributors to the research communities that Chernoff investigates, he argues that there may be some variation in how scholars understand and apply that basic concept. Developing a point Chernoff raises in his concluding chapter, Forsberg emphasizes that being explicit about the precise research question under investigation is just as important as being explicit about the criteria for evaluating an explanation. Forsberg gives particular attention to differences in explanations of generalized patterns of behavior as compared to explanations of individual events, arguing that consensus arises more readily in the former than in the latter. Forsberg illustrates his argument by briefly examining Russia’s behavior in the Ukraine since 2014. He argues that the most influential interpretations are those involving “a simple explanation associated with an established theory, preferably with a big name behind it.” Forsberg argues that debates about Russian behavior in the Ukraine, and other debates as well, are driven more by arguments about theory and a commitment to simplicity than by concerns about empirical adequacy, despite participants’ claims about the latter.[3]
Jérémie Cornut commends Chernoff for focusing on philosophy-of-science issues, particularly in the context of security studies and international relations fields that are often dominated by more narrow methodological debates. He also praises Chernoff for examining actual research practice, and for supporting his prescriptive recommendations with evidence of what researchers actually do and what works. However, Cornut’s sociological perspective leads him to further develop a point raised briefly by Forsberg and criticize Chernoff for his failure to adequately recognize the “politics of knowledge production.” This includes the “psychological, social, and institutional factors” that impede conversation between paradigms, make it harder for new entrants into the field to be heard, and consequently limit the accumulation of knowledge. The problem, Cornut argues, derives in part from the influence (on the IR field and on Chernoff) of Thomas Kuhn’s emphasis on normal science, and in part from Chernoff’s naturalist approach, with the assumption that the model of the philosophy of science adopted by the natural sciences is appropriate for the social sciences.[4] Unlike the Kuhnian world of the natural sciences, where one paradigm can supersede another based on widely accepted criteria, in the social sciences there are “no paradigms or explanations universally accepted as better than others.” This, he adds, is “for the best.” Scientific progress is more likely in contexts that “put scholars in competition rather than in consensus and normal science.”
This line of argument needs to be taken seriously by all scholars in security studies, in the broader field of international relations, in the social sciences more generally, and in historiography as well. Cornut is probably correct about differences in the study of international relations across national boundaries and the lack of interactions across those boundaries.[5] He may also be right about the limited nature of the interactions between paradigms, though I think that depends on exactly how one defines a paradigm in international relations. There has certainly been ongoing interaction and engagement between scholars committed to realist and liberal paradigms, though some would say that both operate within a broader rationalist paradigm.[6] Still, the emergence of constructivism as one of the two or three dominant paradigms in the field, along with the growing influence of postmodernist, poststructuralist, and feminist approaches, runs contrary to Cornut’s argument about the psychological, social, and institutional impediments to change in in the IR field and the difficulties facing new entrants into the field. The IR field is arguably the most diverse in the discipline of political science. That may have been a within-paradigm diversity during early debates between traditionalists and behavioralists and between realists and liberals, but the ‘third debate’ has been inter-paradigmatic and focused on underlying ontological and epistemological issues.[7] Cornut may be right that scholars often become psychologically committed to a certain approach, but there are a large number of exceptions. Consider, for example, the democratic peace research program, which some regard as driven by policy preferences and theoretical preconceptions. It is notable that some of the leading proponents of the democratic peace school started out as critics of the proposition that democratic states rarely if ever fight each other, but were eventually persuaded by the evidence and became some of its leading supporters. These scholars include Bruce Bueno de Mesquita, Stuart Bremer, and Zeev Maoz.[8]
Cornut also exaggerates when he claims that “even when scholars have a similar epistemology and the same broad audience, be it positivist or not, it is uncommon to see genuine engagement.” The emphasis on multi-method research designs—involving some combination of large-N statistical methods, formal modeling, historical case studies, and, increasingly, experimental methods—is now standard in the field, and increasingly expected in doctoral dissertations, at least in the United States.[9] The democratic peace research program has certainly been multi-method, and one can point to many other leading research programs in security studies that have been distinguished by their multi-method character: the diversionary theory of war, audience costs, economic interdependence and conflict, and alliances and war, to name a few. It is worth noting, with respect to Cornut’s comments about “consensus and normal science,” that although each of these research programs is in the “normal science” stage, each is characterized by an enormous amount of competition between scholars.[10]
Forsberg’s claim that debates in the field are often driven more by arguments over theory and a commitment to simplicity than by concerns about empirical adequacy is sometimes valid, though I think there is considerable variation across research programs. As I suggested earlier, I do not think it is valid for the democratic peace. It is worth noting that Forsberg’s argument parallels one commonly made by historians—that international relations scholarship is often driven more by rigid theoretical preconceptions than by the evidence. Historians’ conceptions of the dogmatic use of theory in political science is reflected in Isaiah Berlin’s comment that an “addiction to theory–being doctrinaire–is a term of abuse when applied to historians; yet it is not an insult if applied to a natural scientist.”[11] It is interesting to note in this context that the interpretations of Russian behavior that Forsberg mentions all come from political scientists. It would be interesting to see if the most influential interpretations advanced by historians—of this case and of other cases—are also characterized by “simple explanations.” I think that is unlikely, given the lesser weight historians generally give to the criterion of parsimony.[12]
Now let me return to the democratic peace. I agree with Harrison that system-level approaches to the study of the democratic peace should and will be increasingly important and influential in the future.[13] I also agree with his statement that “For thirty years, dyadic research on the democratic peace regularly produced large and important insights.” I would add that dyadic democratic peace research contributed not only to our understanding of the nature of democratic foreign policies and the relationships between democracies, but also to the research practices of the broader IR field. The robustness of the descriptive finding of the near absence of wars between democratic states played a significant role in leading many scholars to shift away from system-level analyses to the dyadic level in their studies of other kinds of international behavior, after concluding that many system-level analyses were both theoretically incomplete and empirically incapable of accounting for much of the variance in the outbreak or expansion of international conflict. This shift in focus contributed significantly to the emergence of research programs on international rivalries, economic interdependence and conflict, interstate bargaining, and audience costs, among others, and also to new methodolgies of analysis.[14] Harrison is probably right that the pendulum has swung too far in the dyadic direction, and others have reinforced this point in various research areas, [15] but this should not lead us to minimize the past contributions of dyadic analysis.
Harrison’s call for more system-level research on the democratic peace, along with my argument about shifting levels of analysis over time in other research programs, remind us that research programs go through phases. This applies to the theoretical or empirical emphasis in a particular research program at a particular time as well as to the predominant level of analysis. Elsewhere I have suggested a classification of research programs as primarily evidence-driven, primarily theory-driven, or characterized by an alternating sequence of theory and evidence—of conjectures and refutations, to use Karl Popper’s concept.[16] Democratic peace research was driven primarily by evidence in its early stages. For the last decade and a half, theory has played a much greater role, in the form of an intense theoretical competition to explain a near law-like regularity. Other research programs are driven primarily by theory in their early stages, and perhaps longer. This is true of the bargaining model of war and of rational models of international relations more generally.[17] It is probably also true of Waltzian structural realism. Still other research programs are characterized by an alternating sequence of conjectures and refutations. Going beyond International Relations, an example might be decision theory, defined in terms of both formal (normative) decision theory (centered around expected utility theory) and work in behavioral economics on how people actually make choices under conditions of risk (for example, prospect theory).
Many would argue that ideal research programs follow Popper’s model of conjectures and refutations, for which criteria of evaluation ought to reflect some combination of theory and evidence. This raises the question, however, of whether different criteria might be appropriate for the evaluation of different stages of a research program. A system of evaluation that gave too much weight to theoretical criteria, applied to democratic peace research in the late 1990s, or to research on territory and conflict at about the same time,[18] would have significantly underestimated the cumulation of knowledge and scientific progress in each of those research communities. Similarly, a system of evaluation that gave too much weight to empirical criteria, applied to the bargaining model of war, say, in 2010, would have significantly underestimated the scientific progress of that research program and its influence in security studies.
This line of argument suggests an additional layer of difficulty in assessing scientific progress in security studies and in the social sciences more generally. There are multiple paths to the cumulation of knowledge; those paths are not necessarily linear, and a single set of criteria and single set of weights applied to those criteria might not be appropriate. Still, the place to begin any evaluation of competing theories or research program is the specification and conceptual clarification of the criteria for evaluation, as Chernoff has so persuasively demonstrated.
Participants:
Fred Chernoff is Harvey Picker Professor of International Relations at Colgate University. He has also taught at Brown, Wesleyan, and Yale Universities. In addition to Explanation and Progress in Security Studies (Stanford University Press, 2014), he is author of three other books and two dozen journal articles and book chapters in political science and analytic philosophy. He has served in research posts at the Rand Corporation, the Norwegian Institute of International Affairs, and the International Institute for Strategic Studies, London. He holds a Ph.D. degree from Yale University in political science and from Johns Hopkins in philosophy.
Jack S. Levy is Board of Governors’ Professor of Political Science at Rutgers University, and an Affiliate of the Saltzman Institute of War and Peace Studies at Columbia University. He is past-president of the International Studies Association and of the Peace Science Society. Levy’s primary research interests focus on the causes of interstate war, foreign policy decision-making, and qualitative methodology. He is author of War in the Modern Great Power System, 1495-1975 (1983); co-author (with William R. Thompson) of Causes of War (2010) and of The Arc of War: Origins, Escalation, and Transformation (2011); and co-editor of Explaining War and Peace: Case Studies and Necessary Condition Counterfactuals (with Gary Goertz, 2007), The Oxford Handbook of Political Psychology, 2nd ed. (with Leonie Huddy and David O. Sears, 2013), and The Outbreak of the First World War: Structure, Politics, and Decision-Making (with John A. Vasquez, 2014).
Jérémie Cornut holds a Ph.D from École des Hautes Études en Sciences Sociales (Paris). He is currently post-doctoral fellow at the department of political science at the University of Waterloo. His research interests include diplomacy, Canadian foreign policy, and IR theory. His research findings have notably been published in Cooperation and Conflict, International Studies Perspectives, Journal of International Relations and Development, International Journal, Canadian Journal of Political Science and Canadian Foreign Policy Journal. He is currently working on a project on the changing practices of frontline diplomacy.
Tuomas Forsberg is Professor of International Politics at the University of Tampere. He is also deputy director of the Centre of Excellence on Choices of Russian Modernisation at the Aleksanteri Institute of the University of Helsinki. Previously he has worked at the University of Helsinki, at the George C. Marshall European Center for Security Studies, Garmisch-Partenkirchen, Germany and at the Finnish Institute of International Affairs. He gained his PhD at the University of Wales, Aberystwyth in 1998. His research has dealt primarily with European security issues, focusing on the EU, Germany, Russia and Northern Europe. His publications include Divided West: European Security and the Transatlantic Relationship (co-authored with Graeme Herd, Blackwell 2006), The European Union and Russia (co-authored with Hiski Haukkala, Palgrave Macmillan 2016) and articles in journals such as International Affairs, Journal of Peace Research, Review of International Studies, Security Dialogue and Journal of Common Market Studies.
Ewan Harrison is Assistant Teaching Professor in the Department of Political Science at Rutgers University. He completed his PhD in International Relations at the University of Bristol in 1999. He is co-author (with Sara Mitchell) of The Triumph of Democracy and the Eclipse of the West (New York, Palgrave, 2014), and Coeditor (with Annette Freyberg-Inan, and Patrick James) of the forthcoming Evaluating Progress in International Relations (New York: Routledge, 2016). He is currently working on a monograph on non-Western democracies in world politics.
James J. Wirtz is Professor and Dean of the School of International Graduate Studies at the Naval Postgraduate School Monterey, California, and author/editor of many books, including, most recently, Intelligence: The Secret World of Spies, ed., with Loch Johnson (Oxford, 2014). He is currently completing a volume entitled Understanding Intelligence Failure: Warning, Response and Deterrence (Routledge, 2016). He received his Ph.D. from Columbia University.
Review by Jérémie Cornut, Simon Fraser University
What restricts progress and knowledge accumulation in security studies? And why do scholars so often fail to convince their colleagues of the values of their approach? Looking to answer these questions, Fred Chernoff’s Explanation and Progress in Security Studies considers two hypotheses. First, Chernoff proposes that there is no cumulative progress in the field because scholars use different criteria of explanatory superiority and often speak past one another as a result. His second, related hypothesis proposes that the accumulation of knowledge would require contending explanatory schools to use similar criteria. To test these hypotheses, he looks to the philosophy of science and existing definitions of progress and knowledge accumulation. He then analyzes thirty-three prominent works across three domains (nuclear proliferation, alliance formation, and democratic peace) before coming to a mixed conclusion: a uniformity of criteria does help with the accumulation of knowledge, but it is unlikely that this is the only factor preventing progress in the field. Chernoff instead mentions several other possible factors at play.
The book lays bare a fascinating question. Thomas Kuhn famously demonstrated that natural sciences sustain successive periods of ‘normal’ science in which one paradigm dominates, to be interrupted and overthrown by scientific revolutions driven by novel knowledge and paradigm.[19] Given that the social sciences have repeatedly failed to follow this cycle, can we deliberately create the conditions and practices that could allow for a similar evolution? Chernoff’s initial intuition is that lack of clarity around explanatory criteria prevents agreements over them between scholars, and that this lack of agreement in turn prevents knowledge accumulation. In this perspective, progress requires consensus and knowledge accumulation can come only from normal science.
There are two main reasons this book makes an important contribution—and not just to security studies, but International Relations (IR) as a field. First, it takes seriously the criteria we use for assessing and comparing approaches, showing that meta-theoretical clarity has crucial implications for knowledge accumulation and progress in IR. In a field often dominated by methodological debates, it is refreshing to see an empirical and theoretical work focused on the philosophy of science and its implications. Second, it looks to what security studies scholars actually do, beginning its inquiry from their practices. The book is prescriptive—as philosophy of science should be—but only after taking careful stock of what scholars do in practice. Where many meta-theoretical debates are sometimes too esoteric to reach the core of the field, here this is not the case.
I have two main disagreements, both linked to the influence of Kuhn’s work on Chernoff’s. The first stems from Chernoff’s failure to take fully into consideration the politics of knowledge production. This is a central focus for sociologists of science: psychological, social, and institutional factors in the discipline create paradigmatic wars preventing the accumulation of knowledge advocated by Chernoff. Such sociological analyses have already made their way into IR through the works of Ole Wæver, Christian Bueger, Inanna Hamati-Ataya, and Marcus Kristensen.[20] They often find that open dialogue do not easily thrive in a field where competition and opposition are the norm. IR scholars with different epistemologies rarely engage one another in good faith; but even when scholars have a similar epistemology and the same broad audience, be it positivist or not, it is uncommon to see genuine engagement. In a sociological perspective, reversing Chernoff’s logics, the idea of the progress of science should be found in the logic of the scientific field that put scholars in competition rather than in consensus and normal science. As sociologist Pierre Bourdieu put it,
The scientific field always includes a measure of social arbitrariness, inasmuch as it serves the interests of those who are in a position, inside or outside the field, to gather in the profits; but this does not prevent the inherent logic of the field, and in particular, the struggle between the dominant and the new entrants, with the resultant cross-control, from bringing about, under certain conditions, a systematic diversion of ends whereby the pursuit of private scientific interests ([…] in both senses of the word) continuously operates to the advantage of the progress of science.[21]
This is a challenge for Chernoff’s argument: for him, scholars do not know that they should be clear about their explanatory criteria; for sociologists of science, they do not need/want to be clear. Chernoff’s contention that disagreement and lack of clarity create a disciplinary stalemate—his approach-to-consensus on the best theoretical explanation—does not take into account that definition of progress and whether explanatory criteria should be explicit or not are objects of symbolic struggles in the field. The logics of the scientific field and the discipline’s construction create powerful theoretical boundaries. Those looking to successfully position themselves in a scientific field often do so by framing their research in opposition to straw-men of the arguments of others, allowing for more conclusive and infallible demonstrations than an honest engagement with alternative hypotheses would produce. This can explain why scholars do not bother too much with criteria of validity in the works of other researchers, and the impression that scholars are often speaking past each other. Contra Bourdieu, this can also explain stalemates in the social sciences.
Chernoff suggests that if scholars realized a need to be explicit about their explanatory criteria, progress would be easier. Yet, he presents little evidence that democratic peace researchers (the subfield where he considers progress to be most notable) have been more open about their explanatory criteria. An in-depth analysis of this question goes beyond the scope of this review, but we can assume that democratic peace studies have witnessed more consensus on explanatory criteria than the two other subfields of security studies reviewed because of the way it is constituted in the practices of its researchers, not because democratic peace scholars have been more transparent. It might, for example, correlate with the ability of a small number of scholars to wield a large amount of scientific capital in this subfield.
Because consensus is an inherently social construct, it is strange to get a sense that Chernoff treats its formation as logical rather than sociological. In particular, it would have been interesting to see a thorough discussion of the sociology of the discipline in the treatment of alternative hypotheses in Chapter 6. In this chapter Chernoff implicitly admits on several occasions that the social structure of the field contributes to stalemates in security studies, but does not discuss this idea further. For instance, he alludes to the social processes so crucial for establishing what progress means to the discipline when he discusses the need for scholars to maximise their rhetorical and persuasive effects, indicates elements that are expected to be included in scholarly works by customs and norms, emphasises the role of exemplary works on new entrants, and notices how context influences the questions asked by security studies scholars.
Such a discussion would have been important given the book’s explicitly normative and prescriptive dimensions. Had he taken into consideration the scientific field as a field of struggle, Chernoff would likely have formulated very different prescriptions. Clarity and similarity in meta-theoretical foundations will do little to favour knowledge accumulation if the obstacle to progress lies in the ways in which the field is constituted. Problems (and thus solutions) should be found elsewhere: in how scholars construct the discipline in their day-to-day activities, in the general lack of reflexivity of IR scholars, in the emphasis put on making a name for oneself, in the existing strategies of distinction, in career incentives to publish (a lot), and so on. We can hope that clarity of explanatory criteria would favour more honest engagements between scholars, but would we not be better off attempting to deconstruct the strong disciplinary logics that reward distorted engagements with others’ work?
My second disagreement regards Chernoff’s quest for the “best explanation” (1). When he suggests that scholars should clarify their explanatory criteria to settle their disagreements and identify the best explanation, he broadly accepts a conception where paradigms are in competition. His emphasis on consensus over explanatory criteria paradoxically reproduces and reinforces the very war of paradigms contributing to the lack of knowledge accumulation. His invitation to clarify criteria of explanatory superiority takes for granted that theories should be ranked rather than, say, combined.
This is further reinforced when Chernoff implicitly takes a specific side in the epistemological debate over explanatory criteria. The book is sometimes paradoxical. On the one hand, Chernoff’s choice of a research design follows a very positivist framework. He tries to determine objective criteria of selection through building a research design in which hypothesis testing, falsifiability, and empirical fit are crucial. Implicitly, his study aims to identify the regularities of observables in order to make predictions and attempts to meet the accepted criteria of behavioural and quantitative approaches to IR. Presumably, these choices are due to his desire to fulfill the (positivist) criteria of his audience; because he wants to change the practice of mainstream (American) security studies scholars, he is attempting to speak their language. Yet, the concluding chapter seems open to a more pluralist approach to explanations. The question of whether this openness is totally compatible with Chernoff’s own positivist research design and the Kuhn-inspired paradigmatism that the book calls for remains.
Chernoff’s chapters 3-5 give a reader the impression that only a limited number of approaches are able to contribute to knowledge accumulation in security studies. The topics reviewed are typical of mainstream security studies approaches in the US: other approaches to these questions are not seriously considered by this mainstream and there would be no reason to expect a consensus between them to emerge. The thirty-three works reviewed subscribe to a naturalist approach to social phenomena. Yet constructivist, Marxist, post-structuralist, practice theory, English school, or feminist approaches, and many others, can and do say something about these questions. They do produce scientific knowledge and not taking them into account is arguably an important impediment to knowledge accumulation in IR.
Clarity about explanatory criteria is desirable, but IR needs less—not more—paradigmatic thinking. In positivist and post-positivist IR, paradigmatic wars and (failed) attempts to reach the stage of normal science through eliminating competing approaches create more stalemate than progress. Chernoff’s approach is modeled on the natural sciences, where there is a clear succession of paradigms. In the natural sciences, one paradigm can supersede previous ones, according to criteria accepted by all or most scholars in a field at a specific time. This is not the case in the social sciences. There are no paradigms or explanations universally accepted as better than others, and it is for the best. As Chernoff explains, Kuhn himself was sceptical about the application of his analysis of scientific paradigms to the social sciences.
If we want to favour progress in IR—and more generally in social sciences—, rather than praising consensus over explanatory criteria and unwillingly reinforcing paradigmatism, should we not always be careful to keep an open and inclusive definition of what developing knowledge means? Rather than a competition of paradigms imported from the natural science into the social sciences, should our past experience with just how fruitless paradigm wars can be not prompt us to approach social phenomena in an eclectic fashion? Rather than looking for the ‘best’ explanation and equating more consensus with more progress in the field, should we not focus on the conditions in which our own explanations could be complementary and compatible with others’? Would we not be better off with a pragmatic and pluralist approach to the field rather than a Kuhnian-inspired paradigmatism? Some elements in Chernoff’s concluding chapter opens the door to answering to these questions in the positive.
Review by Tuomas Forsberg, University of Tampere
Fred Chernoff has made a notable career in pursuing metatheoretical questions in the field of International Relations (IR). His newest book asks the question as to why there is progress in some areas of security studies but relatively little in some areas: there is a relatively high consensus dealing with the democratic peace question, but there is only modest progress in the balance-of-power debate and even less in research concerning nuclear proliferation. Chernoff’s answer to this puzzle is that progress depends on the degree of agreement about the criteria for good explanation. Moreover, both the degree of consensus about the criteria and progress in general can be related to the conventionality of the measure-stipulation and to the context defined by the intent of the questioners and the background knowledge of the audience: the research on democratic peace has been conducted within a relatively narrow time span addressing a more clearly defined research puzzle than have the debates concerning balance of power and nuclear proliferation. Chernoff concludes that it would be advisable if the researchers were more explicit about the criteria they are using for a good explanation. In Chernoff’s view, this is normally a better strategy for IR scholars than to defend the criteria themselves which is a job that belongs to the philosophers or to leave the criteria implicit or undefined.
Chernoff’s basic research task is fundamental and the way he tackles it by reviewing and summarising systematically the selected sets of relevant literature is admirable. The study of IR is often characterized by the lack of progress and disagreement over valid explanations when it comes to key theoretical issues as well as accounting for concrete historical cases. Only relatively rarely is this question properly addressed. The two reasons normally given for this ‘backwardness’ of the discipline are that first, all the debates are politically loaded and hence scientific consensus is difficult to reach before political agreement, and, second, that the study object itself is so complex as well as historically unique that makes statistical research, experiments and many other research methods difficult to apply. For those reasons, the shared background upon which cumulative knowledge could be built remains thin and contested.
In this light, Chernoff’s contribution is encouraging and sanguine, because he claims that progress in IR and security studies can be achieved also by improving our research practices. We do not have to accept the ‘backwardness’ as a constant feature of the field for which nothing can be done. Yet, the chosen criteria for good theory do not seem to set researchers in opposing camps. Chernoff points out that the disagreements between political realists and liberals do not stem from the fact that they consistently disagree as to what the criteria for good explanation are. Moreover, making the criteria explicit does not necessarily lead to a situation where only one theory or explanation would be accepted and others rejected; theoretical and explanatory pluralism is warranted because there are multiple causes and causal mechanisms at play. Theoretical and explanatory pluralism does not, however, mean that various alternative perspectives would be incommensurable, since they may just address different aspects of the research puzzle.
When listing various criteria for good explanation, Chernoff does not order them hierarchically or make explicit in which cases certain criteria are better than others. He does, however, make his own criteria explicit. First comes empirical fit, then true causes and mechanisms, explanatory unification, falsifiability, and predictive accuracy, while precision, elimination of alternatives, and simplicity are of lower importance for him. It is not clear, however, whether this preference ordering in assessing the criteria for good explanation, if adopted more widely, would bring progress also with regard to the substantial security studies puzzles or whether the ordering should depend on the nature of the puzzle. When researchers use, explicitly or implicitly, certain criteria in justifying their own research, it would therefore be also relevant to know whether they reject alternative criteria. Moreover, it is also unclear to what extent the criteria are understood in the same way. Empirical adequacy is the most widely shared criteria, but it is also the most contested when applied. Many of the other criteria often do not stand alone and often their role is to support empirical adequacy (indeed, it is hard to see that out of thirty-four authors studied, the three [George Quester, Etel Solingen and Stuart Bremer] who did not use empirical adequacy as a criteria would deny its relevance). Empirical adequacy, however, is not a sufficient criterion when several competing theories seem to be empirically adequate or when a suggested explanation is only in a tentative phase and because of lack of evidence cannot be proven to be empirically fit or unfit. The relevance of the criteria often depends on the nature of the question and the pragmatic interests behind it. But if this is the case, then the development of arguments for choosing certain criteria cannot be entirely left to the philosophers.
Being explicit about the exact nature of the research question is hence as important as being explicit about the criteria for a good explanation since the latter depend to great degree on the former. The main issue often is whether the explanation is relevant in explaining the whole pattern of the empirical cases, or whether it focuses on explaining a single case. For practical purposes in world politics, we need both kinds of explanations, since policy-makers need to address the universe of potential issues as well the particular issues at hand, often at the same time. As Chernoff’s book illustrates, consensus is easier to emerge in relation to established patterns of behavior, such as democratic peace, but it is more difficult to form explanatory agreement, when scholars disagree what the pattern to be explained is or try to explain single events, particularly ongoing crises.
In those cases, the difficulty does not primarily seem to come from the adopted key criteria for good explanation since, whether explicitly or implicitly, the key criteria in such cases is empirical adequacy. But the empirical adequacy is often illustrated on a very thin level only, ignoring much of contradicting evidence and not systematically stating contrasts or exploring alternative explanations. The lack of consensus is natural because the evidence is scattered and to an extent unreliable; key pieces of it are typically concealed in security political crises, but in such cases it is even more important to state on what basis and to what extent the theory applies or not.
Let us illustrate some of these problems by looking briefly at one pressing security issue and scholarly attempts to deal with it, namely the Ukraine crises since 2014 and specifically Russia’s behavior—a crisis that only emerged after Chernoff’s book was practically finished. Among the theoretical explanations of Russia’s behaviour in the Ukraine crisis that have been put forward by IR scholars perhaps the most famous is John Mearsheimer’s offensive structural realism.[22] He argues that Russia’s actions in Ukraine are to be seen solely as a self-evident reaction to the West’s aggressive grand strategy of enlarging NATO and the European Union and promoting democracy in the form of western-minded ‘colour revolutions’ in Russia’s neighbourhood. The primary objective of Russia was thus to keep Ukraine out of foreign military alliances and geopolitical blocs. The liberals, such as Michael McFaul, claim in turn that the reasons for Russia’s behavior lie solely in Russia’s domestic politics; Russian President Vladimir Putin reacted to the home-grown threat of political protests and created an external crisis to make it possible to enhance the domestic pressure against the opposition and to rally the nation around the flag.[23] A third position that has emerged can be seen as constructivist; scholars like Robert Legvold argue that the interaction between the two sides, rather than the actions of only one side, is what matters by shaping identity formation and creating the spiral in tensions.[24] Many, many more explanations exist and we can also find attempts at integrating several explanations. Yet, very seldom is the aspect of what the different explanations explain specified in terms of contrasts, even when integrated.
The problem here is that the most cited and therefore allegedly most influential pieces of research are those which put forward a simple explanation associated with an established theory, preferably with a big name behind it. This leads to a situation where the debate is already from the outset framed by such work that may claim to have empirical adequacy as a key criteria, but in reality the criteria for influence is simplicity. While simple positions are helpful in framing the wider debate and progress can be created through counterarguments and replies, it is very seldom that this first round of scholarly research ends in any sort of consensus, since what is immediately at stake are certain rigid but empirically flexible theoretical propositions. Yet, scholars spend much time and use their energy in either criticizing or supporting these simple arguments, because they are expected to do so. Add to this the strong politicized habit of reading research outputs backwards from policy recommendations. As a result, when it comes to explanations, relatively different theoretical positions are often supported by the same people, if the normative implications are regarded as sound. This leaves little room for proper theoretical integration and is surely not a recipe for progress in the field of IR.
It would be desirable if IR scholars would adopt Chernoff’s guidelines for stating explicitly the metatheoretical and methodological claims concerning the criteria for assessing the offered explanation and being more precise about the research puzzle and the explanatory contrast, even if no sudden progress in the field followed. Chernoff admits that there is a trade-off because following certain structured guidelines makes the writing more rigid and cumbersome, and often there are space limits that make it difficult to discuss each criteria properly in a single article. Admittedly, the contests can often not be settled simply by stating the criteria since the disagreements of what the criteria then mean in practice cannot always be resolved as easily. Nevertheless, it is a step forward, assuming that clarifying the goals and the criteria can help to sustain theoretical and methodological pluralism rather than curtail it on the basis of metatheoretical arguments. This is a discussion that not only authors, also reviewers and editors and other gatekeepers, should keep in mind when making decisions about what kind of research is worth publishing.
Review by Ewan Harrison, Rutgers University
Fred Chernoff’s study can legitimately claim to be the most important book in the positivist tradition written on the philosophy of science and International Relations (IR) to date. It offers a potent defense of the possibility of scientific progress in the study of world politics, as well as one of the most comprehensive surveys of cross subfield analysis yet produced. It represents the culmination of decades of research and argument, and Chernoff deserves praise for relentlessly pursing his larger philosophical endeavor to its logical conclusions.[25] Combining the vision and rigor of Patrick James’s influential Scientific Progress in International Relations with the comprehensive disciplinary survey and analysis of Colin and Miriam Elman’s classic edited collection Progress in IR Theory, Chernoff has succeed in taking debates about the accumulation of knowledge in IR to a completely new level.[26] Alongside Patrick Jackson’s The Conduct of Inquiry in International Relations, which provides a non-positivist counterpoint, Chernoff’s book will recast the discipline.[27]
A string of major contributions are offered by Chernoff’s volume. The choice of Pierre Duhem’s “conventionalism” to assess the field, as opposed to Imre Lakatos’s more widely known notion of a scientific research program, is innovative and powerfully insightful.[28] Indeed, his knowledge of philosophical debates about the nature of science is frankly intimidating, even for IR scholars who have worked in this area. The influence of Thomas Kuhn’s descriptive account of how scientific paradigms operate through ‘exemplar’ works is skillfully blended into an overall account of social scientific inquiry which remains firmly within a positivist framework. The resulting combination is both intriguing and highly original.[29] The systematic survey of the state of the art in three major subfields within Security Studies provides a wealth of fascinating data which will trigger much debate and discussion, and offers a unique snapshot of the evolution of the IR as an academic discipline. It will be fascinating to see how this might compare with the future shape of the field, and ensures that Chernoff’s book will have a long shelf life. Chernoff’s engagement with the interpretation of methodological pluralism he offers provides a key statement of a particular way of thinking about this issue for the study of IR. Finally, the sheer tenacity with which he follows his arguments to their inescapable conclusions is evident throughout, and is as compelling as it is remarkable. What Chernoff has attempted and achieved in this volume is profoundly admirable.
A forthcoming edited study examines in depth the many debates opened out by Explanation and Progress in Security Studies, especially in comparison to Patrick Jackson’s book.[30] Despite its strengths, Chernoff’s study is not without weaknesses, and in particular it is possible to challenge the interpretation of progress in democratic peace research that he offers. Chernoff’s argument, in contrast to scholarship in other areas of security studies, is that there has been “a significant degree of approach to consensus in the democratic peace debate with scholars accepting that liberal explanations for dyadic behavior are superior to the realist explanations” (233). The liberal explanation has emerged as the most convincing within the field, and debates center on which liberal explanation is best. To Chernoff this consensus is a remarkable scientific achievement, and it shows that individual researchers are cumulatively contributing to a shared body of knowledge. This in turn demonstrates that positivist methods are possible to apply successfully to IR and provide a template for other subfields. By being clear and explicit about ‘measure stipulation’ (defining key terms in a generally accepted way), democratic peace researchers have achieved something that scholars working in other areas have not (253-255).
However, what Chernoff views as progress may actually be a sign of stagnation beginning in the democratic peace research program. To continue down the dyadic path would lead democratic peace research to encounter the problem of diminishing returns. Scholars are generating knowledge which is increasingly incremental. The reason why analysis of the democratic peace has become a cottage industry is not necessarily that the bandwagon of scientific progress is rolling. Instead, democratic peace research today may simply be driven by disciplinary incentives which favor safe and narrow claims over big and sweeping ones.[31] For thirty years, dyadic research on the democratic peace regularly produced large and important insights. Yet it has now reached a ‘boredom threshold.’ Thus Chernoff’s analysis is too descriptive and documentary, offering an account of democratic peace research that is ultimately backward looking. As such, it is sanguine in terms of its recommendations as to where current researchers might most productively spend their scarce time and research resources (i.e. further dyadic analysis).
My own opinion is that in the future, systemic, rather than dyadic, analysis of the democratic peace will become increasingly important and influential. Consider Erik Gartzke and Alex Weisiger’s recent research on intra-democratic conflicts in a world with a strong democratic community.[32] Their systemic democratic peace argument is that peace between democracies was a product of a world in which the democratic community was small and weak and the autocratic community was large and strong. Since the end of the Cold War, the democratic community has become relatively large and strong and the autocratic community small and weak. Under these conditions, shared democratic identity means less. Hence Gartzke and Weisiger reach the counter intuitive conclusion that intra-democratic conflict will remerge in a strongly democratic world. The crisis in transatlantic relations over the Iraq War, along with the UK parliament’s vote against military action in Syria in late summer 2013, are cases in point. Equally salient are the Euro-Crisis and the fallout from the Snowden affair. None of these developments produced a democratic war, but they are certainly examples of intense geopolitical schisms emerging between democracies. This would seem to be a very rich vein of inquiry and debate, and indeed, that debate seems well under way.[33] Moreover, debates of this nature fit nicely into a Lakatosian view of a progressive scientific problem shift within a research program.[34]
While it is cited and briefly discussed (232), Gartzke and Weisiger’s research could not fairly have been included prominently in Chernoff’s book because it is so recent. It has not had time to establish itself as an ‘exemplar work’ in the field. However, this example of a new arena for democratic peace debate is indicative of a deeper flaw in Chernoff’s application of methodologies derived from the philosophy of social science. By focusing on past achievements, Chernoff’s analysis becomes overly conservative and biased towards established points of reference in the literature. Yet the field needs an account of the democratic peace research program which can more constructively guide its future evolution. Ironically, Chernoff has perhaps underestimated the utility of debates from the philosophy of science in evaluating and guiding inquiry in IR.
Overall, I admire Chernoff’s book immensely. It is powerfully argued and, on its own terms, absolutely compelling. It is as important a work of IR scholarship as we are likely to see in a decade or more on epistemological and methodological debates. As a descriptive guide to the past development of democratic peace research, it is hard to fault. Yet philosophy of science debates and criteria also have the potential to act as a guide to where researchers should expend their finite resources in the future. Here I do not agree with the implications of Chernoff’s study. By continuing down the dyadic path, democratic peace research will end up producing only academic obscurity and ultimately oblivion.
Review by James J. Wirtz, Naval Postgraduate School
International relations theorists, including those who specialize in security studies, lack a paradigm. In other words, there is no general agreement about the phenomena to be explained, how to go about explaining these phenomena, and how to judge when one explanation trumps a competing explanation. As a result, these disciplines fail to become cumulative and fall into simmering ideological, political and theoretical disputes as scholars defend their findings, approaches, and reputations against changing academic fashion. Progress occurs, but the pace is indeed glacial—international relations theorists no longer champion Social Darwinism, and phrenology no longer dominates the study of foreign policy decision-making. Nevertheless, bad theory is more likely to fade out—one dead political scientist at a time—than to be driven out by superior explanations of international issues.
As Fred Chernoff notes, these problems emerge even when the issues are both critical and relatively straightforward. For example, many theories have emerged for why states form alliances, but it is difficult to judge which approach actually provides a superior explanation of the phenomenon in question. Chernoff, however, advances a critical proposition to explain this theoretical mess: progress in security studies is slow because various scholars treat the concept of ‘explanation’ differently in their analyses, and this divergence limits convergence over what constitutes successful theory when it comes to understanding the issues under consideration. In other words, key concepts used in the previous sentence (‘explanation’, ‘theory’ and ‘understanding’) are muddled within individual works and often diverge significantly across competing studies of even relatively simple questions (e.g., Why do some states develop or forego nuclear weapons?). Chernoff presents a theory about theory—an explanation of when convergence (general agreement about relevant levels of analysis and causation) and ‘progress’ (refinement of key elements of accepted theory) is likely to emerge in the social sciences. This is a volume about the philosophy of science that uses recent work from the field of international relations as its case study.
Needless to say, this is pretty sophisticated stuff that will take security studies scholars by surprise. The first two chapters are devoted to a survey of relatively recent developments in the philosophy of science that focuses on various concepts of ‘understanding’ and ‘explanation.’ About a dozen criteria are derived from this philosophy of science literature concerning what constitutes an explanation and these criteria are then used to assess leading works in the nuclear proliferation, alliance-formation, and democratic peace debates that are ongoing in security studies. Chernoff then offers an analysis that validates the manuscript’s primary proposition: when agreement over what constitutes an ‘explanation’ is shared among scholars, theoretical convergence emerges and progress occurs as scholars tend to focus on more detailed and nuanced explanations of the phenomenon in question. In other words, Chernoff offers insight into the emergence of paradigms in the social sciences, an advance on the Structure of Scientific Revolutions. To date, the conventional wisdom is that international relations, to say nothing of security studies, is simply too eclectic to develop into a paradigm. Chernoff identifies an explanation for what is seen by friend and foe alike as so much theoretical mush and offers a method to facilitate the emergence of paradigms in the field of security studies and international relations.
This also is a bear of a book. Chernoff has chosen to summarize scores of works from the philosophy of science and security studies. He clearly states where each of the ideas he deals with comes from, but this makes for a long manuscript that demands much from the reader. To put it somewhat differently, he does not just identify which bricks he uses to construct his argument, he actually describes how each brick varies in chemical composition. Because he is writing for two audiences (philosophers of science and security studies experts), he addresses each audience separately, giving both the amount of detail they would demand as evidence for the points being made. This approach is required by the task at hand. Philosophers of science will want to understand the details behind his selection of the criteria used to assess the security studies literature, while those in security studies will want to see exactly how these criteria are applied. Nevertheless, the book will force most readers to move beyond the well-known arguments and debates in their own field to gain insights from the ideas that preoccupy scholars who travel less familiar intellectual paths.
Explanation and Progress in Security Studies is a rare work in the study of international relations. It is a significant and original manuscript that should be read and seriously considered by everyone who fashions himself or herself as a theorist, regardless of their preferences in today’s intellectual fashions.
Author’s Response by Fred Chernoff, Colgate University
I am very grateful to have the opportunity to discuss the comments of four distinguished scholars of International Relations (IR) and security studies. I thank the contributors and H-Diplo. The participants in this forum have done an excellent job of reading and digesting the core argument of Explanation and Progress. It is gratifying to see the level of support and appreciation the contributors offer, both for the core question that the book investigates and for the argument I present. And it is a pleasure to engage the comments and criticisms that they present.
The comments, not surprisingly given their authors, are thoughtful and bring up important issues. Some comments identify topics that social science and IR debates should pursue in ways that Explanation and Progress specifically recommends (Jérémie Cornut, Tuomas Forsberg). Some take the recommendations of the book farther, which might be desirable but might also be more than most IR scholars are likely to do (James Wirtz). And some identify important questions that are of interest to security studies and IR scholars but are distinct from the core argument of Explanation and Progress (Cornut, Ewan Harrison, Wirtz).
The responses below require that we remember that Explanation and Progress accepts a non-skeptical view of IR, according to which there exists a coherent notion of scientific or intellectual progress. Any discipline capable of progress must have procedures that enable scholars who disagree to distinguish true from false claims, to separate stronger from weaker theories, and to move toward agreed-upon answers (1-2, 24, 266-268). Accordingly, it is possible to produce knowledge of, or at least to provide justification for, statements in at least some areas of IR. Knowledge in this context might i) provide justification to accept new statements, ii) provide justification to reject previously accepted statements, or iii) provide justification beyond what was previously available to endorse statements already accepted.
At first glance this might sound like a narrow and overly-positivist view of social science knowledge. But it is rather, in my view, an extremely minimalist view that would be hard to reject by anyone but a genuine skeptic who doubts social enquiry. It might further lead to knowledge, and to the view that we might know more in a year than we do today. To see this we only need consider what happens if we deny i) – iii). According to this simple reductio argument, imagine that someone asserts that a new discipline should be taken seriously as a natural or social science but admits that inquiry can neither add to what we know nor remove mistaken beliefs, and, furthermore, that there are no procedures for figuring out which side of a theoretical dispute has the stronger position. If this assumption is rejected, then advancing new knowledge disappears as a reason to do research, and learning anything new disappears as a reason to read published work. If all IR scholarship is driven by a desire to gain disciplinary rewards, or to appeal to the emotions or unfounded prejudices of a group of potential readers, then gaining knowledge is not part of what we in IR do.
Jérémie Cornut has two main “disagreements” with the book. Both pertain to the sociology of knowledge and competition between paradigms and “are linked to the influence of [Thomas] Kuhn’s work.” Cornut argues that Explanation and Progress encourages “paradigm wars.” However, the book rejects a view of social science that follows a Kuhnian paradigm-driven view of science, e.g., stating that “the social sciences do not exhibit the same patterns of inquiry” as the natural sciences (14). The only Kuhnian point in the book is a very specific one—about how new scholars learn what the most important works are in a debate, which includes identifying key issues and state of the art methods. Beyond that, the book specifically disavows any acceptance of Kuhn’s view of science and progress and its related application to the social sciences. The term ‘paradigm’ does not even appear after the disavowal in Chapter 1.
The alleged reliance on paradigmism of Explanation and Progress leads Cornut to suggest that the book works against methodological pluralism. However, in the book I write, “methodological pluralism is an assumption of this study” (24; see also 252). The only limitation the book imposes on permissible methods of study is that there must be some coherent argument able to provide a rationale.
Cornut emphasizes the importance of focusing on the sociological and political reasons that lead scholars to endorse various conclusions. But the questions that Cornut raises about the sociology of knowledge constitute a project that is simply different from the one than was undertaken in Explanation and Progress. One can look at the types of evidence and forms of inference that guide scholars to determine what the best answer is to a question, and/or one can look at the social forces that operate over the debate. The former is the subject of the book and the latter is not. Part of social science involves a rational process of formulating hypotheses and theories and evaluating them in light of experience and evidence. This is the part of IR that the book addresses. And while scholars may respond to professional incentives (see also Harrison’s comments), I believe that social forces do not deterministically shape scholarly conclusions: scholars sometimes, perhaps even often, work against disciplinary incentives and social forces to forge new knowledge.
A final point on the paradigm discussion involves Cornut’s claims that Explanation and Progress treats theories in a way that excludes the possibility of combining explanations. The book examines many works in which the best answers are precisely a combination of different answers. In the first debate examined, nuclear proliferation, nearly all of the non-realist approaches involve combinations of different types of explanation (individual psychology, regime success strategies, system-wide norms, bureaucratic “imperatives,” etc.).
Cornut raises the question of the nature of the 33 works in the case studies. The book looks at a sample of 10-12 works in each of three core debates in security studies. As Cornut observes, all of those works are, broadly, in an empirical-positivist vain; none adopts an approach that is thoroughly constructivist, Marxist, post-structuralist, etc. Although many different empirical methods are used in the works studied, e.g., statistical modeling, individual psychology, political economy, etc., it was a surprise to me there were no constructivist or interpretivist arguments in the most influential works, based on my criteria (4-6, 24). The grounds for choosing the most influential works were set out well before any of the 33 publications were analyzed. There was no effort to exclude interpretivist-reflectivist, post-structural, or other sorts of publications. The survey of a range of natural and social science concepts of explanation that comprises chapter 2 is there to support discussion of non-empiricist methods that may be undertaken in future studies, and to provide perspective on what the 33 works studied have in common.
The book deals with issues of justification and grounds for accepting new conclusions in security studies. The intended audience includes scholars who are interested in the question of acquiring more fully justified beliefs and epistemic progress. So when Cornut says that “IR scholars with different epistemologies rarely engage one another in good faith” he refers to scholars who are outside of the intended readership. Those who (self-consciously) lack an interest in what it takes to justify beliefs to any potential reader, whatever her/his epistemology, are not part of the intended audience of Explanation and Progress. Indeed, if scholars should find out that some theoretical disagreement flows from a difference of epistemology, then the next stage of enquiry should move to debate which is the more justifiable epistemology; and this is exactly what Explanation and Progress recommends. Even if further debate does not occur, it is at least clear that the disagreement is not over substance but philosophy. Cornut asserts that many scholars do not want to be clear about the criteria they use, avoiding “honest engagement” with interlocutors.
Tuomas Forsberg’s contribution offers some important insights about progress in the social sciences. He points out that in Explanation and Progress it is possible to accept simultaneously different sorts of explanations of a phenomenon because different researchers may investigate different aspects of an event or process. And, decisions on which criteria are most appropriate and relevant for a particular research question will be shaped in part by the specific interests of the researcher community. But Forsberg argues that the book overlooks the fact that sometimes the same criterion may be interpreted in different ways. This is an excellent point, since some of the criteria, e.g., simplicity, are discussed by so many authors that the criteria have been characterized in somewhat different ways. Following Forsberg’s observation, it would seem wise for IR scholars not only to name the criterion they use but also to cite specific authors who have provided definitions or analyses of the criteria. Forsberg is also correct in pointing out the criteria that are ideal for investigating one sort of question may be different from those in other sorts of question. It follows, then, that IR scholars must understand i) the criteria, ii) the different ways in which the criteria may be interpreted, and iii) that they must be specified properly for each research project (rather than specified once-and-for-all for all research in the field).
Forsberg states that Explanation and Progress leaves the task of finding good philosophical foundations for social science to philosophers. This deserves clarification. The interpretation is somewhat of an overstatement, at least of the intent of the book. IR scholars who seek to advance knowledge must engage, understand, and cite the work in the philosophy of social science. IR scholars should have a solid grasp of the criteria they use and the reasons for using them in particular enquiries. When debate over a theoretical question is seen as hinging on the philosophical and criterial choices, IR scholars should be able to defend the choices they make. It is not feasible for IR scholars to provide arguments for each criterion they use in every publication. And we note here that the when IR researchers make use of methods of study (genealogical inquiry, agent-based modeling, etc.) we rely on the justifications offered by others (Michel Foucault, Thomas Schelling),[35] and need not be defended anew from the ground up in each case. We may view the grounds for using specific sets of criteria in a strictly parallel way.
Forsberg highlights a limitation of the framework of the book by nothing the difficulties involved in approach-to-consensus on questions about very recent events, like the crisis in Ukraine. Explanation and Progress does not claim that all questions are capable of progress (e.g., essentially normative questions), and it does not claim that those that are proper subjects are amenable at every stage as events are unfolding. The study of very recent events may involve evidence claims that are still hard to corroborate, and it may be impossible to unify a set of evidence statements at early stages, since different researchers must rely on still-speculative evidence claims.
Forsberg also correctly notes that I do not present a set of arguments to support the criteria I endorse, but where the criteria are stated, a citation is provided for my more comprehensive metatheoretical book, The Power of International Theory.[36] The book is not an attempt to present a general philosophy of science. The fewer the narrowing assumptions a scholarly work has to make, the wider the audience that is obligated to accept valid inferences (262-64); for that reason, my argument is structured to be consistent with as many doctrines as possible in the philosophy of science.
In Explanation and Progress I identify the criteria I use in order to defend the core hypotheses about necessary conditions for progress. But Forsberg says that in doing so I do not order the criteria. I have three comments on this point. First, while it would be helpful for all scholars both to identify and to order the criteria they use, for the purposes of approach-to-consensus, it is much more important to identify them. Second, while naming and ranking would probably enhance progress, a requirement that authors do both seems impracticable and seems more likely to lead researchers simply to bypass the recommendations of Explanation and Progress. And third, while it is true that I do not rank all of the criteria I mention relative to one another, I do place them in three categories: those most highly prioritized, those used but at a lower level when trade-offs are required with the first group, and those that are not relied upon in the book (60-61, see also comments below on James Wirtz’s contribution).
Ewan Harrison’s very generous praise is followed by two critical comments. One is that the Democratic Peace (DP) research program in dyadic form is, at this point, making only incremental progress. The other is that sociological and professional incentives are driving DP research rather than prospects for successful social scientific progress.
On the first point, Harrison argues that the dyadic DP work has reached a point of diminishing returns. He states that, in his opinion, in the future systemic analysis will become increasingly important. This may well be the case. But an absence of fresh advances upon a widely-accepted theoretical position does not cast doubt on the claim that recent decades’ progress is real. The central question of the book is about current debates in security studies, asking whether overlap of criteria by opposing groups of scholars facilitates movement toward agreement on the best theoretical answer. What may happen in the future is of course of great importance for IR scholars. But a correct forecast answers a different question from the one posed in the book.
In Explanation and Progress I argue that when new approaches, like systemic theorizing, are investigated more intensively, movement toward agreement on the best systemic theory is more likely if opposing authors use a common set of criteria of theory evaluation. And if those authors’ disagreement includes divergence in their criterial preferences, then the debate should turn to focus on the appropriate criteria to answer that particular question.
Harrison argues that systemic analysis, of the sort he and others (e.g., Gartzke and Weisiger 2013) have developed, holds more promise for future intellectual gains than the dyadic approach. If this is so, then it represents an advancement of knowledge.[37] On that view the transition to a mostly democratic world raises the prospect of war between liberal democracies. As Explanation and Progress emphasizes, consensus on a theoretical answer (like liberalism) to a question (like whether, and, if so, why, pairs of democracies are less war-prone than other dyads) in no way implies that the question has been answered once and for all. New events occur, and new evidence is discovered about the past. This is precisely parallel to the natural sciences, where theoretical advances of the most dramatic and transformative nature, however well-corroborated, do not allow us to cease seeking new evidence or to conclude that the question has been answered once and for all.
Disciplinary incentives may in part drive what some people choose to write about, and perhaps even what conclusions they seek. But as long Harrison does not claim that all published work is determined by those incentives, then it is still possible to judge research on key debates, like democratic peace, on their intellectual merits.
Harrison asserts that the analyses of the three case study debates in the book are backward rather than forward looking. However, an assessment of how progress has been made necessarily requires looking at debates as they have proceeded up until now. This can tell us how we can achieve progress (or specific essential components thereof, such as approach-to-consensus). The question of where future effort will be most fruitful is a distinct question. Choosing new directions of research requires creativity and foresight (and luck). Only after research designs in proposed new areas are executed is it be possible to know if progress has been achieved. But in system-level DP studies, or whatever new area or question people in the field pursue, if scholars make explicit the criteria they use, then, as I argue in Explanation and Progress, there is a much greater likelihood of achieving progress.
The point of the chapter on DP studies in Explanation and Progress is not to defend the fact that the most frequently cited scholars have approached the subject dyadically rather than systemically, nor even to argue that the dyadic behavioral hypothesis and the liberal-democratic explanation is correct, true and firmly established for all time. The book clearly states that it does not endorse the view that any answers should be accepted as permanent and beyond challenge. Progress occurs by knowledge-seeking scholars converging on the best answer to a question, given the evidence that is available at the time any particular study undertaken. In the case of the debate over the causes of nuclear proliferation, it was important in 1960s to recognize the persuasive power of realist explanations. But as new events unfolded (like the emergence of the norms forged under the NPT), the recognition that realist power-based explanations do not hold in all cases constituted an advance in knowledge. Thus, while I claim that DP studies represent progress, new evidence might change what is accepted by scholars who seek the most well-founded conclusions (and not those driven to answers by ideology or socio-political pressure).
James Wirtz points out that there is no general agreement in IR about what is to be explained, how explanations should be formulated, and how to compare the relative values of contending explanations. But he does come down clearly on the side that progress in IR is real. Wirtz’s examples cleverly show that some past explanations are, for good reason, no longer accepted by any school of thought. And they show that we should be cautious about how much progress we claim: progress over time has been modest. Wirtz brings to the fore the fact that Explanation and Progress aims to be applicable to forms of argument that are not parallel to natural science enquiry. Interpretivist, reflectivist, historical case study, and other types of methods are perfectly capable of improving our knowledge or the depth of the justification for our IR beliefs in accordance with the assumption in paragraph 6.
The principal criticism Wirtz offers stems from his observation that the book is intended to satisfy both security-studies scholars as well as philosophers of science interested in IR and the social world. He argues that security studies scholars will want to see how the criteria are applied, while philosophers of science would want to see defenses of the criteria. These are good points that deserve a response. In terms of applications, there is a sketch in chapter six of the use to which the criteria are put. The criteria the book uses are listed (60-61): empirical fit, true causes and mechanisms, explanatory unification, falsifiability, and predictive accuracy. Two others used, but with lesser priority, are precision, and the elimination of alternatives. With regard to philosophical defenses of the criteria, that requires a broader meta-theoretical inquiry, which Explanation and Progress was not designed to provide. It does, however, direct the reader to other works, e.g., The Power of International Theory, in which I have offered the sorts of principles Wirtz has in mind.
Notes
[1] As James Wirtz notes in his review, the difficulty of the task is increased by the fact that Chernoff is writing for two audiences, philosophers of science and security studies scholars. There is also a practical issue. As Chernoff notes and as Tuomas Forsberg highlights in his review, space constraints in journals and other publication outlets often prevent scholars from engaging in extensive discussions of criteria for evaluating theories.
[2] Particularly notable existing works are Colin Elman and Miriam Fendius Elman, eds., Progress in International Relations Theory: Appraising the Field (Cambridge: MIT Press, 2003), which adopts a Lakatosian framework; Patrick Thaddeus Jackson, The Conduct of Inquiry in International Relations, 2nd ed. (London: Routledge, 2016), which presents a non-positivist perspective; and Annette Freyberg-Inan, Ewan Harrison, and Patrick James, eds., Evaluating Progress in International Relations (New York: Routledge, 2016), which provides a useful collection of different perspectives. Focusing on structural realism is Patrick James, International Relations and Scientific Progress: Structural Realism Reconsidered (Columbus: Ohio State University Press, 2002). See also Imre Lakatos, “Falsification and the Methodology of Scientific Research Programmes,” in Imre Lakatos and Alan Musgrave, eds., Criticism and the Growth of Knowledge (New York: Cambridge University Press, 1970), 91-196.
[3] Forsberg adds that the problem is compounded by “the strong politicized habit of reading research outputs backwards from policy recommendations.”
[4] Thomas Kuhn, The Structure of Scientific Revolutions (Chicago: University of Chicago Press, 1970).
[5] Arguments about national differences in the study of international relations have been advanced by Stanley Hoffmann, “An American Social Science: International Relations,” Daedalus 106:3 (Summer, 1977): 41-60; Ole Wæver, “The Sociology of a Not So International Discipline: American and European Developments in International Relations,” International Organization 52:4 (October 1998): 687-727, DOI: https://doi.org/10.1162/002081898550725.
[6] John Gerard Ruggie, Constructing the World Polity: Essays on International Institutionalization (New York: Routledge, 1998).
[7] On the intellectual history of debates within the International Relations field see Brian C. Schmidt, “On the History and Historiography of International Relations,” in Walter Carlsnaes, Thomas Risse, and Beth A. Simmons, eds., Handbook of International Relations (Thousand Oaks: Sage, 2014): 3-28; Yosef Lapid, “The Third Debate: On the Prospects of International Theory in a Post-Positivist Era,” International Studies Quarterly 33:3 (September 1989): 235-254.
[8] James Lee Ray, Democracy and International Conflict: An Evaluation of the Democratic Peace Proposition (Columbia, South Carolina: University of South Carolina Press, 1995), 44. For an argument regarding the primarily evidence-driven nature of the democratic peace research program see Jack S. Levy, “Theory, Evidence, and Politics in the Evolution of Research Programs,” in Richard Ned Lebow and Mark Lichbach, eds., Theory and Evidence in Comparative Politics and International Relations (New York: Palgrave Macmillan, 2007), 177-197.
[9] The emphasis on multi-method research designs goes back as least as far as David Laitin, “Comparative Politics: The State of the Subdiscipline” in Ira Katznelson and Helen V. Milner, eds., Political Science: The State of the Discipline (New York: Norton, 2002), 630-659.
[10] One of many examples is the heated debate between proponents of the democratic peace and the “capitalist peace.” Gerald Schneider and Nils Petter Gleditsch, Assessing the Capitalist Peace (London: Routledge, 2012).
[11] Isaiah Berlin, “History and Theory: The Concept of Scientific History,” History and Theory 1:1 (1960), 9.
[12] On differences between diplomatic history and international relations scholarship see Jack S. Levy, “Explaining Events and Testing Theories: History, Political Science, and the Analysis of International Relations,” in Colin Elman and Miriam Fendius Elman, eds., Bridges and Boundaries: Historians, Political Scientists, and the Study of International Relations (Cambridge: MIT Press, 2001), 39-83.
[13] Harrison has already made contributions in this area, in Ewan Harrison, “The Democratic Peace Research Program and System Level Analysis,” Journal of Peace Research 47:2 (2010): 155-165. DOI: https://doi.org/10.1177/0022343309356490.
[14] Stuart A. Bremer, “Dangerous Dyads: Conditions Affecting the Likelihood of Interstate War, 1816-1965,” Journal of Conflict Resolution 36:2 (June 1992): 309-341.
[15] Benjamin Fordham and Paul Poast, “All Alliances are Multilateral: Rethinking Alliance Formation,” Journal of Conflict Resolution 60:5 (August 2016): 840-865. DOI: https://doi.org/10.1177/0022002714553108.
[16] Levy, “Theory, Evidence, and Politics”; Karl R. Popper, Conjectures and Refutations (New York: Basic Books, 1962).
[17] On the bargaining model of war see James D. Fearon, “Rationalist Explanations for War,” International Organization 49:3 (Summer 1995): 379-414.
[18] John A. Vasquez and Marie T. Henehan, Territory, War, and Peace (London: Routledge, 2011).
[19] Thomas Kuhn, The Structure of Scientific Revolutions (Chicago: University of Chicago Press, 1970).
[20] Ole Wæver, “The Sociology of a Not So International Discipline: American and European Development in International Relations,” International Organization 52:4 (1998): 687-727, DOI: https://doi.org/10.1162/002081898550725; Christian Bueger, “From epistemology to practice: a sociology of science for international relations,” Journal of International Relations and Development 15:1 (2012): 97-109, DOI: https://doi.org/10.1057/jird.2011.28; Inanna Hamati-Ataya, “IR Theory as International Practice/Agency: A Clinical-Cynical Bourdieusian Perspective,” Millennium – Journal of International Studies 40:3 (2012): 625-646, DOI: https://doi.org/10.1177/0305829812442234; Kristensen, Peter Marcus Kristensen, “Discipline Admonished: On fragmentation in International Relations and the disciplinary politics of stocktaking,” European Journal of International Relations 22:2 (2016): 243-267, DOI: https://doi.org/10.1177/1354066115586206.
[21] Pierre Bourdieu, “The specificity of the scientific field and the social conditions of the progress of reason,” Social Science Information 14:6 (1975): 32.
[22] John Mearsheimer, “Why the Ukraine Crisis Is the West’s Fault. The Liberal Delusions That Provoked Putin,” Foreign Affairs Online (September/October 2014), http://www.foreignaffairs.com/articles/141769/john-j-mearsheimer/why-the-ukraine-crisis-is-the-wests-fault.
[23] Michael McFaul, Stephen Sestanovich and John J. Mearsheimer, “Faulty Powers. Who Started the Ukraine Crisis?” Foreign Affairs Online (November/December 2014), http://www.foreignaffairs.com/articles/142260/michael-mcfaul-stephen-sestanovich-john-j-mearsheimer/faulty-powers.
[24] Robert Legvold, “Managing the New Cold War. What Moscow and Washington Can Learn From the Last One,” Foreign Affairs (July/August 2014): 74-84.
[25] Fred Chernoff, The Power of International Theory: Reforging the Link to Foreign Policy Making Through Scientific Inquiry (New York: Routledge, 2006); Fred Chernoff, Theory and Meta-Theory in International Relations: Concepts and Contending Accounts (New York: Palgrave, 2007).
[26] Patrick James, International Relations and Scientific Progress: Structural Realism Reconsidered (Columbus: Ohio State University Press, 2002); Colin Elman and Miriam Fendius Elman, eds., Progress in International Relations Theory: Appraising the Field (Cambridge: MIT Press, 2003).
[27] Patrick Thaddeus Jackson, The Conduct of Inquiry in International Relations, 2nd ed. (London: Routledge, 2016); Annette Freyberg-Inan, Ewan Harrison, and Patrick James, eds., Evaluating Progress in International Relations. (New York: Routledge, 2016).
[28] Pierre Duhem, The Aim and Structure of Physical Theory. Translated by Philip P. Wiener, (Princeton: Princeton University Press, 1991); Imre Lakatos (John Worrall and Gregory Currie, eds.), The Methodology of Scientific Research Programmes: Philosophical Papers Volume 1 (Cambridge: Cambridge University Press, 1980).
[29] Thomas Kuhn, The Structure of Scientific Revolutions (Chicago: University of Chicago Press, 1962.
[30] Annette Freyberg-Inan, Ewan Harrison, and Patrick James, eds., Evaluating Progress in International Relations (New York: Routledge, 2016).
[31] John J. Mearsheimer, and Stephen M. Walt, “Leaving theory behind: Why simplistic hypothesis testing is bad for International Relations,” European Journal of International Relations 19:3 (2013): 427-457. DOI: https://doi.org/10.1177/1354066113494320.
[32] Erik Gartzke and Alex Weisiger, “Permanent Friends: Dynamic Difference and the Democratic Peace,” International Studies Quarterly 57:1 (2013): 175-185, DOI: https://doi.org/10.1111/isqu.12046; Erik Gartzke and Alex Weisiger, “Fading Friendship: Alliances: Affinities and the Activation of International Identities,” British Journal of Political Science 43:1 (2013): 25-52. DOI: https://doi.org/10.1017/S0007123412000208.
[33] Allan Dafoe, John R. Oneal, and Bruce Russett, “The Democratic Peace: Weighing the Evidence and Cautious Inference,” International Studies Quarterly 57:1 (March 2013): 201–214, DOI: https://doi.org/10.1111/isqu.12055; Ewan Harrison, and Sara McLaughlin Mitchell, The Triumph of Democracy and the Eclipse of the West (New York: Palgrave, 2014); Crescenzi, Mark J.C. and Kelly M. Kadera, “Built to Last: Understanding the Link Between Democracy and Conflict in the International System,” International Studies Quarterly 60:3 (2016): 565-572, DOI: http://dx.doi.org/10.1111/isqu.12215; Seung-Whan Choi, “A Menace to the Democratic Peace? Dyadic and Systemic Difference,” International Studies Quarterly 60:3 (2016): 573-577, DOI: http://dx.doi.org/10.1111/isqu.12214.
[34] Ewan Harrison, “The democratic peace research program and system-level analysis,” Journal of Peace Research 47:2 (2010): 155-165, DOI: https://doi.org/10.1177/0022343309356490; Ewan Harrison, “Rethinking the Democratic Peace: Competing Accounts of ‘Scientific Progress’ in IR,” in Annette Freyberg-Inan, Ewan Harrison and Patrick James, eds., Evaluating Progress in International Relations (New York: Routledge, 2016).
[35] Michel Foucault, The Archaeology of Knowledge and Discourse on Language, tr. A.M. Sheridan Smith, (New York: Pantheon Books, 1971); Thomas C. Schelling, “Dynamic Models of Segregation,” The Journal of Mathematical Sociology. 1:2 (1971): 143-186. DOI: http://dx.doi.org/10.1080/0022250X.1971.9989794.
[36] Fred Chernoff, The Power of International Theory: Reforging the Links to Policy though Scientific Enquiry (London: Routledge, 2005).
[37] Ewan Harrison, The Post-Cold War International System (London: Routledge, 2004.); Erik Gartzke and Alex Weisiger, “Permanent Friends? Dynamic Difference and the Democratic Peace,” International Studies Quarterly 57:1 (2013): 171-185. DOI: https://doi.org/10.1111/isqu.12046.