ASQ Research Listserver Volume 2, Number 1, January 24, 1996 Current subscribers: 429 General listserver guidelines: 1. Postings and subscription requests: asq@umich.edu 2. Web archive: http://www.umich.edu/~asq 3. Please reply to the moderators, rather than to all subscribers. In this issue: 1. Moderator comments 2. What Theory is...: A Reaction to Sutton and Staw's Essay ``What Theory is Not'' (Robert I. Sutton and Barry M. Staw, Administrative Science Quarterly, September 1995, volume 40, no. 3, pp. 371-384 ) From Michael Masuch, Jeroen Bruggeman, Jaap Kamps, Gabor Peli, and Laszlo Polos (office@ccsom.uva.nl.) ------------------------------------------------------------------------- 1. Moderator comments We look forward to your comments concerning the following discussion of Sutton and Staw's piece in the September 1995 edition of ASQ. --------------------------------------------------------------------------- 2. What Theory is...: A Reaction to Sutton and Staw's Essay ``What Theory is Not'' (Robert I. Sutton and Barry M. Staw, Administrative Science Quarterly, September 1995, volume 40, no. 3, pp. 371-384 ) What Theory is...: A Reaction to Sutton and Staw's Essay ``What Theory is Not'' Michael Masuch, Jeroen Bruggeman, Jaap Kamps, Gabor Peli, and Laszlo Polos Address correspondence to: Center for Computer Science in Organizations and Management University of Amsterdam Saphatistr. 143, 1018 GD Amsterdam, The Netherlands Tel:+31-20 5252852, Fax:+31-20 5252800 Email: (office@ccsom.uva.nl.) In their ASQ-essay ``What Theory is Not'' Sutton and Staw (1995) paint a fairly bleak picture of theory-building in the administrative sciences. Students of organizations, they observe, are getting by with ``weak'' theory (or none at all), provided they can muster enough empirical credibility for their contributions. To encourage ``stronger'' theory, Sutton and Staw argue, journals should ``reconsider their empirical requirements, and ought to be more receptive to papers that test part rather than all of a theory and use illustrative rather than definitive data.'' We agree with their observations, but largely disagree with their diagnosis and prescription. (1) Their indirect approach (``what theory is not'') leaves their concept of theory implicit, when it should be made explicit. (2) Once it is made explicit, it becomes clear that theorizing and theory are two different things: theorizing is a process in the context of (scientific) discovery; theory is a vehicle for (scientific) justification. (3) The distinction between ``strong'' and ``weak'' theory confounds several criteria that should be kept apart---in particular logical, empirical, and relevance criteria. (4) Only logically sound theories are ready for empirical testing; theories with logical deficiencies require other measures. Logical deficiencies, are, alas, a common feature of present-day organizational theory, even of relevant or otherwise interesting theory. As a consequence, (5) rather than weakening the empirical requirements for interesting theories, journals should strengthen the logical requirements for all theories. In sum, if theory-building is the issue, journals should address it head-on. According to Sutton and Staw, too much ``weak'' theory is submitted to journals. Worse, many authors don't seem to know what theory is, since they are submitting references, data, variables, diagrams, and hypotheses instead of real theory. Sutton and Staw do not define their concept of theory, but when arguing what theory is not, they have to invoke an implicit concept of theory nonetheless. It turns out that theory is present when there is a ``stream of logic'', or an ``explication of causal logic'', a ``discussion of logical arguments'', ``logical reasoning'', ``causal reasoning'', or ``causal arguments that are abstract and simple enough to be applied to other settings''. The resultant picture reveals a concept of theory that goes back to Aristotle and has barely changed: by approximation, theory is a set of meaningful statements that expose logical structure. What has changed is the requirement of empirical testing (Renaissance, Bacon, and the consequences), and, more recently, the understanding of the role of logic (Tarski, 1956; Popper, 1959). Logic is needed to justify a theory (e.g., to justify the soundness of an explanation), but it is rarely the medium of theorizing. Thinking up theories is done ``in mental models'' (Johnson-Laird and Wason, 1997; Johnson-Laird, 1983; Johnson-Laird and Byrne, 1991), and proceeds mainly via thought experiments, very much as Weick describes them in his comments (1995: 388). Discovery and justification do indeed go hand in hand, since the justification may fail (necessitating more theorizing), or may succeed (encouraging more theorizing); the interaction between discovery and justification amounts to theory-building. Still, discovery and justification are conceptually different activities, invoking different mental machinery, and different forms of expression. In particular, the mental models of theorizing can remain private and subjective, but the theory must not; it must be presented in a form that supports communication with others. Thus the bottom line requirement is that theories are clearly expressed. Along with clear expression, theories must be consistent, contingent, and sound, and formal logic provides precise criteria for these properties. Inconsistent theories ``cannot have a model''; there is, strictly speaking, no reality to which they could correspond. Non- contingent theories are not falsifiable, and hence not worth testing empirically unless they contain not general statements. Finally, theories with unsound explanations suggest invalid propositions for empirical testing. Logical and empirical criteria are related, but they are not identical. Keeping them apart avoids confusion---and saves money, since logical testing does not require expensive data-collection. Furthermore, empirical tests depend on the operational definition of theoretical concepts for measurement; such definitions are often problematic, especially in the social sciences. Testing logically deficient theories can set off a cycle of superstitious ``learning'', where operational definitions are modified so as to make logically deficient theories testable. Logical testing cannot replace empirical testing, but it should precede it. Along with logical and empirical criteria, there are pragmatic criteria for the success of a theory. Preferably, theories should be relevant, interesting. Relevance is highly contextual, depending on time, place, forum, reputation, etc. It may elude formal testing, yet overlap with logical and empirical issues (e.g., elegance, parsimony, generality). Still, its context dependence makes relevance a thing apart, so one should distinguish between the three sets of criteria, particularly when theory in question appears ``weak'', and its authors need help and direction. Now, according to Sutton and Staw, many authors indeed do need help---an observation we can confirm on the basis of our own, though more limited, editorial experience. But we disagree with their prescription (relax the requirements of empirical testing for potentially ``strong'' theories), because it treats only symptoms. If the cause of weak theory is poor or uneducated theory-building, then theory- building itself should be made the issue, not only in the correspondence between editors and authors (the back-alleys of science), but also in the published works. Theory-building an important and difficult issue, not a side issue to be hidden from the public. Consider two classical examples of theory-building in a simple domain, geometry. Is the fifth postulate in Euclid's geometry independent or not? {Note: This postulate says that a straight line s and a disjoint point p determine exactly one straight line s' that goes through p but has no common point with s. This postulate is intuitively less obvious than the other postulates, and so it was thought that it may be derivable from the other postulates. All these attempts failed, even though they resulted in several interesting equivalent formulations of the postulates, such as: any three points are either on a straight line or on a circle. The problem was finally settled when Bolyai, Lobachewsky, and Gauss realized that the negation of the fifth postulate, together with the other four, has geometric models as well, e.g., the surface of a sphere} This question remained open for more than 2000 years, and occupied more scholars than ever practiced organization theory, but when it was finally settled in the 19th century, it opened avenues for non-Euclidean geometries that are, for instance, at the basis of relativity in physics. A second example from geometry: Euler's polyhedron theorem was flawed in its original formulation{Note: The theorem says that the number of vertices of a polyhedron, plus the number of faces minus the number of edges is always equal to 2}, since his definition of a polyhedron allowed for counterexamples. The theorem was proposed in 1758, and it took a long series of stricter and stricter definitions until a definition was finally suggested in 1899 that would satisfy Euler's theorem (Lakatos, 1976). Consider some other examples closer to home. Arguably, the theory of Organizational Ecology (OE) (Hannan and Freeman, 1989) depends on the First Theorem of organizational inertia (inert organizations carry an evolutionary advantage above flexible organizations) because this theorem establishes the crucial analogy between natural and organizational selection (Hannan and Freeman, 1984). Is this theorem correct, that is, does OE actually prove this theorem? No, the reasoning in (Hannan and Freeman, 1984) is logically incorrect; it confuses necessary and sufficient conditions (Peli et al., 1994). A second example: the relative fitness of generalist vs. specialist organizations is predicted in niche-width theory (Hannan and Freeman, 1997; Freeman and Hannan, 1983; Freeman and Hannan, 1987). The predictions are based on a mathematical model in which two dimensions (trait y, environment) are confused, and the verbal argument provides insufficient assumptions to derive the theory's conclusions (Bruggeman, 1995; Bruggeman and O Nuallain, 1995). A last example: as is well- known, OE strives to avoid rational, circumspect agents as explanatory factors of organizational structure. Can one avoid invoking such agents when the relative success of ``efficient producer'' organizations vs.``first mover'' organizations is concerned? No---at least not if one sticks to the common operationalization of the size of organizational populations (Peli and Masuch, 1994; Peli and Masuch, forthcoming). We are not trying to pick on Organizational Ecology. We chose OE as a target of logical analysis because of its relevance, and because it appears to provide relative solid theory; having analyzed various other theories, we have no reason to revise our evaluation. All other theories that we analyzed in the last couple of years Polos, 1994; Masuch and Huang, 1994; Polos, 1995) also exhibited, on closer inspection, logical problems---often problems that would not be solved by minor changes, that would be difficult to detect by empirical testing (and certainly not by relaxing the requirements of empirical testing), and that require active theory-building. Logical problems are not unique to the social sciences, as the examples from geometry show. Arguably, geometry constitutes a much simpler domain than the social sciences, so one may expect logical problems to be more severe and more widespread in our field. Do not blame the social sciences for this; blame social scientists for ignoring, or sidestepping, the issue. In fact, there is some evidence that logical problems receive even less attention now than they did 30 years ago---perhaps because the armchair social sciences went into a crisis at that time, while large-scale empirical testing, newly possible through computers, was widely seen as the way out. Thirty years later, explicit theory-building has largely ceased; instead, we have the situation described by Sutton and Staw, with ``weak'', theory dominating the pages of social-sciences journals. Many social scientist have accepted this situation, having convinced themselves that the ``softness'' of their field is either unavoidable, or prejudiced talk, or both. While the ``stats'' of modern research have become very advanced, the logic is still done superficially, if at all. And who cares? Why has nobody ever discovered that trait y and environment are confused, even though the first paper containing the error is one of the most widely cited articles in the social sciences (Clemens et al., 1995). Or perhaps somebody did, but couldn't get the results published.{Note: The flawed model was published in AJS three times. Our mathematical analysis of this model was rejected by AJS on the ground that it was a research note, and, as the editor wrote, ``we do not publish research notes.''} Who does not have a close colleague who has published a paper in a leading journal with an important table completely misprinted and nobody ever says a thing? How can we build theory under such conditions? How can we build theories on theories and create cumulative knowledge? The discourse in our field is still one of scientific progress, but how can we know whether scientific progress occurs if newer theories may be just as shaky as the older ones that they are supposed to replace? Weakening the empirical requirements for potentially strong theory is not the way out. Theory building must be addressed head on; it should be allowed to take a prominent position in published contributions, and allowed (again) to become a legitimate subject in its own right, not only a side issue of empirical confirmation. Whether the journals will endure such a challenge is a different matter, of course. The institutional pressures may not encourage theory- building as much as the institutional rhetoric may suggest. Theory- building is necessary, but it is an academic affair, usually boring, seldom ``interesting'' for a broader public. To encourage theory- building, journals may have to publish papers about theory building. Let's hope someone will find the time to read them, if only in the elevator on the way down to the next executive class. Acknowledgments Thank you to Scip Garling and Babette Greiner for their comments on an earlier draft. Bibliography Bruggeman, J. (1995). Niche width theory reappraised. Journal of Mathematical Sociology, forthcoming. Bruggeman, J. and O Nuallain, B. (1995). A niche width model of optimal specialization. Technical report, CCSOM, University of Amsterdam. Clemens, E.S., Powell, W.W., McIlwaine, K., and Okamoto, D. (1995). Careers in print: Books, journals, and scholarly reputations. American Journal of Sociology, 101:433--494. Freeman, J. and Hannan, M.T. (1983). Niche width and the dynamics of organizational populations. American Journal of Sociology, 88:1116--1145. Freeman, J. and Hannan, M.T. (1987). The ecology of restaurants revisited. American Journal of Sociology, 92:1214--1220. Hannan, M.T. and Freeman, J. (1977). The population ecology of organizations. American Journal of Sociology, 82:929--964. Hannan, M.T. and Freeman, J. (1984). Structural inertia and organizational change. American Sociological Review, 49:149--164. Hannan, M.T. and Freeman, J. (1989). Organizational Ecology. Cambridge, MA. Harvard University Press. Johnson-Laird, P.N. (1983). Towards a Cognitive Science of Language, Inference, and Consciousness. Cambridge University Press, Cambridge. Johnson-Laird, P.N. and Byrne, R.M. (1991). Deduction. Essays in Cognitive Science. Lawrence Erlbaum Associates, Publishers, Hillsdale USA. Johnson-Laird, P.N. and Wason, P., editors (1977). Thinking. Cambridge University Press, Cambridge. Lakatos, I. (1976). Proofs and refutations. The logic of mathematical discovery. Cambridge University Press, Cambridge, England. Masuch, M. and Huang, Z. (1994). A logical deconstruction of organizational action: Formalizing Thompson's organizations in action into a multi-agent logic. CCSOM Working Paper 94-120, Center for Computer Science in Organization and Management. Peli, G., Bruggeman, J., Masuch, M., and O Nuallain, B. (1994). A logical approach to formalizing organizational ecology. American Sociological Review, 59(4):571--593. Peli, G. and Masuch, M. (1994). The logic of propagation strategies: Axiomatizing a fragment of organizational ecology in first-order logic. In Moore, D.P., editor, Best Papers Proceedings of the 54th Annual Academy of Management Convention, Dallas, Texas, pages 218--222. Peli, G. and Masuch, M. (forthcoming). The logic of propagation strategies: Axiomatizing a fragment of organizational ecology in first-order logic. Organization Science. CCSOM Preprint 94-100. Polos, L. (1994). A formal approach to Gallhofer and Saris. Argumentation from the perspective of decision theory. In Proceedings of the IIId ISSA Conference. CCSOM Reprint 94-115. Polos, L. (1995). The metaphysics of organizations. Technical report, CCSOM, University of Amsterdam. Popper, K.R. (1959). The Logic of Scientific Discovery. Hutchinson, London. Tarski, A. (1956). Logic, Semantics, Metamathematics. Oxford University Press, New York.