Complex Systems, Complexity, Chaos Theory,Complex Adaptive Systems,Complexity and Strategy,Organisational Change,Self Organisation,Complex Systems and Knowledge Creation,Brain,Mind,Complex Systems Resources,Complexity and Chaos Resources,Organisational Form, Complexity Theory,Consultancy


T
hat's another fine mess you've got me into: The Value of Chaos in
Organisational Analysis.

Frances Storr, Humberside Training and Enterprise Council

Science and Organisational Analysis

The dominant paradigm in science has always been a key influence on the theories and frameworks we use to understand organisations. For example two key assumptions, derived from Newtonian science, which still predominate are reductionism and materialism. Reductionism is the essence of many organisational management approaches such as goal setting, appraisal systems, budgets and operational plans. The belief is that one can divide the organisations operational plan down into its component parts, allocate responsibilities, sum the resulting actions and the overall aims of the plan will be achieved. Materialism, ie emphasising things rather than the relations between things, is observed very commonly for example in organisations HR practices. Companies attempt to become more sophisticated at "measuring" people for the purpose of selection, appraisal and promotion. The emphasis on things rather than relationships is so strong that processes become reified and the words "budgets", "operational plans" and "appraisals" come to signify the piece of paper on which it is written rather than a cyclical process or a conversation. This is also apparent in recent developments in knowledge management which assume knowledge to be a thing which can be "captured".

Darwin's theory of evolution is another major scientific theory of our time which pervades our thinking and our thinking about organisations. It has given us terms like survival of the fittest and emphasises competition, incremental adaptation to ones environment, selfishness and survival as the driving forces in evolution. This same language pervades our models of organisations in their environments and concepts such as the Learning Organisation are based on the idea of evolving to adapt and respond to ones environment. The basis of the learning organisation theory is that the rate of learning within the organisation must be greater than or equal to the rate of change in the environment or the organisation is in decline. Complexity theories on the other hand suggest that the organisation and its environment are inseparable in that they are part of the same system, each affecting the other.

This paper describes a newly emerging field of science called complexity, which is a science of non linear systems, and explores its relevance to understanding organisations. Complexity theories represent a challenge to many of our current models for understanding the world such as the theories of Newton and Darwin.. These models have served us well for a long time but scientists are realising that they offer a very incomplete understanding of the world we live in. The launching of a spacecraft to land on the moon several days later relies on Newton's linear equations of motion and the same is true for fielders in a cricket match. Most of nature, however, is non linear and not easily predicted. Weather is a classic example: many components interacting in complex ways leading to notorious unpredictability. Ecosystems and economies are also examples of non linear systems which defy mathematical analysis or simulation. The more that is learned about complexity and complex evolving systems the more relevant it appears to be to our understanding of organisations and how they work.

The Science of Complexity

Complexity refers to the potential for emergent order in complex and unpredictable phenomena. The economy, ecologies, the human brain, developing embryos and ant colonies are all examples of complex evolving systems. Each of these systems is a network of many agents acting in parallel. In a brain the agents are nerve cells, in an ecology the agents are species, in an embryo the agents are cells. In an economy the agents might be individuals or households. If you were looking at business cycles the agents might be firms or if international trade the agents might be nations. In all of these systems, each agent finds itself in an environment produced by its interactions with the other agents in the system. It is constantly acting and reacting to what the other agents are doing. And because of that, essentially nothing in its environment is fixed.

The central discovery concerning non linear dynamical systems is that they can be driven by a set of simple sub processes. Chaos theory (Gleick 1988), is a central concept of non linear dynamic systems. Complexity theory is a wider field and a complex evolving system, the basis of the theory, can be represented as follows.

Characteristics of Complex Evolving Systems.

The components, or agents, in a system interact locally and these interactions may be governed by simple rules. From the interaction of the individual components comes some kind of global property or pattern, something you could not have predicted from what you know of the component parts. For example, in the brain, consciousness is an emergent phenomenon which comes from the interaction of the brain cells. Global properties flow from the aggregate behaviour of individuals. Furthermore, the control of a complex evolving system tends to be highly dispersed. There is no master neuron in the brain or master cell within a developing embryo. Any coherent behaviour in the system arises from competition and co-operation among the agents themselves. Even in an economy, the overall behaviour of the system is the result of myriad economic decisions made by millions of individual people.

Collectively, the research is indicating that complex evolving systems have a number of characteristics which are described below.

Connectivity


Complexity arises from the inter-relationship, inter-action and inter-connectivity of the elements within a system and between a system and its environment. This means that a decision or action by one element within a system will affect all other related elements but not in any uniform way. It is important to note that there is no dichotomy between a system and its environment. The notion to be explored is that of a system closely linked with all other related systems making up an ecosystem. Change needs to be seen in terms of co-evolution rather than adaptation to a separate and distinct environment. (Mittleton-Kelly 1997).

Co-evolution and Fitness Landscapes

Stuart Kauffman (1993) described this co-evolution by his notion of fitness landscapes. For a particular system - call it X - the fitness landscape covers the array of all possible survival strategies open to it. The landscape is made up of peaks and valleys, the higher the peak the greater the fitness it represents. Xs evolution can be thought of as a journey across the fitness landscape, the purpose being to find the highest peak. If the strategy is incremental improvement then X is likely to get stuck on the first peak it comes to as any subsequent steps will lead downhill. When system X changes its strategy other interconnected systems will respond and the landscape heaves about, changing constantly.

Positive feedback, sensitive dependence on initial conditions

Edward Lorentz studied the solutions to equations describing weather patterns and with the aid of a computer he traced out the solutions on a screen. Lorentz realised that he was dealing with a radically new type of behaviour pattern, that very small changes in initial conditions in the weather system can lead to unpredictable consequences, even though everything in the system is causally connected in a deterministic way. Knowing what the weather is now is no predictor of what it will be in a couple of days time because tiny disturbances can produce exponentially divergent behaviour.

The consequences of this mathematical discovery are enormous. Since most natural processes are at least as complex as the weather the world is fundamentally unpredictable in the sense that small changes can lead to unforeseeable results. This means the end of scientific certainty, which is the property of "simple" systems (the ones we use for electric lights, motors, electronic devices). Real systems, particularly living ones such as organisms, are radically unpredictable in their behaviour. Long term prediction and control, the hallmarks of the science of modernity, are not possible in complex systems. (Goodwin 1994).

Emergent order

For many years the second law of thermodynamics, that systems tend toward disorder, has been generally accepted. Ilya Prigogine's (1977) "dissipative structures" showed that this was not true for all systems. Some systems tend towards order not disorder and this is one of the big discoveries of the science of complexity.
Computer simulations of complex evolving systems demonstrate that it is possible for the order of new survival strategies to emerge from disorder through a process of spontaneous self organisation (Kauffman 1995). The order arises form non linear feedback interaction between agents where each agent "does its own thing" without any overall blueprint or prior programme. It seems that self organisation is an inherent property of a complex evolving system.

A readily observable example of emergent order is flocking behaviour in birds. Research using computer modelling has shown that one can model the flocking behaviour of birds by using a few simple rules such as the distance each bird maintains between itself and other birds and other objects (Reynolds 1987) What was striking about these rules was that they were entirely local. None of the rules said "Form a flock". If a flock was going to form at all it would have to do so from the bottom up, as an emergent phenomena. And yet flocks did form, every time the simulation was run, which could fly around obstacles in a very fluid and natural manner.

Far from equilibrium

Nicolis and Prigogine (1989) showed that when a physical or chemical system is pushed away from equilibrium it survives and thrives, while if it remains at equilibrium it dies. The reason is that when far from equilibrium, systems are forced to explore their space of possibilities and this exploration helps them to create new patterns of relationships and different structures.

Medical cardiology is undergoing something of a revolution as a result of exploring this concept in relation to the study of normal and abnormal heartbeat patterns. Nothing is more orderly than the rhythmic beating of the heart but combined with this order there is a subtle but apparently fundamental irregularity. In healthy individuals and particularly in young children the interval between heartbeats varies in a disorderly and unpredictable way. If the heartbeat interval is regular then this is a sign of danger (Goldberger 1996). Too much order in heart dynamics is an indicator of insensitivity and inflexibility. Complex evolving systems function best when they combine order and chaos in appropriate measure.

A state of paradox

A typhoon may well be the unforeseen consequence of a small change in a weather system but a typhoon is not itself a chaotic weather pattern: it has a highly organised dynamic structure. So the dynamics of weather combines both order and chaos. Other research on complex evolving systems has reinforced this finding that bounded instability or the edge of chaos is characterised by paradox: stability and instability; competition and co-operation; order and disorder.

Using Complexity for Organisational Analysis

Very little research has been undertaken on complexity in social systems. Brian Arthur has related these theories to economics (1990) and Stacey (1996, 1995) explores complexity in relation to organisational strategy. Stacey talks about the legitimate system i.e. the formal polices, procedures and processes and the shadow system i.e. the informal systems, grapevines, networks etc of an organisation. An example of the legitimate system could be a Board meeting and an example of the shadow system would be the discussions that go on in the corridor just before and after the meeting. He suggests that it is only when these two systems are in tension with each other that the organisation can be at the edge of chaos or, as he calls it, the space for creativity. It is only here that the organisation is changeable because it is only here that it is capable of double loop learning. The edge of chaos is characterised by creative tension and paradox. Evidence shows that when organisations resolve the paradox, they eventually fail (Miller 1990), whereas those that sustain the paradox and operate in nonequilibrium states are more likely to survive (Pascale 1990).

Stacey identifies five control parameters which he believes determine whether an organisation occupies the space for creativity or not. They are:
(a) Information flow. Prigogine's research in chemistry showed that dissipative systems require higher levels of energy to sustain them. Stacey suggests that as organisations move up to a new level of operating so they require higher levels of information flow to sustain them. This parameter is considered to reach a critical point when it becomes impossible for formal systems in the organisation to retain the necessary information about changes in the fitness landscape. The shadow system then comes into play as its informality can retain faster flows of information. Past the critical point of information flow even the shadow system will be unable to retain enough information to cope with competitors moves and the organisation can tip into the unstable zone.
(b) Degree of diversity. In biological systems this is known as requisite variety. For a system to explore its possibility space it needs to continually generate new behaviour. A shadow system characterised by conforming members produces stable organisational dynamics. At some critical point between the extremes the organisation has enough diversity to provoke learning and creativity but not enough to cause anarchy and disintegration.
(c) Richness of connectivity. As discussed earlier, connectivity is a key concept in complex evolving systems. In organisational terms few connections bring stability and many bring instability. Between these extremes there is a critical point where connections are rich enough to produce endless variety in behaviour. The other important dimension is the strength of those connections. Strong ties bind people together making it more likely that behaviour will become repetitive and uniform. Weak ties on the other hand provide bridges to other parts of a network through which variety may be imported. This parameter reaches a critical level at some intermediate point between weak and strong and many and few connections.
(d) Level of contained anxiety. When anxiety is so firmly contained that it is avoided altogether, for example, by strict adherence to the requirements of hierarchy, then an organisations shadow system operates in the stable zone. The critical point of this parameter is when anxiety levels are contained at a relatively high level and members are able to be creative. When the anxiety level becomes too high it is disabling.
(e) Degree of power differential. In the spectrum ranging from concentrated power exercised in an authoritarian manner to equally distributed power hardly exercised at all, a critical point is reached where one can find both containment of anxiety through clear hierarchical structures and directing forms of leadership, on the one hand, and freedom to express opinions and risk subversive, creative activity without fear on the other.

This is one model of how the science of complexity relates to organisational analysis and much more research is needed in this field.

Conclusions

Most models of management ignore the reality of organisations as non linear feedback systems and complexity theory suggests a new approach to organisational analysis. Theories of complexity offer a new way of thinking and a new way of seeing the world. In a non linear system where slight variations amplify into unpredictable results the long term future is unknowable. Therefore the skill is not to predict the future but to see patterns. Margaret Wheatley (1993) suggests organisational analysis should focus on identifying patterns over time, rhythm, flow, direction and shape. One should remain aware of the whole and resist analysing the parts to death.

Complexity theory is relatively new. It originated from the Santa Fe Institute in New Mexico in the mid 1980's. The application of complexity theory to organisational analysis is still embryonic and there are many important research questions to address such as: How can phase transition/edge of chaos/space for creativity be identified ?; How does one know when an organisation is in it ?; What does self organisation and emergence mean in organisational terms ?; How do self organisation and leadership fit together ? What is the role of the leader ?; What is the role of redundancy and slack resources ?; What kind of management and consulting interventions make sense in the space for creativity ?
What complexity tells us already is that organisations cannot move according to some blueprint. Vital strategic outcomes emerge from spontaneous self organised groups and strategic direction emerges from complex interaction. Therefore controlling an organisation from the top is an illusion but to trust to self organisation is a huge leap of faith. There are few published examples of organisations applying this framework but Ricardo Semler (1994) in his book "Maverick" describes how Semco has developed as a "natural" (sic) organisation and this has created both economic success and the most popular workplace in Brazil.

References
Gleick, James (1988) Chaos Heinemann, UK
Goldberger A L (1996) Non linear dynamics for clinicians: chaos theory, fractals and complexity at the bedside. Lancet 347, 1312-1314.
Goodwin, Brian (1994) How the leopard changed its spots Phoenix
Kauffman, Stuart (1993) Origins of order: self organisation and selection in evolution. Oxford University Press
Kauffman, Stuart (1995) At home in the universe, Oxford University press.
Lewin, Roger (1993) Complexity: Life on the edge of chaos Macmillan
Miller, D (1990) The Icarus paradox: How excellent organisations can bring about their own downfall. New York: Harper Business
Mittleton-Kelly, Eve (1997) Organisations as co-evolving complex adaptive systems Paper presented to British Academy of Management annual conference 1997.
Nicolis G and Prigogine I (1989) Exploring complexity W H Freeman
Pascale, R (1990) Managing on the edge: How successful companies use conflict to stay ahead. London: Viking Penguin.
Prigogine, Ilya (1977) "Order through fluctuations: self organisation and social system" in Jatsch and Waddington, Evolution and consciousness: human systems in transition, Addisson-Wesley London.
Reynolds, Craig (1987) quoted in Waldrop (1992) below
Semler, Ricardo (1994) Maverick Arrow Books
Stacey, Ralph (1996) Complexity and creativity in organisations Berret Koehler, San Francisco
Stacey, Ralph (1995) "The Science of Complexity: An alternative perspective for strategic change" in Strategic Management Journal, vol 16 no 6 pp 477-495
Waldrop, M (1992) "Complexity: The emerging science at the edge of order and chaos" Penguin
Wheatley, Margaret (1993) Leadership and the New Science Berret Koehler