Share |

The new science of leadership

Emerging research provides insight into the unseen forces that influence leadership effectiveness in turbulent environments

Even with thousands of scientific studies and unlimited writings on leadership, the complexity of leadership remains a mystifying concept. Emerging thought and discoveries in psychology, neurology, quantum physics, and complexity theory are providing new perspectives that could either illuminate the nature of leadership or illuminate the reasons why leadership is such an elusive process.

In this two-part series, I will consider perspectives on leadership that are emerging from new discoveries and philosophies in science and psychology. This article, "The new science of leadership", will provide insight into the unseen forces that influence leadership effectiveness in turbulent environments (Wheatley, 2006). The next article, "The new psychology of leadership" will demonstrate how the values and opinions of followers hold a key to effective leadership (Reicher, Platow, & Haslam, 2007).

 

Newtonian leadership models winding down

Exploring leadership through emerging discoveries in systems theory, chaos theory, and quantum mechanics, Margaret Wheatley (2006) argues that traditional leadership models are ineffective in complex and dynamic social systems of the 21st century. To Wheatley, the clockwork universe presented by Newtonian physics is an adequate model in a static world that thrives on predictability. The hierarchical structures of modern organizations and contemporary models of leadership tend to reflect a mechanical Newtonian perspective.

The Newtonian universe offers an objective reality that leaders can measure and control by carefully planning to predict the future, partitioning people and processes into parts and organizing them hierarchically, providing sufficient rewards and punishments to get people to implement the plans, and isolating cause and effect to correct direction. However, Wheatley (2006) argues that the turbulence of global society and culture is forcing organizations to realize that the models and habits developed for a stable environment may not work in a dynamic world. Imposing static and mechanical processes on an organization in a turbulent environment can submerge the organization until it implodes under pressure. To survive in a turbulent environment, the complexity perspective holds that organizations must continuously change and adapt with their environments. Imposing a model on an organization stifles change. Reality exists only in the context, and reality changes with the context. One expert interpretation or best practice does not apply in all situations. This means that organizations and their members must continuously adapt with or “co-evolve” (p. 163) by interacting with the environment.

 

Leadership as a landscape of connections

Where the Newtonian perspective attempts to understand the system by isolating its parts, the new science takes a holistic perspective that attempts to understand the system by seeing the relationships within the networks. Understanding the “landscape of connection” (Wheatley, 2006, p. xxxvii) presents a view of the behaviors that emerge from the interacting elements within dynamic processes. At this point, the new science of leadership starts to sound like the old religion and the ancient wisdom. In a quantum universe, “relationship is the key determiner of everything” (p. xxxviii); nothing exists separately from another.

Understanding leadership is no longer a matter of isolating elements, behaviors, traits, or situations. In the new science, building blocks disappear and the unseen connections among separate entities become the “fundamental ingredient of creation” (p. xxxix).

Chaos leadership

When considering how complex systems organize and operate or how innovations occur, Russ Marion (2002) offered what seems to be a simple answer: “interaction.” However, the theories that explain complex interacting systems are far from simple.

Chaos theory explains interacting systems, while complexity theory explains interacting and adaptive system. A science of turbulence, chaos theory shows how unpredictable behavior may not be random. Attempting to use a computer model to predict weather, Edward Lorentz (1963)found that small differences in initial conditions had a large impact on how events unfold. This “sensitive dependence on initial conditions” (Gleick, 2008, p. 23) would become known as the Butterfly Effect, and served as the foundation for chaos theory.

The ideas of interconnectedness and chaos were familiar with ancient philosophy and religion, but they were new concepts to scientists who viewed reality as Newtonian clockwork. Traditional science had attempted to identify and explain the predictable, while dismissing irregularities as errors in measurement or as anachronisms. That is, until researchers like Lorentz started to notice universal patterns in irregularities. Using computer models, Lorentz was able to identify structure and patterns in turbulent systems. Where once was chaos and unpredictability, computers helped scientist see that stability and unpredictability coexist.

Like most revolutionary discoveries in science, Lorentz’s observations were hardly original; but his unique contribution was that the same kind of sensitivity to small changes can affect even simple systems. Chaos theory helps to show how chance appears in a deterministic world; that predictability requires perfect knowledge of the universe and exact laws of nature. Even in the unlikely case that all laws become clear, humanity will not likely ever know the state of the entire universe (Strogatz, 2008). As stated by Lorentz, (1963) “any physical system that behaves non-periodically is unpredictable” (Gleick, 2008, p. 18).

A key concept that chaos theory offers for leaders is that understanding the current state of an organization will provide little information about its state tomorrow. The more complex a system, the less predictable it becomes, because the relationship between cause and effect are not constant.

The weakness of chaos theory for leadership applications is that it accounts for neither adaptation nor intelligent behavior. In her attempt to apply chaos theory to leadership, Wheatley (2006) offered an analogy of a river that has ability to adapt to and change the environment. She suggests that the river knows how to make things happen because it has a “need to flow”. The river holds a clear mission and multiple strategies for dealing with obstacles because it “realizes” that many ways exist for reaching the ocean, and has the “faith” that it can achieve its mission (p. 16). Anthropomorphizing a river in attempt to define leadership applications of chaos theory has a key limitation: a river is neither human nor conscious, and the river’s environment is a natural system not a social system.

Marion (2002) suggests that chaos theory helps to explain phenomena in environmental systems, like fluid turbulence and weather; but it does not account for adaptation and intelligent behavior. Although chaos theory initially offered interesting metaphors for leadership applications, it has limited relevance to human social interactions.

Complexity leadership

Complexity theory is a derivative of chaos theory that addresses the limitations of chaos theory. Like chaos theory, complexity theory is “a science of large interactive networks and nonlinear cause and effective” (Marion, 2002, p. 302).

Unlike chaos theory, complexity theory accounts for the rational and deliberate changes that systems make to their environments. The complex behavior in systems operates under cause and effect rules that allow for a degree of predictability and control. Both chaos theory and complexity theory “are about interactions among different actors and how that interaction generates both innovation and fitness” (p. 303).

While traditional theories tend to focus on the material (for example, people) versus the immaterial (for example, task), complexity theory shows that people and task are not separate entities; they must be understood together. Social realities interact with material realities to fit with the environment. Complexity predicts that this adaptive interaction results from associations. This suggests that a key role for leaders is to build and strengthen networks to facilitate fitness between the organization and the environment.

Through the complexity perspective, mechanical leadership practices that advocate strong and visionary leadership “are wrong” because strong leaders “shut down” adaptable behavior through control. Under forceful leadership “the group behavior can be no more creative” than the leader (Marion, 2002, p. 315). The effective leader is technically competent, manages and develops networks, cultivates interdependencies within and around the organization, and serves as a catalyst for change and adaptability. Making sense of complexity requires a systemic perspective that can recognize the nonlinear relationship between cause and effective.

Challenges of the new leadership perspectives

A key challenge of applying the new sciences to leadership is that the relationships, forces, and waves that influence patterns of behavior are invisible; researchers can neither directly observe nor measure the dynamic and intangible phenomena. Lack of measurement means that the assumptions of the new sciences are not only difficult for typical researchers to understand, they are also difficult to validate. This does not mean that insights offered by the new science are without merit; it likely means that contemporary science simply lacks the means to measure dynamic phenomena.

The history of leadership research provides an example of how trait theory was universally dismissed until researchers developed new techniques that could test its assumptions. New research techniques and statistical models allowed researchers to not only validate elements of trait theory, but also to expand it (Kirkpatrick & Locke, 1991; Goleman, 1998; Yukl, 2010). Lex Donaldson (1996) provided evidence to support this point when he suggested that the lack of empirical support for dynamic processes may be due to simplistic analytical models, not the processes that the models attempt to measure. In addressing critics who argued that contingency theories could not be validated with research, Donaldson integrated divergent contingency theories to develop a model for analyzing dynamic processes in organizational environments. Using his model, Donaldson was able to validate key elements of contingency theory that appeared universal among 87 organizations in five countries. This is not to say that Donaldson's model can be used to verify applications of quantum physics and complexity theory to leadership, but it does suggest that the lack of empirical support may say more about the method of measurement than it does about the process being measured.

Works Cited

Donaldson, L. (1996). The normal science of structural contingency theory. In S. R. Clegg, C. Hardy, & W. R. Nord, Handbook of organization studies (pp. 57-76). London, England: Sage Publications.

Gleick, J. (2008). Chaos: Making a new science (Second ed.). New York: Penguin Books Ltd.

Goleman, D. (1998, November-December). What makes a leader? Harvard Business Review , 93-102.

Kirkpatrick, S. A., & Locke, E. A. (1991). Leadership: do traits matter. Academy of Management Executive , 5 (2), 48-60.

Lorentz, E. N. (1963). Deterministic nonperiodic flow. Journal of the Atmoshperic Sciences , 20, 131-141.

Marion, R. (2002). Leadership in education. Long Grove, IL: Waveland Press, Inc.

Strogatz, S. (2008). Chaos. Chatilly: The Teaching Company.

Wheatley, M. J. (2006). Leadership and the new science: Discovering order in a chaotic world (3rd ed.). San Francisco: Barrett-Koehler Publishers, Inc.

Yukl, G. (2010). Leadership in organizations (7th ed.). New York, NY: Pearson Prentice Hall.