Exploring Agile

A brief history of agile

Agile has its roots in software development. A landscape historically dominated by industrial views and beliefs, specifically the Taylorist scientific management conviction of separating people from processes(separating the people that do the work from the decisions on how to do and plan that work).

This was exemplified by the widely adopted use of the waterfall development model. A linear and sequential method, promoting upfront planning with a focus on predictability and control. The adoption of this industrial paradigm in software development eventually revealed its flaws, in some cases contributing to projects running massively over budget.

In a search for better ways of developing software a meeting was organised between a group of leading software development professionals that shared a sympathy for the more lightweight software development methods they were using at the time. Collectively they outlined a set of 4 values and 12 principles that captured the shared characteristics of these lightweight approaches and the mindset of their practitioners. This meeting resulted in the well-known Manifesto for Agile Software Development (2001).

As Bob Martin, one of the original Manifesto signatories, recalls: The most disagreement was over what to call it. The name of the meeting itself was “Lightweight Methods Summit,” but no one liked the term “lightweight.” After proposing and discussing a number of alternatives, they used a show of hands to adopt Jim Highsmith’s Suggestion of “agile.”

Over time, the word agile started to be used as a noun. Agile methods were packaged and sold as a management revolution with big “A” Agile as a brandname. Daniel Mezick describes the birth of an “Agile Industry” in his article The Agile Industrial Complex:

The “Agile Manifesto” was born, and with it, a movement. The movement was lively and brisk. The results generated by those who used Agile methods was impressive. The Agile methods created great outcomes, real human engagement and even happiness in the workplace. People started buzzing about Agile software development. Before long an “Agile community” sprouted. Training became widely available, and suddenly “certifications” appeared. Consultants started specializing in “Agile coaching.”

Mezick goes on to describe how the original intentions and spirit of the Agile Manifesto were difficult to understand for executives that were looking for ready-to-implement solutions promised by Agile methodology.

This transition from agile to Agile could be described as Pournelle’s Law in action (all cultures evolve until they are controlled by people who don’t share the values of that culture). In his blog post, “The Agile Imposition”(2006), one of the original Manifesto signatories Martin Fowler expresses his concerns:

As a methodology or design approach becomes fashionable, then we see a lot people using it, or teaching it, who are focusing on the fashion rather than the real details. This can lead to reports of things done in agile’s name which are a polar opposite to the principles of movement’s founders.

Despite all misconceptions and oversimplifications of agile (which persist to this day), the manifesto has managed to remain impressively relevant due to the timeless concepts it promotes. Not necessarily intended as such by its creators, the Agile Manifesto has grown into a landmark in the history of Software Development and the Agile community in general. In response to the Agile Industrial Complex, we see more and more initiatives (like agnostic agile) emerging that aim to point back to the original intentions and spirit of the Manifesto’s creators instead of zooming in on specific technical implementations or frameworks.

Even though Agile originated in software development, some of its beliefs have been exported outside of it. Most notably, the lean startup that combines Agile and Lean to rapidly discover the viability of a proposed business model (test-driven development at company level). Successes demonstrated in the startup community have inspired more and more larger companies to embark on their own Agile Transformation.

The IT perspective

The IT perspective on Agile is strongly influenced by the Manifesto for Agile Software Development. Ironically, while some of the contributors of the manifesto shared a motive to heal the divide between technology and the business, it partly resulted in the adverse effect of people outside of the software development community starting to perceive agile as “an IT thing”.

Adaptive approaches to Software Development have largely become the norm (even though waterfall approaches still persist), with Scrum being the most widely used Agile Software Development Framework (next to Kanban and Extreme Programming.

Adaptive approaches acknowledge that Software Development lives in the empirical world and complex domain (you can only know how long something took after it has been completed). Agile software projects have a fixed schedule and resources while the scope varies and its delivery is forecasted as opposed to planned. Success is defined by value delivery, and course correction is based on empirical evidence gained from user testing.

Another notable difference is that adaptive approaches like Scrum, don’t have a classic project manager role, instead these responsibilities are distributed across the Scrum Team, making for a self-organised development team that is focussed on delivering maximum business value, supported by a Product Owner. A Scrum Master “manages” the Scrum process (as opposed to managing people), and coaches the team into self-organization.

Many things have greatly evolved since the creation of the Agile Manifesto, like the vast optimisation of release management processesand devOps automation. The manifesto already mentioned in one of its principles:

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Continuous Integration and Continuous Deployment (CI/CD) have evolved to a level where continuously integrating and deploying software has now become effortless (or jokingly described as boring).

The “Agile Industrial Complex” has somewhat distorted the IT perspective on Agile, narrowing Agile Software Development down to “fast and early delivery of functionality”. This led to a popularisation of “officially named”Agile approaches, imposing them on teams and enforcing structure in order to increase efficiency without understanding the underlaying values. These dogmatic, cargo-cult approaches to Agile practices have left many developers (and companies) with a bad taste for Agile.

However, Agile done right (focussing on early user value creation through iterative delivery and continuous improvement through learning) generally leads to better results and happier teams. Successful, motivated and self-organised teams that operate in healthy Agile environments, are increasingly multi-disciplined and focused on exceptional outcomesinstead of acceptable outputs. These teams want to go beyond just delivering working software for their clients, moving from a client-vendor relationship to a partnership of shared success. This evolution is reflected in initiatives like modern agile, making the idea behind the Agile Manifesto more universally applicable with a focus beyond software practitioners.

The design perspective

Agile from a design perspective is often mixed with scepticism, since most of the Agile practices emerged from within the development community. A great example is the Scrum Framework, prescribing that a Development Team should be cross-functional (should have all the skills necessary to create a product Increment), but doesn’t recognise titles for Development Team members. Effectively, this would mean that designers should join a Scrum Team, but would be called… a Developer?

To some degree this is understandable, since most of today’s digital design disciplines simply didn’t exist at the time that Scrum, and later, the Manifesto, were created. From a designer’s perspective, Agile Frameworks that have their roots in Software Development can be experienced as restrictive due to their incrementalism and focus on output and delivery, leaving relatively little room for ideation and creative processes, let alone a holistic design approach. Creativity simply doesn’t scale that well.

In traditional and linear approaches (waterfall), designers were positioned upstream from the development process, creating big designs up front, handing them over to a development team in order to turn these designs into a working product (often resulting in a parallel reality).

In an adaptive and iterative approach, design-skills are required at all levels of the process, offering (or at least encouraging) the right design activities at the right moment and at the right speed (considering both short-term and long-term goals). Arguably the biggest “agile challenge” from a design perspective is that of design-led change, breaking free from a silo’d approach, making good design everyone’s responsibility by explaining its business value and helping agile software development practices move to a more design-driven approach.

Not many designers would necessarily refer to their practices as Agile. Because of the strong connotation with software development, the design perspective on Agile is commonly reduced to a methodology for software-development. In reality, Agile (in the broader sense) is much more than that and the most used framework, Scrum, is actually the opposite of a methodology. When exploring fundamental beliefs behind Agile like self-organization, cross-functional teams, customer collaboration, embracing change, empiricism (learning through experimentation) and continuous attention to quality, one can‘t help but notice a large overlap with the values associated with Design Thinking (and to be fair, it’s easy to see how a designer would more likely gravitate towards a name such as this).

The business perspective

Agile from a business perspective is probably closest to the literal meaning of the word. Organizations are increasingly striving for an ability to adapt to a changing (business) environment.

Large traditional corporations are starting to acknowledge that last centuries management approaches and rigid organizational structures are no longer serving them well in an increasingly service-based economy. They are increasingly experiencing serious competition from “small players” in their market with the agility to outpace them in terms of innovation and time-to-market. In a faster moving and more demanding world, organizations need to be increasingly customer-oriented, moving from silo-based to team-based organizational structures.

However, true agile transformations are hard and not only underestimated, but misunderstood. Agility is often falsely perceived as something that can be acquired, rolled out or implemented by an Agile Coach as if it was another management methodology. This has resulted in a lot of “fake agile” going around. Many agile transformations are exclusively focussing on a faster time to market (the most commonly mentioned reason for adopting Agile). The adopted practices to achieve this goal are often agile in name only and rarely render the expected results.

In his article “Projects and Products in Scrum”, Agile Consultant Ian Mitchell describes one of the most common reasons for agile transitions to fail; the lack of executive sponsorship:

Unfortunately the appetite for deep and pervasive enterprise transformation is rarely strong, and any sponsorship for real change is typically weak.

This could be compared to paying a personal trainer, but not understanding you are expected to break a sweat. When traditional corporate hierarchy and politics remain unchanged, no real transition will ever take place. For managers that have longtime operated in a traditional (vertical) management culture of top-down decision making, the concept of Agile can be hard to grasp. In his article “Why Do Managers Hate Agile” Steve Denning accurately describes these two different worlds:

*The world of “management” is vertical. The purpose of this vertical world is self-evident: to make money for the shareholders, including the top executives. Its communications are top-down. Its values are efficiency and predictability. The key to succeeding in this world is tight control. Its dynamic is conservative: to preserve the gains of the past.

The Agile mindset is horizontal. Its purpose is to delight customers. Making money is the result, not the goal of its activities. Its focus is on continuous innovation. Its dynamic is enablement, rather than control. Its communications tend to be horizontal conversations. It aspires to liberate the full talents and capacities of those doing the work. It is oriented to understanding and creating the future.

The reality is that creating an environment and culture required for high performance might require a drastic reduction, or even an absence of traditional management culture and the adoption of supportive leadership with the courage and trust to empower teams by decentralising decision making and distributing authority. Leaving the vertical dynamic of top-down command-and-controlism behind and embracing a network of small autonomous teams.

A truly successful transformation might simply start with some form of organizational self-reflection, identifying organizational patterns or behaviours that could be considered non-agile and laying out a plan to discontinue, un-learn and and transform these, moving from doing agileto being agile.

The holistic perspective

The IT, Design and Business perspectives on Agile provide some insight into the different meanings of Agile. Because some of these perspectives commonly originate from certain disciplines or theories, they often have a strong focus on specific tools or processes.

Agile practitioners that view agile from a Holistic perspective apply a wider focus on Agile, considering an (enterprise-wide) shift in mindset as the ultimate goal (an almost zen state of Agility). This mindset is based on the belief that agile is something you are, not something you do. Any practices, tools or processes are a natural expression of that mindset, not a goal by itself.

In the second part of this article, we’ll be exploring the Agile Mindset, what defines organizations that have this mindset and wether or not this mindset can be adopted.