Agile has been around for a long time. There are many who still claim that while agile may be a useful concept for software development, it cannot apply to other types of projects. In this article, we will delve into agile a bit. First, we will introduce the agile manifesto itself, try to explain what makes any process agile, and discuss process purity. Second, we will explore a few “core” agile concepts. Finally, we will take a look at how they may expand beyond the “just for software development” argument.
What is Agile?
For the purposes of this article, Agile is derived from the agile manifesto. Way back in 2001, a small group of influential people got together to discuss how the traditional approach of managing software development projects was failing far too often, and there had to be a better way. They came up with four value statements which became known as the agile manifesto:
That is, while there is value in the items on the right, we value the items on the left more.”
“But wait,” you may say. “Where is the part about daily scrums? Where is the part about time boxes? Where is the part about user stories? Where is the part about (insert favorite Agilism)?”
At its core, “being agile” or “doing agile” is about trying to follow these value statements. It is not about a particular method or process itself, but it is about how we approach our processes and methods.
What Makes Any Process Agile?
Aside from the overly flippant answer of “because said so”, there is not a conclusive, objective set of criteria to determine if any process is “agile”. Some of the most popular, and some might argue “pure” agile methodologies include – Scrum, Extreme Programming (XP), and Dynamic Systems Development Methodology (DSDM) (especially popular in UK/Europe). Each of these build upon the values in the agile manifesto, providing frameworks for execution, as well as offering additional structure.
Unfortunately, as soon as you start defining specific practices or components within any given methodology, you introduce the concept of “process purity” which inevitably leads to the theological arguments of whether or not you are “really doing” any one of them.
This has led to a nearly infinite number of hybrid processes, based off of pieces and parts of any particular methodology, including traditional waterfall practices. We have a myriad of colloquial names that often mash together names of their primary components for these “impure” processes, some of my favorites include: Wateritive, Scrumbut, and XP + (plus). Of course, there are also actual “spin-off” methodologies that try to merge what they believe are the best components, but that isn’t overly germane to this article.
The irony, of course, is that by applying structure through any methodology, with its associated official guides, courses, and certifications, that people are incented to be more rigid in whether they consider any process to be in compliance with the standard.
Purist’s View vs. Realist’s View
There are two primary perspectives to this conundrum. On the one hand, it is important to be able to objectively identify any particular process and its compliance to the standard. I’ll call this the “purist” view. On the other hand, having an understanding about how any given organization operates or project is being run and trying to align a “best fit” requires some degree of flexibility against the standard. I’ll call this the “realist” view.
In my experience, a purist usually comes from someone who either operates theoretically (such as an academic or one who works for a specific methodology’s organization or certifying authority), or is someone who has completed a course of study or certification, but has yet to have significant practical experience in application of said methodology.
I hear people, primarily those who have recently attended a class, talk about how agile won’t work for their particular, unique project, organization, or circumstance. Maybe they are convinced that they can’t get enough business or user buy-in. Maybe they are concerned about the size and scale of what “going agile” means. Maybe they are simply confused by the semantics. For sure, there are many agile practitioners who like to label things as “not agile”. But agile is a concept and an idea. Agile comprises a spectrum.
However, the purely black and white view of the purist can devolve into: “If you fail to implement this single (relatively minor) part, you are not doing scrum/XP/DSDM.” Or worse “you added this other (non-agile) practice, therefore you are not doing scrum/XP/DSDM or even agile.”
The other side of this coin may experience pitfalls as well. The realist, one who tries to “make agile fit”, can lose sight of the details of a given methodology, and when making too many accommodations, may start to violate of some core tenets which enable the methodology to realize its purpose.
In practice, each of the statements in the agile manifesto represents a continuum. For example, working software over documentation can mean anything from the “no documentation” for a mobile platform application all the way through significant documentation for an FDA-compliant system. Producing the documentation itself, given the context of the work, does not negate the purpose of the statement.
Core Agile Concepts
Needless to say, there are a number of common concepts proposed by and supported across many agile methodologies. However, most, if not all, are not really unique to agile, and many have been around for years.
As has always been true, none of these are a silver bullet in and of themselves.
But when applied in conjunction, these techniques can and do provide foundational support and powerful propulsion on the road to agile success. It is beyond the scope of this article to discuss the specifics of how these apply to each flavor of agile, but we will briefly explain the merits of each. The below techniques comprise a sampling, and the set is certainly not exhaustive.
Horizon of Predictability & Just In Time
There are a few expressions of the “horizon of predictability” and just-in-time paradigm. The foundation of this technique is that one should not even attempt to detail anything too far into the future, as the further out you go, the less accurate any predictions will be. But as that horizon gets closer, the picture becomes more clear, and you can provide details which start to predict with much higher accuracy.
The primary example of the need for this is in project planning and a detailed schedule extending out months and months or longer. The justification is that you will spend way too much time upfront on the schedule, and then continue to spend way too much time trying to keep it up to date. Instead, you should create a tiered approach of rolling plans.
One example might be:
There is a corollary to the planning aspects reflected in this horizon of predictability related to the project requirements themselves. Only produce enough details based on how close you are to implementation, starting with very high-level milestone goals, breaking those down into features, breaking features into component stories, and finally fleshing out detailed stories.
Time-Boxed Iterations & Cadence
Perhaps one of the most superficially recognized aspects of agile is short, time-boxed iterations – often 1 or 2 weeks in duration. It bears pointing out that, in this case, an iteration includes all the work needed to produce a product (planning, analysis, design, development, test). Without delving into any reasoning behind “less is more”, there is benefit from the time box itself. Providing a recurring cadence to the team allows for habits to form and productivity gains from the familiarity of expectations.
Further, in compliance with the horizon of predictability, having a short-term target allows the team to make reasonable commitments, which in turn causes greater team accountability. By forcing a defined end to a short iteration, the team will organically become focused and motivated to complete work. Also, time boxes tend to encourage teams to get more items “done” and minimize the number of items “in progress.”
Continuous Business or User Engagement
Actively working with your customer is always a good idea. The fewer intermediaries you go through to transmit “the message”, the more likely that you will be able to deliver products which ar “fit for purpose”. This concept has never been in dispute. However, the emphasis from an agile perspective is to have ongoing engagement throughout the life of the project. This means that there is more time spent together than the more traditional bookend approach of requirements and user acceptance.
Agile takes this construct even one step further by introducing the business or user into the project team itself, as a member. This removes the “us/them” tension and promotes a sense of ownership. Coupled with the time-boxed iterations and horizon of predictability, this engagement maintains focus on providing details only when effective along with near real-time feedback. If any step of the iteration misses the mark, that feedback helps reduce wasted effort. The adage “fail early, fail often” applies.
Everyone knows about using agile for software development. It was the genesis of the agile manifesto and that is still where it sees the most traction.
Attend any sort of agile course, and you are likely to hear anecdotes and stories of how agile can apply to other endeavors, such as scrum being used on everything from building a dog house to planning a vacation. That’s all well and good, but you don’t hear the nation’s leading home builders using scrum, nor travel agents. Just because you can use a particular tool or method does not mean that it is ideal.
However, there are a number of situations or projects that are ancillary to software development; projects that involve many of the same IT professionals that would work on software who may already be familiar, or even skilled, at “doing agile”. In these cases, many of those agile techniques can be applied with great success.
Such various endeavors might include: DevOps, Product Configuration, Systems Integration, Data Operations – ETL and migrations, and even good old Process Definition / Improvement itself. While each of them has their own idiosyncratic details, none of the above-referenced techniques would be misplaced in their execution.
In some of these examples, the project team may have some (if not most) very specific, detailed requirements at the get go. That doesn’t mean that there won’t be a need for “horizon of predictability” to apply to longer term planning. In fact, there is still benefit to working through a few iterations to understand team dynamics and getting into a strong cadence of delivery. At project inception, it may still make more sense to have some iteration goals for the longer term, while allocating specific requirements into just the first two iterations, maintaining a prioritized backlog, and reacting to customer feedback.
I often hear people say that some of these projects can’t even show (or deliver) anything to the customer until a certain set of (or even every) requirement is implemented. Hogwash, I say! You just need to set appropriate expectations up front. Further, you need to ensure that only that small subset of functionality that was built can provide customer value. Often times, that implies a shift in the way we organize those requirements. Take a rules-intensive project. Instead of building every permutation of a single step before proceeding to the following step, you can create a single thread all the way through the entire process. Will that deliver the whole intent of the process? No, of course not. But it will demonstrate that your team is making progress for that single thread, and it allows the customer to start to see an end result.
Bringing It All Together
Recently, I worked on a large-scale, enterprise systems integration project for a client. The client had a traditional waterfall-based process and had trepidations about applying agile for these mission-critical, core to the business application projects. However, they were also intrigued by how agile methodology might apply.After providing some background materials and walking through the proposed process, they agreed to try an agile approach, though there was still much hand-wringing.
Needless to say, the project ended up as smashing successes, with some of the highest approval and feedback heard throughout the organization. So, you might ask, what was the secret sauce? First and foremost, it involved taking a look at the client organization, its practices, processes, and proclivities, evaluating how much appetite they had for agile, or really change in general, and applying the methods, tools, and techniques best suited.
Core agile tenets were applied. Releases were broken down into sprints with specific goals, allowing for time boxes and cross-department planning and coordination. Using a rolling planning cadence, each impacted part of the organization had a target for project participation and could operate at a scrum-of-scrums level to stay informed and engaged. Two-week sprints allowed for short intervals between customer demonstrations, including a standard feedback loop, which dovetailed into an organizational change management thread that accelerated understanding of what was coming, and enabled the users to take ownership of the new functionality. Senior and executive management received status reports based on defined goals, allowing the flexibility of specific scope items to shift in response to the ongoing feedback while maintaining an overarching view of schedule and budget.
Are any of these ideas really unique? Of course not, but they were applied in a specific way after cooperative discussion with a specific organization. Could this specific mix apply to a future engagement? Of course. But the key is to understand each project and client organization, and only then determine the most effective course of action in applying agile, without blindly using a cookie cutter set.
This article has discussed a few things about agile, from its foundation of the agile manifesto through some core concepts. At a minimum, I hope that this article has provided some food for thought in the application of agile to non-software development projects, and has allowed you to think about creative ways that you might apply agile.