Letting them build it

February 27, 2019

Agile methods like scrum and XP are very exciting, especially when you are first introduced to them. There is something very common sense about the ideas in them that seems to resonate for a lot of people. I know it was that way for me. I’d looked at a lot of different project management methods before settling on XP (thank you Steve McConnell). A lot of those methods looked interesting, but XP was the first one that just made sense. For a young project manager looking for a new way to do things, it was an easy choice. 

Now when you look closely at a method like XP you learn very quickly that it is actually a collection of practices, many of which have been around for a very long time. The thing that makes XP work, is the way that this particular set of practices or, as I like to think of it, this big agile bag full of cats works together. For instance, iterations by themselves have been around for a very long time under a different name: time boxes. Pair programming on the other hand, was a relatively new innovation as far as I know (although not entirely unheard of). And while continuous integration had actually been around in some form or another for a while, it was certainly best articulated and demonstrated by the proponents of XP. On their own I would argue that each of these ideas had plenty of merit, but the real magic happens when you combine them together. Each of these practices, and in XP there were roughly 13 of them, complements and overlaps one or more other practices in the set. So as a whole, you have a system of related ideas that have some redundancy and interconnection. You can see this in Ron Jeffries’ diagram of XP.

Now this gives you a package offering of interrelated ideas that many, including all XP practitioners I’ve ever met, say you need to adopt as a whole. You can’t just pick and choose the bits you like and expect to get great results. Why not? Well, I would go back to the redundancy and interrelated ideas. Let’s suppose for just a minute that you adopted all 13 XP practices, but you found that continuous integration for one reason or another was “too hard” or “not a good cultural fit” or for some other reason wasn’t going to work for your team. What might happen? Well, in all likelihood, in the short term you might not see any immediate effect. In fact, you might find that the team goes a little faster because they aren’t struggling to build continuous integration into their process. But hang on, we’re not done yet. You see there are practices that depend on continuous integration in order to work. For example, test driven development (TDD) and continuous refactoring. TDD relies on CI to give the developers quick feedback on their tests. That can’t happen without CI. So, developers are going to lose feedback on their tests, which means they aren’t going to get as much value from doing the tests in advance…and therefore they aren’t likely to keep doing TDD. Quality may start to suffer. And if they don’t have CI and TDD, then they don’t have the safety net of tests that they need to do continuous refactoring…so they are going to be less likely to try refactoring because it feels too risky.  By removing CI we have undermined quality and the resilience of the system we are developing (because we’re no longer refactoring). 

The impact of removing practices, especially in a pre-packaged set of methods has some rather insidious consequences. Things don’t immediately fall apart. Instead there is a gradual erosion of benefits that causes a cascade of related and also seemingly unrelated problems. You may still be getting some benefit from the remaining XP practices, but the system is now much more fragile and less resilient. You have removed some of the reinforcing mechanisms from the method that helped insure it is robust. When the team encounters a crisis, some sort of emergency in production where they need rapid turnaround and depend on high feedback, they aren’t prepared. They are slow to respond, introduce more defects and likely to struggle. At which point someone is liable to point out that this process sucks. Congratulations! Of course it does, you made it suck.

This is the reason that adherents of pre-packaged methods tend to sound so religious about the unequivocal adoption of all their practices. You have to adopt all the practices, otherwise you aren’t doing XP, Scrum, Kanban, and so on. I want to pause for a moment, because I don’t think that’s the end of the story. 

If we were to stop for a moment and look at development and management practices (agile and otherwise) we might find that there are practices that tend to have similarities that might cause us to group them together. Testing and QA practices like TDD, BDD, and others do share many similarities. Estimation practices like story points, ideal developer days, and others also share similarities. My point is that for any given meme or idea that we have in XP or in agile in general, there are multiple supporting practices that may fit. In addition, some practices are sophisticated enough that adoption can be measured by degree rather than in absolutes (we are 30% toward CI rather than all or nothing). My point is that there are multiple options for many of the key elements of popular frameworks. And even within many of those options there is a matter of the degree of adoption. After all, as so many agile advocates often say, it’s a journey, not a destination. Therefore, if I’m 30% of the way along the path, that must be worth something.

All of this is to say that we can substitute our own practices with some judicious caution. We’re allowed to do that, despite what the more religious might say. In fact, we can mix and match to find the elements that work for us. Now this is really hanging our toes out on the radical edge. Ivar Jacobson has something he calls essential methods. Basically, it is a catalog of development methods that you can combine and recombine to build your own framework. Now, you can still screw up. Remember that the reason that frameworks like XP and scrum have been successful is that they have concepts that are interlocking and support each other. The DIY approach is much riskier (practices may or may not support each other), but for some groups that may be the best way to go.

The important thing is to understand why these frameworks work as well as they do. They are composed of a series of practices that support each other, making them robust in the face of a world full of disruption and challenges. You mess with them at your own risk. Or…you build your own. Just know that you need to understand what you are building. If you do it poorly, it very likely won’t work.


Team Genetics

September 28, 2014

dna-163710_640

Today I was listing to “The Splendid Table”, a great cooking show on NPR. They were talking about variation in growing heirloom tomatoes. Somehow, that got me thinking about agile teams (probably time to see the therapist again). It occurred to me that ideas like Agile are memes.

Richard Dawkins defined a meme as “an idea, behavior, or style that spreads from person to person within a culture.” and Agile certainly fits that definition. Agile has spread from obscurity to worldwide acceptance within 20 years. In another 20 years I fully expect that waterfall, plan driven methods will be nothing but a footnote in the history books. Despite some early prognostications to the contrary, Agile has grown at a very healthy rate over the last several years.

“Richard Dawkins invented the term ‘memes’ to stand for items that are reproduced by imitation rather than reproduced genetically.”

While much of the credit belongs to the teams that actually do the hard work of making a new process work, there is also the business that has arisen around evangelizing and introducing Agile to companies that deserves a great deal of the credit. Agile training and consulting has done a remarkable job of spreading the meme throughout the software development world.

I think there are characteristics of Agile training that have made Agile “sticky” as a meme. Much of the Scrum certification is based on plenty of hands-on exercises. Training and certification has yielded a decent business. I’m not sure if anyone has the numbers, but I’d be surprised if it wasn’t a multi-million dollar enterprise worldwide. Strangely enough, much of that spreading has been through imitation. There is no shared agenda for the training, much of it is simply imitated from trainer to trainer.

When trainers and others spread the meme they are like Johnny Appleseed sowing Agile ideas across fertile corporate soil.

Genes change with each generation, and so do ideas. They go through a mixing and blending each time they are shared. Parts of the idea are forgotten, other new ideas are grafted on. Soon the original idea is unrecognizable. I think that’s kind of what has happened with XP. Extreme Programming originally contained a collection of ideas that were very potent. Things like pair programming, continuous integration and others all served as core ideas within XP. Over time, those ideas have been co-opted and found their main expression in Scrum. Today, almost no one trains teams in XP, Scrum is the dominant process that is trained and introduced to teams.

“Memes do this through the processes of variation, mutation, competition, and inheritance, each of which influence a meme’s reproductive success.”

So too does Agile. In recent years methods like Kanban and ideas like No Estimates have arisen and are becoming a meaningful part of the software development landscape. These are evolutions of the Agile meme. Agile is evolving, I wonder where it will go next…