Letting them build it

February 27, 2019

Agile methods like scrum and XP are very exciting, especially when you are first introduced to them. There is something very common sense about the ideas in them that seems to resonate for a lot of people. I know it was that way for me. I’d looked at a lot of different project management methods before settling on XP (thank you Steve McConnell). A lot of those methods looked interesting, but XP was the first one that just made sense. For a young project manager looking for a new way to do things, it was an easy choice. 

Now when you look closely at a method like XP you learn very quickly that it is actually a collection of practices, many of which have been around for a very long time. The thing that makes XP work, is the way that this particular set of practices or, as I like to think of it, this big agile bag full of cats works together. For instance, iterations by themselves have been around for a very long time under a different name: time boxes. Pair programming on the other hand, was a relatively new innovation as far as I know (although not entirely unheard of). And while continuous integration had actually been around in some form or another for a while, it was certainly best articulated and demonstrated by the proponents of XP. On their own I would argue that each of these ideas had plenty of merit, but the real magic happens when you combine them together. Each of these practices, and in XP there were roughly 13 of them, complements and overlaps one or more other practices in the set. So as a whole, you have a system of related ideas that have some redundancy and interconnection. You can see this in Ron Jeffries’ diagram of XP.

Now this gives you a package offering of interrelated ideas that many, including all XP practitioners I’ve ever met, say you need to adopt as a whole. You can’t just pick and choose the bits you like and expect to get great results. Why not? Well, I would go back to the redundancy and interrelated ideas. Let’s suppose for just a minute that you adopted all 13 XP practices, but you found that continuous integration for one reason or another was “too hard” or “not a good cultural fit” or for some other reason wasn’t going to work for your team. What might happen? Well, in all likelihood, in the short term you might not see any immediate effect. In fact, you might find that the team goes a little faster because they aren’t struggling to build continuous integration into their process. But hang on, we’re not done yet. You see there are practices that depend on continuous integration in order to work. For example, test driven development (TDD) and continuous refactoring. TDD relies on CI to give the developers quick feedback on their tests. That can’t happen without CI. So, developers are going to lose feedback on their tests, which means they aren’t going to get as much value from doing the tests in advance…and therefore they aren’t likely to keep doing TDD. Quality may start to suffer. And if they don’t have CI and TDD, then they don’t have the safety net of tests that they need to do continuous refactoring…so they are going to be less likely to try refactoring because it feels too risky.  By removing CI we have undermined quality and the resilience of the system we are developing (because we’re no longer refactoring). 

The impact of removing practices, especially in a pre-packaged set of methods has some rather insidious consequences. Things don’t immediately fall apart. Instead there is a gradual erosion of benefits that causes a cascade of related and also seemingly unrelated problems. You may still be getting some benefit from the remaining XP practices, but the system is now much more fragile and less resilient. You have removed some of the reinforcing mechanisms from the method that helped insure it is robust. When the team encounters a crisis, some sort of emergency in production where they need rapid turnaround and depend on high feedback, they aren’t prepared. They are slow to respond, introduce more defects and likely to struggle. At which point someone is liable to point out that this process sucks. Congratulations! Of course it does, you made it suck.

This is the reason that adherents of pre-packaged methods tend to sound so religious about the unequivocal adoption of all their practices. You have to adopt all the practices, otherwise you aren’t doing XP, Scrum, Kanban, and so on. I want to pause for a moment, because I don’t think that’s the end of the story. 

If we were to stop for a moment and look at development and management practices (agile and otherwise) we might find that there are practices that tend to have similarities that might cause us to group them together. Testing and QA practices like TDD, BDD, and others do share many similarities. Estimation practices like story points, ideal developer days, and others also share similarities. My point is that for any given meme or idea that we have in XP or in agile in general, there are multiple supporting practices that may fit. In addition, some practices are sophisticated enough that adoption can be measured by degree rather than in absolutes (we are 30% toward CI rather than all or nothing). My point is that there are multiple options for many of the key elements of popular frameworks. And even within many of those options there is a matter of the degree of adoption. After all, as so many agile advocates often say, it’s a journey, not a destination. Therefore, if I’m 30% of the way along the path, that must be worth something.

All of this is to say that we can substitute our own practices with some judicious caution. We’re allowed to do that, despite what the more religious might say. In fact, we can mix and match to find the elements that work for us. Now this is really hanging our toes out on the radical edge. Ivar Jacobson has something he calls essential methods. Basically, it is a catalog of development methods that you can combine and recombine to build your own framework. Now, you can still screw up. Remember that the reason that frameworks like XP and scrum have been successful is that they have concepts that are interlocking and support each other. The DIY approach is much riskier (practices may or may not support each other), but for some groups that may be the best way to go.

The important thing is to understand why these frameworks work as well as they do. They are composed of a series of practices that support each other, making them robust in the face of a world full of disruption and challenges. You mess with them at your own risk. Or…you build your own. Just know that you need to understand what you are building. If you do it poorly, it very likely won’t work.


Time Machine

February 26, 2019

OK, Mr. Peabody, where are we going today?

Well Sherman, Any time I explain what Scrum or XP is, I start with time boxes. The time box method has been around a really long time. The earliest record I can find in a casual search is where they were used at DuPont in the 1980s. I suspect that time boxes are much older than that. The time box basically applies a constraint to the system. It creates an arbitrary start and end date, usually on the smaller side. You commit to a fixed amount of work and when the end of the time box is reached you are done, no matter what the completion state of the work. Work that is complete is counted as done within the time box, work that still remains to be finished is either scope that gets dropped or perhaps that work is continued in the next time box.

This technique has some benefits:

  1. Deadlines, even arbitrary 2 week time boxes, help keep everyone focused.
  2. Deadlines force the question of prioritization. Not everything will fit in the box.
  3. Small time boxes create a short heartbeat or pulse that is useful for measures of capacity and throughput.
  4. It forms a useful skeleton for the OODA improvement cycle

There are also some challenges:

  1. Small time boxes demand that you figure out how to break work down into smaller, but still valuable pieces. Many teams find this hard to do.
  2. Small time boxes means that it is almost inevitable that scope won’t be delivered sooner or later. How the business manages this scenario says a lot about how the benefits of time boxes are perceived.
  3. Much of the angst of estimation is due primarily to the fact that teams are struggling to fit work to their limited capacity in ways they didn’t have to prior to the time box.
  4. It doesn’t work if you can’t break the iron triangle of scope, schedule, and quality. Scope usually has to be compromised in some form or another in order for time boxes to work (it’s kind of what they are based on)

Like so many other things, a time box is useful in the right context, but not all contexts. I’ve seen a few projects where a time box would not work (hardware constraints, legacy mainframe applications, an organization that wasn’t willing to give up the iron triangle, etc.). All too often we force the time box on the team and tell them that they suck if they can’t overcome the challenges. Sometimes that’s true, other times it isn’t. It’s a judgement call. Beware, and don’t let yourself get caught forcing a round peg into a square hole (I’m looking at you Scrum).


Test Driven Transformation

January 28, 2019

The introduction of agile methods has brought a wave of innovation in the business world that some might argue has revolutionized thinking about how organizations should be structured and how people work together. However, as it stands today, much of the promise of agile methods is wrapped up in preconfigured frameworks that offer a one-size-fits-all solution for every business challenge that a company may face. This is despite the fact that the modern organization is a highly complex structure, bordering on chaotic, that is often not best served by the application of frameworks. We see this manifested most commonly today in the failures to scale agile methods within large organizations.

The conversation about failure rates in the world of transformation is similar to prior discussions about the failure rates of projects and programs: both are notoriously vague and poorly defined. Almost all of the surveys that you find (PMI, etc.) use an embarrassing amount of anecdotal evidence to back up their assertions. The very definition of failure is usually so broad as to be completely meaningless. So, with that said, I think it’s important that we are careful with any assertions that transformations are failing or succeeding. In fact, my experience is that when we are talking about transformations within organizations, we are working at such a high level, that it is never clear what is entirely successful or failing. After all, In a good transformation, there is a lot of failure. You experiment, try things out, and find out that they don’t work. I’m not sure I trust anyone who tells me that 100% of their efforts are always successful. That tells me that they aren’t really changing much.

When I speak of frameworks, what exactly do I mean? Well, I’m thinking globally. I’m not just talking about those large scaling frameworks like SAFe and LeSS (that’s easy), I’m also pointing the finger at small scale, team level frameworks like Scrum and XP. And it’s not that these frameworks can’t work or can’t be useful. In fact, I’ve seen them applied and applied well. However, more often than not, they aren’t applied well. I know there is bitter and acrimonious debate on this subject. I’ll leave that battle for others and simply say, “We can do better.”

We need to step back and reassess how we engage with organizations from the very earliest stages of the engagement. It’s no longer sufficient to make prescriptive, framework-oriented recommendations and have any reasonable expectation of those proposals having any kind of success. In fact, I think we may well find they are often more harmful than helpful. Framework oriented approaches give the false promise that their solutions will solve every problem, and when they fail, they leave the customer having wasted tremendous time an energy, without anything to show for it. To make matters worse, consultants implementing such transformations will simply say that the organization didn’t have the right “mindset”, effectively blaming the customer for the failure of the transformation. This allows the consultant to wash their own hands of any responsibility for the failure as they move on to the next engagement with yet another set of pre-packaged proposals.

It’s time that we brought an end to such thinking and begin to focus on how we can properly understand the problems in the organization before we even begin to make recommendations. Then, like with any prescription for a complex system, we need to apply trial experiments not broad frameworks to address the specific problems that we find. Of course, in order to do this well, we need to have reliable means of assessing the health of the system. We need to treat the system like what it truly is, a complex organic structure, that lives and breathes, composed of living elements interacting with each other and participating in flows of ingestion, respiration, and value production for customers. This requires a first principles approach to understanding organizations. We need to understand exactly what organizational health looks like before we can make any kind of decent assessment of the system. To make any recommendations without that sort of understanding is irresponsible.

So what’s our target? Achieving some hypothetical state of agility is not a meaningful or useful target for a transformation. Agility has no objective meaning that a business person finds useful. Instead it is an end state in search of a meaning. In short, it has none.

Alternatively, there are those who propose that we should start from a place of experimentation. That also is an insufficient starting point for working with organizations. A company is not a consultant’s toy to be experimented with. And no one wants to be the subject of experiments. The experimental approach, while well meaning, signals rather strongly that you not only don’t understand the problem, but also that you have no idea what the real solution is. This experimental approach should be considered by any business owner of integrity as completely useless.

What organizations need is a clear eyed and objective assessment of what the problem is. It should be the sort of analysis that allows us to measure our effectiveness against that of our competition and our customer market in some meaningful fashion. Furthermore, based on that data, we should know what the prescription for change should be with a very high degree of confidence. Organizations are not looking for your best guess. They want to have confidence that any change or transformation effort has some reasonably provable possible outcome.

Another way of putting this is to think of it as test driven transformation. We must have some idea of a reasonable set of tests for assessing the relative health of a system. The results of those tests should give us some clue to the different kinds of problems that may afflict the system. They must be quantifiable, and like a doctor, we must have some notion of what the results of the tests imply. It doesn’t mean that we know for sure what the outcome will be, but it also doesn’t mean that we are taking a random shot in the dark. A good doctor will use multiple diagnostic tests to build a picture of the problems with the patient. Based on the results of those tests, the doctor is able to narrow down the treatment to a subset of commonly recommended approaches. Nothing about this is random experimentation, but rather it is a systematic, data-driven approach to understanding the nature of the problem.


Team Genetics

September 28, 2014

dna-163710_640

Today I was listing to “The Splendid Table”, a great cooking show on NPR. They were talking about variation in growing heirloom tomatoes. Somehow, that got me thinking about agile teams (probably time to see the therapist again). It occurred to me that ideas like Agile are memes.

Richard Dawkins defined a meme as “an idea, behavior, or style that spreads from person to person within a culture.” and Agile certainly fits that definition. Agile has spread from obscurity to worldwide acceptance within 20 years. In another 20 years I fully expect that waterfall, plan driven methods will be nothing but a footnote in the history books. Despite some early prognostications to the contrary, Agile has grown at a very healthy rate over the last several years.

“Richard Dawkins invented the term ‘memes’ to stand for items that are reproduced by imitation rather than reproduced genetically.”

While much of the credit belongs to the teams that actually do the hard work of making a new process work, there is also the business that has arisen around evangelizing and introducing Agile to companies that deserves a great deal of the credit. Agile training and consulting has done a remarkable job of spreading the meme throughout the software development world.

I think there are characteristics of Agile training that have made Agile “sticky” as a meme. Much of the Scrum certification is based on plenty of hands-on exercises. Training and certification has yielded a decent business. I’m not sure if anyone has the numbers, but I’d be surprised if it wasn’t a multi-million dollar enterprise worldwide. Strangely enough, much of that spreading has been through imitation. There is no shared agenda for the training, much of it is simply imitated from trainer to trainer.

When trainers and others spread the meme they are like Johnny Appleseed sowing Agile ideas across fertile corporate soil.

Genes change with each generation, and so do ideas. They go through a mixing and blending each time they are shared. Parts of the idea are forgotten, other new ideas are grafted on. Soon the original idea is unrecognizable. I think that’s kind of what has happened with XP. Extreme Programming originally contained a collection of ideas that were very potent. Things like pair programming, continuous integration and others all served as core ideas within XP. Over time, those ideas have been co-opted and found their main expression in Scrum. Today, almost no one trains teams in XP, Scrum is the dominant process that is trained and introduced to teams.

“Memes do this through the processes of variation, mutation, competition, and inheritance, each of which influence a meme’s reproductive success.”

So too does Agile. In recent years methods like Kanban and ideas like No Estimates have arisen and are becoming a meaningful part of the software development landscape. These are evolutions of the Agile meme. Agile is evolving, I wonder where it will go next…


Slowing Down

February 12, 2013

photo

Last week I led a session at Agile Open Northwest called, “Slowing Down”. The idea for this session was inspired by my own struggles with becoming quite over-committed to a variety of things (my job, my hobbies, etc.) and the resulting stress and crisis it has created for me. You see, the funny thing about it all was that even though I was perfectly aware of what I was doing by over-committing like crazy, I couldn’t seem to stop.

The Introduction

So I came to this session, not as an expert selling a solution, but rather as a novice seeking help. Since I really didn’t know where things were going to go, I simply started with the session title. I wrote “Slowing Down” on the whiteboard and introduced myself to the small group of people who had joined me for the session. I started with a story of my own. It was a bit like what I imagine an Alcholics Anonymous conversation starts like, “Hi, my name is Tom and I can’t slow down…”

Fortunately for me, many in the audience had a similar story. Since we are a bunch of software development types, it didn’t take long for the concept of sustainable pace to be mentioned. Of course we all knew full well what sustainable pace means. It is a term that I originally encountered in Xtreme Programming. I could ramble on for hours about the importance of keeping the pace and duration of your work under control so that you can sustain your creative energy and not burn out. Easy. But I can’t seem to do it worth a damn. That’s the interesting bit. Why? Why is it that, even knowing the importance of maintaining a sustainable pace, I and others like me seem to struggle so hard with it?

Why?

A few interesting ideas for why we get sucked into this dynamic were suggested during the session:

its-mine

Ownership – Feelings of ownership can make it hard for people to let go of tasks and delegate them to others. For example, it is very easy for project leaders to feel a very strong sense of ownership and commitment to the success of projects that they are working on. This can be quite normal – often our organization want this kind of commitment from us. However, like many things, this can go too far. The undesired dynamic plays out as a feeling that you and only you are personally responsible for the success or failure of the project (what happened to the team?). When challenged, people who struggle with ownership issues will often look with incomprehension when asked to give up some part of a project, “If I don’t do it, who will?” I think that in some cases this inability to give up ownership can also manifest as heroism (ownership + adrenaline junkie). Perhaps at its heart, ownership issues are tightly tied to ego. They seem to manifest as a very selfish view of project success or failure.

nun-with-habit

Bad Habit

Habit – We form all sorts of bad habits that contribute to the stress in our lives. For example, I’ve gotten into the habit of checking my email compulsively throughout the day. Often even when at home. Habits like this that tether us to the office and constant communication serve to raise our overall stress levels. Other examples include habitually taking home the laptop with you every night and carrying the work phone with you wherever you go.

Culture – One major reason for difficulty with slowing down is the work culture you live in. People shared many different stories of how the expectations at work made it hard or almost impossible for them to escape the pressures of the office. Everything from evil bosses that demand attendance over performance to co-workers who make snide comments when a colleague dares to leave the office at 5:00. Some places even provide rewards for those who make decisions that put work above any other activity. Examples of these sorts of influences in the workplace abound.

All of these influences are very common reasons why people find it hard to slow down. It is no wonder that there are many who struggle to maintain a sustainable pace of work at the office. Understanding why you are feeling that pressure is critical to understanding what strategies to use to manage the problem. The strategies where where we ended up going next.

Strategies

As we moved along in our discussion, people identified strategies that could be used to deal with slowing down and establishing a more sustainable pace. We captured and expanded upon those strategies as we wove the narrative of slowing down.

Setting Boundaries

boundary_full

The first strategy that came up was setting boundaries. Setting boundaries is fundamental to establishing control over your own schedule and pace. Fail to do this and all the rest really doesn’t matter. People told many stories about how they managed to establish meaningful boundaries in their work lives that helped them to keep a meaningful sustainable pace. Some made their 9 to 5 work hours non-negotiable. They never offered the longer hours that many fall into. You get me for 8 hours a day, and the rest of my life is not for sale. It was remarkable to hear the strength of some of these voices. Others refused to take work home or turned off the cell phone after 5.

Basically, what I heard were people establishing a service level agreement for their participation. One benefit that I noticed from this sort of boundary was that it made visible to everyone just what they could and could not expect from you. Visibility is a strongly held value in the agile community and it struck me that making my boundaries more visible would be a uniquely agile way of dealing with the issue (I’m closing the door now…). Another way of making my boundaries and limits visible would be to use a personal task board mechanism like personal kanban in order to not only make my existing commitments visible, but also to review them myself and keep tabs on how the work load is balanced (or not).

Reflections

Diana Larsen did a great session last year at Agile2012 on personal retrospectives. As team facilitators, we are pretty well versed in running team retrospectives, however I never do them by myself. That is exactly what Diana proposed: do self-retrospectives on a periodic basis in order to reflect on your progress toward your goals, and where you want to go next. I think this would be a useful tool for many, whether it is only at the end of the year or much more frequently. I know that my own responsibilities feel like they have changed quite dramatically in the last year. Stopping to assess those changes might just give you the opportunity to recognize stressful trends and start to do something about it. You can start to do it now, or wait until a crisis imposes that reflection. Your call.

This is just my summary of what I saw and heard during our talk. Looking at the sheer number of topics that we covered it’s quite apparent to me that we covered a broad number of subjects. Many of them are worthy of deep investigation. Perhaps, as the mind map suggests, we have created a map of the terrain of the topic of slowing down. Others may have different take aways. I certainly hope so. I appreciated everything that the group brought to the conversation and I hope that I was able to serve as a reasonable scribe for what was said.


XP2011 Day 4

May 13, 2011

Sessions

Silo Busting w/Tom Perry and Lourdes Vidueira

Yeah, that’s me. It was our big session. And just for the record, we rocked the house. In fact, the people attending our session made so much noise that people in sessions in the rooms adjacent to us complained about all the noise. What did I think? I think that means I’m doing a good job as a facilitator. Especially given the fact that there were only 10 people in the session. It was awesome! The feedback we received was nothing short of phenomenal. I’m extremely grateful to those who participated.

I was pretty exhausted after running the session. 4 hours seems like the equivalent of running a 220 yard dash. It’s not a sprint and it’s not a marathon. You have to keep things moving fast and you can’t lose your focus. We went out on the town afterward in Madrid and had a grand celebration. I had seafood that would give Louisiana a run for its money, and the people were just as friendly, if not more so.

The conference has been a good one. I’m probably too tired to do a decent recap of everything that happened today, but I’ll give it a shot tomorrow. Signing off from Madrid.


XP2011 Day 3

May 12, 2011

Conversations

What can I say? The restaurant open bar last night was epic. Actually I wasn’t saying very much at all to anyone this morning…

Sessions

Keynote: What Forms of Work and Life Make Sense for Us? w/Brian Marick

As usual Brian’s keynote was eccentric, enlightening, and above all else, unique. At about the halfway point he actually had the entire room stand up and he gave a tango lesson (which was no surprise, he had been talking about it on twitter for weeks). Still, there were a lot of European men arm-in-arm dancing with each other. Perhaps not so unusual. The talk itself covered some interesting subjects.

First he talked about gift economies vs. money economies. The way I understood it, he described the agile team as using a gift economy. Favors are exchanged freely with no exchange of money. However outside the team and especially within the corporation at large, it is a money economy. I think the point was to suggest that we need to be conscious of the different economies at work and adjust our expectations and behavior accordingly.

He also talked about the influence of context on behavior, basically debunking using assessments like Myers-Briggs for any predictive purpose. Instead, there is plenty of evidence to suggest that context matters much more when it comes to predicting people’s behavior. So, my take away from that is that we need to create the right environments for people to be successful.

Grumpy Old Agile Coaches w/Rachel Davies

This was a fun session with Olav Lewitz, JB Rainsberger, Kati Vikki, Mike Hill all sitting on park benches and Rachel Davies acting as moderator. While the conversation was good, I have to agree with some of the participants, that the grumpy old agile coaches looked pretty happy for a bunch of grumps! I was interested to hear about the lonely-coaches-sodality google group which I definitely want to check out.

Agile at Scale

Apparently this was another fishbowl – I enjoyed it and jumped right in with the big fish. It was fun to bounce some of my thoughts off the group and get their perspective. Mary Poppendieck was a hoot and provided some lively conterpoint and tough questions of her own. Jutta Eckstein was the moderator and did a great job.

After Hours

It’s a quiet evening for me. I have to give my Silo Busting tutorial tomorrow morning, so it’s early to bed.