Friday, July 23, 2010

How the Underdog Outperformed the Champ

This post is a textual version covering the information I covered in my Agile2010 talk in Orlando.
I want to tell you about 3 teams and show how implementing agile practices and slowly begining to understand the principle behind those practices helped them deliver value faster. One of the strange occurrences that happened though was how a more junior team outperformed a more senior team for a very long time.

The story starts with a high profile internet booking engine for a low cost airline. The project was having a rough start and was not making progress. Most of the development was being done in a new office that was located 7 hours away from the office we were located at in Europe. I, an agile coach, moved to this location and a director for the product spent <>

The team started as 1 team that we quickly split into two teams. Soon after we added a third team at the office in Europe.

All 3 teams were new to agile. The teams were setup with 6-7 developers and 1-2 testers. The teams shared two customer representatives/business analysts. They had worked in iterations but the iteration was never a commitment but simply a place to track what was currently being worked on. The original two teams were relatively junior with mostly front-end experience. The third team we added had more years of development experience and more balanced experienced across all tiers of an application.

We started implementing extreme programming practices with the two teams at the same location. The initial focus was on planning and we moved to a 1 week iteration with daily standups. We started doing retrospectives as well and this allowed us to start make fast improvements. Which quickly identified problems with our user stories and acceptance criteria.

We did some user story training, based on Mike Cohn's material, with the team. Then we started working with them to breakdown the stories. This actually took a couple of iterations to get the stories into a size and correct split so that they could be developed and completed in a one week iteration.

The two teams at the original location had a couple of attributes I really appreciated. The first was they were open to change. They wanted to improve and they listened to everything we would recommend as issues came up. One thing that took a bit more convincing but we were able to get them to try was working in a cross discipline way. The whole team focusing on testing when testing took more effort than development or testing was behind.

The team that implemented these ideas the best became very efficient and continually solved their issues. This lead them to have a very consistent velocity. However, it was not only the most consistent it was the highest of all the teams.

The second team struggled with consistency. As we started to implement the process they did have an initial upward turn but they did not consistently solve the root issues that were occurring and struggled to have a consistent output of value.

I will start with a retrospective on the high performing, "underdog", team. The items they were doing that really helped them included focusing on finishing the work in an iteration. The iteration became a commitment that they desired to keep. The divergently searched for ways to do this and improve this.

One of the first issues that really made all the teams suffer was story size, completeness and how they were split. This was a team effort to get this right. The business analysts watched how the team worked and the development and testing team help get stories into a size that both delivered value and was completable in a one week iteration.

At times stories were still too big and were not complete until late in an iteration. This caused the testing to be very high risk. The testing team need assistance and the whole team would help as required. Generally speaking they had, or rather developed, good cooperation between all roles on the team.

As I said earlier the team that did this the best also had a very high output of completed stories and fewer defects.

This team was open to new ideas and wanted to learn. They wanted to be better at planning and technical practices although with the pressure on the time line the technical ended up getting less focus.

One of the issues was the practices were seen as absolutes and understanding the values and principles driving these practices was not completely grasped. This lead to struggles when I left the team for a few months. When I returned they told me what all of their issues were but they were not solving them. No one on the team really developed into the leader, coach and evangelists to keep improving and refocus the team back to the goals of the practices.

There were many technical practices that we did not implement well. One of these that we spent a few weeks on trying to get going was ATDD/BDD. I am a big proponent of doing this and we struggled to get the team to take the time to learn tools and techniques to do this well and it was dropped. Of course the normal problems of not have an automated suite of test came up with each release including many defects and repeated defects and longer manual regression periods that mostly focused on positive and negative checking.

We should have added or developed someone on the team to be the leaders that could have kept the team focused without a full time agile coach.

One of the things I am very thankful for was we did have management support for changing the way we were working all the way to the Senior VP level. They not only allowed us to do it but were removing as many of our roadblocks as possible. Many of these roadblocks were with internal and external teams we integrated with and the team had very little influence or relationship with. They gave the team the contacts to develop the relationships and the support to influence those teams leaders to help meet the teams needs.

Another thing I am thankful for is how well the team accepted me. I came in with authority to make changes and it is very difficult to accept an outsider who is asking you to make huge changes. They quickly allowed me to be a part of the team which allowed me to help them become a better team. This also gave me new friends to help adjust to a new country and culture which is not an easy thing to do.

Now we go to the third team in the story. This team was added a couple of weeks after we had started implementing the changes on the team. They stepped into the system we were creating from a completely different system. They had a few more years experience than the other 2 teams both within our company and outside. They had come from a team that had some level of success working in a different way. They called this way agile but as I mentioned before this was iterations that did not have the commitment of an agile iteration. They were use to working in long release cycles where mostly technical stories and upfront design were done first versus stories that delivered small pieces of business value where the design evolved and refactoring was continuous.

They really fought against the change. They did not see nor did they put effort into understanding the value of how we were working. This ultimately lead to a long period of very low performance. It was not until the last iterations that they became more consistent at delivering high quality value for the customer to use.

There were some things the team did that I liked a lot. They had better design skills and understanding of good design principles. They also fought to get refactorings and removal of some large pieces of technical debt.

But unfortunately there was a lot of room for improvement.

The team did not focus on finishing iteration commitments. This was not a priority. They did not focus on breaking the stories to fit in the iteration. If development was done they did not assists with testing. This lead to an animosity between the developer and the tester and a lot of defects.

The did understand design principles but the idea of simple design and working in small sets of changes was not understood. Refactoring was seen as a big story when things got so bad you could not longer make changes without breaking everything else. This lead to a couple of delayed releases due to large refactorings that would take a week or so to fix all the defects created. Obviously better automated tests of all types would have helped this.

They never worked as a team. Each developer owned their own stories and worked on it alone. The communication between the developers, testers and business analysts was very bad at the beginning. It took a long time to improve this.

They did not find and solve real issues. They usually identified symptoms and avoided digging to the real cause of these symptoms. This meant issues carried on for a long time before they were actually solved. I think part of this was they knew what the real issue was but did not like the possible solutions, like slowing development to help with better testing or working in smaller batches of work.

This team had no one leading for a long time. There was no one on the team who had worked in an agile way and was focused on continuous improvement. Someone was added to help lead the team after a few weeks of struggles but it took a while to get the team turned around.

The team was built from a group of people that really did not understand agile values, principles and practices. This meant myself and one of the business analysts were the only ones trying to show them the value of what we were trying to do with minimal success. One thing we did mid-way through the project was have the team that was performing well do a session with the other teams to tell them what they had been doing and how it helped solve their issues. This helped a bit.

One of the things we tend to struggle with is creating new teams for a project that is growing. We tend to start with too many people at the beginning and when we need to grow a new team we build a whole team from scratch. I think it is "eXtreme Programming Explained" by Kent Beck that says split and existing team to grow a new team. This is more likely to give you a team that is starting with people who understand the application, the current design, the process and the system as a whole.

The less experienced team wanted a set of practices to start with. A senior team needs to have more say. Most people want to improve. Let them fail early but in a way they can get quick feedback. They may still need some guidance and some help asking the right questions but the freedom will reduce the reluctance to change. I believe in most cases the team will fill the pressure to correct the problems they create.

One thing I enjoyed was the debates I had with the leader that was brought on to help the team. He did not agree with everything we were doing, similar to the rest of the team, however he wanted to discuss and understand why. We had long discussions about things we were trying to do and things that were going wrong. In the long run he ended up being a great help in convincing the team to try new things.

So what have I learned on this project.

Getting started with a new teams is hard! There is so much that needs to be considered. Our teams had domain experience and experience with part of our architecture. However, they had not worked as a team together.

The first thing I think you need is a strong leader. By a strong leader I do not mean a dictator. I mean someone who understands the system you are in. The leader should understand the process you will be starting with and understand the values and principles behind that process. This leader must be able to explain those and guide the team towards understanding and using those values and principles in order to continuously improve.

In order to continuously improve the leader and the team must know how to dig deep into the issues and find the root cause of their issues. This requires courage and openness. It is much easier to point at and blame symptoms and/or others for the issues the team has. The team will want to say the problem is "the short iterations", "pair programming" or "the open environment" when the issue is stories are too big and are not clearly understood. Their is a communication problem between developers and testers. The team is guessing when they do not understand versus having a discussing with the customer and/or business analyst. Or many other issues that are usually people related and system related.

One thing I found that helped the team see issues were coming was watching and limiting the amount of working in progress.

Limiting work in progress, like having a goal to avoid carryover, does not fix your problems. However, it is a great indicator that there are issues.

Too much work in progress hides issues and delays feedback. To much work in progress means testing is not fully done, or maybe not done at all, and you are hiding quality issues. It also means the customer cannot see and use the work yet so it delays getting customer feedback. It delays the discovery of all types of communication issues on the team and between the team and third parties they must interact with. To much work in progress also hides the progress. It is difficult to tell where you are when work is not developed, tested and accepted.

One of the interesting things I saw and the reasons I decided to write and talk about this was how I saw experience work against a team and how inexperience "helped" another team. We actually had 2 teams at a couple of different points and they both had the performance issues I discussed earlier.

With a less experienced team you can and probably should give them more of a starting point. A team not familiar with agile planning needs some absolute starting practices to try out. But be-careful that you are always explaining why you are recommending a certain practice. It is a must to mentor them and develop a leaders who understands not only what you are doing but why. If there is no one on the team who can be this find someone you can bring on to the team and develop into this role.

Senior teams need to have a bigger say! They do have some experience so make sure to spend time getting their buy in and let them help set the starting process for the team.

The way we built the 3rd team would have made this difficult because we did not build the team with a base from the existing teams. It would have been extremely difficult to manage teams with different iteration schedules from a release perspective. However, this team was so inconsistent we could not really promise what they would have done in any set of iterations anyway so we still struggled with release scope issues.

Experience is a broad term. I do not think many people believe time based experience is the only or even best measure. Someone that has done something many times and done it the exact same way each time does not have experience. It is unlikely they would be able to or would even try to adapt what they have done when the context changes.

I really like the four levels of experience that Andy Hunt used to describe different levels of experience in his book "Pragmatic Thinking and Learning":

Novice = "...have little or no previous experience in this skill area. ... They can, however, be somewhat effective if they are given context-free rules to follow"

Advanced Beginner = "...can try tasks on their own, but they still have difficulty troubleshooting"

Competent = "...can now develop conceptual models of the problem domain and work with those models effectively. ... Competents can troubleshoot."

Proficient = "need the big picture", "will be very frustrated by oversimplified information" and "can self-correct"

A team lead I worked with recently had great success limiting his small team to 3 stories in progress at once. When he moved to lead a team that was more than double the size he wanted them to keep the same WIP limit as the small team. The team was very unhappy because they were not staying busy and they knew they could do more than they were doing. He had learned a practice but could not adjust it to the new context yet.

I hope this helps you and please comment and ask questions. I am still learning too!!!