Archive for the 'Agile Software Development' Category

Software Development and Chaos Theory

After watching an excellent BBC documentary called The Secret Life of Chaos, I was interested to see if anyone had written about the association between chaos theory and the unpredictable nature of software development.

I came across these papers written in 1995 by L.B.S. Raccoon:

The Chaos Model and the Chaos Life Cycle

Raccoon (1995) The Chaos Model and the Chaos Life Cycle, in ACM Software Engineering Notes, Volume 20, Number 1, Pages 55 to 66, January 1995, ACM Press. (PDF available here).

The chaos model combines a linear problem-solving loop with fractals to suggest that a project consists of many interrelated levels of problem solving. The behaviour of a complex system emerges from the combined behaviour of the smaller building blocks.

The chaos life cycle defines the phrases of the software development life cycle in terms of fractals that show that all phrases of the life cycle occur within other phrases.

This suggests an iterative, outside-in development process, which is one of the fundamental principles of Behaviour-Driven Development (BDD).

The Chaos Strategy

Raccoon (1995) The Chaos Strategy, in ACM Software Engineering Notes, Volume 20, Issue 5, Pages 40 to 47, December 1995, ACM Press. (PDF available here).

The main rule in chaos strategy is always resolve the most important issue first. This reflects the Just-in-Time production ideology as promoted by Lean Software Development.

It’s interesting to note that today’s popular agile software methodologies use some of the principles of chaos theory as presented in these papers. It seems that software development is recognised as an example of complexity at work.

Agile software development follows a natural, evolving, chaotic cycle. The smallest variation in conditions can result in a massive difference in outcome. There will never be a way of accurately predicting the outcome of a complex software project. We simply have to accept chaos as a fact of life and embrace an evolutionary development process.

Are Burndowns Evil?

Agile teams often use a burndown chart to track the progress of a software project. The burndown chart provides an indication of work done over time. I have used burndowns on many projects and I have come to believe their use can negatively impact the quality of the software we deliver, without providing much benefit to the outcome of the project.

Firstly, a burndown can create schedule pressure early in the iteration. This is especially apparent on a new project with a new team. It takes time for a new project to get under way and for team members to get up to speed. This time can be very difficult to factor into an iteration.  Even though progress will improve, seeing the burndown flat-lining immediately can cause a lot of negative pressure on a team.

A burndown chart is linear and has no room for variations in team size, unscheduled meetings, unforeseen technical problems and other issues outside the control of the team. The burndown doesn’t take into account the unplanned, but necessary work that needs to be done to ensure the success of the project.

A burndown can be very unforgiving. One bad day and the progress line goes off course. This can cause pressure within the team to cut corners to get back on track. This is detrimental to the quality of the software and encourages developers to get a story done quick-and-dirty just to satisfy the burndown progress.

The progress of a burndown can be taken the wrong way by project managers and customers. No stories complete == no work done, which is usually not the case. If we need to track progress, we can simply look at the task board! How many stories are done? How many are still to do? How much time is left in the iteration? What needs reprioritising? Talk about it. Discuss the issues. The task board is a great place for the team to get together and talk about the progress of the project. If required, create a throw-away progress chart. But we shouldn’t drive the development process from it.

Use the number of points/ideal-days completed to estimate the team’s velocity. The velocity should be calculated on work already completed. For a new team and a new project, it is almost impossible to predict a velocity for the sake of creating a burndown, so why bother? It can cause more harm than good.

Another problem with the linear nature of a burndown is that it doesn’t factor in breakthroughs. A breakthrough is a fundamental shift in the understanding of a software design. This is a very important step in improving the design, quality and maintainability of the software. If a breakthrough is discovered by the team, then taking the time to refactor should be encouraged. Breakthrough refactorings can be hugely beneficial for the future development of the software.  A burndown can discourage refactoring and improvement by promoting a linear track.

The focus on a burndown is on reaching a predetermined end-point in time. Instead we should be focusing on delivering value to the business without compromising quality.

Experienced teams working on new or familiar software might not have any of these problems. I have been on projects where the burndown was very accurate and the project went very smoothly. But this was not because of a burndown. The burndown didn’t really provide any benefit to the outcome of the project. The project work was easy to estimate and so the burndown was always on track.

I’m not saying never use a burndown. They are often required by project managers to report on progress. Just don’t let it become the focus of the project, as it can potentially do more harm than good.

New Books

Here’s a few new titles I have added to my wish list:

Clean Code – Robert C. Martin

Implementation Patterns – Kent Beck

Pragmatic Thinking and Learning – Andy Hunt

The Productive Programmer – Neal Ford

Sizing Up Software

There is often debate as to whether to use ideal days or story points to estimate agile software projects. My personal preference is for story points.

Software estimation is the measurement of relative size and complexity, not duration.

Story points are a pure measure of the size of a feature. They do not state how long a particular feature will take to implement. The combined ability, or velocity, of the team determines how many points can be achieved in a development cycle, or sprint.

By working with size, rather than directly with duration, we can focus on expressing how much effort is involved in implementing a particular feature. This makes estimating a much quicker process and reduces the likelihood of an estimate being turned into a guarantee. Also, story points tend to belong to the whole team, whereas estimates in days or hours are often targeted towards the abilities of a particular person.

Knowing the relative size of a feature allows us to better determine its place in the release alongside other features. This gives us a forecast for a release that we can use to prioritise tasks and construct a release schedule.

During sprint planning, the team can then choose stories to implement in the sprint period (usually 2 or 4 weeks), based on business priorities. These stories can then be broken down into their constituent tasks and delegated to team members, or put into a task pool. By working in short iterations, it is much easier to determine the tasks required to develop a particular story and the estimated number of hours to complete each task, than if we were breaking down the entire project.

You don’t want to have to re-estimate because your initial estimate was wrong. If you estimate that you can complete 20 story points in a sprint and you only complete 15, you can assume you can complete 15 on the next sprint. If you estimate a story to take 5 days and it takes 10, then what about the other estimations on the board?

By sizing up a software project using story points, we get a better understanding of the complexity of a project and therefore determine a more realistic schedule that can adapt according to the progress of the team.

For more information on this topic, I highly recommend the book Agile Estimating and Planning by Mike Cohn.