One Study It aims to show that accelerated software development has a 268 percent higher chance of failure. Such huge losses should really make everyone stop executing any accelerated project immediately.
Advertisement
(Picture: Eberhard Wolff ,
Eberhard Wolff is Head of Architecture at SWAGLab and has been working as an architect and consultant for over twenty years, often at the interface between business and technology. He is the author of numerous articles and books, including Microservices, and is a regular speaker at international conferences. His technical focus is on modern architecture and development approaches such as the cloud, domain-driven design, and microservices.
How do the numbers come about? The study first found that the following factors improve the success of software development projects:
- Requirements are clear before the project begins (97 percent more successful projects).
- Perceived problems can be discussed and resolved quickly (87 percent).
- Project requirements are based on real problems (54 percent).
- The project has a complete specification or requirements document before implementation begins (50 percent).
- There were no significant changes to requirements later in the development process (7 percent).
However, whether people worked on one or more projects at the same time had no significant effect.
The study defines accelerated development as projects that:
- Development begins even before the requirements are clear,
- There is no absolute specification and
- The project has undergone significant changes of late.
Overall, this leads to 268 percent more failures and 65 percent more failed projects. The data are also equipped with appropriate statistical values and are therefore reliable.
What is “success”?
The first problem of the study is the lack of a definition of “failure”. A possible criterion would be that the projects were not delivered within budget. But this is a strange goal for an individual software project. If you want the software to be as cheap as possible, you should not develop it yourself, but rather buy standard software. This makes sense for areas where you do not need the flexibility of your own implementation. For example, it would not be clear to me why you would implement financial accounting yourself.
Perhaps the higher costs are associated with higher value as more features are implemented and perhaps even more sales and profits are achieved? So is the project a failure?
Another way to define “failure” would be if the client is disappointed with the outcome at the end of the project. That may make sense, but it is not a rational criterion. The client may be disappointed for many reasons. Different stakeholders may have different levels of satisfaction with the outcome.
The definition of “failure” is by no means without practical implications. I’ve worked on projects that were officially successful, but subjectively had significant problems or did not achieve pre-defined project goals. It gets worse: some projects have no real project goals – how do you decide whether they are a failure or a success?
Actually, you can stop looking at the study in more detail now. But there are many other problems with the study.
big surprise!
The study claims that it is helpful if the requirements are known before the project starts, the specification is complete and critical changes late in the project are avoided. I have to admit: my surprise at this as well as at the other results of the study is limited. More information at the beginning and fewer changes during the project certainly make life easier for projects. You could describe such a project as boring. From the point of view of the implementers, this is great because the project is more likely to succeed. However, from the point of view of the customer, it does not seem that such a project has any high value. For example, if you want to enter a new market with a new digital product faster than competitors, the requirements are probably unclear, the specification is incomplete and there will be late changes in the project because it is not at all clear what exactly the customers want and what the market looks like. Nobody can know this. Customers only take action if they like the project.
This is where agility can show its strength. I immediately admit that such projects often deviate from the original plan and can be called a failure. But what is the result? Wouldn’t you simply deal with such projects and take advantage of such a market opportunity? Take measures to pretend stability but then hinder the project?
In other words: the projects that fail more often, according to the study, may be projects that generate particularly high value.
What is Agile?
Apparently based on the definition of agility from the study Agile manifesto. So the Agile Manifesto should say something like: “If the requirements are clear at the beginning of the project and there is a complete specification, delete this document and make sure that nobody remembers the requirements and specifications – otherwise the project does not meet this manifesto.” And it should also say “If there are no significant changes later in the project, invent some – otherwise the project does not meet this manifesto”. I looked it up: this is not in the Agile Manifesto and there is no paragraph that can be interpreted this way. On the contrary: the Agile Manifesto weighs options like following a plan and reacting to changes – and focuses on one thing, in the example reacting to changes, without evaluating the other option as unimportant – simply less important. Reacting to a change actually seems to be more worthwhile than sticking to a fixed plan that may not have even considered the change yet.
Even more problems
By the way, this study is based on a survey of 600 software engineers. Surveys can only reflect the subjective image of the person surveyed – in this case, of software engineers. Since software development is an economic investment, it would be advisable to also survey other groups, especially the various stakeholders. In the end, they are the ones who turn over projects. So who better to assess whether a project is really a failure?
And the study leaves another question unanswered: How many projects have clear requirements? 10 percent? 25 percent? 90 percent? The only thing that can be seen from the study is that the number should be large enough to make a statistically significant statement. If a lot of projects have unclear requirements, that would be a sign of a significant problem. Whether or not you can solve it and how is the next question–and it’s not so easy to answer. Requirements can’t just be wishy-washy, but, as mentioned, they can be vague because of their very nature.
and now?
The study is not really worth a blog post. It is a transparent maneuver to discredit agile software development and promote a different approach. But the study is still impressive – I have found references to the study in two different contexts. It makes sense: 268 percent is an obvious number and the result is surprising because agility should actually lead to more success. You just have to look inside! If you actually do, it becomes immediately clear what nonsense it is. A Youtube video The numbers were used to criticize other aspects of agility, not the problems with requirements as described in the study. Regardless of whether the criticism is fair or not: the video uses the absurd numbers from the study to attract attention.
If we act this way in our industry, we should not be surprised if we do not learn what really helps and therefore make no progress. Important work with sources is taught in school.
We cannot answer from the study whether agility is “good” or “bad”. But: The fact that we have to build software in iterations became clear in practice very early – in principle, when people started developing software in teams. Even if, unlike agile projects, you have relatively stable requirements. This becomes clear when you look at such precursors Professor Christian Floyd listensI also have this topic A presentation Held. It should also be clear that there are projects that perform well, for example, by reacting to changes rather than following a plan. The real problem is probably that agility as a term is now disappearing because genuinely “agile” processes are being introduced that do not conform to the original ideas and are not particularly helpful.
tl;dr
Our industry is vulnerable to biased studies. That’s sad. Clear claims attract attention.
(RME)
