A lot is improved since Agile Manifesto. Still, one of the major complaints from non-Agile community is architecture and long term quality. The usual statement is that Agile people don’t care about architecture.
The price of short cycled incremental and iterative delivery of business value is paid with low long term quality and a mess in overall corporate IT landscape.
There is some truth in this, and there is a solution – already applied in corporate world- which does not only remove this argument against Agile methodologies, but delivers a better long term perspective than we ever had before.
The way IT department looks at software product development – custom-made, COTS, or something in between – is based on total time and cost a software product is delivered in a project, and operations’ time and cost. It is usually divided in 30% development and 70% operations.
Because of this division, many IT departments tend to put more effort in optimising and emphasising operations at cost of software development. Also, during operations, a developed product can not actually be used to serve business. Therefore, software development itself is seen as something to be done as little as possible, since it disturbs day-to-day operations.
Of course, the irony of costly operations because of bad product development is noticed, but difficult to solve by itself. These IT departments find costly operations perfectly normal, until business becomes really unsatisfied with IT and them saying “No”, or being late so often. This type of IT departments is rapidly becoming part of the history.
In many of these cases, management wants to finish a project so badly, that project is stopped prematurely at cost of quality. So called operations, full of fixes and other repairs, becomes even more expensive. These repairs are usually not real solutions, but patches full of workarounds, creating a big ball of mud.
In other cases, a similar IT department must continue with a never ending project . Business demands a working product. So, after many years, it is finally finished. Unfortunately, used architecture and requirements are already antiquated, and both business and IT are forced to start all over again, right after delivery in production. Costs are skyrocketing, not much value is produced and everybody is frustrated.
Suddenly, someone mentions Agile and Scrum, which will solve this project management nightmare. Business becomes happier, because the long wait is over, and real features are delivered a bit faster. Depending on how fast delivery in production is, you can actually use them sooner than normal.
But, IT departments, and mainly CIO and architects, are usually not that happy. First, these Agile teams are not really part of IT department, and neglect operations’ policies and existing architectural constraints. The primary focus is delivery of business value, at cost of anything vague as enterprise architecture, IT governance, etc. Operations teams are even less satisfied, since they could hardly handle delivery every 6 months. Suddenly, they are asked to do it every 3 weeks. Impossible, they say. There is a constant conflict. Many “Agile teams” accept this “fact of life” and deliver after 6 months or longer, which isn’t really Agile anymore. I dare to say that majority of Agile teams in corporate world is stuck in this or very similar situation, or building something insignificant.
So, IT departments make another painful decision and start experimenting with DevOps teams. This reduces directly felt mess in production, since any problem had to be solved right away. The continuous inspect and adapt cycle, driven by directly occurring issues brings faster delivery and other improvements.
For most IT departments, the switch to DevOps can be huge, since the whole notion of being in stable operations mode compared to disruptive development phase is dissolving. It feels like being in constant chaos. New apps on any possible platform are popping up everywhere, and they cannot keep up. They complain:
Our IT landscape is becoming a chaos. We are still constantly replacing old with new with no real thought about long term goals. It is worse than what we used to have.
Quite often, this is true. In fact, yearly IT costs could be larger than what it used to be, while overall IT governance is becoming really difficult. An example is usage of cloud solutions to circumvent normal IT operations with in-house infrastructure. For a big enterprise, using cloud with very little governance is after 2-3 years more expensive than using own existing infrastructure. At this crossroads, which may also happen earlier, IT management has to make a decision:
- Limit this whole Agile thing to software development disciplines, and use architects and operations teams to enforce the quality, or
- Embed architecting as a discipline in a similar way as operations
Difference between the two options is huge. The companies I’ve seen choosing the first option, have slightly more satisfied business, but overall improvement in terms of effectiveness, speed and quality, is limited to 10-15 %. In this situation, there is a normal IT department with traditional operations, and there are multiple separate Agile teams working with business and in constant conflict with IT department people about going in production and dealing with architecture.
Fortunately, some companies and teams choose to embed architecting as integral part of DevOps. This means continuous care of overall IT landscape, collaboration between teams and experts, root cause analysis on any problem in software structures and integration between components, refactoring on architecture level, knowledge exchange etc. This continuous architecting becomes part of every iteration and normal software development. Normal inspect and adapt cycle goes much deeper than patching this current issue in production.
A penalty for this way of working is extra time every team will spend on dealing with these aspects, every iteration at cost of delivering features. From what I’ve witnessed, most customers don’t even notice. And if they do notice, the effect is increased respect for a responsible way of dealing with our profession.
But, the real advantage is the overall quality of IT landscape. The need for large and disruptive technology replacement projects is being removed. Instead, the overall landscape is continuously evolving with many small steps.
These small steps together are much simpler and less costly than big replacement projects. In fact, this might be the solution to a decades old problem in so many large enterprises: making a bold promise with costly technologies in a multiyear project, failing to deliver this promise, and making new promises in even larger replacement projects.