Lead Time

Lead time is how long it takes to get an idea into production, fully supported by operations. How can Dev Ops thinking reduce this time to the minimum?

When we have to, our development teams can make changes quickly and get them released to production faster than our regular, monthly release. We can do this in a matter of days, or less, if required urgently. Not bad, right? Could be better, though. An awful lot better.

Look at the likes of Netflix, Etzy, or Google. Their continuous delivery models allow them to make hundreds of releases every day. In fact each time code is checked in to trunk, it could trigger a release to production.

So we may be able to get code out there fast relative to the slow, regular fixed software releases. But we could not do so on a consistent basis. While our processes have automation doing much of the complicated bits, there is enough manual work to make daily, or even more frequent releases, a real headache that would lead to developer burnout if it continued day after day.

Not Everything is About Automation

If we focus only on the parts of the process that can be automated, from development through to release and monitoring, we may miss areas that increase lead time. There can often be a lot of delay and wastage before the new features get into the development process proper. I want to concentrate on those delays in this article. We’ll come back to automation later.

The lead time for our regular project work tends to be measured in months. There can be long gaps where no one is working on something, for example, we can be blocked by a subject matter expert that has not answered an outstanding question. Another example is waiting for approval coming from the executive level. But really delays can happen anywhere along the chain.

One issue we see regularly is the delay between analysis completing and development starting. The team initially spends time on a user story to analyse, estimate and agree the requirements with the business. But it could be some weeks before the story is agreed, signed off and scheduled.

There are many reasons for this, but the main thing is that it is hard to get people working in different areas, and with their own separate priorities, to contribute to this one shared thing. It is simply not high enough on everyone’s to-do list to be completed efficiently.

Another reason for delay is that a higher priority user story comes in at the last minute and so pushes out one that is ready to go for at least one sprint. This is often the case where the team had prepared a user story to remove technical debt. It gets pushed down the priority ladder to make way for a business led feature.

By the time a delayed user story enters the sprint, the development team can feel that they are unfamiliar with it. Then it takes time (or wastes time) going through the details again, but this is needed to re-familiarise the team with the requirements.

We have already mentioned in Focused Teams that having less changes on the go at once would improve a team’s productivity, but how can we avoid having to wait for externalities such as questions to be answered or approvals?

The Dev Ops Handbook proposes the formation of dedicated teams that include everyone that is needed to complete a project. This could include the business people, we can call them the subject matter experts, required to answer all questions.

We also want to make sure that someone from the operations team is in the group. Potentially an architect and security experts could also be included, but it depends on the change. These roles, like the subject matter expert, might change over time.

Having an executive be part of this team is also helpful. Or at least allow a direct communication channel to an executive so that approvals can be quickly achieved.

Co-locate them together. This is a key to making the group act as a team. They need to be separated and relieved from their daily duties while working on the project.

With such a team as the one above, focused on achieving a shared project goal, it should be possible to dramatically reduce the lead time for changes.

Let’s list some of the potential advantages:

• New initiatives will get to market faster. Opportunity cost is what we call a delay to entering the market; by getting to market earlier, we reduce the opportunity cost, and potentially gain market share.
• React to competition faster. We all want to be the innovators, but if another company is stealing customers from you with some shiny new feature, you may need to build the same or similar capability, quickly, just to stem the tide.
• Innovation. Having business people work day to day with IT opens the door to innovation. Ideas can simply happen out of a conversation. IT may have ideas they don’t know how to market. Business people have ideas they don’t think are possible. Mix the two groups and new ideas are sure to result.
• Shared goals. The group will have a shared goal that is more important than any of the other demands placed on them, allowing them to focus on this one thing, the next most important thing.

From a purely IT point of view there are also some key benefits:

• quick response time for queries about functionality from developers
• users can see the feature emerge as it is developed and offer feedback early in the development process, in time to course correct, resulting in better overall quality
• the business user can see the automated test cases and determine their worth in proving the system, and trust them to regression test future changes, speeding up the next development
• the business user can get a feel for how the development team works and it will help to make future requirements gathering smoother
• considerations for building and releasing the feature can be built into the development
• security issues that come up during development can be addressed immediately
• architectural decisions can be made quickly
• new products or tooling can be approved by having access to executive decision making
• lead time will be reduced

I focused in this post about reducing the lead time for getting a new feature released, but in particular the time it takes to get the work into the development team. With the subject matter expert sitting with the team, the knowledge transfer to developers does not rely on weekly meetings or exchange of email, it can be achieved almost immediately by turning around and talking.

As well as getting the requirements understood more quickly, I contend that there will be an increase in quality by frequent feedback to and from the business owner. Also security, architecture, and delivery can be improved by having access in the same location to skill sets that do not ordinarily sit in a development team (architects, security experts and operators).

The executive, or at least their nominee, completes the team by being able to get decisions made, whether on overall strategy or budget calls. They will ensure that the development brings value and is aligned to the high level strategy and goals of the company.

Actions:

1. Look at a recent feature that you developed. When was the proposal first put into words?
2. How long did it take until the idea became a user story?
3. How long did it take for the team and stakeholders to agree that the Definition of Ready was complete? (See this post for a discussion on Definition of Ready).
4. How long did it spend in development? How much rework was there? Did development and testing go well?
5. How long did it take to be released after development was done (i.e. met all the requirements for your definition of done — you do have one, don’t you?)?
6. Were operations willing to support the new feature immediately? What handover was necessary? What supporting documentation did you produce to assist operations?
7. Was the feature development a success? Is it being used in production? Can you even measure its use? How many production issues relating to the feature have been logged?

I hope that you are happy with the answers you gave to the above questions and that you found everything easy to answer. But if you were not happy or were not easily able to answer the questions, then perhaps spending a little more time on that area will help.

Make sure to involve those closest to the issue in the discussion. If operations had a difficult time with the feature release, get them in the room to figure out why. If the requirements were poor, work with those involved to improve the next feature they are involved in.

Everyone involved from idea inception to release and support can impact the speed of releasing new features. Work together, if not as a dedicated team, then at least by participating in regular reviews, to try to identify problems and come up with ways to alleviate them. Embrace your constraints and use them to create a better software development life cycle.