To meet the Definition of Ready, most users stories require a lot of analysis to be completed. Team members need to understand the requirements and they will measure success. Backlog Grooming (also called refinement) is the process of getting the story ready.

In my post, Agile Analysis, I discussed the benefits of getting analysis of user stories completed before starting work on them, compared to some of the risks with leaving the analysis until later.

In this post I want to talk about getting this analysis done within the team and the general process my team uses to get to completion.

A Product Owner should be responsible for ensuring that the team’s backlog is populated with relevant, estimated user stories that will serve the company’s goals in some way. These should have an overall priority but some may be ready sooner than others and get completed earlier.

The process of managing this list is usually referred to as Backlog Grooming. The product owner cannot do this task alone, however, it often takes input from the scrum team or the business analyst to fully specify the requirements, setting dependency ordering between stories, raising questions about any remaining unknowns, and ensuring that the conditions of satisfaction are complete.

I have found that it is useful to involve the whole team in the process of managing the backlog. Since the Agile philosophy intends that any team member should be able to pick up any task, it is critical that the team know the user stories well, and also how they interact with other user stories and the backlog in general.

User Story Grooming

It takes a lot of time, however, and it is important to involve the team at the right times. Here’s how our team might groom a user story.

A requirement is born

We don’t have formal Product Owners in my company. We do have a number of subject matter experts accessible to the developers. To decide on the priority between such groups of users, we need to escalate the call via Project Managers to the next level up.

This is not ideal; it would be better to have a single expert calling the shots for each of our scrum teams. It does give us access to users that really know their requirements well, people working with the systems everyday. So there are positive elements also.

Given a business need and the right priority I will be informed of the new requirement. Sometimes I provide a high level estimate before the priority is available, and at other times the requirement is mandatory and simply has to be done. The estimate sometimes helps to define the priority.

After an initial question and answer conversation with the expert, and in consultation with a business analysis, we create an epic. We use JIRA to capture requirements. Sometimes the change is obviously small and one or two user stories will suffice.

Before bringing the details to team, and in my team leader capacity, I spend time building up the explanation of what’s needed, often using a business analyst to gather data or find answers.

Estimation

We are usually under pressure to give story point estimates for all stories within a project at once. It can cause a lot of disruption to a sprint, using up all the time for normal grooming which can result in starvation of well analysed stories, or making us miss our sprint commitments.

I think the cause is that there is a worry that if developers see the requirement before it has been formally approved then the time wasted will have a high cost. And so we are not given the stories to estimate until after the lengthy approval process is complete.

On large projects, however, there is often a small number of features that are definitely going to be approved; the haggling and delay in approval is over the nice to have features. If project managers would give us the requirements that are guaranteed to come our way anyway as soon as possible, even if the entire project’s scope is still open, I’d be happier.

We usually close the estimation session to include team members only, however, sometimes we do need the subject matter experts there to answer questions. A business analyst will usually attend, but since our BAs float between scrum teams, they may not be familiar enough with our story point sizing to take an active part in the estimation.

The session will take the stories one by one, and play them back to the team. As we go through them, questions are raised, dependencies are pointed out and we call out any assumptions. Everything gets documented in the user story as we go. This is not exhaustive at this point, but better to write down anything that we come up with.

We use planning poker to come up with an estimate of the user stories. We try to stick to the Fibonacci series: 1, 2, 3, 5, 8, 13, 21, but if we hit the high end of this list, it is usually a reliable sign that the story needs to be broken up. We try to keep the maximum story size to eight points or lower.

Most of us use one of the plethora of free apps but we have some decks of cards lying around. Fingers on the count of three and writing on a back of a post-it note are also used from time to time.

These sessions can take time. There can be a lot of question raised, and we try to phrase them as assumptions that will need to be validated if the story size could be impacted.

The story points do not simply reflect how many hours the development is likely to take. Rather they can represent the complexity of the work. We take into account how difficult testing might be, especially if the group of users will demand more assistance to set up data, for example.

Analysis

While we are put under a lot of time pressure to give estimates, it can sometimes be months before the stories actually need to be delivered. This is project management trying to figure out when multiple streams will most likely coalesce and trying to avoid one team getting to the finish line too early (or worse, too late).

I believe this is a failing in our Agile methodology, since we often need a refresher session, again with our subject matter expert, to get everyone back up to speed. It causes unnecessary delay, but can give time to have assumptions validated and dependencies delivered before really getting down to work.

There are bound to be more questions. Conditions of satisfaction take time to be written, spoken about and negotiated. Sometimes a technical investigation is required to identify the areas of the code base that have to change.

We use an initial group session to come up with as many of the questions or unknowns as possible. These will either need to be resolved internally by the team, or we will have an external dependency, and we log it as such.

If it is clear that the analysis effort required is going to be high, then we may need to raise a spike. Spikes are outside the scope of this article and I will revisit in a future post. Usually this would be overkill, however.

In normal cases, I will assign one of the team as the technical lead on the story. They will figure out what is missing, chasing the subject matter expert if necessary, working with the QA and BA to document conditions of satisfaction and escalating delays to me or the scrum master.

The technical lead brings the story back to the team when they think they have documented all the details required to complete the story. It is their job to play it back to the team, answering any new questions. This may take a few iterations if there are details missing.

The difficulty with this approach is that the time spent can quickly build up. You might find yourself in a position that it would have been more straightforward to create a spike, have someone work on it as part of the sprint delivery and not simply as extra work to get done.

Splitting stories

We try to keep user stories below a certain size. Analysis has shown us that any stories of thirteen points or more is likely to be carried (for my scrum team at least).

Stories of thirteen points or over tend to simply limp from sprint to sprint, with endless new issues and delays emerging, and would sometimes keep going for two or three full sprints. This lead us to take the action to break down any story of this size or larger into one or more stories.

Doing this takes practice. The aim is to split the complexity, and to deliver some value with each of the pieces.

Early attempts where we split the front end from the back end parts of a system did not work. It was extremely difficult, if not impossible, to test one without the other.

A better approach is to take a simple scenario, all the way from back to front, with few variables, and use that as the first story. It should be possible for a user to see some of what they want working. In theory this functionality can be released without the remainder, delivering value early.

Later stories can add the rest of the features, but will build upon an existing base rather than be built from scratch. Also the testers have a better view of how they will test things and can offer feedback from the initial story on how to improve testability.

All going well, automated testing will be built to allow easier regression of the first deliverable. Even without this, the overall build will be smoother.

Definition of Ready

When is the analysis done? I included a brief list of items that we check off in Agile Analysis, mentioned above. We have used this in varying forms but are flexible with it, bending the rules as demands need.

In addition to these points we make use of JIRA sub-tasks to break the work down into smaller items. The amount of detail in sub-tasks can vary considerably. Each sub-task will be estimated in hours and this helps to track our burn down rate on a daily basis.

It is important to play back the user story at this stage to a product owner or subject matter expert. It may have undergone some changes, for example, we break down large stories into two or more from time to time. They need to review and to make sure that there is value being delivered.

At the final session before bringing the story into the sprint, which is often the planning meeting itself, we review all the details, double checking that it is indeed ready. All team members are responsible for this.

Problems

There can be a lot of work involved in getting the details above. Playing back stories to the team at least three times (initial estimation, grooming, and when accepting the story as ready) is especially expensive. Getting time from product owners and users can be equally costly. And all this is before any real development is done.

We use a few techniques to speed up the process.

Assign a technical leader

Every story our team looks at is assigned to a single team member to be the technical expert. This is usually a developer, since they can read and interpret the code, but occasionally it is the QA, especially when the bulk of the effort required is to add conditions of satisfaction.

In fact the technical ownership can change hands depending on the expertise needed at that time, or to allow for load balancing among the team. It is not practical to have team members trying to add details to more than one story at a time, unless they are closely related.

It is usually possible for a single person to come up with all the questions the team might ask, document them, and find the right person to get answers from. If they get stuck they can escalate the issue. If they miss something, then the final playback session should discover it and it will need to go back for further analysis.

Ultimately the goal of this person is to get the story into a state where all the boxes on the definition of ready list are ticked.

Example mapping

Example mapping is an exercise that is best done with the stakeholders in a meeting room with plenty of thick markers and index cards of four different colours. Here’s a good explanation of how to run one of these sessions.

Without repeating the details in the article above, this strategy will quickly uncover any remaining questions that exist, should help with a rough break down of the stories that are required, and give a good start on the various scenarios that exist.

We’ve used this technique recently with different sets of users - the reaction has been great. And the feeling in the team is that we get closer to a user story that meets the definition of ready sooner than using any other approach.

The Three Amigos

To avoid involving the entire team in every session, we find it useful to get the three amigos together. This usually involves the development team leader, the QA and a BA. There is a description of this on the Agile Alliance website.

Together we can get a raw story quickly into a usable story where we have documented the outstanding questions and who owns them and identified what tasks are remaining to complete the story preparation, such as adding conditions of satisfaction and doing technical code analysis.

Off line grooming

Getting people together in a room, or around a single computer, or even remotely inevitably includes delays and interruptions. Computers can be flaky and let you down. Some people get quiet and nervous in groups and don’t speak up. For others, the social animal in them can hijack the conversation and waste time talking about trivialities.

A useful technique is to do the grooming off line. Let one person prepare changes and then publish them to the team, highlighting the changes and asking for additional questions. A response indicating agreement on the changes should be asked for.

The latter is important, if you don’t get agreement then there may be something a team member doesn’t like and it will come up as an issue later. Or worse, they may not have read the changes.

Conclusion

Getting all the analysis done so that a user story can be developed as smoothly is time consuming. But having a defined process with recognisable steps and allowing individual team members to take ownership of the process can enable the team to become efficient at the process.

The most common technique of all within scrum is to use a task called a Spike, a time boxed, clearly defined, and of small scope task that can be done alongside user stories during a sprint to bottom out requirements. I’ll post more on this in a future article.

Try to keep in mind that Agile is run in Sprints. Fast paced development is one of the key goals. In order to get off to a flying start each sprint, try to ensure that you have removed all obstacles from your path.

Having a Definition of Ready as a key artefact of your Scrum team and checking it for every story will ensure that there are no surprise hurdles in front of you. If you have to bend the rules, only do so when there is simply no other choice. Document why and try to avoid it becoming the norm.

Actions

1. How does your team define readiness for a user story? Do you have a formal Definition of Ready? Do you subject each story to it before committing to it in a sprint?
2. If not, take a look a recent user story that you worked on that took longer than expected. Analyse it using the Definition of Ready here. Were there items missing from the analysis that would have reduced some of the delay?
3. If you do have one, how often do you review it? Do you train new team members on why it is important?