Project Retrospectives

A kinder, gentler, more productive way
to learn from past mistakes

Most every organization, at one time or another, has experienced the pain of a Project Post-Mortem. The story goes something like this:

From the beginning, the Project Team knew that not only was the schedule unrealistic, but the requirements were ill-defined, the resources were far less than what was needed, and commitments were being made to key customers without consulting R&D.

Everyone worked as hard as they could on the project, often spending 60-80 hours per week. In the end, the product was delivered several months later than planned, had fewer features than were promised, and had too many bugs.

Customers were not happy with what they received since it was months late, didn’t have all the promised features, and crashed or locked up frequently.

Management asked for a Post-mortem to review what had happened. The Project Manager scheduled a meeting about a month after the software was released. People were given the usual short notice that this was happening.

At the Post-mortem, the Project Manager tried to control the discussion by asking for examples of things that worked well and things that didn’t work well. After about a half-hour of tense discussion, the blaming and finger pointing started. QA blamed Development for being 6 weeks late and for delivering code that was very buggy. Development blamed project management for not spending enough time clarifying requirements up front and for allowing requirements to change right up to the last weeks of the project. By the end of the meeting, everyone was angry and nothing of value was recorded.

On the next project, the same mistakes were made again.

This “story” unfortunately happens far too often. Often, when post-mortems are held, people tend to either say very little or not show up. As a result, nothing is learned and nothing changes.

The term “post-mortem” has some negative connotations to it. People have come to expect that “post-mortems” are nothing more than forums for exacting retribution and venting frustration. In terms of return on investment (ROI), “post-mortems” usually rank low since little of value is learned. While Management may be satisfied that a post-mortem was held, the organization continues to make the same mistakes on the next project because nothing was learned and nothing was changed.

Here then is the first thing to learn about project post-mortems:

If you always do what you’ve always done,
you’ll always get what you’ve always gotten.

Thankfully, there is a better way.

Enter Project Retrospectives…

The better way is called Project Retrospectives and was developed and refined by Norman Kerth [1]. Norm has developed a much more effective way to glean “wisdom” from projects.

Post-mortems are meetings that typically occur with no planning, no preparation, are led by a project team member who may or may not have good facilitation skills, and may take only a few hours at most.

Project Retrospectives on the other hand, are “events”. Here are some key attributes:

• They are planned

• People come prepared – click here to learn how to prepare…

• An experienced facilitator plans the event and leads the discussion

• Retrospectives typically take 2 -3 days (yes, days)

By making an investment in learning from the past, organizations can significantly reduce the likelihood that they will repeat the same mistakes on the next project.

Planning a retrospective is critical to maximize the ROI. The facilitator meets with the project manager and selected project team members to get a sense of what happened, to gather relevant project data, to understand the organization (roles, responsibilities, and reporting relationships), get a sense of the culture, and to identify any hot button issues. The more familiar the facilitator is with these issues the better able he or she will be to handle them when they surface.

Searching for Wisdom

Wisdom is defined as:

• accumulated knowledge or erudition or enlightenment

• trait of utilizing knowledge and experience with common sense and insight

• ability to apply knowledge or experience or understanding with common sense and insight

When project teams work together, they learn stuff. Some stuff is not that important and some stuff is very important. The important stuff has the potential to become “wisdom”.

How do our experiences on projects become wisdom? Well, when we collectively can discuss our experiences in a safe and open environment, those discussions can often lead to “light bulb moments” – when people realize how their actions, their procedures, their behavior, etc., affects others and, as a result, affects the product.

Safety and Trust are Essential

One of the key problems associated with your typical post-mortem is that many people may not feel that they can speak openly for fear of retribution or because they don’t know how to tactfully say what’s on their mind. Kerth addresses this key issue by introducing the notion that safety and trust are essential. He says:

“Regardless of what we discover, we must understand and truly believe that everyone did the best job he or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand.” [1]

Kerth’s Project Retrospectives are based on this notion and everyone participating must buy into this. Respect for individuals is also a very important aspect of Project Retrospectives. Kerth [1] defines some ground rules that are set out at the beginning:

• When someone is speaking, we will try to not interrupt them

• We will accept everyone’s opinion without judgment

• We will talk from our own perspective, and not speak for anyone else

• There will be no jokes made at the expense of anyone in the room

• When someone is holding the designated object, then only that person may speak

• While everyone is encouraged to contribute, your participation in discussions and exercises is optional

The designated object should be something silly that helps defuse any tension that may be present. As a facilitator for a recent Project Retrospective, I used a Yoda Pez Dispenser as the designated object. At the end of the Retrospective, everyone received their very own Pez Dispenser as a reminder of the event.

Once the ground rules are established, the team can now work on creating an environment of safety and trust. Kerth recommends doing this with some exercises.

The “Create Safety” Exercise [1]

The objective of this critical exercise is to help create an atmosphere where everyone feels comfortable talking openly about important issues. Without a safe environment, we can’t get to the real issues that need to be discussed.

Kerth [1] suggests measuring safety by using a secret ballot and a comfort scale…

Based on the results of the vote, the group determines if:

• Everyone feels reasonably safe talking openly

• Or, additional steps are needed to increase the level of safety…

The secret ballot is held and the facilitator records the results. If they are mostly 4’s and 5’s, there’s no problem and the team can move on. If the results are mostly 2’s and 3’s, there is a problem and the facilitator needs to address it.

To address the issue of safety, the group is asked to stand and form natural affinity groups. This is done by having the team members move near people with whom they have worked closely with on this project and away from people with whom they worked very little. Team members may only move themselves and may not talk in this exercise. Once the affinity groups are formed, each group finds a corner of the room and they brainstorm ideas for how to increase the level of safety. Each group brings a list of items that are then discussed with the whole group. Once the whole group reaches consensus on these ideas, a second secret ballot is taken and hopefully, the level of safety is now 4’s and 5’s…

The bottom line is the Retrospective can’t begin until everyone feels safe talking openly and honestly about their experiences. Once everyone feels safe, we can move on to the next exercise…

The “Define Success” Exercise [1]

Kerth [1] says that purpose of this exercise is to “establish the idea that the project’s degree of success is not a relevant measure to use while people are trying to learn from their experiences.” Every project, even acknowledged failures, have aspects about which the project team should be proud of – in addition to things that can be improved upon.

This is a good exercise to use immediately after the Create Safety exercise because it helps get people talking and listening. An experienced facilitator can tell if people really do feel safe by the nature of this discussion.

We begin the Define Success Exercise by asking the project team members for their definition of success. This activity results in the first of several lists that are created during the Retrospective. Each response is listed.

We then discuss this definition:

A successful project is one on which everyone says “I wish we could do that over again – the very same way.”

Based on this definition, the project team is asked: Was this project a “success”?

The facilitator leads the discussion on this question – ensuring that everyone has the opportunity to talk – coaxing those that seem quiet – to help build the feeling of safety and trust.

We then look at some (albeit dated) industry data [2].

• 31% of projects are canceled before they ever get completed

• 53% of projects cost 189% of their original estimates

• On average, only 16% of software projects are completed on-time and on-budget

• For small software companies, 78% of their software projects are deployed with at least 74% of their original features and functions

We now compare what the project team accomplished against this backdrop. Again, the goal is to get people to start talking about the project in a way to help people see that there were successes as well as failures – all is not bad.

Indiana Jones meets CSI

The real work in a Project Retrospective is a lot like a cross between Indiana Jones (archeology) and the TV show Crime Scene Investigators (forensic science). Each person is asked to bring to the Retrospective specific artifacts that they felt were important to the project from their perspective. Artifacts can be documents, e-mails, coffee mugs, photos, or anything that has specific meaning relative to the project. We make sure to bring project schedules (planned and actual) to discuss. We go around the room and each person discusses the artifact they brought, it’s significance to them and to the project.

Once this discussion is completed, the team is ready to start building the Project Timeline…

The “Develop Time Line” Exercise [1]

Part of the problem with post-mortems is that most people tend to remember only what occurred during the last part the project. The end of most projects is often the most contentious and stressful.

We need to uncover and discuss evidence of what actually occurred throughout the entire project – not just the end. The way to do this is to discuss the artifacts and then build the project timeline based on factual information.

The facilitator asks the affinity groups to move to their corner of the room. The work is inclusive not consensual which means everyone’s opinions are valid. Each group identifies significant events that occurred during the project as follows:

• Write each event on a post-it note along with approximate date…

• Anyone who thinks an event was important creates a post-it note for it…

• Use the artifacts present in the room to stimulate your memory…

While the affinity groups are working, the facilitator hangs flip chart paper on a large wall in the meeting room and months are noted along the top.

Once the affinity groups are finished, they start placing their events on the timeline. As a result, much discussion ensues.

Eventually, all the post-it notes are placed and everyone is general agreement with the result. Now, stand back and see what actually happened from start to finish!

The “Mining for Gold” Exercise [1]

Once the timeline is created and its significance absorbed, it’s time for the team to perform the last and most important exercise. For this exercise, the facilitator creates five lists on five flip chart easels.

• What worked well that we don’t want to forget
  
  This list is for those things that most agree worked well and that we want to continue doing on the next project…

• What did we learn
  
  Here is where we document cause and effects that have lead to learning…

• What should we do differently on the next project
  
  This list is for issues that need to be done better next time

• What still puzzles us
  
  This list is for issues that the project team doesn’t have enough information or skill to resolve. The issues on this list often require training or more thorough investigation…

• What do we need to discuss in more detail
  
  Issues on this list are things that need further discussion, clarification, information…

Usually, the discussion will take off on it’s own. If necessary, the facilitator can help jump start the process by having the team look at the timeline and asking questions such as:

• What event is most significant?

• What event surprised you?

• Are there any patterns that you can see?

• What events don’t you understand?

As the discussion progresses, the facilitator (or a volunteer) adds items to the five lists as agreed by the team. This is the most important part of the process! By spending time to build a safe and trusting environment, you’ll get the most valuable results. This just can’t happen using the old post-mortem approach.

Bringing Closure to the Process

The Project Team determines when the Retrospective is done. When that happens, the facilitator can capture the information from the five lists along with other important information from the event. This information is reviewed with Management. Specific action items are identified along with a responsible person and a date for each action item.

This is the part of the process that leads to change. If Management resists change, I would suggest reminding them of my favorite mantra:

If you always do what you’ve always done,
you’ll always get what you’ve always gotten.

Till next time…

On a sad note…