Percent Complete – The Seven-percent Solution

He raised his eyes languidly from the old black-letter volume which he had opened. "It is cocaine," he said, -- "a seven-percent solution. Would you care to try it?"

- Sherlock Holmes, in "The Sign of the Four"

In my previous post I extolled the virtues of ETC for estimating remaining work on a task.  But percent complete is so much more frequently used, it must have some virtues.  So, yes, it has the virtue of being easy.  Easy to calculate or easy to get a task owner to report.  So easy that it is addictive.  Yet the accuracy is so bad as to make it worthless.  Further it reinforces the wrong behavior on the task owner.

Most popular project management tools will calculate percent complete for you.  Depending on the task type, the tool can calculate percent complete on duration or work.  These calculations offer no predictive value, so they offer nothing to the PM as an early warning indicator.

Another way to get percent complete is very effective for tangible work.  Percent complete originated in the construction industry and it’s pretty easy for the foreman to walk around and see how complete a job is just from looking at the framing, siding or flooring.  So percent complete might also work acceptably with testing, if you carefully track total number of test cases and number of test cases executed.  But this doesn’t work for most software development tasks.

Alternately, the PM can ask the task owner how far along they are.  But the software development literature is littered with examples of problems from this.  I’ll describe a few.  First is that it is subjective:  unlike ETC, there is no way to validate the estimate or know on what it is based.  The PM is entirely dependent on the good faith of the task owner to get a meaningful estimate.

Second, it can progress to successively larger numbers without converging.  Three weeks into a four week task, the owner might report being 80% complete.  Since it is ahead, this is good.  The next week, though, the report is 90%.  Now the task is behind.  At the end of the fifth week you are told it is 95% complete and the next week that it is 98% complete.  What you really want to know, though, is “When is it going to be done?”  And you really wanted to know that around the second or third week.

By knowing how much work (effort) has been put into the task and getting the ETC (work remaining), you would know the answer.

The most important reason, though, that percent complete fails as an early warning indicator is that it rewards bad behavior.  A programmer can get a task to the 80% or 90% state and feel good.  They may have put in 20 or 30 hours, but have a stumbling block removing that last almost inconsequential error that would allow them to declare it complete.  At that point, they can get a lot more satisfaction – positive reinforcement – by getting another task to the 80% or 90% state in the same amount of time that it will take to complete this task.

ETC works effectively because the reward scenario is exactly opposite.  If the task owner gets to that point where there are just a few hours remaining to complete the task, they can get a lot more satisfaction for completing the task rather than starting a new task and having multiple tasks with just a few hours remaining on them.

If you do have to use percent complete – and sometimes we just can’t avoid it – there is a best practice that alters the reward system.  Instead of allowing the task owner to estimate percent complete, use a rule-based system such as:

·         Before the task starts, it is zero percent complete

·         Once started, it is ten percent complete

·         If it is near completion, it is 50% complete

·         Only when it is complete complete does it get set to 100% complete

With this approach, task owners, from my experience, are much more likely to complete their tasks without letting them drag on indefinitely.  This may not be a 100% fix, but, with apologies to Arthur Conan Doyle, it is at least a seven percent solution.

Do you have any examples of good or bad early warning systems?

Ode to ETC – The Task Overrun Early Warning System

History is a vast early warning system.

                                                - Norman Cousins (15Apr1978)

Sometimes you have to use the tool you have available, whether it is any good or not.  So it is with getting task completion status as percent completes.  Much better though is using the best tool for the job.  This blog exists to share project management best practices, so today I extol the virtues of ETC.

As PMs, our job includes delivering a project according to plan, knowing when the project is off plan (and acting to get it back), and communicating status to stakeholders.  Other people – our project team mates – do the important work and we rely on them for the information to do our job.  To be successful, we need a tool that helps our team mates communicate that information and for us to receive it accurately.

The well-defined task is the starting point.  Knowing whether the task has not started, has started (is in progress), or is completed is beneficial.  It helps us with history, with past events, but doesn’t communicate enough about the future.  Looking back is easier and more comforting, but it’s not the most beneficial nor the most important activity.  The most important objective with task status is to look forward, to be able to determine that the task will progress to plan.  Only by looking forward effectively are we appropriately serving our stakeholders.

The best tool for tracking progress is Estimate to Complete (ETC).  ETC works best when tasks are estimated, scheduled and reported by effort (as they should be), but even with duration scheduling is still superior to other methods of communicating what remains to complete the task.  When you ask the task owner “How much more work is (or how many more hours are) required to complete this task,” it forces them to re-estimate the remaining work, but with the advantage of everything they’ve learned by progressing to the current state.  The values get progressively more reliable.  Further, as the task gets closer to completion and the ETC value gets smaller, the task owner has an incentive to complete the task (just to get it off their plate), as opposed to the negative incentive system with percent complete reporting (see my next post).

No predictive system works effectively if it is rule based.  That is, if the task was originally estimated at 40 hours and the owner has completed 24, they can’t satisfy the ETC just by reporting 16 hours to go;  that defeats the predictive value of the technique.  They have to actually re-estimate the remaining work, which could be more or less than 16.  Tip:  be very suspicious if a task owner just keeps reducing ETC by the number of hours worked on the task.

ETC, when used consistently and properly, is a PM’s best friend.  ETC is an early warning indicator into task delays, under-the-cover scope creep, or a task owner that may be over their head or not performing.  As an early warning indicator, ETC is much more reliable than other methods.

One thing I’ve found using ETC is the necessity to frequently reinforce that the task owner must re-estimate the task each week.  Do you have any examples of problems using ETC?

The Root Cause of IT Project Failure

Art is making something out of nothing and selling it.  Frank Zappa.

There exists anecdotal evidence that Information Technology (IT) projects fail at a greater rate than other types of projects.  Why is this?  Are we that bad at delivering projects?  Or is it that we’re that bad at planning?

Have you ever asked an artist to predict how long it will take to create a novel piece of art?  Or tried to get a marketing team to plan the schedule for coming up with the next campaign?  Why are we as IT teams so ready to come up with these precise, scientific – even optimistic – schedules?  What we do is often just as novel and just as creative.  But we’re ready to plunk down numbers on a schedule with the confidence of divine inspiration.

Project Management is a special kind of management.  That is, project management is the poster child of Management by Objectives (MBO).  The unique aspect of MBO – and thusly project management – is the objectives (milestones) with which are either tracking or off track.  To have objectives entails planning.  In project management, we call the objectives milestones and they are points on a (time or cost) schedule.

I want to distinguish here among varieties of projects.  Like the abstract water colorist who can knock out wilderness landscapes with the reliability of an assembly line, some shops do many very similar projects.  With those conditions your planning can get fairly precise.  It’s those other projects I’m talking about.  Not high risk, no, not those.  Rather, the high novelty projects.

Projects where you and the team have never really done anything like this before.  You’re breaking new ground and every design activity is a unique creative effort.

How can you say up front how long this will take or how much it will cost?  But we do.   And we commit to these numbers.  Why can’t we be like the artist or the marketer?  Why are they so much smarter than us when it comes to these commitments?  Is it because they’ve learned from their mistakes?

I assert that what we’re really predicting in our plans is not that it will take this long, but rather that it will take at least this long.  That it can’t realistically be done in less time.  And some of the engineering work can be planned.  For example, we can lay out that we will complete requirements in x days.  Or that development will take so many weeks.  But how long does it take to research alternatives?  How many prototypes will we have to do before we get it right?

IT projects – that is, novel IT projects – are just as creative as the artist carving the block of marble or the ad team brainstorming the perfect campaign.  And we should be just as resistant as those artists to committing too much (too little?) in advance.  We do ourselves, our team, and our stakeholders a disservice otherwise.

The root cause of IT project failure:  committing to a plan too early.  We don’t fail on delivery.  We fail trying to deliver to a faulty plan.

What happened when you promised when you should’ve waited?  What stories can you tell about trying to deliver to the wrong plan?

Project Failures & The Chaos Reports

Diane, the Chief Financial Officer, had invited the partner of the local PwC office for this meeting.  She had only done that once before, so he knew this discussion was special.  “How do I guarantee project success?” she asked.  He didn’t need it explained that this was probably a make-or-break project for the company.  He grasped all of the nuances of this question.  After carefully considering his thoughts, he responded “There are many ways to improve the probability of project success, but none of those guarantee success.  In fact, you really can’t guarantee project success.  But there is another way:  Call the activity an experiment or research or a prototype.  Regardless of the outcome, you can claim success.  Just don’t call it a project.”

I want to take a couple of posts and discuss project failures.  In this post I’ll discuss the often quoted Chaos reports, which applies to all types of projects, and in the next post will discuss more specifically IT project failures.

Let me start this discussion by stating up front that I’ve never read a Chaos report.  The Chaos Report is published by The Standish Group and costs hundreds of dollars (or more) to purchase.  My guess is that most people who reference the Chaos Report are, like me, only familiar with the press releases, headlines, and secondary references.

Because I haven’t read the report, I can’t criticize it.  I can, however, criticize all the people who reference it to make a point about project failure.  Allow me to present a couple of examples of why the headline number of project failures is misleading and misrepresentative.

In the first example, I will use an analogy.  Assume that most people who get a dog want it trained and that most of those people choose to do it themselves rather than hire a professional.  Further, most people who go it themselves don’t dedicate enough time, don’t have the experience, don’t follow through, and thus don’t succeed.  Now, let’s survey all the people who bought dogs last year and ask them about the success of their dog training efforts.  The results, of course, will show that most dog training efforts ended in failure, despite that virtually all professional training was successful.  The headline (most failed) completely misrepresents the success of professional dog trainers.

My second concern is with the definitions of success and failure.  “Strategies for Learning From Failure,” (Amy C. Edmondson, Harvard Business Review, April 2011) describes a spectrum of failures ranging from blameworthy to praiseworthy.  Just because it didn’t meet the original (optimistic) objectives does not mean that the project was a failure.  In addition, different perspectives of the project can produce completely different opinions.  For example, both the Sydney Opera House and the Ford Taurus went tremendously over schedule and budget, but both today are regarded as tremendous market successes.  In another example, a project could be started to introduce a new product into the market.  During the exploratory phase, the team determines that it cannot meet the business objectives and it’s cancelled. The brand manager considers the project a failure because the product did not launch;  the CEO, however, considers the project a success because the company did not over invest into a failing proposition and could redirect the funds to other more promising opportunities.  So how can I accept the Chaos report headlines when the same project can elicit either success or failure results depending only on who I ask?

There are many other reasons why the headlines should be ignored:

·         If the organization has failed projects, what is their success with operational efforts (i.e., what is the organization’s success baseline in general?)

·         How does organizational project management maturity correlate with project success?

·         How do the respondents differentiate between project vs product success and failure?

·         How does The Standish Group address different viewpoints of success vs failure on the same project?

·         How does project manager experience, autonomy and authority correlate with project success?

Taken together, if a barely profitable organization with poor manufacturing processes and that had no experience delivering projects asked one of their middle managers to lead an effort to bring out a revolutionary new product, told the manager how much was budgeted, how long it would take, and who would comprise the project team (but the team members would continue to report to their current units), who would be surprised when, after a few months, they cancelled the project and called it a failure?  How many of the projects in the Chaos Reports fit this model?  If The Standish Group reported that most projects with these criteria failed, would it be newsworthy?

I have every expectation that The Standish Group conducts their research professionally and competently.  Again, I’m not criticizing The Standish Group or their Chaos Report.  I am saying that we can’t use the headlines to understand project success or failure rates;  we need the demographic details that are contained in the detailed report.  Those details, I am sure, would provide clarity on something much more important than the frequency of project failure – they would provide insight into conditions for project success.

Wouldn’t you bet those details show that projects in organizations with mature PM processes and run by qualified project managers are successful at a much higher rate?

The Project Manager’s Cycle – Redux

“What does a project manager do?  What value do I add to your project?  It’s true that I don’t contribute directly to the product or service for which the project exists.  However, if I’m doing my job well, then I add significantly to the productivity and value of the team by improving communications, eliciting priorities, and focusing the team on the key deliverables.  Further, I am your agent, seeing that we perform to plan and communicating back when we diverge from it.”

The Project Manager’s Cycle includes these activities:

·         Validate the metrics

·         Validate task status

·         Incorporate approved change control

·         Reschedule/Replan

·         Check variances and progress/productivity

·         Communicate planned/expected resource utilization to resource managers

·         Develop earned value reports

·         Review risks and mitigation actions

·         Review issues and action items

·         Review commitments

·         Conduct client/sponsor status meeting

·         Conduct project team meeting

·         Publish the project status report

A baker’s dozen posts on the core project management elements of monitoring and controlling a project.  This series on The Project Manager’s Cycle been a pleasure to write and has also been beneficial for me.  I developed the outline of the series several years ago, but this journey required that I really flesh out the details, relationships, and components.  I appreciate all of my reader (yes, that would be you) who stayed with me through this.

A picture is worth a few words, it’s said, so I’ve tried to rough out the series in a diagram.  If I ever figure out how to make it available as a download, I’ll enable that.  In the meantime, leave a post or drop me an email and I’ll reply with a full-size PDF of version 1.0 of the diagram.  Next time I’m feeling ambitious, I’ll add the inputs and outputs, though I’m concerned that will make the diagram too busy.

We stepped on the ferris wheel together in January to start this ride.  We’ve gone around the cycle and now it’s time for us to step off.

What of this series has been the most beneficial to you?  Where have I not been clear and need to expand or improve the discussion?

The Project Manager’s Cycle – Publish the Project Status Report

“Project status this week, in a nutshell, is that we are continuing to stay on schedule, but this is despite your team’s contribution.”  After the initial chit-chat, Srini intentionally opened the status meeting with a statement to get everyone’s attention, especially the Sponsor and Owner Jack.  “Our project team has been working around the delays of returning document reviews and approvals.  We’ve pretty much exhausted the continuing work, though.  As you can see on the status report, there are several deliverables over due for approval and the draft requirements document is held up.  If you think these durations will continue, then I recommend that we extend the time your team is allocated for these activities and extend the time line.”

With this post, we conclude documenting the activities in The Project Manager’s Cycle, the Monitoring and Controlling activities that the project manager performs each reporting cycle for the life of the project.  All of these activities (with the exception of the Client Project Meeting) build each week to the Project Status Report (PSR).  The PSR encapsulates all of the information the Sponsor/Client/Owner and all Stakeholders need into a compact package.

The PSR should document both the current status of the project (in summary) and progress since the last report.  The essential elements of the PSR include an Executive Summary, Accomplishments, Planned Accomplishments (in the next reporting period), Deliverable/Approval Status, Change Request Status, and Issues.  Accomplishments and Planned Accomplishments are reported relative to the plan (project management is the classic example of Management by Objectives).

Note that traffic lights (  Red,  Yellow,  Green) are not in my list of essential elements of the PSR.  I am ambivalent about them.  When used properly they are often valuable, but they are even more often used improperly.  If you use traffic lights, the meaning of each color must be defined clearly and all of the stakeholders reading the PSR must know what each color means.  Further, a single light is really not meaningful.  After all, the ultimate purpose of the lights is to show where external action is needed and one broad status doesn’t provide the specificity needed.  Rather, what I’ve seen that works best are lights representing Schedule, Scope, Budget, Resources, and Closure/Acceptance repeated for each project phase or (preferably) each deliverable.

The PSR is probably the most important document the PM produces and I can’t cover all of the nuances in one post, so expect to see more on other elements of the PSR in the future.

Do you have a project status report best practice to share?  How about a PSR disaster?

The Project Manager’s Cycle – Conduct Project Team Meeting

“Alright, let me take a few minutes to update everyone from the Sponsor meeting.  You got the minutes, but I’ll drive into some of the content between the lines.  Then we’ll go around and let everyone give their update.”

I like to hold my project team meetings early in the week – Monday afternoon or Tuesday morning – and focus on what will be accomplished by the end of the week.  Over the years I’ve attended many other team meetings.  Those experiences, like my own early experiences where I focused on getting a status from everyone, provided a very accurate after-the-fact picture of project slippage.

We near the end of this series on The Project Manager’s Cycle.  The project team meeting is a different animal than the Client Project Meeting.  It doesn’t need the formality.  It’s where you surface the problems and develop an action plan for responding to them, so that you can go to the client with solutions, rather than problems.

I found that meeting early in the week and asking the team members what they were going to accomplish by the end of the week was effective at focusing their energies as well as driving out issues before they occurred.  There are only a few possible responses.  For example, if a team member explained that they wouldn’t complete a task on schedule because of a blockage that I could eliminate, I could do my job, keep them productive, and avoid problems.  Or if they commit to accomplish things that aren’t in the schedule, that creates an opening for me to ask questions to understand the variance.

The best case is if they commit to complete activities that are aligned with the schedule.  For most people, that duration is short enough that they have visibility into anything that would prevent them from completing the task.  So I can be fairly confident that if they say they’ll get it done that week, they will.

There’s one more question I ask, though, that builds on the commitment.  “What can prevent you from completing this task on schedule?”  This question is subtly powerful.  The inexperienced or naïve may say “Nothing.”  But this is a trap.   If they subsequently fail to make the commitment they cannot then roll out a series of excuses.  They have already stated before their peers that the only reason they would fail to deliver is their own.

The experienced, sharp, connected team members will use this opportunity to list all of the possible causes of failure or delay, which is just what I want.  I write the list down, then go through each item one by one.  These are the detailed task risks that you don’t generally hear about until they are brought out as excuses after missing the date.  By exposing them before the due date, as a team we can qualify them, prioritize them, determine the appropriate response, and assign an owner before rather than after the fact.  If something does happen, we have evidence of anticipating the problem and proactively responding.  Further, it gives me the opportunity to ramp up client or management resources in anticipation of a problem.

Further, when team members see that I am using the information to make them successful, they become even more open and trusting.

In terms of process, the project team meeting consists of the activities publish the meeting agenda;  conduct the meeting; and publish the minutes.  Inputs are team member individual status reports, the project schedule, and minutes from the Client Project Meeting.  Outputs may include updates to the schedule, Project Status Report, risk register, and issue log.

Most team members are forthright and dependable.  The challenge, though, is recognizing and motivating the remainder.  What do you do in your project team meetings to improve success, even from the challenging members?  Do you have other best practices to share for optimizing the success of your project team?