Which one? #1

If you have been involved in any large project procurements, I'm sure you've encountered the use of weighted scale ranking of proposals.

These attempt to bring objectivity to the evaluation of proposals while dealing with a large amount of diverse information in the proposal.

They usually end up in a matrix such as:

I've grabbed this from an evaluation of an IT system.

There are numerous problems with the approach:

In most cases the weights are arbitrary: unscaled, uncalibrated and without repeatable reference to the real world. There is also scale compression for low weighted scores vs high weighted, misleading scorers as to their scores. They are nothing like a rating for uni grades where a score is weighted by the proportion of the academic program that the course represents.

The score can be sensitive to low weight 'herding': a number of high scores on low weight factors can overwhelm a high score on a high weight factor, such as is illustrated below.

Factor 'value' for proposal 'b' scores '30' for 'value', presumably an important factor, but is the proposal with the lowest score for this factor. It is nevertheless catapulted to the highest score by high scores in some low weight factors.

One could argue that the weightings as  a whole deal with this in terms of the objective of the project, but without evidence that the problems of scaling, calibration and variability arising from arbitrary assignment, the vulnerability of the scheme to even inadvertent manipulation or outright error is unaddressed. I wonder how many procurements of large projects have gone off the rail due to this approach?

There are a few ways of overcoming this. I will address them in the next post on this topic.

10 factors 8: risks

8.    Risks. Risk management is a great proactive way to solve potential problems before they occur. Identify risks early in the project and continue to manage risks throughout the project.

Tim Lister is sort of famous for his statement that risk management is how grown-ups manage projects. What does this mean?

Let's consider a major project risk: the delivery team will not be capable of achieving the project outturn performance (performance of the completed project output). We manage this risk by bringing people with the appropriate skills and experience onto the team.

A major risk is that there will be cost and time over-runs. How do we manage those?

Using 'reference class forecasting' we assess the liklihood of failure along the dependency lines of particular work package items. We then time the  related activities, with suitable coordination buffers, to allow for risk; so an activity might be timed all going well to take 2 weeks. But we know all does not go well, based on history, so, to achieve a 95% probability of performance, the activity is  provided a period of 3 weeks to complete. The project accommodates the 15%  at risk in a buffer.

We do not do risk management by sitting in a circle dreaming up risks and allotting them 'probability of occurance and magnitude of damage' to do a sum to produce a 'risk rating'. That's bumbledom. The great icon of risk bumbledom is a 'risk matrix. Cox has dealt with these. Others have also commented.

10 factors 7: issues

7.    Issues. Jump on issues as soon as they are identified. Prioritize and resolve them before they impact on your project. Take pride in keeping issues to a minimum.

Minor matters that the PM and his or her team must deal with pop up ALL THE TIME on any active project of non-trivial scope.

People usually end up with an 'issues log' sitting in an Excel spreadsheet. I advise against this. Excel is far from bullet proof and using it for mission critical information is courts disaster, even if you have a rigorous nightly redundant back up plan.

Using a bug-tracking tool from our IT developer friends might be a solution, or a custom built Access database  could be suitable. You could, of course, go the whole way and build a project information management system in Access that ties issues, risks and actions to project elements, work items, contributors (suppliers, partners, owners, approvers, etc.) and deliverables.

Examples are: Meridian Systems and, at the simpler end,  Project Perfect.

Prominent issues need to be tracked in production meetings and quickly updated. However, issues are only part of it. Decision items need to be tracked as well. What decision is required when and from whom. "Issues" aside, decision delays can be worse than an unresolved question on a project.

10 factors 6: quality

6.    Quality. Understand the expectations of your customer in terms of quality and put a plan in place to meet their expectations.

To borrow from Kant, who taught that  existence is not a predicate, I argue that 'quality' is not a predicate of projects. The correct predicate is 'performance'.

A customer seeks a certain level of performance, and possibly needs the PM's assistance to develop acceptance criteria for the performance sought.

Quality, care of the 'TQM' fad and its relations of past years, is not a thing separate from performance. As soon as it is treated as an overlay on production management failure is being courted because it is an 'add on' not an inherent part of an activity to achieve a performance outcome.

I also wonder at the functional difference between 'plan to meet their expectations' and 'put a plan in place...' The former is a very popular piece of clumsy writing that usually adds nothing to the simple present tense of the verb.

5. Control the work in progress

The overall control system I use in practical terms is the production horizon: when will 'x' be finished, compared to when it needs to be finished.

A look ahead program will help discuss such questions and Earned Value Analysis will help tell you where you've been, but you need leading indicators of success as well.

Commitments is one such: have you committed dollars to agreed activities sufficient for their timely delivery (e.g. you've got the next sub-contractor signed with sufficient time for him to mobilise to start work in time.).

10 factors 5: communication

5.    Communication. Make sure you keep everyone informed by providing the right information at the right time. Produce status reports and run regular team meetings.

Communication is more than just communication in projects, particularly the more complex projects.

The PM/PM office needs to have a very clear appreciation of the information flows and needs of the project, the sources and destinations of information, and what 'meta-information' to keep track of.

Regular systems of meetings are part of the deal, for sure, but they are only hubs in an extensive communication/information network that links the project to its environment, its own performance and its players.

Document management is an important part of this, with currency of documents being tracked to ensure all are using current information.

A project 'data dictionary' to maintain the project's set of definitions and information baseline are important.

If a building/construction project, than a 'BIM' system should be considered, with a system to catalogue and organise the huge flow of information that such projects need.

4. Build a one team one goal approach

If the goal is the overall project and its mission; then 'yes'. But local goals change from time to time. Usually they concerned with task completion, hand off and collaboration, but the goals intertwine. A project is not that neat that at the operational level there is one goal at any time.

Of course, scale might have something to do with this. On a $500m hospital project with the total project team running to dozens of firms and hundreds of people and a WBS of several thousand items, then many goals all the time!