Does The Project Manager Need To Be A Specialist?

Question

“To be an effective Project Manager it is essential to have knowledge of the project topic – only civil engineers can be PM’s for civil engineering projects”

Yes

Do Project Managers need to have specialist technical knowledge? I would argue that they do. That is not to say that they need to understand the minutiae of the detail but a good solid background experience and a broad knowledge of the technical products will turn a so-so project manager into an excellent one. I have spent thirty years running projects of various shapes and sizes but mainly in the software and services industries. Should I be appointed to a similar sized construction project? I will be able to do it, eventually, but it will probably cost more, be riskier and a lot more stressful for all concerned. A project manager with experience and knowledge of the job in hand will be able to

• Hit the ground running, they will be able to formulate plans, build a delivery coalition and be able to understand how efficiencies and excellence can best be achieved
• Have credibility amongst the team and other stakeholders. This may exist in someone new to the discipline but it would be a brave client who commissioned a new nuclear power plant from a firm whose project managers were excellent at software projects and nothing else
• Converse sensibly with contractors and others about the problems that they face and help to convert these into a sensible contractual solution
• Identify and predict major risk areas with far more effectiveness than someone without these attributes, this will save time, money and too many ‘blind alleys’.

There are dangers of course, the primary one being that of the PM who simply has to get involved in all the detail, slowing things down, duplicating effort and not extracting the best for the team. There are also grave dangers where too much experience and knowledge mean that we keep re-inventing the problems of old and get ‘stuck in a rut’. Like most things it is a balance, whilst some knowledge is useful for the reasons identified above, too much knowledge can get in the way,

No

So now I’m intrigued. Was the manager of the T5 project an expert in luggage conveyance systems or an expert in modern building methods. Or thinking on, perhaps they were an expert in aviation transport logistics or even retail outlets. For that matter, expertise in restaurant management or even anti-terrorism security may be of the essence.

The answer, of course, is that it doesn’t matter.

Only the simplest of projects involve just one technical discipline or topic. By far the majority of projects have a myriad of such specialisms and to imagine that the only person who can manage such projects is someone with recognisable expertise in all of them is clearly ridiculous. Apart from any other considerations, which project could afford anyone with this many qualifications?

Project managers manage projects. They do not manage departments, or teams of individuals, sharing the same expertise. If they did then they would be “Functional Managers” and as students of organizational theory can explain, these resemble “Project Managers” in the same way that chalk resembles cheese.

The management of projects is about the management of the overall objective. It is about delivering the big picture by bringing together all the pieces that make up that picture. By necessity the role requires a broad perspective, sometimes referred to as a “helicopter view”, of the overall endeavour. The management of individual specialisms relates only to how these individual “pieces” fit together and involves defining their input, asking after and supplying the necessary support and interfaces they need, and then verifying that they have completed the work satisfactorily. The minutiae of how individual specialists achieve their objectives is something the project manager is drawn into at her or his peril. Not only is it an impossibly big task for all but the simplest of projects but it involves “landing your helicopter” and all the narrowing of perspective that this entails.

In fact, such a scenario is a common cause of strife within projects when the technically expert Project Manager concentrates only on those project elements that coincide with their own technical discipline. It ensures some project elements are neglected whilst upsetting the specialists in question who resent being micromanaged.

The full story of T5 is yet to be written but it does seem that the teething problems were not due to technical failures within any discipline mentioned above. The failure seems be associated with a failure to recognize the need for training of staff. There was a piece of the jigsaw missing: the sort of thing that stands out only when viewed from above. And while you are in your helicopter, “any sign of my suitcase from up there”?

To Train or Not To Train?

In light of the new “Time to Train” legislation which came into force on the 6th April 2010, all employees now have the right to request time off for training, many organisations have been feeling somewhat unnerved by what it could mean for them given the current economic conditions. The legislation is currently aimed at employers with 250 or more staff but will be extended to smaller employers in April next year.

Employees are entitled to request time away from their core duties to undertake training, which can be accredited, leading to a recognised qualification or even just to develop specific skills relevant to their current role or business.

Employees can request any training they believe will improve their performance and/or the business. They can request as much time as they require for relevant training and employers can only decline the request if, the training does not improve the performance of the business, the burden of additional costs is too great or the employer cannot reorganise work among existing staff.

Despite the fact employers are not obliged to pay for the training or the time off, the impact of losing a key member of staff for any length of time could still be substantial to a business and they will have to ensure they can provide the correct support cover for their time away from work. Employers may even have to pay for extra staff to come in, all of which can be costly and disruptive for any business, regardless of its size.

The best approach businesses should take is one of planned support. Employers need to help and advise staff about what their training options are, what would be best for their particular role, development objectives and to plan ahead for potential courses and training that staff my decide to take at a later date.

Organisations that are not supportive in these decisions could risk losing employees to competitor companies offering more benefits and internal support for staff looking to train and develop within the organisation. It is important to offer staff as many training options as possible and to consider shorter, more cost effective and distance learning packages that will be constructive to staff and employers and have the minimum effect on the business during training – the perfect compromise.

Courses such as Parallel Project Training’s Distance Learning package are the perfect solution as it offers not only a shorter APMP training course, which requires only three days out of the office as opposed to five, it also allows students to study for the APM project management qualification from their own home or office and at their own pace allowing staff to be more flexible in how they do their training, which is beneficial to both the employee and employer.

These types of courses are ideal for important and busy staff that may be unable to devote five consecutive days out of the office for training. Businesses should do their research now to find out which courses would be relevant to their employees and help up-skill staff with the minimum impact on the business before the inevitable staff training requests begin.

Value Management Approach to Clinical Trials Project Management, a White Paper by Paul Naybour and Roger Joby

The Cost of Drug Development

There is some debate about the real cost of drug development but what you can be sure of is that it is a very large number and increasing year on year. Back in 2003 the US Journal of Health Economics (DiMasi, 2003) published a figure of US$ 802m as the estimated cost of a pharmaceutical companies bring a new drug to market. This figure was based on the research and development of 68 drugs launched by 11 pharmaceutical companies. More up to date estimates suggest that these cost have risen to US$1.3 billion or even as much as US$1.7 billion (Collier, 2009). According to CenterWatch less that 10% of clinical trials are completed on time. The largest causes of delays are contract and budget negotiation, patient recruitment and enrolment and protocol design and refinement. (CenterWatch, June 2009). With a typically drug trial costing between $5m and $15mm each, these delays lead to a significant overspends. We argue that the root cause of these delays and overspent is insufficient commitment to the feasibility studies needed to optimise the study protocol and commercial relationships between the sponsor (Pharmaceutical Company), clinical research organisation (CRO) and investigator sites. We that propose a more collaborative approach to these feasibility studies based on the principles of value management could significantly reduce these overspends and identify significant reductions in cost. 

Reasons for delays and overspends

The typical clinical research procurement process has a major limitation because the CRO project managers and clinical research associate (CRA) who have all the experience implementing studies are often not closely involved the development of the initial protocol and procurement process.  This leads to the following issues:

1)    Project objectives and protocol are often defined by the Sponsor without input from either the in-house or out-sourced research teams hence these objectives can be arbitrary and unrealistic.

2)    The feasibility is sometimes done in isolation from the CRO which are going to complete the study or the CRO is asked to do feasibility as part of the selection process. As a result many assumptions about the best way to implement the protocol are made within either no or insufficient input from the CRO. The assumptions made at this point in the process may not reflect the most practical approach to key areas like recruitment of patients, selection of centres etc, which can have a profound impact on timelines and costs at a later stage.

3)    During the procurement process CROs rarely have longer than two week to tender for the work and the contracts are usually some form of fixed price. The aim of the CRO at this point is to win the contract and limit the liability. Very little time exists to do any proper feasibility study into the proposed protocol and the CRO is in sales mode so is unlikely to be honest about a poor quality approach. The more likely strategy is to ensure that the scope is tightly defined and CRO is protected from the weakness in the protocol by clearly defined assumptions which pass the risk back to the sponsor.

4)    During implementation phase virtual all project plans need to be amended, often in response to a more realistic assessment of site set up times, patient recruitment and practicalities of the protocol.  The CRO will expect the sponsor to cover the costs of these amendments. Many of these changes could have been readily foreseen in the feasibility phase if the people involved in completing the study were involved in the design of the protocol.

Furthermore the design may not allow for new innovations in techniques which could have saved significant time and increased patient recruitment and retention. Typical examples include:

• The protocol does not reflect the typical medical practice in the selected country.
• Site set up is delayed by contract negotiations with investigational sites.
• Competitive studies change site selection.
• Over optimistic recruitment due to insufficient risk analysis.

In summary may studies fail because they set unrealistic timelines, don’t do adequate feasibility, sometimes they end up starting projects well into the process. As a result many trial runs late and the cost overruns become significant. For a £10m study an overspend of 10% is not unusually. We seem to have forgotten the advice of Abraham Lincoln “Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”

We believe that more collaborative but challenging approach to the design and procurement of clinical trials, with more focus on the feasibility phase, could results in significant savings and a reduction in the delays by addressing the root causes early in the process.  Value Management is a widely recognized framework to improve the collaboration and cooperation between the parties in projects.

Value Management; a Step by Step Approach to Collaboration

Many sectors facing the same challenge of getting the best value out of the supply chain, have moved towards a value management approach. The UK office of Government Commerce (Office of Government Commerce, 2007) has summarised a wide range of case studied in which value management has saved significant project cost. For this approach an integrated feasibility team (from both the sponsor and preferred supplier(s)) focus on value not cost by:

• Understanding and challenging the relative importance of the sponsor objectives. This helps define realistic criteria for the success of the project from the outset and prevents the adoption of arbitrary timescales. Value management ensures that everyone understands the relative importance of the project objectives, together with the project constraints and risk from an early stage.
• Identifying alternative courses of action including different techniques, approaches and methods which may be more cost effective. The early input from CRO project managers and CRA’s into the design of the protocol can significantly improve the selection of the most appropriate investigative sites, diagnostic techniques and data collection processes.
• Analysis and evaluation of different approaches against defined criteria to select the best option. This practical and collaborative input to feasibility reduces the chance of delays and additional costs introduced by amendment of the protocol once the study has started.

The change however in not just procedural it is also cultural with sponsors and suppliers adopting a more open approach to collaborative working. Some of the areas for change are:

• Joint leadership and commitment from the top, with a collaborative approach to project management. This eliminates the need to duplicate project managers in the Sponsor and the CRO with the formation of an integrated study team.
• Rewards for CRO against balanced scorecard which measure the overall success of the study financially, in terms of quality, for the staff and the customers, not just the just cost. Ideally CRO and Phama staff incentives would be linked to the successful completion of the project within the agreed objectives.
• An integrated knowledge management database using shared electronic data management systems. Much has been done in this area to use technology to improve the flow of information within the study teams but an integrated approach using an integrated system has potential to further improve the follow of information between the sponsor and the CRO. 
• A flexible work force with strong team working (secondments etc) between the partners in the project in this way the members of the Phama team get an appreciation of the challenges faced by the CRO and vica-versa.
• Multi disciplinary team working across all phases of the study to bring the key people involved in the design, feasibility and implementation of the study together at critical times.  
• Most controversially the use of open book accounting in which the true costs of the study are open and reward is based on a more holistic assessment of performance.

Bridging the Gap between Feasibility and Implementation

The processes for value management are well defined and include clear steps completed at each of the stages. A number of workshops are conducted at each stage of the project. The aim is to bring together the different teams in a collaborative view of the projects objectives and strategy. The main studies conducted are:

VM 1: stakeholder needs, objectives and priorities

The primary objective of value management at concept stage is clarifying why the need exists, to ensure that the project objectives have been thoroughly analysed and understood. The workshop would be conducted early in the development of the protocol with input from the CRO who will be bidding for the contract to complete the study. It gives the CRO the opportunity to understand the sponsor’s objectives and influence the approach for the protocol to avoid unnecessary complications and unrealistic constraints.

VM2:  project definitions and options

The purpose of VM2 workshop is to ensure that the project options are evaluated and the one which offers best value for money is selected.

Selecting the best value option requires:

• Review the validity of the objectives.
• Agree modifications.
• Evaluate the feasibility of options.
• Develop the preferred option (best value).
• Determine if the preferred option (best value) can be improved or enhanced.
• Agreed recommendations about the preferred option.
• Produce a programme for development.
• Develop an action plan.

This workshop would ideally be conducted during the feasibility phase just prior to the issue of the RFP so that the CRO organisations who are bidding can bring their experience to the design of the protocol.

VM3/4: value engineering to reduce cost

Value Management during the feasibility stage is termed ‘Value Engineering’. The objectives are to ascertain elements of the study design and specification that add no value to the project in terms of satisfying the brief and objectives or those which could be provided through another method thus increasing value. It could be completed between the submission of proposals for the study and the award of contract. Typically one preferred bidder would be asked to participate in the value engineering workshop.

Value Engineering provides a structured approach to ensure that the most cost-effective means of implementing the project are identified.

VM 5/6 Value Reviews

Value reviews repeat the exercise undertaken in VM 3/4 with during the implementation of the study. Value reviews shall be undertaken as part of the project’s change control mechanism to allow the opportunity to value manage / the project to determine if the functionality could be altered and be more cost effectively so the project may be delivered within the original budget.

Ideally a value review must be undertaken at least once during start up and implementation phases to identify more effective ways of working. More frequent reviews may be necessary due to the complexity of the project.

Conclusion for Clinical Research

Value management is a tried and trusted way to reduce the overall cost of projects and has a successful track record in many sectors. It brings together the complete project team to understand the challenges and identify smarter, quicker and more cost effective ways of completing the project, often by simple changes to the approach and requirements. The clinical research and development sector could learn much from these more collaborative and value focused approaches. For more information including clinical trials project management training visit www.parallelprojecttraining.com

1.   Bibliography

CenterWatch. (June 2009, June). U.S Sites Rate Medpace, Kendle and ICON at TOP CROs in 2009. The CenterWatch Monthly .

Collier, R. (2009, February 3). Drug development cost estimates hard to swallow. Canadian Medical Association Journal , 180(3): 279–280.

DiMasi, H. G. (2003). The price of innovation: new estimates of drug development costs. Journal of Health Economics , Vol 22, pp. 151-185.

Office of Government Commerce. (2007). Achieving Excellence in Construction. Retrieved from Office of Government Commerce: http://www.ogc.gov.uk/documents/CP0064AEGuide4.pdf

APMP Distance Learning from Parallel Project Training from £300

Study for the APMP by Distacne Learning Using the integrated Parallel Learning System Parallel Project Training has launched a distance learning package for the APM project management qualification APMP.

Read more...

Earned Value Management is it worth the effort?

This is a summary of the presentation to the South West Branch of the APM on the 19^th May 2010. It summarises the presentation and the feedback from workshop sessions to identify the barriers to the implementation of earned value.

Introduction

Earned Value Management (EVM) is a fundamental process in both the APM’s and PMI bodies of knowledge and has its roots in the United States Department of Defence (DOD) Cost/Schedule Control Criteria System (C/SCSC), which has been specified for all DOD contracts for which the government carries the risk since 1967.

Earned Value Management is a simple practical concept with significant benefits for project managers including:

• An integrated view of the three key elements of project status, planned cost, project progress and actual expenditure.
• The ability to rapidly determine the status of a project based on performance indexes.
• The ability to rapidly highlight work packages that are overspending and / or behind plan.
• An estimate of cost at completion and completion date based on current performance.

And yet very few project managers’ use earned value management. Those that do, tend to be mostly in the defence, nuclear and some transport sectors. In this paper we argue that successful implementation of EVM is closely linked to a development of an organisations project management maturity.

Barriers to Wider Use of EVA

Based on research with PMI and APM seminars we can identify the barriers to adoption of EVM as being focus around the three elements of process, people and systems and the implementation of the fundamental principles of basic project management. In summary these barriers are:

Embedded processes to establish project scope, define realistic but challenging norms based plans, rules assessing progress and effective control of project change are all vital processes that must be in place to support the implementation of EVM. Without these fundamental processes then the earned value data will soon lose consistency and soon become discredited.

People within the organisation need the aptitude, training and discipline to implement effective EVM. They need the ability and training required to understand and use the data produced, they need to be familiar with the jargon used and trust it to make decisions. Some project managers see the increased visibility provided by EVM as a threat to there ability to manage the clients and their own senior managers expectations. Data needs to be provided within an understood context.

The systems which support EVM need to be fed with high quality baseline data with integrate cost and time information within the plan and accurate and timely cost information in a structure consistent with the project Cost Breakdown Structure (CBS). The latter of these often requires an interface to the organisations finance systems and possible modification to coding structures.

During the workshop we identified many potential barriers and aids to the implantations of earned value in organisations. These include:

Road to EVA Maturity

Critical to the successful implementation of EVM is the effective application of the core fundamentals of the project control cycle. In particular the following key elements need to be in place for each stage of the project control cycle

Plan the Work

The plans used for EVM need to implement key elements of good practice. They must integrate cost and time information through a resource and cost loaded Gantt chart. The work packages within this plan must be integrated with the CBS against which we are going to track costs. Finally it is useful if a rolling wave approach is used to development the detail within the plan a stage by stage.

Work the Plan

It is a basic tenet of project management that we work in accordance with the plan, but in many organisations this can be a real challenge, either because the plan lacks credibility, buy-in or commitment from the project team. For these reason many individuals or teams execute tasks in different sequences from the planned activities, complete work not included within the plan or ignore it all together. Detailed integrated baseline review of the project plan before baseline can ensure that the team has a detailed understanding of the project plan and iron out difficulties before implementation starts.

Measure Progress

Progress can be measured or estimated in several ways, from measurement of physically completed work to estimating the work left to complete. However in all but the most physical projects this estimate progress is too subjective. To effectively implement EVM we need defined rules for assessment of progress on work types and these need to be understood and adhered to across the organisation. For example for a design or software project we might make a rule that activities only earn value once they are complete and which point they earn all the value associated with that task.

Take corrective action and manage change

Uncontrolled or sluggish control of change can cause major problems for the implementation of EVM. Completing activities not shown on the plan, changes in scope or changes to the sequence of activities need to be captured and incorporated into the baseline through an efficient change control system. Without a timely and rigorous change control system the reports and performance measures produced by EVM systems will be inaccurate, misleading and could lead to incorrect management action.

For an organisation to benefit from the implementation of EVM the organisation needs the level of PM maturity which enables it to:

1. Prepare and agree robust plans integrating both time and cost
2. Execute work according to the plan
3. Accurately measure progress and record cost in a format consistent with the plan
4. Implement efficient and effective change control process.

These in fact are the corner stones of good project management. So the question remains as to what level of maturity is required to successfully implement EVM?

The Project Management Maturity Model

A Project Management Maturity meets these needs, drawing on established concepts from existing models such as the Capability Maturity Model (CMM) from Carnegie-Mellon Software Engineering Institute and the Business Excellence Model from the European Foundation for Quality Management (EFQM). Project Mangement Maturity describes four levels of increasing project management capability, termed Naïve, Novice, Normalised, and Natural. The aim is to provide a structured route to excellence in project management, with recognisable stages along the way which organisations can use to benchmark themselves against.

The various levels are defined as follows:

1. The Naïve project management organisation (Level 1) is unaware of the need for management of projects, and has no structured approach to projects. Management processes are repetitive and reactive, with little or no attempt to learn from the past or to prepare for future threats or uncertainties.

1. At Level 2, the Novice project management organisation has begun to experiment with project management, usually through a small number of nominated individuals, but has no formal or structured generic processes in place. Although aware of the potential benefits of managing projects, the Novice organisation has not effectively implemented project management processes and is not gaining the full benefits.

1. The level to which most organisations aspire when setting targets for management of projects is captured in Level 3, the Normalised project management organisation. At this level, management of projects is built into routine business processes and project management is implemented across all aspects of the business. Generic project management processes are formalised and widespread, and the benefits are understood at all levels of the organisation, although they may not be fully achieved in all cases.

1. Many organisations would probably be happy to remain at Level 3, but we can define a further level of maturity in project management processes, termed the Natural project management organisation (Level 4). Here the organisation has a fully project-based culture, with a proactive approach to project management in all aspects of the business. Project-based information is actively used to improve business processes and gain competitive advantage.

Earned Value Management and Project Maturity

We are proposing that earned value management requires a foundation of project maturity before it begins to add value.

For organisations at the “Naïve” level of maturity EVM will be over complex and
inappropriate, project management development efforts should concentrate on basic roles and responsibilities, definition of project objectives, scope and basic timeline planning and control.

The ad hoc processes within a “Novice” organisation are unlikely to respond well to the rigour and visibility introduced by the introduction of EVM, significant problems are likely with robustness of baseline plans, progress reporting, understanding within the organisations of EVM performance measures. However if the organisation has the commitment to develop to a normalised level of performance then EVM, especially required by a client, may be the vehicle to achieve this improvement, although it may take some time for the benefits to be realised.

EVM is ideally suited to a ‘Normalised’ organisation seeking to move towards the best practices exemplified by a ‘Natural’ organisation. The rigor, integration and visibility introduced by EVM will rapidly highlight any weakness in the organisations people, processes or systems and act as a driver to stimulate management action to further improve the organisations capability.

Organisations that have reached a ‘natrural’ level without using EVM for project control are likely to benefit little from its introduction. It is something that they may do if they have a client need or particular requirement. With a mature organisation then the changes required to implement EVM should be relatively painless giving increased levels of confidence in the project delivery capability.

Conclusions

EVM offers project management the promises of significant benefits, however very few project managers have adopted it because of significant barriers within their organisations. These barriers are linked to the project management maturity of an organisation. In order to be ready to adopt EVM organisations should be at least at a ‘Normalised’ level of project management and for these organisations EVM will give significant benefits and support their development. In addition, to enable really effective decision making EVM needs a maturity of understanding and support by Senior Managers. Project Managers will then, not only be helping themselves and their projects to understand whether they are ‘on target’, but also their Senior Managers. This should prevent the necessity of Project Managers to continually having to sell and explain the concept to their superiors.

Earned Value Presentation for the APM 18th May 2010

Earned Value

Is it worth the effort?

Earned value promises significant benefits to project managers including

•    An integrated system to measure cost and schedule performance.

•    The ability to measure efficiency and progress towards the project goal.

•    Ability to quickly identify underperforming work package and instigate corrective actions.

•    Accurate forecasting of outturn cost and time.

Despite these valuable benefits earned value management (EVM) is not widely adopted by project managers. In this interactive seminar we will explore why earned value is not used as much as it should be. What are the barriers to wider adoption of EVM? Will technology help us overcome these barriers?

The seminar will start with a review of EVM principles, benefits and implementation processes. We will then several review case studies of organisations who have adopted EVM, the benefits they experience from the introduction of EVM, barriers they have experienced and the critical enablers of successful EVM implementation.

Click here to download slides

The business case for project management training

The need for capable and professional project managers is clear to many businesses. With ever increasing complexity, risk and financial implications, modern society demands that projects are undertaken with a clear and apparent expectation that the project not only finishes as planned and to budget, but that any potential hitherto unknown opportunities are converted into material financial benefit. In order to do this the hypothesis is that we need professionally competent project managers.

It is virtually impossible to be categorical about the benefits of training. This is because it is rather self evidently impossible to re-run a project and compare it with a control project.

In the final analysis we will never really know, but this argument could be used in any sphere of professional life. Would we want our heart operations carried out by unqualified people, should our train drivers be people without the requisite training? Of course not. If training does not improve competence then let’s stop training aircraft pilots and see what happens.

There are three main perspectives on a project and the advantages to be gained from having a qualified and competent hand at the rudder of the project in which they have an interest.

The Customer – where a significant investment has been made in the creation of some new facility, service or product, the customer is the one who has paid. What most customers expect is the delivery of what they asked for, when they wanted it and for the price they expected to pay. They expect at some point for the cost to be overtaken by the benefits derived.

A competent project manager will be equipped to push out these boundaries to extend wherever possible the benefit derived from the known scope either by under spending, over delivering or beating the schedule. They will also seek out opportunities to extend the scope, after discussion and with due consideration to take advantage of changing circumstances or new information.

The Supplier – in a lot of organisations the project manager is delivering on behalf of the supplier be they an internal service provider or on the other end of an external contract. Either way the customer organisation is a separate entity to the project managers’. The suppliers want to get rewarded in line with their expectations set early in the concept stages. They do not want to be extricating the project manager (and themselves) from difficult conversations at every turn. They expect the job to get done without fuss and difficulty.

A competent project manager will make sure that they are fully aware of the needs and constraints of their own organisation. They will go to great lengths to understand the nature of the requirement on the part of the customer and make sure they are set up to deliver those components. They will also make sure that there is as little uncertainty as possible both in terms of the required deliverable and also the manner in which it will be delivered. It is this risk reduction that marks out the truly competent project manager.

The project manager – wants to deliver the job on time and on budget and to get rewarded for doing so. Apart from immediate praise and financial benefit, most competent PM’s have an eye to the bigger picture. They will continue to be able to deploy their excellent project management skills only for as long as they are offered the opportunity. The best way to continue this virtuous circle is to deliver the current one as well as is humanly possible.

Competent project managers will continue to seek out new and challenging opportunities to maintain their own marketability and remuneration potential. They will be lifelong learners, eager to demonstrate their continued professional development and contribution to the general knowledge and capability of their own profession.

Why should an organisation seek to develop its project managers? What’s in it for them?

The benefits of project management training for organisations are:

•    They will have a pool of qualified individuals from which to draw the best placed individual for the job in hand. This pool will be comprised of the trained individuals. This will grant an organisation the option to choose who does the next job, knowing they are all just as capable as each other.

•    The amount an organisation spends on its project training will have a direct result on its project success. If an organisation spends 7% of its project budgets on training successful achievement of its objectives will rise from 30-70% (source IDC).

•    The trained project managers will be familiar with concepts and terms in a consistent and clear way. They will be able to repeat their experiences with reference to their successes of the past and develop a huge lessons learned capability. This standardized approach reduces re-work and cost.

•    The project maturity of an organization is partly a function of the competence of the people employed and it has been shown that there is a direct correlation with a nominal maturity level and the overspend (or otherwise) of that organizations projects.

•    In an environment where there is a rigid structure of financial incentives to manage projects well, the introduction of personal development can be a huge factor in the retention of good staff and their motivation. Training staff can be a relatively low cost mechanism for providing motivation when other forms are not available.

•    An organisation with a reputation for good or even excellent project management will bring with it a following of potential recruits and plaudits, raising morale, profile and confidence. This will result in easier recruitment, a higher calibre of staff and teams and a groundswell of satisfaction.

Why would an individual want to undertake project management training?

What is in it for the individual?

•    Individuals can improve their capability and learn new knowledge, so they can practice new skills in the workplace. This use of a wider range of tools and techniques will raise their own confidence giving them greater confidence in their ability.

•    They can recognize their role as a profession and relate directly to others in the same situation. This allows a direct synergy in building support groups and networks, giving a second order access to advice and help.

•    They will be able to pursue aspirations to develop a professional career, with the increased recognition of professional standards and certification. This brings with it necessary continual professional development, but this merely acts to cement lifelong development and focus.

•    Where their acquired knowledge is transferable they will have the opportunity to move organizations and adopt similar roles elsewhere. This is self evident.

What is the value of a PMO?

David seems to have stirred up a bit of hornets’ nest with his piece about the negative value of the PMO, in project magazine. Having led a couple of PMOs and worked with many I can see the value in David’s arguments. A poorly run PMO can do little or nothing to support the delivery of successful projects in an organisation. They can be over obsessive about corporate reporting processes, following the process for the sake of the process and confuse accountability for the reporting of project progress. A well run PMO can provide support to project managers then they find makes a significant contribution to the successful delivery of the project, through providing the proper tools and skills to deliver the projects. This is especially true in organisation with lots of small and diverse projects which are not significant enough to invest in their own processes and system. However the PMO needs to remember it is there to improve project performance.

Advantages and disadvantages of a PMO

How the measure the value of a PMO?

How can we evaluate if the value of PMO exceeds its cost? Would the project managers pay for the PMO as part of the project costs? One way of evaluating the performance is to use them as an internal consultancy team, with targets to win support assignments from project managers. In this way they can generate income to invest in services systems, processes and training to offer the project managers. These project managers are only likely to buy the services that they perceive as adding real value.

Measuring Return on Investment in Project Management Training

Many organisations invest significant time, cost, and management effort in project management training. Project Management is about a team delivering clearly defined objectives to achieve a business benefit. How can we apply the same principles to Project Management Training Programmes? How can we measure the benefits of the project management training? How do we assess the acquisition of new knowledge and skills and the application of these in the work place? How do we measure the business benefits that flow from these new approaches to project management? Can we produce a realistic but prudent assessment of Return on Investment for a project management training programme? This paper looks at the current thinking in training evaluation and how it can be applied to Project Management development programmes. 

Overview of Training Evaluation

‘The first important contributions and still the most influential’ framework for training evaluation (Bramely, 1996) was proposed in 1959 by Donald Kirkpatrick’s “Techniques for Evaluating Training Programmes”(Kirkpatrick D.L., 1996). He identified four levels against which objectives should be set and evaluated. These are: 

Level 1: The reaction of the trainees to the programme 

Level 2: Measuring the amount of Learning of princples, facts, skills and attitudes 

Level 3: Changes in Behaviour
in the job 

Level 4: Results or changes in organisational effectiveness. 

More recently, Jack Philips (2003) proposed return on investment (ROI) as the fifth level in the Kirkpatrick framework. Measurement of ROI is important in Project Management Training Programmes because they must demonstrate a return from the significant investment in training. Other models exist such as CRIO (Warr, Bird and Rackham, 1974) and Peter Bramely’s evaluation against Effectiveness, Behaviour, Knowledge, Skills and Attitudes. Neither of these approaches have the clear link to the measurement of the impact on the business. CRIO focuses on the event itself while Bramely’s model lacks the structure and the link to return on investment. Hence, for this paper we will use Kirkpatrick’s four levels as extended by Jack Philips to include ROI and it’s application to project management training programmes. 

Measuring ROI for Training Programmes

To measure the ROI from the programme we need to follow the steps shown in figure 1. In summary the first step is to develop plans and baseline data, this includes plans for data collection at each of the lower four levels in the model (reaction, learning, application and business impact). Next, we must isolate the effects of the programme from other changes taking place in the business. Once these have been isolated, we can convert the business improvements into a financial return and together with a record of the full cost of training (both direct and indirect) we can calculate an ROI. Benefits that do not deliver a measurable financial return are intangible benefits. In the next section, we identify specifically how to measure ROI for a Programme. 

Application of ROI Model to BNFL ES Programme

Stage 1 Develop Evaluation Plans and Baseline Data

A number of data collection methods are available at different levels (see box 1). For example, tests are exclusively limited to the evaluation of learning (level 2) while focus groups capture data on application and business impact. The first step is planning which method of data collection is appropriate to the organisation and tabulate these decisions in a data collection plan as shown in Table 2. This defines the objectives, measures, data collection method, timing and responsibility for the data collection. 

Table 1 Data Collection Methods used at each Level
 

Table 2 Sample Data Collection Plan for Project Management Training Programmes

Level 1 reaction Level Evaluation

Evaluation of programme at level 1 uses a questionnaire completed immediately at the end of the courses. Typically the questionnaire covers the areas of briefing, course material (including content), delivery, outcomes, administration and further training requirements. To ensure 100% of the forms are returned they are collected at the end of every session. A target is usually set of achieving an average of at least Good (4 out of 6) across all these area. 

Level 2 Learning

Three mechanisms that are proposed to measure the effectiveness of learning. First, the programme includes a number of formal external exams and assessments and second the programme design was based on a knowledge based training needs assessment / competence assessment by line managers. Finally workplace assignment can assess the extent to which the knowledge can be applied in the workplace . 

The first approach utilises the progressively graded examinations and assessment administered by the Association of Project Management (APM) as shown in figure 1. These summative criterion references tests, evaluate that participants have acquired knowledge at the levels defined by the APM. 

The second approach will be to repeat, after say 12 months, the initial knowledge based Training Needs Analysis, based on tools. These norm-based tests will evaluate the extent to which the knowledge of the participants has improved. 

The final approach is to use work placed assignments. These can be designed so that participants have to demonstrate that they are able to apply the knowledge gained in a work place setting. They also support the application of the knowledge gained which is evaluated at the next level. 

Level 3 Application

The level of application cannot be evaluated until some time (3 to 6 months) after the training has been completed. However, three sets of measures are proposed. First, a review of action plans using telephone interviews 3 to 4 weeks after the training. During the interview participants should be asked about specific examples of how they have used the new knowledge from the training. The weakness of this approach is the difficulty validating the benefits of the individual actions. Second, focus groups could be used at each site to discuss the application knowledge gained during the training programme, these should be completed 3-4 months after the training. While the focus groups will provide significant feedback on the application of the learning, they may lack external credibility and may be difficult to get full coverage of a large population. The third measure is external audit and/or internal review to determine the extent to which the skills and processes have been implemented within day-to-day management of the projects. This approach, while expensive, has external credibility. 

Level 5 Business Impact

The ultimate value of the programme is its impact on business performance. The measures at this level need to reflect the objectives of the organisation. However they would typically include: 

1. Completion of project milestones to plan
   

1. Completing the full scope of work
   
2. Accelerating work through change control
   
3. Meeting Environmental, Health and Safety Requirements
   

The targets for these measures need to be agreed based on current performance and anticipated improvement. However not all the changes in these measures will be due to the training programme, so we must find ways to isolate the impact of the training from the other changes that are taking place in the organisation.
 

Isolating the Effects of Training

It is very unusual for training to be the only initiative or change taking place in a business. In fact the demand for learning is often triggered by an external change in the business environment or an internally driven change from within the business. It is unjustified to claim that all the improvement (or deterioration) in business performance is due to a training programme. Philips (2003) identifies several ways to isolate the effect of training, see Box 2. If baseline data exists, then trend analysis is the most convincing and reliable measure. It would be worthwhile supplementing this with focus groups and estimates from senior managers to validate the results of the trend analysis and improve buy-in. 

Converting to Monetary Data

To calculate ROI we need to both measure the return and the full cost of the training programme. The return can be often derived from the improvement in business measures however Philips has identified a number of guiding principles to ensure that the results are prudent. These are: 

1. When a higher level evaluation is conducted it must be supported by data collected at lower levels. (i.e. poor quality training (level 1) is unlikely to lead to significant business improvement)
2. When an evaluation is planned at a higher level, the previous level of evaluation does not need to be comprehensive. (i.e. if current business measures are being used to monitor the implementation then the need to project audit is reduced)
3. When collecting and using data only use credible sources, it is better to have a lower but justifiable ROI than a measure based on poor quality data.
4. When analysing data always choose the most conservative among alternatives
5. At least one method must be used to isolate the effects of training
6. If no improvement data is available for a population assume that no improvement has been occurred.
7. Estimates of improvements should be adjusted to use worst case estimates to account for potential errors
8. Extreme data items or unsupported claims should not be used for ROI calculations
9. Only the first year of benefits should be used for short term programmes
10. All costs should be loaded into the ROI calculation
11. Intangible benefits are those that are deliberately not to be converted into robust monetary benefits.
12. The results of ROI measurement should be communicated to all stakeholders.

 Following Phillip’s guiding principles ensures that the reported ROI is conservative and will be credible with the sponsors of the programme. 

Discussion and Conclusions

The initial very positive level 1 reaction feedback is a critical initial step towards the overall success of the programme. Equally, if not more important, is supporting and stimulating the application of the new knowledge within projects. Monitoring and evaluation of these higher levels will be a vital part of the feedback loop that ensures the overall success of the programme. Plans for monitoring these higher levels using trend line analysis of existing business measures, telephone interviews, focus groups and estimates from senior managers have been proposed. 

Bibliography

Bramley, P. ‘Evaluating Training’ Chartered Institute of Professional Development, 1996, ISBN 0-85292-636-7 

Philips J, ‘Return on Investment in Training and Performance Improvements Programs’, third edition 2003, Butterworth-Heinemann, 2003, ISBN 0-7506-7601-9 

Kirkpatric, D. L. ‘Techniques for Evaluating Training Programs’, Training and Development, American Society for Training Directors, Jaunary 1996. 

For more information on the benefits of project management training

Can social networking help ALLEVIATE some of our project complexity issues?

The social media revolution

The last ten years have see an explosion in social media with an estimated 307.4million unique visitors to social network sites in December 2009, an increase on 82% on the previous year (Nielsen Company, 2010). Many dozens of sites exist but the most recognisable and relevant to project managers are probably LinkedIn and Twitter, but these goliaths of the cyber world are not alone. Social networking sites abound, the growth of easy to use ‘blogging’ sites and the availability of open source software (such as WordPress) means that just about anyone armed with very little technical knowledge can express and exchange ideas in an interactive way with many thousands of others in a matter of days. The combined social networks and blogs are now the most popular form of online activity, according to the market research company Nielsen.

Social networks do not need to be external to an organisation like LinkedIn. An internal social network is a closed or private community that consists of a group or people within a company, association, society, education provider or other selected list. These facilities are often created for specific groups to support the real social networks within an organisation to help establish communities of practice to share knowledge, information and experience about a particular area or business interest.

The project management community have been part of this revolution with 2.8million project managers registered on LinkedIn alone. Resourceful project managers are always looking for tools to help them manage projects more effectively but are these social networking tools a help or an unwanted diversion with the potential to detract from delivery focus? In this article we will look at how social networks could be one part of the solution to the management demands of our increasingly complex projects.

The Growing Complexity of Global Project Management?

Projects have not escaped the globalisation trends. Modern technologies enable teams to work around the world with apparent ease. But in so doing they introduce for themselves a whole raft of new problems than did not emerge when teams were all co-located in a single geography or even office. This dispersal is a one of the key factors in the nature of the projects complexity.  The problems created as a result stem from several factors all of which have their roots in ambiguity and uncertainty. Research in the Australian Commonwealth Defence Department identified seven themes which contribute to this increased complexity (Remington, 11-13 October 2009) which can be summarised as:

These complexity factors put our traditional classical assumptions of project management under strain. We assume that we can reduce the complexity by de-composing the project into ever more tightly defined and simpler compartments. Paradoxically it has been shown that these reductionist approaches in fact lead to increased complexity within the project as each further level of breakdown generates just more interfaces that need to be managed and controlled.

A “clockwork universe” in which the future is predictable and repeatable with the outcomes proportional to the inputs does not exist for anything other than relatively simple projects. As the complexity increases these rules of predictability become even less reliable. We work in vain to increase formal control over workers activity to obtain more predictable outcomes whereas in reality this merely leads to more bureaucracy and less control in the real world of the project.

We also assume we can eliminate all major risks through identification, assessment planning and management but fail to recognise the dramatic effect of the ‘unknown unknowns’ associated with projects involving diverse and geographically distributed project teams.

In reality, apart from very simple co-located projects these factors were never true but we never really faced up to the fact. As teams become more diverse these paradigms are stretched to breaking point and now demand a re-think.

The Management of Complex Projects

The management of complexity is a topic of much debate (Whitty SJ, 2008) and a full discussion is beyond the scope of this article but although the guidance is still developing, current thinking would steer the project manger to consider a number of key areas;

Each of these would justify a volume of work on their own. The social networking theme of this month’s magazine drives us to consider the last topic here, that being the contribution that a developed social networking facility can make to reducing complexity and making the project mangers efforts more effective.

Can a social network help?

Every project relies on an informal communications network to support the more structured approaches enshrined in communication plans, stakeholder management plans, reporting procedures and so on. These informal meetings used to happen in the corridor, at the ‘water cooler’, coffee shop, elevator, in the car park. They are a key component of the social glue which keeps the project team moving in the right direction, highlighting risks, resolving issues and cementing progress before they get anywhere near the more formal systematic project management umbrella.   This glue holds a project together. It represents the linkages between individuals and coalitions within the wider project team.  Each social network has a ‘social capital’ which forms from the sum of the knowledge in the network, the strengths of the links between members, the alignment of the actors objectives, the collaboration between them and the effectiveness of the team leadership. (Hanifan, 1920). The skill of the best project mangers has always been to balance the use of the formal project approach with empathy for the social network to build a high performing team. With increasingly divers geographic teams we have lost much of this social glue.

Social Networking in the Virtual Team

Recognising the importance of the project social network is the first step in understanding how fundamental it is to the successful delivery of project. In a virtual team it is not possible to bring groups together on a regular basis (if at all), individuals cannot be forced into interacting with people they have never met. Could it be that the use of the social network tools will allow us to replace the lost interaction of the co-located team with a surrogate version hosted in one of these systems? Could they offer a replacement for that chance corridor meeting, the informal conversation over the skinny latte with extra caramel, the beer after work?

It’s possible that as with a traditional co-located team, debates and discussion could take place outside the constraints of formal communication, with on-going discussion of project issues and risks using the informal networks between team members. It might just be that we do not need to provide a procedure so individuals can interact with each other. Now there’s a thought.

Would I put my project on facebook?

A public forum like Facebook or LinkedIn may not be the most appropriate place to build a project network for obvious security and confidentiality reasons. The technology is so cheap and available though that this functionally can be easily replicated on a company intranet (available only within the organization) of extranet (which is available to partners). Typically based on propriety or open source platforms they have the potential to support and facilitate this valuable informal communication and knowledge sharing across a geographically dispersed project team. Many organistions already use internal discussion boards to provide technical support or project blog sites to keep a diversely distributed project team in touch with the latest project news. The extension of these approaches to develop a project social network is a relatively easy step.

If it’s such a good idea, why don’t we all do it?

We have been socially interacting for millions of years and arguably doing reasonably well as a result. As soon as a new mechanism, concept or idea comes along there is a natural human tendency to seek out opportunities from it but at the same time a reluctance on the part of the ‘establishment’ to change. Eventually we adapt but it can take a while. This cultural reticence stands in the way of an immediate and wholehearted adoption of things (like) social networking systems. This will over time wash out of the system as people reared and inculcated on these new and innovative techniques take their place amongst the traditionalists.

For now though there are a few things that need to be considered by an organization considering adopting such a system;

Conclusion

Modern communication technology has enabled projects to become increasingly fragmented and multinational adding exponentially as they go to the complexity of trying to manage them using the traditional methods. The co-located social networks that used to support the development of a performing team are increasingly more difficult to develop and maintain. Social networking systems seem to offer one way of maintaining this loose, unstructured but essential contact so as to underpin a more diverse project team. These tools are not without their dangers but rest assured that the downsides will not be solved through misguided attempts to regulate them. How effective will they be? It has taken thousands of years to become used to the written word, hundreds for telephones, tens for visual tools, perhaps the dawn of a new and more liberated way of interacting may be closer than any of us think. Ask a young adult just out of university, here’s a quote from one of them.

“I think social media, when used effectively, can be very useful for project management. Project teams are increasingly composed of young professionals. Rather than stifle their interest in social networking, why not utilize the creativity and initiative? Applications such as Twitter and Facebook can be effective tools for communicating in real time with project team members. Information sharing tools such as Digg, StumbleUpon, and Google Buzz are great ways to send relevant articles and research papers to the team as well as promote your project”

Watch this space!

Bibliography

Hanifan, L. J.-1. (1920). The Community Cente. Boston: Silver Burdett.

Nielsen Company. (2010, January 22). Led by Facebook, Twitter, Global Time Spent on Social Media Sites up 82% Year over Year. Retrieved March 2, 2010, from Nielsen Wire: http://blog.nielsen.com/nielsenwire/global/led-by-facebook-twitter-global-time-spent-on-social-media-sites-up-82-year-over-year/

Remington, K. a. (11-13 October 2009). A model of project complexity :distinguishing dimensions of complexity from severity. Proceedings of the 9th International Research Network of Project Management. Berlin.

Whitty SJ, M. H. (2008). And then came Complex Project Management (revised), . Int J Project Management , doi:10.1016/j.ijproman.2008.03.004.

Parallel Project Training

John Bolton and Paul Naybour both co-founders of Parallel Project Training the newest APM accredited training provider are enthusiasts in the adoption of any mechanism to make learning more accessible and effective and are committed to the development of a profession of project managers. Parallel Project Training uses modern technology such as podcasts, e-learning, communities of practice, moderated forums and webinars to complement traditional printed study guides and training workshops. Join them at a free webinar on the 19th April to discuss project complexity, social networking and the issues raised in more detail. Visit www.parallelprojecttraining.com to book your place.