Are we there yet? This is one of the key questions that a Post-Implementation Review (PIR) can answer. Typically performed about half a year after an IT project or program has been implemented, the PIR reviews how well the investment accomplishes its intended purpose.
The PIR can and should be a decision point to answer additional questions: Should the recent deployment of the investment be continued without change? Does the project or program need adjustments to improve performance? Is it feasible to improve the functional value of the investment? Do users need assistance or additional training? Or, in a few cases, is the outcome unsatisfactory—suggesting that the enterprise should consider alternatives.
The PIR provides an opportunity to document lessons learned, which should be added to an enterprise IT knowledge base for improving future investment management. In addition, the PIR can establish a comprehensive baseline of data that should provide the framework for future, periodic Operational Analysis Reviews.
An organization may also choose take advantage of the PIR to look at specific issues in depth: How do actual costs compare with initial estimates and the budget in the Performance Measurement Baseline? To what extent did the benefits estimated in the early Cost-Benefit Analysis actually accrue to the enterprise? Could there have been improvements in the procurement or deployment process? Do the technical results comply with the specifications and test criteria?
A PIR may also be conducted immediately following investment termination, providing lessons learned for future decision making.
The Problem
Once a major investment is deployed, there is a natural tendency for IT shops and business sponsors to celebrate and declare success, because they have a great deal invested in the hard work and drama of bringing a complex project or program to fruition. Flying in the face of these emotions, however, is the reality that even a well-planned, developed, and deployed IT initiative will have a few warts and kinks. The average initiative, though performed by hard-working, capable people will have a number of issues; and a few investments are likely to result in disappointing outcomes.
For a PIR to be productive, it is important to be constructive: all projects yield lessons learned, and most investments can be improved (sometimes with little additional cost). During the course of a PIR, be sure to emphasize that criticism of specific project individuals is undesirable and counter-productive. A PIR should not be used to find fault or apportion blame. A professionally performed PIR will provide a balanced assessment focusing on positive and negative feedback. This focus on a constructive review process should be emphasized.
The Solution
A productive PIR should be planned with a clear methodology. There needs to be a schedule for orientation, reviews, findings, decisions, and closure. It often helps to have a consulting facilitator/coach to lead the review in a constructive, positive manner. Documentation from the investment’s initiation, planning, and early deployment stages should be identified and available (preferably available using SharePoint or another online repository).
A PIR Team should be identified that includes decision makers and members of the Integrated Project Team (IPT). Technical personnel with in-depth knowledge of the investment and its development/ deployment should be available for providing input at review sessions.
For Federal IT investments, the Office of Management and Budget (OMB) suggests that common elements to be reviewed during a PIR should include:
- Mission alignment
- IT architecture including security and internal controls
- Performance measures
- Project management
- Customer acceptance
- Business process support
- Financial performance
- Return on investment
- Risk management
- Select and control phase performance ensuring initiative success
- Gaps or deficiencies in the process used to develop and implement the initiative
- Best practices that can be applied to other IT initiatives or the CPIC process
The PIR should include recommendations and decisions about the investment. The PIR should be documented and include lessons learned. These outputs should be accessible in a Corporate Knowledge Base.
What’s New
The PIR process should be applied to major investments in cloud computing including Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS).
QT Plan
In our experience as facilitators and coaches, the PIR takes 1 – 2 months. However, several additional months may be required if documentation must be assembled and/or a user satisfaction data collected.
There should be a written plan or charter for the PIR, along with a review methodology, schedule, and plan for documenting the PIR. It is helpful to engage an independent coach or facilitator to run the process. The PIR scope should include milestones for:
1. Agree on scope and Plan or Charter for the PIR
2. Conduct an organization meeting for PIR Team
3. Finalize methodology and plan schedule
4. Identify, collect, and organize investment’s documents
5. Review customer satisfaction (or conduct survey)
6. Review problems (test reports, help desk trouble reports, risk log)
7. Orient the PIR Team about the methodology and structured approach to be used
8. Review Performance Measurement Baseline (PMB) and performance reports for development and operations
9. Conduct PIR review sessions(s), organized on the basis of key topics that correspond to the PIR scope
10. Evaluate and score results by key topics and develop recommendations for the future
11. Define any needed follow-up actions
12. Identify record-keeping data to be used for subsequent Operational Analysis Reviews (assuming that a decision is not made to terminate the investment)
13. Develop a decision package based on PIR findings
14. Prepare and gain consensus on a final PIR report including findings, recommendations, and decisions
15. Close the PIR and add the final report to the enterprise IT Knowledge Base
16. Update the performance measurement baseline if necessary.
Reference
Office of Management and Budget, Capital Programming Guide, Version 3.0, July 2012. See Subsection III.3.3.2 Post-Implementation Review.
More Help
Jim Kendrick and the P2C2 Group, Inc. provide Facilitator/Coach and Subject Matter Expert services in this Quick Task area: kendrick@p2c2group.com.
Last Word
Even with the best system development process, it is quite possible that a new system will have problems or even major flaws that must be rectified to obtain full investment benefits. –OMB, Capital Programming Guide.
