Tracking and Evaluation
Assess the extent to which strategic targets are being achieved. Review all relevant programmes using qualitative and quantitative measures (such as satisfaction surveys, programme progress, and KPIs), and generate input for strategy reviews.
Improvement Planning
Practices-Outcomes-Metrics (POM)
Representative POMs are described for Tracking and Evaluation at each level of maturity.
- 1Initial
- Practice
- Practices are ad hoc and occur on a best endeavour basis.
- Outcomes
- An increased awareness of the IT function's contribution to other business units may start to occur.
- This should help gain approval for future IT investment programmes.
- Practice
- Practices are ad hoc and occur on a best endeavour basis.
- Outcome
- The IT Strategic Planning process, may start to see some evidence of a built-in improvement or remediation mechanism.
- Practice
- Practices are ad hoc and occur on a best endeavour basis.
- Outcome
- Dependencies between programmes may not be as overlooked, leading to less rework and wasted resources.
- Practice
- Practices are ad hoc and occur on a best endeavour basis.
- Outcome
- IT Strategic Planning processes will be less likley to remain at their primitive stage.
- 2Basic
- Practice
- Implement a basic process to review delivery of the intended results for major strategic programmes.
- Outcome
- More programmes stay on track, and, where they deviate, are likely to be quickly brought back on track.
- Metrics
- At the level of the IT strategic plan:
- # of programmes terminated before completion.
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- Practice
- Implement programmes to provide individual tracking/evaluation data from the IT strategic plan for review as per a scheduled frequency.
- Outcomes
- Strategic plan is more likely to remain relevant.
- Programmes that deviate from strategic plan are more likely to be identified and the situation remedied.
- Metrics
- At the level of the IT strategic plan:
- # of programmes terminated before completion.
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are the needs of each business area being sufficiently covered?
- Practices
- Assess all strategic programmes for strategic compliance during the approval of the outline business case.
- Thereafter, review performance against the business case, usually by the programme manager in conjunction with IT function management.
- Outcome
- An improved match between the programme portfolio and the IT strategic plan occurs.
- Metric
- # of programmes terminated before completion as % of overall programme portfolio.
- Practice
- Selectively measure the efficiency of the IT Strategic Planning process based on qualitative metrics, e.g. through interviews.
- Outcome
- The IT Strategic Planning process becomes increasingly effective.
- Metrics
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- 3Intermediate
- Practice
- Closely track IT strategic programmes and remediate where necessary.
- Outcome
- Strategic programmes that stray off course are quickly identified and remedial action taken.
- Metrics
- For each programme in the current strategic plan:
- Status (e.g. not started, in progress, complete, terminated).
- Budget consumed vs. expected.
- Progress achieved vs. expected (e.g. stage achieved, %complete).
- At the level of the IT strategic plan:
- # of programme change requests/proposals approved (e.g. scope, budget, schedule).
- # of programmes terminated before completion.
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- Practice
- Ensure reviews of strategic programmes match actual against planned achievements for all programmes in the IT strategy plan.
- Outcome
- The strategic plan stays relevant, regardless of deviations from original targets.
- Metrics
- At the level of the IT strategic plan:
- # of programme change requests/proposals approved (e.g. scope, budget, schedule).
- # of programmes terminated before completion.
- Average satisfaction rating for completed programmes (e.g. on time, performance to budget, deliverables meet functionality expectations).
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are the needs of each business area being sufficiently covered?
- Practice
- Ensure IT governance is in place to check compliance of programmes included in the IT strategic plan.
- Outcome
- The on-going alignment of the programme portfolio with strategy is ensured.
- Metrics
- # of programmes terminated before completion as % of overall programme portfolio.
- % of resources used on programmes in IT strategic plan vs. expected.
- Practice
- Implement metrics to assess plan effectiveness.
- Outcome
- The efficiency and effectiveness of IT Strategic Planning in critical areas is enhanced.
- Metrics
- Process cycle time & trends.
- Total staff time consumed in plan development & trends.
- Total cost of plan development & trends.
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- 4Advanced
- Practice
- Undertake post-implementation reviews on strategic IT programmes.
- Outcome
- Lessons learned can inform the improvement of the IT strategic plan and other relevant processes.
- Metrics
- For each programme in the current strategic plan:
- Status (e.g. not started, in progress, complete, terminated).
- Budget consumed vs. expected.
- Progress achieved vs. expected (e.g. stage achieved, %complete).
- At the level of the IT strategic plan:
- # of programme change requests/proposals approved (e.g. scope, budget, schedule).
- % of objectives achieved for completed programmes.
- # of programmes terminated before completion.
- Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations).
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- Practice
- Use feedback from implementation of strategic IT programmes to jointly and regularly review IT strategy.
- Outcome
- The IT strategic plan 'flexes' so as to respond expeditiously to other business unit dynamics.
- Metrics
- At the level of the IT strategic plan:
- # of programme change requests/proposals approved (e.g. scope, budget, schedule)
- # of programmes terminated before completion
- Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations)
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are the needs of each business area being sufficiently covered?
- Practice
- Conduct oversight of portfolio compliance to the IT strategic plan by an IT function and other business units management governance body.
- Outcome
- Non-compliant programmes are highlighted at an early stage in their lifecycle.
- Metrics
- # of programmes terminated before completion as % of overall programme portfolio.
- % of resources used on programmes in IT strategic plan vs. expected.
- Practice
- Use advanced tools such as internal benchmarking to measure the effectiveness of IT Strategic Planning.
- Outcome
- Objective measures of process performance feed into a programme of continual review and improvement.
- Metrics
- Process cycle time & trends.
- Total staff time consumed in plan development & trends.
- Total cost of plan development & trends.
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- 5Optimized
- Practice
- Monitor programme performance through continual oversight/governance.
- Outcome
- continual insight results into the progress, risk and impact of strategic programmes.
- Metrics
- For each programme in the current strategic plan:
- Status (e.g. not started, in progress, complete, terminated)
- Budget consumed vs. expected
- Progress achieved vs. expected (e.g. stage achieved, %complete).
- At the level of the IT strategic plan:
- # of programme change requests/proposals approved (e.g. scope, budget, schedule).
- % of objectives achieved for completed programmes.
- # of programmes terminated before completion.
- Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations).
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?
- Practice
- Monitor programmes continually for achievement of intended objectives and feed the results into the strategy review process.
- Outcome
- Field data from programme implementation informs continual strategy review.
- Metrics
- At the level of the IT strategic plan:
- # of programme change requests/proposals approved (e.g. scope, budget, schedule).
- # of programmes terminated before completion.
- Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations).
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are the needs of each business area being sufficiently covered?
- Practice
- Manage compliance with IT and business strategy through an interacting set of audit and oversight processes, e.g. business case assessment, IT governance and organization governance.
- Outcome
- Non-compliant programmes are detected in a timely manner.
- Metrics
- # of programmes terminated before completion as % of overall programme portfolio.
- % of resources used on programmes in IT strategic plan vs. expected.
- Practice
- Match performance of IT Strategic Planning against external industry benchmarks.
- Outcome
- IT Strategic Planning process is continually improved.
- Metrics
- Process cycle time & trends.
- Total staff time consumed in plan development & trends.
- Total cost of plan development & trends.
- Qualitative measures of efficacy of IT Strategic Planning process may include:
- Are we encouraging our people sufficiently to engage in Strategic Planning?
- Are the needs of each business area being sufficiently covered?
- Are the right people involved?