IVI Framework Viewer

Tracking and Evaluation

B5

Assess the extent to which strategic targets are being achieved. Review all relevant programmes using qualitative and quantitative measures (such as satisfaction surveys, programme progress, and KPIs), and generate input for strategy reviews.

Improvement Planning

Practices-Outcomes-Metrics (POM)

Representative POMs are described for Tracking and Evaluation at each level of maturity.

1Initial
  • Practice
    Practices are ad hoc and occur on a best endeavour basis.
    Outcomes
    • An increased awareness of the IT function's contribution to other business units may start to occur.
    • This should help gain approval for future IT investment programmes.
  • Practice
    Practices are ad hoc and occur on a best endeavour basis.
    Outcome
    The IT Strategic Planning process, may start to see some evidence of a built-in improvement or remediation mechanism.
  • Practice
    Practices are ad hoc and occur on a best endeavour basis.
    Outcome
    Dependencies between programmes may not be as overlooked, leading to less rework and wasted resources.
  • Practice
    Practices are ad hoc and occur on a best endeavour basis.
    Outcome
    IT Strategic Planning processes will be less likley to remain at their primitive stage.
2Basic
  • Practice
    Implement a basic process to review delivery of the intended results for major strategic programmes.
    Outcome
    More programmes stay on track, and, where they deviate, are likely to be quickly brought back on track.
    Metrics
    • At the level of the IT strategic plan:
    • # of programmes terminated before completion.
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
  • Practice
    Implement programmes to provide individual tracking/evaluation data from the IT strategic plan for review as per a scheduled frequency.
    Outcomes
    • Strategic plan is more likely to remain relevant.
    • Programmes that deviate from strategic plan are more likely to be identified and the situation remedied.
    Metrics
    • At the level of the IT strategic plan:
    • # of programmes terminated before completion.
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are the needs of each business area being sufficiently covered?
  • Practices
    • Assess all strategic programmes for strategic compliance during the approval of the outline business case.
    • Thereafter, review performance against the business case, usually by the programme manager in conjunction with IT function management.
    Outcome
    An improved match between the programme portfolio and the IT strategic plan occurs.
    Metric
    # of programmes terminated before completion as % of overall programme portfolio.
  • Practice
    Selectively measure the efficiency of the IT Strategic Planning process based on qualitative metrics, e.g. through interviews.
    Outcome
    The IT Strategic Planning process becomes increasingly effective.
    Metrics
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
3Intermediate
  • Practice
    Closely track IT strategic programmes and remediate where necessary.
    Outcome
    Strategic programmes that stray off course are quickly identified and remedial action taken.
    Metrics
    • For each programme in the current strategic plan:
    • Status (e.g. not started, in progress, complete, terminated).
    • Budget consumed vs. expected.
    • Progress achieved vs. expected (e.g. stage achieved, %complete).
    • At the level of the IT strategic plan:
    • # of programme change requests/proposals approved (e.g. scope, budget, schedule).
    • # of programmes terminated before completion.
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
  • Practice
    Ensure reviews of strategic programmes match actual against planned achievements for all programmes in the IT strategy plan.
    Outcome
    The strategic plan stays relevant, regardless of deviations from original targets.
    Metrics
    • At the level of the IT strategic plan:
    • # of programme change requests/proposals approved (e.g. scope, budget, schedule).
    • # of programmes terminated before completion.
    • Average satisfaction rating for completed programmes (e.g. on time, performance to budget, deliverables meet functionality expectations).
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are the needs of each business area being sufficiently covered?
  • Practice
    Ensure IT governance is in place to check compliance of programmes included in the IT strategic plan.
    Outcome
    The on-going alignment of the programme portfolio with strategy is ensured.
    Metrics
    • # of programmes terminated before completion as % of overall programme portfolio.
    • % of resources used on programmes in IT strategic plan vs. expected.
  • Practice
    Implement metrics to assess plan effectiveness.
    Outcome
    The efficiency and effectiveness of IT Strategic Planning in critical areas is enhanced.
    Metrics
    • Process cycle time & trends.
    • Total staff time consumed in plan development & trends.
    • Total cost of plan development & trends.
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
4Advanced
  • Practice
    Undertake post-implementation reviews on strategic IT programmes.
    Outcome
    Lessons learned can inform the improvement of the IT strategic plan and other relevant processes.
    Metrics
    • For each programme in the current strategic plan:
    • Status (e.g. not started, in progress, complete, terminated).
    • Budget consumed vs. expected.
    • Progress achieved vs. expected (e.g. stage achieved, %complete).
    • At the level of the IT strategic plan:
    • # of programme change requests/proposals approved (e.g. scope, budget, schedule).
    • % of objectives achieved for completed programmes.
    • # of programmes terminated before completion.
    • Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations).
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
  • Practice
    Use feedback from implementation of strategic IT programmes to jointly and regularly review IT strategy.
    Outcome
    The IT strategic plan 'flexes' so as to respond expeditiously to other business unit dynamics.
    Metrics
    • At the level of the IT strategic plan:
    • # of programme change requests/proposals approved (e.g. scope, budget, schedule)
    • # of programmes terminated before completion
    • Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations)
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are the needs of each business area being sufficiently covered?
  • Practice
    Conduct oversight of portfolio compliance to the IT strategic plan by an IT function and other business units management governance body.
    Outcome
    Non-compliant programmes are highlighted at an early stage in their lifecycle.
    Metrics
    • # of programmes terminated before completion as % of overall programme portfolio.
    • % of resources used on programmes in IT strategic plan vs. expected.
  • Practice
    Use advanced tools such as internal benchmarking to measure the effectiveness of IT Strategic Planning.
    Outcome
    Objective measures of process performance feed into a programme of continual review and improvement.
    Metrics
    • Process cycle time & trends.
    • Total staff time consumed in plan development & trends.
    • Total cost of plan development & trends.
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
5Optimized
  • Practice
    Monitor programme performance through continual oversight/governance.
    Outcome
    continual insight results into the progress, risk and impact of strategic programmes.
    Metrics
    • For each programme in the current strategic plan:
    • Status (e.g. not started, in progress, complete, terminated)
    • Budget consumed vs. expected
    • Progress achieved vs. expected (e.g. stage achieved, %complete).
    • At the level of the IT strategic plan:
    • # of programme change requests/proposals approved (e.g. scope, budget, schedule).
    • % of objectives achieved for completed programmes.
    • # of programmes terminated before completion.
    • Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations).
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?
  • Practice
    Monitor programmes continually for achievement of intended objectives and feed the results into the strategy review process.
    Outcome
    Field data from programme implementation informs continual strategy review.
    Metrics
    • At the level of the IT strategic plan:
    • # of programme change requests/proposals approved (e.g. scope, budget, schedule).
    • # of programmes terminated before completion.
    • Average satisfaction rating for completed projects (e.g. on time, performance to budget, deliverables meet functionality expectations).
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are the needs of each business area being sufficiently covered?
  • Practice
    Manage compliance with IT and business strategy through an interacting set of audit and oversight processes, e.g. business case assessment, IT governance and organization governance.
    Outcome
    Non-compliant programmes are detected in a timely manner.
    Metrics
    • # of programmes terminated before completion as % of overall programme portfolio.
    • % of resources used on programmes in IT strategic plan vs. expected.
  • Practice
    Match performance of IT Strategic Planning against external industry benchmarks.
    Outcome
    IT Strategic Planning process is continually improved.
    Metrics
    • Process cycle time & trends.
    • Total staff time consumed in plan development & trends.
    • Total cost of plan development & trends.
    • Qualitative measures of efficacy of IT Strategic Planning process may include:
    • Are we encouraging our people sufficiently to engage in Strategic Planning?
    • Are the needs of each business area being sufficiently covered?
    • Are the right people involved?