IVI Framework Viewer

Researching User Experiences

A1

Assess user behaviours, needs, and motivations — for example, through surveys, focus groups, individual interviews, direct observation, and usability testing.

Improvement Planning

Practices-Outcomes-Metrics (POM)

Representative POMs are described for Researching User Experiences at each level of maturity.

1Initial
  • Practice
    User experience not formally sought.
    Outcome
    Anecdotal evidence used to build understanding of user's experience.
    Metric
    Assessments conducted prior to project Y/N?
  • Practice
    No formal approach to conducting assessments in place.
    Outcome
    Requirements are at best anecdotal.
    Metric
    Assessment gathering approaches the same for all projects Y/N?
  • Practice
    No resources allocated to conduct requirements capture.
    Outcome
    No resources in place to support requirement capture.
    Metric
    % of IT projects identifying resources to conduct assessments as part of their project plan.
2Basic
  • Practice
    Assessment focused on technical functionality.
    Outcome
    Main focus is to understand the technical functionality of the systems and develop the interface around that understanding.
    Metric
    % improvement in system performance criteria.
  • Practice
    Basic structure for conducting user assessments adopted for all projects.
    Outcome
    Improved quality of response concerning user experience.
    Metric
    % of projects complying to adopted/agreed assessment method.
  • Practice
    Select technical specialists to provide end-users assessments.
    Outcomes
    • Selective requirement gathering.
    • Mainly focused on small-subset of users ( e.g. specialist users)
    Metric
    % of end-users included in requirements gathering process.
  • Practice
    Technical requirements dominate requirement assessment.
    Outcome
    Improving technical performance of IT systems is a key design driver.
    Metric
    Average number of performance requirements versus UI requirements captured in the assessment.
3Intermediate
  • Practice
    Need and scope of assessment determined at project development stage.
    Outcome
    Allows needs assessment to be planned for during project implementation / update phase.
    Metric
    Average number of variation in requirement gathering methods by project.
  • Practice
    Assessment conducted against functional / Business Unit requirements.
    Outcome
    Assessment process more efficient as it takes into consideration broader requirements of organization.
    Metric
    % of assessment of requirements prioritized against identified business need.
  • Practice
    Assessment looks to include user-centric as well as technical requirements.
    Outcome
    How users interact with systems now considered important part of requirements gathering.
    Metric
    % of assessment of end-user requirements based on usability aspects of system performance.
  • Practice
    Development of more structured approach in assessing user-centric requirements.
    Outcome
    Formalized approaches developed to more effectively capture end-user system interaction requirements.
    Metric
    Variation in interaction design approaches across projects.
4Advanced
  • Practice
    Assessment includes feedback on quality and delivery of user experience.
    Outcome
    Improved planning around delivery and format of current and future assessment of user needs.
    Metric
    % satisfaction/dissatisfaction with assessment process based on relevance of questions to user.
  • Practice
    User experience assessments collated and analysed centrally.
    Outcome
    Assessments are collated centrally in order to ensure user needs are prioritized in line with business need.
    Metric
    % of requirements collated centrally versus those collated locally.
  • Practice
    Standard approach to assessing user requirements across organization.
    Outcome
    Common approaches ensure improved requirements gathering across organization.
    Metric
    Average number of variants being used to gather and define end-user requirements per project.
  • Practice
    Assessment of requirements used to develop a broader understanding on user needs.
    Outcome
    Requirements used to develop a better understanding of user needs across organization.
    Metric
    % or requirements gathered by user job category, skill level, and capability.
5Optimized
  • Practice
    Assessed needs captures user technical and business capability and skills profiles.
    Outcome
    Current and future user needs proactively shaped by current capabilities which can be broken down by end user job type, position, previous training, and business unit requirements.
    Metric
    Skill/Job/Capability profile of assessed users versus Skill/Job/Capability profile of total population of expected system users.
  • Practice
    Assessment shaped by prior assessment analysis.
    Outcomes
    • Assessments can be optimized based on data from prior assessment for similar projects.
    • Thus reducing time taken to conduct assessment.
    Metric
    Time taken to conduct assessments.
  • Practice
    Assessment used to determine user system response and decision-making capability.
    Outcome
    Requirements not just assessed from a usability perspective but also for a decision making perspective.
    Metric
    % Assessment of speed and quality of response by users of currently deployed systems.
  • Practice
    Qualitative and quantitative requirement gathering techniques used.
    Outcome
    Qualitative techniques used to develop a ‘richer’ understanding of end-users ability and desire to interact with systems.
    Metric
    % of projects using focus groups for refining design criteria.