Researching User Experiences
Assess user behaviours, needs, and motivations — for example, through surveys, focus groups, individual interviews, direct observation, and usability testing.
Improvement Planning
Practices-Outcomes-Metrics (POM)
Representative POMs are described for Researching User Experiences at each level of maturity.
- 1Initial
- Practice
- User experience not formally sought.
- Outcome
- Anecdotal evidence used to build understanding of user's experience.
- Metric
- Assessments conducted prior to project Y/N?
- Practice
- No formal approach to conducting assessments in place.
- Outcome
- Requirements are at best anecdotal.
- Metric
- Assessment gathering approaches the same for all projects Y/N?
- Practice
- No resources allocated to conduct requirements capture.
- Outcome
- No resources in place to support requirement capture.
- Metric
- % of IT projects identifying resources to conduct assessments as part of their project plan.
- 2Basic
- Practice
- Assessment focused on technical functionality.
- Outcome
- Main focus is to understand the technical functionality of the systems and develop the interface around that understanding.
- Metric
- % improvement in system performance criteria.
- Practice
- Basic structure for conducting user assessments adopted for all projects.
- Outcome
- Improved quality of response concerning user experience.
- Metric
- % of projects complying to adopted/agreed assessment method.
- Practice
- Select technical specialists to provide end-users assessments.
- Outcomes
- Selective requirement gathering.
- Mainly focused on small-subset of users ( e.g. specialist users)
- Metric
- % of end-users included in requirements gathering process.
- Practice
- Technical requirements dominate requirement assessment.
- Outcome
- Improving technical performance of IT systems is a key design driver.
- Metric
- Average number of performance requirements versus UI requirements captured in the assessment.
- 3Intermediate
- Practice
- Need and scope of assessment determined at project development stage.
- Outcome
- Allows needs assessment to be planned for during project implementation / update phase.
- Metric
- Average number of variation in requirement gathering methods by project.
- Practice
- Assessment conducted against functional / Business Unit requirements.
- Outcome
- Assessment process more efficient as it takes into consideration broader requirements of organization.
- Metric
- % of assessment of requirements prioritized against identified business need.
- Practice
- Assessment looks to include user-centric as well as technical requirements.
- Outcome
- How users interact with systems now considered important part of requirements gathering.
- Metric
- % of assessment of end-user requirements based on usability aspects of system performance.
- Practice
- Development of more structured approach in assessing user-centric requirements.
- Outcome
- Formalized approaches developed to more effectively capture end-user system interaction requirements.
- Metric
- Variation in interaction design approaches across projects.
- 4Advanced
- Practice
- Assessment includes feedback on quality and delivery of user experience.
- Outcome
- Improved planning around delivery and format of current and future assessment of user needs.
- Metric
- % satisfaction/dissatisfaction with assessment process based on relevance of questions to user.
- Practice
- User experience assessments collated and analysed centrally.
- Outcome
- Assessments are collated centrally in order to ensure user needs are prioritized in line with business need.
- Metric
- % of requirements collated centrally versus those collated locally.
- Practice
- Standard approach to assessing user requirements across organization.
- Outcome
- Common approaches ensure improved requirements gathering across organization.
- Metric
- Average number of variants being used to gather and define end-user requirements per project.
- Practice
- Assessment of requirements used to develop a broader understanding on user needs.
- Outcome
- Requirements used to develop a better understanding of user needs across organization.
- Metric
- % or requirements gathered by user job category, skill level, and capability.
- 5Optimized
- Practice
- Assessed needs captures user technical and business capability and skills profiles.
- Outcome
- Current and future user needs proactively shaped by current capabilities which can be broken down by end user job type, position, previous training, and business unit requirements.
- Metric
- Skill/Job/Capability profile of assessed users versus Skill/Job/Capability profile of total population of expected system users.
- Practice
- Assessment shaped by prior assessment analysis.
- Outcomes
- Assessments can be optimized based on data from prior assessment for similar projects.
- Thus reducing time taken to conduct assessment.
- Metric
- Time taken to conduct assessments.
- Practice
- Assessment used to determine user system response and decision-making capability.
- Outcome
- Requirements not just assessed from a usability perspective but also for a decision making perspective.
- Metric
- % Assessment of speed and quality of response by users of currently deployed systems.
- Practice
- Qualitative and quantitative requirement gathering techniques used.
- Outcome
- Qualitative techniques used to develop a ‘richer’ understanding of end-users ability and desire to interact with systems.
- Metric
- % of projects using focus groups for refining design criteria.