user experience index
Insight

Do You Know Your XIS?

Agile research can be a catalyst for immediate impact though quick prototype evaluation and ongoing iteration. It is also one of the best ways to ensure that you’re making the right thing for your user base.

But we soon realized that one of the biggest challenges of iterative 2-week design sprints wasn’t completing the research deck in time, it was being able to compare research results and recommendations between sprints. It can be difficult to track progress or regression, and it can be even harder to advocate for your Experience Strategy recommendations when you have no solid, consistent way of tracking your research results.

The User Experience Index Score (XIS)

To try and solve the pain of tracking prototype progress, we started experimenting with different ways of bringing some quantitative measurement into our qualitative UX research. We immediately recognized the value that doing so brought to our client and team. By keeping track of some simple, but key metrics, we were able to easily, objectively measure changes in the user experience or prototype. And thus was born the User Experience Index Score (XIS); a holistic evidence-based way to evaluate experiences, fueled through quantitative measures of the user experience.

The “XIS” (pronounced “size”) has been integrated into our UX research practice, and we now take quantitative measurements in each sprint, even when working with the very small sample sizes that in-depth qualitative research demands. The aggregate data that is collected over time into the XIS can then be considered to help us make decisions about prototype evolution from sprint to sprint, as well as to surface larger patterns and themes over time.

What’s your XIS?

So how might you start integrating quantitative measurement into your qualitative research in order to build your own XIS? The answer to that question depends on variable factors, as your XIS will be tailored to the specific questions, contexts, users, and business objectives of your product. But there are some standard tools you can use to help you measure key factors. For example:

Usability

How usable is your experience? Can you put a number on it? SUS can! SUS, or System Usability Score is a quick and replicable 10 question survey that has a proven ability to assess a system’s usability. And since there’s a strong link between SUS, NPS, and ROI, it’s a metric you don’t want to miss out on measuring.

Satisfaction

A simple, effective tool that you should consider adding to your XIS is the tried-and-true CSAT, or customer satisfaction question. It’s deviously effective at getting a user’s gut reaction on an experience. And if a score is low, you can always follow up with, “Why?” to understand where your experience is missing the mark. At BeyondCurious we find this question so effective that we ask our own clients to rate their satisfaction every week when we’re working together on a project.

Comprehension

Remember when you were in school and your teacher would spring a pop quiz on you? There’s a reason for that: because a quiz is an excellent way of quickly measuring comprehension. So ask yourself: what is it that you want to understand about a given experience? What are the key business and design drivers that will remain constant throughout prototype evolution? And then write a quiz that measures comprehension of those elements in your experience. Track comprehension sprint-over-sprint, and you will start to understand where your experience is succeeding and where it needs a little extra help.

Engagement

No matter what you’re making, it’s a safe bet that you want to measure engagement. One excellent way to do that is to measure time spent engaging with the experience. Even if your metric is that you want your user to quickly get in and out, time spent is still a key metric. So get out your stopwatch!

Business Objectives

Finally, the most critical: you must know what business objectives your experience is attempting to accomplish and measure them along the way. An experience could be beautiful, amazing, and life-changing to the user, but if it does not accomplish the business objective, it will be considered a failure by your client. Some simple ways of measuring whether your experience has achieved its business objectives are measuring purchase intent if your experience is selling something, or Net Promoter Score (NPS) if you are trying to understand the experience’s effect on loyalty to the brand.

A User Experience Index Score that tracks quantitative measures of qualitative UX research is an incredibly useful tool for measuring user experience. It can effectively help you track progress, evolution, and performance of an experience sprint-over-sprint in agile design research. When combined with the rich, in-depth analysis of user behaviors, values, contexts, and needs that qualitative UX research yields, a quantitative XIS can be an invaluable tool to help manage risk, advocate for Experience Strategy recommendations, and ensure that you’re making the right thing for the business and for your end users.

Contact us to find out how to create and improve your own User Experience Index Score (XIS).

Subscribe for more insights.

Related Insights

What is Living Strategy, and Why Do You Need One?

  Carrie Yury  / 

What companies have done for decades is put together a strategy. This is their best guess for how to achieve their vision for where they want to go. They have done that at planning meetings once a year—built 3-5 year strategies, gone out and tried to execute to that strategy, then met up again at the end of the year to see how they performed against their goals, tweaking their approach if needed. This approach to strategy doesn’t work anymore. The world is moving way too fast to have a set-it-and-forget-it strategy.

Read more