User testing makes all the difference, folks
Everything you need to know about user testing — what it is, the benefits, metrics & all the methods to conduct them effectively.
With over 125 million active premium subscribers, Spotify has been ranked No1. brand according to a recent study on top-performing apps in the world. However, a few years ago it was not necessarily the most beautifully designed apps. This is because of their poor navigation experience where the users had to tap multiple times through the hamburger menu to get to a particular section they wanted.
This was not an intuitive experience. Finally, after several rounds of user testing, the hamburger menu was ditched and it instantly became so much better. Spotify later reported a 30% increase in app engagement after removing their hamburger menu.
After this, Facebook notably switched to a tab bar after a ton of user testing, and many other apps soon followed. This is why User Testing is vital, especially in scenarios where apps just follow a popular trend.
What is User testing?
Design is often considered as an iterative process and users are at the center of any design process because, in the end, the users interact with the products or websites, not designers. It is really vital for a successful product to be validated, not just the final product but also the innovative concepts, new ideas before actually working on the development.
So, User testing commonly called Usability testing can be defined as the process through which a website or a product can be evaluated by real users who go through a series of tasks in realistic conditions.
✅ User testing must be happening at every stage of a development process to build successful websites.
Benefits of User Testing?
- ⏱️ Identify the average time taken to perform important tasks by the user groups.
- 💸 It saves a lot of cost and time in the long run because rebuilding a product or a website is not an easy task.
- 👵🏻 It provides the best user experience for the users of all demographics.
- 📈 Companies can know about useful insights on how well satisfied the users are with their product.
- 💯 Gives a Benchmarking score for your product which can be compared with industry standards.
Usability Metrics
Metrics are the most useful data points, insights around the testing and usability. These are very much important as qualitative research only covers things like verbal feedback from the users and usability cannot be actually measured just using qualitative responses. It’s a mix of both quantitative and verbal feedback, with which one can get an idea of why the problems occur and how to fix them with design changes.
👌 Task Success Rate
This is to measure to which level the users are successfully able to complete a given task when using a product/website. Failure will lead to Score 0 and Success often means 100. You can tweak it accordingly.
⌛ Time on Task
This is the average time taken by the users to finish a usability task which starts from the time they enter the flow and stops when they reach the last screen in the flow. Be careful about the traps such as users trying too hard to finish the task, etc.,
⚠️ Rate of the Errors
This is a measure of the most common errors which tripped up the users of the experience. They can either be severe — the errors which often prevents users from finishing the task and non-severe — the ones which generally lowers the efficiency of the experience.
🔊 NPS Net Promoter Score
In this, the users are asked a single question on how likely is they would recommend a brand/website/service to a friend or colleague and users would rate is on a scale of 1–10.
Types of Testing
Are you ready to do this?
🧪 Scripted Testing
This is a type of usability testing in which there is more control and these tests are suggested for understanding specific user behavior in an experience. They produce a lot of numerical data which will be useful for usability metrics mentioned above. They can also get qualitative data based on how much control one wants the script to be. They are 2 types of scripted tests, moderated and unmoderated.
Moderated Usability Tests
Researchers believe that you get high-quality data when you have enough time and personnel to conduct the moderated tests as the moderators try to dig deeper into understanding by asking layered questions, helps users to stay focused and also clarifies the confusion during the test.
When to Conduct them: Early stages of the design/research process, building high-level products, developing products with security concerns, etc.,
Unmoderated Usability Tests
These tests are useful because we want the users to experience a product in their natural environment at their workplace, coffee shop, etc.,
Advantages: Saves lots of time, lowers the costs as you don't need any rooms to conduct the tests, hire moderators, and simpler coordination efforts.
When to Conduct them: Getting quick user feedback about products/interfaces, get a lot of quantitative data for supporting the design, etc.,
To conduct an unmoderated test, a user script has to be crafted carefully which explains the testers what needs to be done when they look at a product and walk them through all the screens with guidance.
Also, users are encouraged to speak aloud when they interact so that their sessions could be recorded and shared later.
Tree Testing
This is a specific type of user test which is used to understand how well the users of an interface can navigate between the different pages and uncover any problems with menu items in the navigation.
Advantages: To validate design changes, fixing the information architecture of the website, evaluate the findability and glanceability of features.
When to Conduct them: In the early stages of creating the interface to understand the navigation patterns.
In a tree test, we examine only the information architecture and the naming of the content, nothing else. A tree test asks the users to perform a list of tasks using a sitemap which is clickable. Their sessions are recorded to record the task success rate. Tree tests are highly beneficial as they are scalable to any number of items.
Tree testing in addition to card sorting will solve all the navigation-related issues.
🧪 Non-Contextual Testing
Most of the time we want to validate an idea with the users without the actual product. Non-contextual testing is geared at finding your user’s motivations and attitudes on a product that will help in generating a lot of ideas. Even though these ideas are just conceptual, they are vital data points in driving the design. These are some of the most popular, cost-saving testing methods.
Card Sorting
This is one of the simplest yet meaningful testing methods where the researchers write all the elements of a product on post-it notes and then the users/participants have to organize them in a way that makes the most sense.
Advantages: Information Architecture understanding, uncovers most of the navigation issues, the grouping of elements and naming.
When to Conduct them: This should always be always combined with tree testing.
There are 2 variations open and closed card sorts.
In Open sorts, the groups are not defined to the users, so they have their freedom to name the groups
In Closed sorts, users are given a list of items and they have to categorize and group them into preexisting groups.
Things to remember: Never mix parent & child categories, set time limits to the participants and provide some blank cards too for additional categories.
User Interviews
These are the best ways to understand what users think and these directly engage you with the direct user base of the product and you can also control what kind of qualitative data you want. Always make sure that the questions being asked are around the user’s perspective. Some of the things which can be asked are things like
- List of websites/interfaces they use regularly and why,
- The user’s process of searching and
- How they interact with similar products,
- Why users like certain products/apps, an ideal experience, etc.,
Heuristics Evaluations
This is a testing methodology that would act as a scorecard for your website’s usability and experience. Heuristic evaluation breaks down usability into subsections which would be graded individually and this scorecard as low, medium or high.
Advantages: This is a very0 low cost consuming method which would be a helpful test to benchmark your product regularly and can be used to compare with other competitors.
🧪 Natural Environment Testing
These are the types of user tests that do not have a written script and generally occur in the user’s natural environment. This is a very informative form of testing because there is no interference from the mediator and these tests work very well with broad ethnographic data etc., These are easy to set up and they are conducted remotely.
A/B Testing
In an A/B test, as the name indicates different sets of participants are given different (usually 2) choices of variations of a single feature. It is considered to be a scientific test which means, in the variation only one variable is changed and the others remain constant. This is because if many variables are changed in the variation when the results come out we wouldn't understand which variable was responsible for better performance.
Few of the elements which can be tested using A/B testing are form input types, call to actions, headlines in a page, images, promotional offers, and pricing, etc.,
First Click Testing
This testing tries to examine what a user will click to complete a task when presented with an interface or a website. This also allows users to test the effectiveness of IA, navigation, and visibility. Based on usability.gov, a user who clicks on the right path on the first click will be 87% successful and a user who clicks on the wrong path on the first click will be 46% successful.
However, it is vital to write down clear task scenarios, and also time each task as it is being performed. Also, additionally, you can ask your users and rate them on a scale of confidence to find out navigational issues. Open-ended questions can be asked after a test to find out more about an ideal experience.
Heat Maps/Eye Tracking Tests
User tests like diary studies and actual field studies give you a context on how products and websites are being used in the user’s location. But, to go into further detailing what people are actually looking at, how they are navigation between pages and track user’s eye movements — Heat maps are really useful.
Also, heat maps are really easy to interpret and understood by most people. There are lots of design tools out there that can be used to run heatmaps and get proper design direction.
In the end, Usability testing allows us to understand how the users are interacting with the product at every step of the dev process. Ultimately, this helps in building better products which in turn will make it easier to market, onboard users and improve retention rates.
Comment below about your ideas and thoughts. Also, check similar articles on the UX design below.
For any work-related queries check www.vamsibatchu.com or email me @vamsibatchuk@gmail.com
