Everyone who works on online research projects should use (or at least be familiar with) qualitative methods, too. Even if you are a data scientist who trusts the numbers more than anything else, you can learn great things if you run a few analyses with smaller sample sizes, but more in-depth understanding. Today I am going to show you one of these great qualitative research methods: five second testing!
What is a 5 second test?
Five second testing is a qualitative research method in which you show a test-user a screenshot of your website or product – for five seconds only. Then you take away this picture and ask different questions about what she has seen – or thinks she has seen – in the image!
Your tested media could be anything. Quite often, it’s the above-the-fold part of your landing page, but it could be a banner, an app menu, a complete UI, anything.
It’s much easier to show it than explain it, so here is how a five second test looks from the tester’s perspective:
It’s all nice and simple. It’s not a big job — you can set one up in around 10 minutes — yet it provides you very valuable feedback from your users.
Why exactly 5 seconds?
We all know that nowadays people browse online crazy fast. So it’s kind of self-explanatory that when you build an online product, first impression is key!
With a 5 second test you can get immediate feedback about your design and copy – based only on this first impression.
- Do you users get your main message or not?
- Does your brand look trustworthy or not?
- Do people understand your main product or not?
I’d like to emphasize the “immediate feedback” part.
To set up a proper 5 second test you don’t need to write a single line of code. It’s enough to build a prototype in PhotoShop that you can show to your testers — you are already ready to gather insights.
When is a 5 second test useful?
You can usually take advantage of five second testing in two cases.
1. Before product release
The first one is when you’re building a brand new website, product, campaign, etc.
In these projects, it’s essential that you get early feedback about the overall user experience. These quick tests are just perfect for that. They’ll help you to go in the right direction from the very beginning of the product development process. Using them, you will be more agile and you will understand your users much better even before your first big release.
The process looks like this:
- You come up with the product idea.
- You (and your team) brainstorm about the different possible designs and copies.
- You come up with 5 to 10 different variations.
- You run five second tests for each version.
- You get feedback, learn and iterate.
Note: I usually like to use multiple methods when running research — even in these early phases. So besides five second testing, I really recommend adding usability testing to the mix. And later, when you release your product, other data-driven methods can come into the picture (e.g. funnel analysis, segmentation, A/B testing). I wrote more about this in my research framework article.
2. Before an A/B test
A 5 second test can become handy in another common situation: before you start an A/B test.
A useful A/B test is always based on comprehensive research. Many A/B tests fail because they are designed based on a random idea of a random person at your company. (I’ve seen too many of these to know that…) On the other hand, winning A/B tests are backed by both data analysis and qualitative research (e.g. five second tests and usability tests).
Let’s say you have done your job well, you have done all the research and you have a useful A/B test. Great! Even so, before you actually set up and start your split test, you should do a proof check and run some quick 5 second tests, too.
Why? Let me explain!
A/B testing takes time.
It takes engineering time from your developers. But more importantly, it takes time while you are waiting for the data to flow in. Most small or mid-size online businesses don’t have enough visitors/users to do their A/B tests fast enough… Usually they have to wait 2-3 weeks, a month or even more(!) to get significant results. It takes even more time if they come up with three, four or five competing versions and not just two… That’s too slow for a small business, right?
But if you run a five second test before your A/B test, you can get immediate feedback about your different variations and get rid of those that perform really badly on this qualitative part. Of course, since you are doing your five second tests with only 20 or 50 test participants, your results won’t be significant. But they will get you on the right track.
E.g. if you see that your five-second-testers really don’t get your messaging in your version C, but they get it in version A and B, then you can just go ahead and skip version C and leave it out of your upcoming split test.
Your developers won’t have to build a version that won’t win anyway. And you won’t have to run as many different variations in parallel – so you will get significant results faster.
Everybody’s happy.
How to do it? Which tool to use? How much does it cost?
It’s not magic to build a basic five second testing platform for yourself, but if you don’t want to, you can use the same tool that I do: https://usabilityhub.com/five-second-test (I don’t have any affiliation with them, and they are not sponsoring this article; I just simply love their platform.)
Their pricing is fair.
An optional $99 monthly for premium features (I usually don’t use these anyway) and $2 for every participant they hire for you.
Yes, they can hire participants for you!
If you want to target specific segments, then it’s $3 per person.
You can hire testers by yourself as well – that’s free. (If you know where your target audience is, I suggest this option.)
Anyway, setting up your first test in UsabilityHub is super easy:
STEP #1 You have to specify a “scenario” and/or add some instructions. Something like: “Imagine that you are about to rent a car and you find this website…”
STEP #2 Then upload your design (e.g. the above-fold part of your landing page)
STEP #3 And the final step is to specify the questions you want to be answered. E.g. “Does this brand look trustworthy?” (I’ll list my favorite 5 second test questions below!)
And done! Send out your five second test to the target audience – or hire participants via UsabilityHub.
In 1 hour you will have the results.
Five second test best practices
#1 – What to test?
As I mentioned above, most of the time I am testing landing pages and banners (e.g. for Facebook ads). If you have an SaaS-type (Software as a Service) web product, you can get feedback about your UI too. If you have a very early stage product, you can even upload wireframes.
Basically what you research here is up to your creativity, but I’d recommend focusing on the first touch-points – since a five second test is about first impressions.
(It makes a bit less sense to five second test the bottom of your website, because the people who scroll down there have already a first impression about your stuff – most probably a good one since they didn’t bounce without scrolling.)
#2 – Best 5 second test questions
Do not bias your results! In qualitative research you should not usually ask closed-ended questions (ones that testers can answer with a yes or no.) In a five second test it’s better to ask something around what the tester can recall or what grabbed their attention. My favorite five second test questions:
- What do you think this page was about?
- What product do you think this company sells?
- What’s your first impression about the site?
- What grabbed your attention?
- What words or sentences can you recall?
- Etc…
It’s always good to have hypotheses before the test. Having expectations will enable you to decide whether the answers are in line with what you want to see or completely off.
#3 – The situation and the instructions
You can (and should) give instructions before the test. Keep this simple. And make sure you don’t bias your results here either – so don’t give any information here that can directly affect the answers of the users.
Sample size and evaluation
The perfect sample size for a five second test is somewhere between 20 and 50 participants. Don’t forget, this is a qualitative method – so you are not looking for significant and representative results here, but for an educated guess.
Keep this in mind when you evaluate. Even if you will present charts to the management at the end of your analysis, I recommend that you go through every answer one by one. You might find interesting and useful bits and pieces there.
Note: you can print a word cloud from the received answers as well, if you have to present results to stakeholders.
Conclusion
Good research takes a mix of quantitative and qualitative methods. You might be a data analyst, but at the end of the day, what you are looking for is to understand the overall user experience and to see the full picture.
As you can see, five second testing is not rocket science at all. So if you want to create better online products, please just do it! Combined with data analysis, usability testing, A/B testing and other methods, this will be a powerful weapon in your research arsenal!
- If you want to learn more about how to become a data scientist, take my 50-minute video course: How to Become a Data Scientist. (It’s free!)
- Also check out my 6-week online course: The Junior Data Scientist’s First Month video course.
Cheers,
Tomi Mester
Cheers,
Tomi Mester
