Every online data analyst and/or researcher should use – or at least be familiar with – qualitative methods. Even if you are heavily into analytics and even if you trust numbers more than anything else, you can learn great things if you run a few researches with smaller sample-size, but more in-depth understanding. Today I am going to show you one of these: Five second testing!

What five second testing is?

Five second testing is a qualitative research method, where you show a test-user a screenshot about your website or product – for five seconds only. Then you take away this picture and ask different questions about what she/he has seen – or think she/he has seen – there!

Your tested media could be anything. Mostly it’s the above the fold part of your landing page, but it could be a banner, an app menu, a complete UI, anything.

This is how a five second test looks from a tester perspective:

I’m a big promoter of five second testing, because it’s super easy and super quick to set up. And at the same time it gives you many highly valuable feedbacks.

Why 5 second?

We all know, that nowadays people are browsing between websites crazily fast. It’s self-explanatory, that first impression is key here! With five second testing you can get immediate feedback about your design – based exactly on this first impression. Do they get your main message or not? Does your brand look trustworthy or not? Do they understand your main product or not?

I’d like to emphasize here the “immediate” word. To set up a five second test you don’t need to build the code part of your website. It’s enough if you have a design in PhotoShop, that you can upload, test and validate. It can all be done very fast.

When is it useful?

I usually use five second testing in two cases.

The first one is when we work with designers on building a brand new website, product or campaign. We want to get early feedbacks about the overall user experience. And these quick tests always help us to go into the right direction from the very beginning, and be more agile and emphatic with the users even before the first release. Usually the process looks like this:

  1. Brainstorming.
  2. Come up with 3-5 different design variations.
  3. Five second testing each version.
  4. Iterate.

(I usually like to use multiple methods, not just five second testing, I’ve written about it here: Online Research Framework.)

Five second testing becomes pretty handy in another common situation: before you want to release an A/B test.
A good A/B test is backed both by data (eg. funnel analysis, cohort analysis, segmentation, etc…) and qualitative researches (eg. usability testing and user interviews), but before you’d start it, you should do a proof-check and do some quick five second test too. This is important, because A/B testing is a big investment in terms of time. Most of the small or mid-size startups and e-commerce companies have to wait one or two weeks (or even more) to get significant results. For five second test results, you have to wait 1-2 hours tops. Of course, it won’t be significant, but it will give you at least a hunch. If the answers from the test are very off, maybe it’s worth reconsidering the A/B test release too.

How to do it? Which tool? How much it costs?

It’s not a big magic to build a basic five second testing platform for yourself, but if you don’t want to, you can use the same testing tool, that I do: https://usabilityhub.com/five-second-test (I don’t have any affiliation to them.)

Their pricing is fair. An optional $99 monthly for premium features (anyway I usually don’t use these) and $1 for every test-person they hire for you. (Yes, they can hire test-people for you!) If you want to target specific segments, then $3 per person. You can hire users by yourself as well – that’s free. If you know, where your target audience is, I suggest to do this latter.

Anyways setting up a test in UsabilityHub is super easy:

  1. You have to specify your scenario and/or your instructions. Something like: “Imagine that you are about renting a car and you find this website…”
  2. Then upload your design (eg. the above-fold part of your landing page)
    five second testing - upload your design
  3. And the final step is to specify the questions you want to be answered. Eg. “Does this brand look trustworthy?” (I’ll list out my favorite 5 second test questions below!)
    five second testing questions

And done! Send out your five second test to the target audience – or hire people via UsabilityHub. In 1 hour you will have the results.

Some five second testing best practices

#1 – What to test?

As I’ve mentioned above, most of the time I am testing landing pages and banners (eg. for Facebook ads). If you have an SaaS-kinda (Software as a Service) web product, you can get feedback about your UI too. If you have a very early stage product, you can even upload wireframes. Basically it’s up to your creativity, what you’ll research here, but I’d recommend to focus on the first touch-points. (It makes a bit less sense to five second test the bottom of your website, because the people, who scroll down there have already a first impression about your stuff – most probably a good one as they didn’t bounce without scrolling.)

five second testing data workshop in berlin

I’ve five second tested these banners for my Data Coding Workshop in Berlin (back to 2016)

#2 – Best 5 second test questions

It’s important not to bias your results! Thus we don’t like to ask so called “closed-ended” questions (the ones, that they can answer with yes or no.) It’s better to ask something around what the testees can recall or what grabbed their attention. My favorite five second test questions:

  • What do you think this page was about?
  • What product do you think this company sells?
  • What’s your first impression about the site?
  • What grabbed your attention?
  • What words or sentences can you recall?
  • Etc…

It’s always good to have hypotheses before the test. Having expectations will enable you to decide if the answers are in line with what you want to see or they are completely off.

#3 – The situation and the instructions

You can give instructions before the test. Keep this simple. And make sure, you don’t bias your results here either – so you don’t give any information here, that can directly affect the answers in the questions part.
five second testing instructions

Sample size and evaluation

In my opinion, the perfect sample size for a five second test is somewhere between 20 and 50 people. Don’t forget, this is a qualitative method – so you are not looking for significant results here, but for an educated guess.
And keep this in mind also when you evaluate. I recommend you to go through every answer one by one – but you can print a word cloud from the received answers as well, if you have to present results to stakeholders.

five second testing results wordcloud

Usability Hub – Results in “wordcloud format”

Conclusion

Never forget: a good research is a mix of quantitative and qualitative methods. You might be a data analyst, but at the end of the day, what you are looking for is to understand the overall user experience and to see the full picture.

As you can see, five second testing is not rocket science at all. So if you want to optimize your online surfaces – or you want to release a new product or website, please just use it! Combined with analytics, usability testing, A/B testing and other methods this will be a powerful weapon in your research arsenal!

If you want to be notified first about new content on Data36 (like articles, videos, handbooks, etc.), sign up for my Newsletter!

Cheers,
Tomi Mester