User testing - no problem: anyone can ask questions!
That's true! But do you get the answers you need as a start-up to develop a successful solution on the market? Who is the right person to ask questions to? Which questions should you ask? And how do you formulate these questions correctly? In the various JU-KNOW training modules on user testing, interviews and moderation, start-ups learn everything they need to know to learn as much as possible from future users. This also includes how to talk sensibly with future users about innovations that they cannot yet imagine using. Training participants can immediately try out the instructions and tips with business actors and familiarise themselves with various conversational tactics and different tools hands-on. With the help of the training sessions, start-ups become fit to talk to future users not only about their solutions, but also about attitudes, motives, decision-making factors and emotional aspects, thereby gaining valuable insights for the development of their innovations. In the following, we have summarised the most important basics from the beginner's module of our programme in a DIY guide - in other words, everything that generally needs to be considered during user testing:
Preparation of the user testing:
#01 obligatory: Preparing interviews
- Develop a 'topic guide' with a 'common thread' for the aspects to be discussed (keyword 'funnel model':- start with more general topics and become more specific / detailed as the conversation progresses)
- Formulate an open introductory question for each topic
Tip: only formulate the introductory question - write down the other points in keywords so that the questions can be freely formulated during the conversation
- Do not number questions consecutively, it is better to work with bullet points to prevent a rigid 'rewinding' of the questions
- If more than one person is conducting the conversation: note important information or instructions at the appropriate points to ensure that the conversation is conducted in a uniform manner
#02 Clarify the guideline for the interview to all interviewers
- The aim is to understand the 'big picture' of future users
- therefore: do not talk about the idea / solution itself from the outset - but focus on these potential users, on typical situations in which the idea could be applied
- proceed impartially, do not take needs for granted, do not ask leading questions
#03 optional: Conduct test interviews
- Objective: familiarise yourself with the guidelines, gain confidence and aplomb in conducting discussions
- Do not read out questions, better speak freely
- Adapt the order of the questions / sub-items to the course of the conversation, not necessarily 'work through' point by point
- work with a supervisor if possible
- Objective in the case of several interviewers: to ensure that they all work in a comparable way
User testing itself
#1 never pitch - at no point in the conversation
- Do not imagine yourself as a developer or expert for the solution
- do not explain or explain the idea (in detail) - leave the access to the future user (even if this is sometimes difficult)
- Proceed neutrally, do not show your own enthusiasm - so as not to nip sceptical or critical comments in the bud
#2 Don't forget the introduction and rules of the game
- At the beginning of the conversation, provide information about what the conversation is generally about
- point out that ...
- there are no 'right' or 'wrong' answers, but everything that comes to the interlocutor's mind is of interest (keyword: we want to learn)
- positive and critical statements are very important
- The personal opinion of the dialogue partner counts - not what 'others' think about it
- Build trust through a 'warm up' - Choose an 'easy' introductory topic that fits in with the overall theme
#3 have an open, relaxed conversation.
- do not play 'question-answer ping-pong'
- not be an (investigative) interviewer, but a curious dialogue partner
#4 Listen more than talk yourself
- Formulate introductory questions in such a way that no yes/no answers are possible and 'flow' can develop
- Allow pauses in the conversation, do not immediately ask the next question
- Signal non-verbal interest to keep the 'flow' (nod, smile, eye contact)
- Pay attention to non-verbal signals from the other person
#5 precise questions
- Ask short, concise questions
- Do not formulate nested sentences
#6 ask clearly
- do not ask either - or questions
- Do not ask leading questions that already direct attention in a certain direction
#7 Do not provide answers, do not articulate your own opinion
- The expert role is played by the answerer, not the questioner
- Allow diversity of opinion and always remain neutral yourself
#8 Focus on the present
- even potential users cannot predict the future - but they can say a lot about what they are doing today and how, and whether the experience and result is satisfactory for them
#9 Learning from the concrete
- not: 'Where would you like to go on holiday?
- better: 'when you think about your last holiday - what were the highlights?
#10 Do not be satisfied with the first answer
- Ask for reasons and motives, for the causes 'behind the scenes'
#11 Understanding why something is important
- use the direct "why?" sparingly
- better explore the context with various W questions: how, what, when, what for, where, with what, why, where from, who / for whom
#12 Understanding what is true
- undermine the mechanism that people talk about doing or wanting to do certain things - but their reality is often different
- Provide a frame of reference: e.g. instead of 'how often do you go jogging? last week jog?'
#13 Do not ask directly about willingness to pay
- EURO amounts can only be transferred to a later market situation to a limited extent - it is clearly not advisable to draw conclusions about future willingness to buy
- the focus on the value of the result that the user sees by using the solution is far more meaningful (keyword: expected benefit)
- Pricing should be developed using suitable standardised procedures
#14 Defining the competitive environment 'outside the box'
- not only include the existing or expected market supply
- also identify unconventional 'work-arounds' of potential users that they currently use
#15 at the end of the conversation: swap roles
- Leave the final question(s) to the dialogue partner: is there anything to add from the perspective of the future user that has not yet been discussed?
- Now you can answer questions about the solution, give explanations and share your own enthusiasm!
Last but not least: #16 Ask the right people
- In the first step, work out potential target groups / personas in as much detail as possible
- Do not look for dialogue partners in your own social environment - a positive bias is likely here
- If necessary, commission a recruitment agency to find the right target persons
- Better to 'deep dive' into one / a few potential target groups than to collect superficial information from different target groups
#17 Create a good overall atmosphere
- Be friendly
- Be attentive
- Do not allow any distractions (smartphone on flight mode)
- Agree on a time frame in advance - and stick to it
- Make sure that all questions were asked - so that a comparable analysis of all interviews is possible
#18 Guarantee confidentiality and comply with data protection regulations
- Analyse conversations in anonymised form only
- do not associate any personal data with the statements
- In order to do everything right and avoid trouble, it is best to consult the appropriate sources on the Internet
The more interviews are conducted, the more advisable it is to record the interviews. This requires the consent of the interviewees, which is best obtained when the appointment is made. When recording, make sure that no personal information can be heard on the recording. Alternatively, a second person from the startup team can create a simultaneous protocol during the interview - or the interviewer can write a memory protocol.
Ursula Kloé
Founder and Managing Partner JU-KNOW GmbH Heidelberg, Market Research Consultant; New Mobility Enabler; Lecturer at SRH University of Applied Sciences Heidelberg and at Mannheim University of Applied Sciences / inno.space on User Testing & Need Finding, Marketing & Sales, Soft Skills, Design Thinking