Get feedback to see if your dialog is working
1) Quick and dirty WOZ experiment
2) Standard WOZ experiment
For the most realistic experience, simulate the persona’s role by playing the persona’s prompts using the TTS Simulator in the Actions on Google Developer Console. Download the audio to have it ready to play on demand.
This version requires four things:
- A conversation script that provides directions on what the persona should say after each user response. The high-level flow (or a simplified version of it) is ideal for this.
- Downloaded audio of all the persona’s spoken prompts. Use file names that will help you quickly identify the correct file to play.
- Someone to play the "user." This should be someone who’s unfamiliar with your Action.
- Someone to play the “wizard." This should be someone highly familiar with your Action.
Have the wizard start the conversation by playing the audio for your Action’s greeting, for example, “Welcome to your launchpad for all things Google I/O. The festival's underway right now. Are you one of the lucky attendees?” The wizard will then wait for the user to respond, hopefully with a synonym of “yes” or “no”. Once the user has responded, the wizard will have to quickly consult the high level-flow to determine what prompt to play next, then find and play the correct audio file.
3) Standard usability experiment
Of course, once you’ve started building your Action, you should test it often using the the Actions Simulator in the Actions on Google Developer Console. Have your friends, family, or colleagues test it too!
|No matter what experiment you use, be sure to do the following:|
|Talk it out||Since your goal is to update your design to reflect what works best for real users, you want your WOZ prototype to be as close to reality as possible. What looks good on paper doesn’t necessarily sound or feel natural in real conversation, so make sure users are hearing your prompts and speaking their response.|
|Record your sessions||Get permission to record your sessions so you can go back and listen to them. Take note of any issues that arose during the session.|
|Ask for feedback||Ask the user to describe their experience in their own words. How did it meet or fail to meet their expectations? Did anything surprise them? Were they satisfied? Remember that the focus is on their behavior, not their opinion.|
Running a WOZ experiment allows you to understand how people will engage with your design. You may find that users were doing something very different than what you had expected, requiring you to alter the design to better align with their needs and expectations.
Bottom line: Focus on the usability of your design (and not on users’ opinions). Iterate based on user behavior, and test again if time permits.
|Things to look for (and how you might improve your dialog):|
|Natural conversation||Pay attention to the way users naturally ask for things. Do they feel like they can only speak in short keyword-like phrases, or do they sound more conversational? Do they sound hesitant or confident when speaking to your persona? Does the flow make users feel like they can only provide one piece of information at a time, or does it encourage them to provide multiple details in one sentence?|
|User confusion||Look for places where users look confused or are unsure of what to say or do. Examine the previous prompts to see where you could make some clarifications. Was the call to action clear?|
|Unexpected utterances||Users might say something you didn’t expect. Take note of it and add handling for it in your design.|
|Signs of frustration or impatience||This is typically a sign that the interaction is too long-winded. Review your prompts to see if you can be more concise. Are there details that can be omitted?|
|Observe who’s speaking the most||Do users seem to be in control of the conversation? If not, how can you change that?|
Robust testing is essential for developing high-quality software and creating user satisfaction.