INTRODUCTION
Google Home Mini gives you answers, and personalized help with your daily tasks, and hands-free your entertainment all by recognizing your voice.
My test goal is: How Well It Responses to Our Commands through different tone of voice, various wording choices, long sentences or precise use of keywords/phrases.
From the findings, we are looking for potential recommendations and improvements through four key cognitive dimensions: efficiency, consistency, hard mental operations, and end-user satisfaction.
OBJECTIVES
The test aimed for:
Efficiency: Google Home Mini is a product based on English speaking. However, some users come from different languages and cultural backgrounds with the different ability of English speaking.
Can it understand various ways of asking the same question?
Can it understand a command that is lack of grammatical structure?
Does it have a wide range of languages in the system?
Consistency: A consistent and smooth conversation is the key to a successful voice-controlled speaker.
Can it complete a long and complicated conversation/command?
Hard mental operation: Does the language barrier cause any unnecessary failures during conversations?
End-user satisfaction: Does the language barrier cause repeated failures that lead to frustrations?
METHOD
-PARTICIPANTS-
Five ages from 25-30, non-English speakers, 3-5 years of working experiences, work and study related to art design, web design, coding, programming, etc.
Accepting new technology, and willing to challenge and learn new products.
-PROCEDURE-
I chose to use Remote Testing for this test. All the equipment was set up in the participant’s home 30 minutes before the test. First, I gave a brief about the process and pre-test demonstration on how to voice-activate the device. Then, I used the walkthrough method for the test process.
Each participant had three scenarios, and each scenario contains 5-6 tasks. Thirdly, I chose to use Likert Scale as a post-test questionnaire to collect their satisfaction rate. To follow up their inner thoughts, I also asked a few questions. I used one camera during the test to record his/her emotional reactions and voices/comments.
-METRICS-
1. Qualitative data: I used their verbal comments, expressions and actions to find both hard-mental operations and satisfaction.
2. Quantitative data: I used time-to-completion and repeated action rate to measure consistency, questionnaire to find satisfaction level, and completion success rate for finding efficiency.
FINDINGS
1. When the question is way too long, Mini is not catching any keywords. The long and complicated sentence confuses its thinking process, eventually, lead to –
A) “Sorry, I do not understand.” / “Sorry, not sure how to help. I am still learning…” – which means there are too many hot words, and Mini does not know which one to pick.
B) a wrong answer – which means Mini did not grab the right keywords from the user’s question. The intention went wrong.
2. This function is not available in Google Assistant yet.
Such as: - remove beef from the shopping list.
- what is the busiest hour of this restaurant?
3. Translation issues.
- Translates when: it is an English sentence, long or short because it is an English based device. Only if your pronunciation is relevantly good with no errors, and speak clearly and smoothly. Then, Mini will translate the whole sentence to the language you want to translate to.
- Cannot translate when: your sentence/words/phrases are in other language and want to translate to English.
Therefore, I think it is not very user-friendly at this point. A lot of non-English speakers may want to use the translation in both ways.
4. Sometimes, participants can have a fluid conversation with Mini back. Usually can take up to 5 sentences. Participant does not need to say, “Ok Google” to activate it, context will guide Mini to the right answer. However, many times, Mini loses the context, but participants thought it still on track and will keep talking. In this case, Mini gets confused and either give wrong answers or end the conversation on unintentionally.
5. Voice-control: activation.
A) Participants tend to forget to say, “Ok Google” before each conversation. The repeat action is causing a small level of frustration.
B) Mini may not hear the participant’s voice when it is ringing.
6. The use of Google Assistant’s natural language processing:
A) when participants are not English speakers, their sentences may not sound as natural as native speakers. Therefore, Mini may miss the keywords or misunderstood user’s intention, provides wrong answers.
B) The natural language processing has its code which is not very human-like enough, so if the participant did not use the words that are already programmed in its system, then the answer may get wrong.
7. When Mini is hearing, try to speak fluidly and clearly; avoid any tiny blank of time. Otherwise, Mini will either assume you complete the sentence already and start the process and provide the wrong answer, or stops the conversation because it did not receive any keywords.
CONCLUSIONS
From my observations, Google Home Mini requires a high level of learnability, and more suitable and useful for people who adopt new functionality fast. Here are three recommendations that I think will help Google Home Mini to expand the range of their user groups and make its usability easier for future users.
Google Assistant system needs to be personalized more. Automatically learn and add user’s voice, tone, wording preference, new vocabulary. So the future conversations will slowly get smoother, and task completion rate will increase gradually.
Expand the databases on languages so that Google Home Mini could recognize the user’s pronunciation more accurately. Moreover, also, be able to translate more vocabulary and even long sentences, paragraphs.
Add a Helping function – when the user failed to ask a question more than two times, which means Google Home Mini did not catch the correct meaning. Then, it automatically provides 2-3 choices of what it thinks is the right answer and let the user pick. Add the user’s option to the user’s customized profile data for future use.
Add a screen in the future design, so more functions will be available for users. Search Google images, watch videos, transfer images, video call other users. Etc.
