Voice Search
Voice in Apps: Amazon Shopping
In-depth analysis of the voice search feature in Amazon India Shopping app.
Till date, most of the attempts to add voice search to e-commerce apps have been half-hearted and half-baked. Most of these companies just tied up the mic button in the search bar and Google’s Speech Recognition together and called it a night. Tata, Goodbye :) Not Amazon, they went a step ahead and added ‘Alexa’ to their shopping app.
Voice Search is taking over India. It looks like Amazon took notice of Google’s Year in Search report 2018 which showed 28% of all the searches are happening through voice. That’s not all, voice searches are growing at 270% year on year in India. Hindi voice searches are growing at 400% YoY. It shouldn’t come as a surprise, when Jio added Google Assistant to its Jio Phone, Google Assistant’s usage in India jumped by 6X.
In this week’s ‘Voice in Apps’, we are breaking down the Amazon Shopping App. For the uninitiated, Slang Labs has started a new(now pretty old) series called ‘Voice in Apps’. Every week(we try to!) we take an app which has integrated voice inside it and break it down. We are doing this because we think that it’s important to give recognition to the trendsetters. We want to surface how and why these businesses are adding voice inside their apps and what is the result of it. We have already broken down voice features in ‘My Jio’, ‘Gaana’, YouTube and Paytm Travel.
Visual Breakdown:
Mic Icon
We have a standard easy to understand old school mic icon on the top right side of the title bar of the Amazon App. The icon is filled with white which gives a clear contrast to the background colour. Placement of the mic icon at such a prominent place in the app indicates the seriousness and value Amazon believes in-app voice shopping can deliver to their customers.
Difference of Mic Icon in Amazon US vs Amazon India app
Amazon has added Alexa’s logo in the US app for voice shopping feature instead of the mic button. This is probably done due to a higher brand recall of Amazon Alexa. Alexa’s mindshare is much higher in the US when compared to India because of the high market share of Echo devices running Alexa by Amazon.
Onboarding Flow:
Users are introduced to this voice search mic by a simple coach mark. On clicking the mic button user sees a dialogue box which explains what the user can do with Voice. On pressing the continue button, the user needs to grant mic permission. Alexa is triggered after the permission screen and the characteristic blue wave appears. You can also look at the videos of onboarding on our YouTube channel.
‘Speak To Shop’ Dialogue Box:
Speak To Shop is the biggest element on the dialogue box attracting instant attention. This helps to set the context to the user and introduce them to the feature.
Try Saying has a couple of utterances shown as an example to train the user. This helps in setting the right expectations for the user. Even, in the Slang surface, we show these sentences to guide the user.
Amazon tells the user to enable Mic permissions for this feature. In Slang, we speak out the purpose for the permissions and thereafter show it.
UI Breakdown
It might not seem like much but there is quite a bit to breakdown here. Let’s get started.
On clicking the mic button a user gets to see a bluish-green moving wave at the bottom of the scene with help utterance right above it. This helps in setting the context and guiding users on what they can ask Alexa.
Listening mode: While the user is speaking the greenish wave vibrates from the centre-out.
Processing Mode: When Alexa detects silence or end of the utterance, the greenish waves go all the way till the end. The waves pulsate slightly as well.
Speaking Mode: This mode the wave appears to be more of a horizontal line filled with moving blue colour.
Auditory Feedback: As soon as the mic is clicked, Alexa starts listening after a ‘ting’ which acts as an auditory cue. The same ‘ting’ is triggered when Alexa stops listening. We have implemented the same thing in Slang as well. We have a different ting sound when we start and stop listening. We hope users get trained overtime and become familiar in knowing when to start and stop speaking.
Error Handling:
After an unrecognized utterance, Amazon shows this help screen which tells the user, Alexa couldn’t find what user was looking for and shows a bunch of utterances use can speak.
This dialogue box is opaque and has an Alexa button to speak again. Food for thought — why did Amazon not add the mic button here instead? Why add a button which users haven’t been exposed to yet, instead of one which they recognize.
Functional Breakdown
Accuracy
Full points to Amazon on accuracy for Alexa, It’s highly accurate even when spoken in different accents. It even recognized long brand names and difficult product names(like mamy poko pants ) with almost no recognition error which is surprisingly good. Be it long-form product names or produce name with just one simple word they have managed to outdone themselves. Accuracy is top-notch.
Speed
Alexa excels over here as well. Breaking down the speed of exact components like ASR and NLP is not possible as Alexa doesn’t let you look under the hood. If it did, we would have been able to get more insights. Having said that, Alexa in the Amazon Shopping app is extremely fast. Search results pop up almost in under a second. Kudos to the team at work who made this happen.
Natural Language Processing
This is one area where Amazon is still falling behind. “I want to see toys Alexa” searched for ‘Toys Alexa’. Removing fluff and stop words from an utterance is not that difficult of a problem to solve. This is one area where Voice search in Amazon app has been disappointing. Lack of a good NLP engine brings down the Voice Augmented eXperience in Amazon app.
What can I do with Alexa?
You can also ask Alexa, what you can do with it using voice. It tells you the two things it can do via voice. Instead of just replying with a voice-only answer, Alexa should utilize the screen and show the user all the things it can do both visually. Currently because of the only three choices
Limitations around Voice Search in Amazon App
Alexa’s Boundary issues
During voice search onboarding Amazon tells us two things we can do with the mic icon — Track orders and search items. Since it’s Alexa that is embedded in the app and not just another shopping assistant, it can do a lot more than tracking and searching orders. This ability feels more creepy than useful. If I speak something(maybe I am talking to someone with mic icon clicked) which is not a product in Amazon’s database, Alexa starts speaking out search results which feel awkward and weird.
Sometimes bowing down and saying, I did not understand that is just fine.
Not reproducible, but it even tells me about the weather and random facts without even asking for it. Imagine while shopping, Alexa speaking out…”According to NHS, you should at least wash your hands for 20 seconds..”.
Support for Vernacular Language
If you switch to Amazon India app to Hindi, mic icon disappears. Vernacular support is still missing. India being so linguistically diverse, supporting vernacular languages should be a high priority for Voice Shopping.
Better UI and surface design
Although highly subjective, but in my opinion, Alexa’s UI could be much better. Right off the bat, the text in the help window can be increased to improve readability. Hints shown on the Alexa surface in the listening mode could be bigger too.
Amazon’s Voice search is hands-down the best Voice Augmented eXperience in any eCommerce app. (Probably not for long. *Wink*). Even in its current avatar, it’s miles ahead of any other competitor.
Slang has recently introduced ‘Slang for Grocery’. It’s VAX experience built specifically for the Grocery domain. It’s an off the shelf integration which voice enables your app in 4 different languages — English, Hindi, Tamil and Kannada(Malayalam in beta). With ‘Slang for Grocery’, you can voice-enable your grocery app in less than 2 days.
If you would like to add the most accurate multi-lingual Voice search to your Grocery app in just a couple a days, let us know at 42@slanglabs.in
