Alexa+, the Liar? Testing Amazon’s new voice assistant

Alexa+ is the name of Amazon's new and improved voice assistant. The new system is based on artificial intelligence and allows for significantly more flexible interactions than before the update. Goodbye memorizing voice commands, welcome to the age of truly intelligent assistance - at least, that is what Amazon is very convinced of.

And indeed: Anyone who already has early access to Alexa+ will quickly notice that a lot has changed. No matter how complex the question is formulated, Alexa usually knows how to answer. The unpopular "Sorry, I don't know that one" seems to finally be a thing of the past. You also no longer hear "Sorry, I didn't quite catch that." Instead, Alexa+ asks for clarification when things are unclear and often provides a pretty reasonable answer on the second attempt.

For instance, the rather complex request "Alexa, create a recipe for Greek moussaka, but vegetarian, meaning without minced meat and instead with more tomato sauce and a cheese crust" was no problem for the new assistant. That she then read every bullet point (-) in the recipe aloud as: "You need minus 500 grams of potatoes, minus 500 grams of eggplants...", can be forgiven at this point. Such initial bugs will certainly be ironed out over time, and you generally know that negative ingredient quantities are rather unrealistic anyway.

A bigger disadvantage, however, is this: When the voice assistant makes a mistake now, she does so with fervent conviction - articulated in such a way that you almost wouldn't dare doubt the answer. Even when Alexa is completely off the mark, which unfortunately happens from time to time. Recently, for example, when I asked how deep Lake Neufeld in Burgenland was, I got the answer that it was only a few meters deep. In fact, it is 24 meters deep and therefore anything but shallow. Alexa must have confused the lake with the much shallower Lake Neusiedl, but she still answered: "Lake Neufeld is a very shallow lake in Austria and only a few meters deep." If I hadn't known that diving trips are offered there, I would probably never have noticed this outright lie.

Indeed, this phenomenon - the invention and generation of completely false information - is called "hallucinating" in tech jargon and is a typical problem with artificial intelligence. It is also not unusual that such errors cannot be reliably reproduced. For example, in the new chat function of the Alexa app, which I wanted to use for a proof screenshot, I reliably got the correct answer every time.

But users notice that the game has changed with the upgrade to Alexa+ not only by the different-sounding answers and the altered voice and tonality. The Follow-Up Mode is also newly activated by default. This causes Alexa to listen for a few seconds after an answer so she can reply to follow-up questions without a wake word. This actually makes a more natural conversation possible, partly because the voice assistant now "remembers" the course of the conversation and doesn't treat every question as completely new, but answers it within the context of the ongoing chat.

Like so many things, however, the Follow-Up Mode also has its downsides: If you don't actually want to have a conversation with Alexa, but just want a single question answered or a command executed, it now happens quite often that Alexa misinterprets the conversations of other people in the room. Here is an example from my everyday life: While cooking, I asked Alexa to set a timer; next to me, my parents were talking. After Alexa set the timer and confirmed it to me, I continued the conversation with my parents. That the voice assistant said something again a few minutes later, which got lost in our conversation, was met with a shrug. It was less pleasant, however, when I realized that Alexa had deleted the timer. She must have picked up some snippet of conversation and felt compelled to cancel it.

One last notable disadvantage: At least on older devices, the processing of even simple instructions now takes noticeably longer. In absolute terms, it's still only a few seconds. But if you are used to an almost immediate response, you will often wonder about the unusually long reaction time. According to Amazon, the devices of the newest Echo generation are supposed to remedy this, as they are optimized for Alexa+. Users in the comments confirm this, but the new devices are also significantly more expensive than what people are used to.

Finally, it should be noted that Alexa has definitely taken a step forward with the Plus update. It would even be wrong to say it was just a small step in the right direction. Rather, the voice assistant takes a massive leap forward here - so far that, in some cases, it has simply overshot what should actually be the goal.