Toyota’s new ‘smart assistant’ learns voice commands and gets smarter over time with machine learning –

Author: Eric Walz

When a driver calls for help, Toyota’s Intelligent Assistant listens and responds to the conversation in natural language.

As vehicles come with more technologies and features, using some of them on the road can become a distraction. Toyota has therefore developed a voice-activated assistant to assist with “Toyota Connected” vehicle services.

Toyota’s new in-car “intelligent assistant” is a highly advanced voice assistant that can learn driver commands and get smarter over time. It is available for both Toyota Audio Multimedia and Lexus Interface, including the 2022 like the Toyota Tundra and Lexus NX.

Toyota offers connected services trials for each vehicle, which include Drive Connect services, which offer Intelligent Assistant, Cloud Navigation and Destination Assist.

Toyota says rolling out these smart connected services will transform it from an automotive company into a mobility company.

According to Ryan Oehler, Product Owner at Toyota Connected, an independent software and innovation company, who leads the team responsible for Toyota’s new smart assistant.

When a driver calls for help, the smart assistant listens and responds to the conversation in natural language. For example, if a driver says “I would like a coffee”, the system will display a list of nearby coffee shops on the vehicle’s infotainment screen and then ask the driver if they would like to drive to one. for a lunch break.

Suppose a driver searches for a nearby Starbuck and the assistant displays several listings, then a driver can say “take me to third” and the navigation system will launch the turn-by-turn directions.

To aid navigation, the map can be displayed on the infotainment screen at any time by saying, “Hey, Toyota, show the map.” Drivers can also ask the assistant to zoom in on the map for easier reading.

Each request is fulfilled using a complex web of commands and behind-the-scenes processing on the vehicle itself and the cloud to complete it.

Oehler said there has been a steady ramp-up of the smart assistant to cater to both mainstream US customers as well as tech-savvy “power users” with intuitive features for drivers. He also said that even more advanced features are in the works.

Not only does Intelligent Assist listen to the driver’s voice using Automatic Speech Recognition (ASR), it also examines the waveforms in the audio. When the driver presses the talk button in the vehicle, it transcribes that speech into text both in the vehicle’s on-board computer and in the cloud.

By analyzing the audio waveforms, “the system is able to understand, phonetically, what that translates to and formulate transcriptions based on that audio input,” Oehler said.

The technology is even able to recognize commands for different accents, dialects and even different pitches, to recognize what to do next. Since Intelligent Assistant is available in the United States, Canada, and Mexico, it is designed to recognize commands in English, Spanish, and French.

Once a command is transcribed into text, it is processed by Toyota Connected machine learning models in the cloud. Machine learning models can determine the intent of words used, from audio commands to windshield wipers or even find a nearby five-star restaurant.

From there, it is sent back to the vehicle where the notes are compared between the built-in and cloud voice assistants to ensure that the car most accurately executes the command the user actually wants.

“That’s where the Toyota Connected magic comes in,” Oehler said.

The system first determines “primary intent”, which includes common vehicle controls such as temperature, audio and navigation. The next step is to determine the “sub-intent”, such as a specific temperature, song title, or frequently visited cafe.

If a driver requests a certain song while already listening to their favorite streaming platform such as Spotify, the head unit will attempt to find the song on that specific platform. If not found, the main unit will switch to the next best option, such as satellite radio. Drivers can also instruct Intelligent Assist to play specific genres of music by saying “Hey Toyota, play some rock music.”

The system gets smarter by looking at accuracy and adjusting controls to add more ways to request specific functions, according to Toyota. Drivers can also instruct the system to send a text message to a saved contact by simply saying the person’s name and then using their voice to send a message.

“Some audiences expect to interact in completely different ways,” Oehler said. “Younger audiences who are more familiar with their modern voice assistants tend to operate more fluidly, while less familiar people would say ‘Navigation’ and then ‘Texas’ in a series of steps,” for example. To help the user, we have added more contextual elements, such as prompts, to guide them through this new user experience. One of the other big things we learn is what our customers want to use every day.”

The intelligent assistant is already available in Lexus NX, LX and Toyota Tundra pickups for the 2022 model year, but will be introduced in other Toyota models for 2023 in North America. As it rolls out to other models, the rate at which the machine learning system gets smarter will only accelerate, according to Toyota.

Among the planned novelties is the fact of asking the intelligent assistant to tell a joke. A sense of humor was added to the virtual assistant’s repertoire after the engineering team observed customers asking for it.

“There are a lot of opportunities,” Oehler said. “There’s value with a voice assistant because it adds depth to the experience, it’s smart, and it can and will likely build consumer trust as more and more people use it and evolve in future iterations.”

Using the smart assistant is also optional on Toyota vehicles. For drivers who prefer not to use it, the vehicle’s built-in voice assistant can be used instead for more basic voice functions, such as adjusting the interior temperature.


Source link

Comments are closed.