Beyond Prompting: How Voice Will Define the Future of AI

Author(s): Yaksh Birla

Originally published on Towards AI.

Remember when we thought the pinnacle of AI interaction was crafting the perfect text prompt? Well, buckle up all you “prompt engineers”, because we’re about to leap into a world where your AI assistant isn’t just reading between the lines — it’s speaking them out loud. And trust me, this isn’t your grandma’s Siri we’re talking about.

The Silent Revolution Gets Vocal

For the last 2–3 years, we’ve been hammering away at our keyboards, trying to coax the perfect response from our AI companions. Entire companies and jobs were created with the sole purpose of mastering “prompt engineering”. And don’t mistake me — it is very useful. AI systems still need a certain degree of structure to generate desired outputs, so prompt engineering is not going away anytime soon.

But let’s face it, typing is so last decade. People are impatient (I sure as hell am) and do not want to experiment with multiple different prompts to get what they want.

News flash: Most people aren’t wired to be prompt engineers. People are, as it turns out, wired to speak. And tech giants are catching on fast.

Therefore, the real revolution is happening right now, and it’s all about voice. It is a deliberate effort in abstracting away the need for prompt engineering and enabling more intuitive human-AI interactions and outputs. As Eric Schmidt, former CEO of Google, prophesizes:

The internet will disappear. There will be so many IP addresses, so many devices, sensors, things that you are wearing, things that you are interacting with, that you won’t even sense it. It will be part of your presence all the time. Imagine you walk into a room, and the room is dynamic. And with your permission, you are interacting with the things going on in the room.

Why Voice is the Future of AI Development and Human-AI Interaction

Voice interaction isn’t just a minor convenience — it’s a fundamental shift in human-AI interaction. Let’s break down why voice is the future:

1. It’s Natural: We’ve been talking for millennia. It’s time our tech caught up.
2. Context is King: Advanced AI can now grasp nuance, tone, and even sarcasm.
3. Personalization on Steroids: Your AI will learn your quirks, preferences, and possibly even your mood.
4. Multitasking Magic: Imagine planning a party while cooking dinner — all hands-free. Voice assistants will seamlessly manage smart devices and apps.
5. Goodbye, Robotic Chats: Think less “computer interaction,” more “knowledgeable friend.”
6. Accent Adaption: Accommodating different cultural nuances and offering global accessibility.

The Voice AI Arms Race: Who’s Leading the Charge?

The race to dominate the voice AI space is heating up, with tech giants and startups alike vying for supremacy:

Google

Google has recently launched Gemini Live, a new AI voice assistant focused on natural, free-flowing conversation. Key features include:

• Ability to interrupt and change topics mid-conversation
• Choice of 10 distinct voice models
• Integration with Google’s productivity tools
• Available on Android devices with a Gemini Advanced subscription

Google is positioning Gemini Live as a “sidekick in your pocket” capable of handling complex tasks and research. Here’s a video displaying just a sliver of Gemini’s voice capabilities:

Apple

Apple has not yet released a new voice AI assistant, but is taking a measured approach with a focus on privacy and security and a promise to overhaul Siri slowly, but surely. Recent efforts include:

• Apple plans to market its new AI capabilities under the name “Apple Intelligence”.
• On-device AI processing for enhanced privacy and scalability
• Exploring integration of AI with iOS and macOS, allowing Siri to control individual app functions using voice commands for the first time.

Apple is expected to announce major AI updates, including potential voice AI advancements, at their upcoming events.

OpenAI

OpenAI has introduced Voice Mode for ChatGPT, pushing the boundaries of natural language and human-AI interactivity. Key features include:

• OpenAI’s Voice Mode enables real-time, natural voice interactions with ChatGPT, allowing users to engage in back-and-forth dialogue and change topics seamlessly.
• The system supports multiple languages and various accents, utilizing OpenAI’s Whisper for accurate speech recognition and transcription.
• Voice Mode leverages GPT-4o, combining audio and text processing capabilities, and features human-like voice responses generated through a dedicated text-to-speech model.

Anthropic

Amazon has a $4 billion minority stake in Anthropic that will, no doubt, lend itself to the Amazon-Alexa ecosystem. This is still my best guess, but their approach could include:

• The integration of Anthropic’s advanced language models could potentially improve Alexa’s natural language understanding and generation abilities.
• Amazon’s various voice-enabled services, from shopping to customer support, could benefit from the advanced AI capabilities provided by Anthropic’s models.
• New voice AI features: The collaboration might lead to the development of novel voice AI features that leverage Anthropic’s expertise in safe and steerable AI

Each of these companies brings unique strengths and approaches to the voice AI landscape, from Google’s data-driven insights to Apple’s privacy-focused on-device processing, and from OpenAI’s cutting-edge language models to Anthropic’s emphasis on ethical AI.

Try experimenting with different voice AI assistants to understand their strengths and weaknesses. This will help you choose the best one for your needs as they evolve.

Other Notable Mentions

• Samsung Bixby: Samsung’s native voice assistant offering device control, task automation and natural language understanding.
• Yandex Alice: Russian-language voice assistant offering integration with Yandex services and smart home devices.
• IBM Watson Assistant: Enterprise-focused AI assistant for customer service and business applications customizable for specific industry needs.
• Mycroft: Open-source voice assistant that can be customized and installed on various devices, including Raspberry Pi.
• SoundHound Houndify: Voice AI platform that allows developers to add voice interaction to their products.
• Huawei Celia: Integrated into Huawei devices as an alternative to Google Assistant.

The Multimodal Future: Beyond Voice

While voice is leading the charge, the future of AI interaction is, of course, likely to be multimodal. If you start projecting out the next 5 — 10 years, we can easily imagine a future where AI can:

• See: Interpret visual information and gestures.
• Hear: Process and understand speech and environmental sounds.
• Feel: Respond to touch inputs or even simulate tactile feedback.
• Understand context: Combine all these inputs to grasp the full context of a situation.

Amy Stapleton, Senior Analyst at Opus Research, envisions a future where

The technologies of machine learning, speech recognition, and natural language understanding are reaching a nexus of capability. The end result is that we’ll soon have artificially intelligent assistants to help us in every aspect of our lives.

This multimodal approach will create more intuitive, responsive, and helpful AI assistants across all areas of life.

Ethical Considerations in Voice AI

Before we get too starry-eyed, let’s talk ethics. This voice-powered future comes with some serious questions:

1. Privacy: Is convenience worth sacrificing personal space?
2. Data Security: How do we protect sensitive voice data?
3. Bias and Fairness: Will AI understand diverse accents and languages equally?
4. Transparency: Should AI always disclose its non-human nature?
5. Emotional Manipulation: As AI gets better at reading emotions, how do we prevent misuse?
6. Dependency: Are we outsourcing too much of our thinking?

Sarah Jeong, deputy editor for The Verge, offers a prudent reminder:

Artificial intelligence is just a new tool, one that can be used for good and for bad purposes and one that comes with new dangers and downsides as well. We know already that although machine learning has huge potential, data sets with ingrained biases will produce biased results — garbage in, garbage out.

The Conversational Singularity: A New Human-AI Paradigm

We’re heading towards what I call the “Conversational Singularity” — a point where AI becomes so adept at natural interaction that it fundamentally changes how we relate to technology and each other.

This isn’t just theoretical. We’re already seeing the beginnings of this with the rise of AI personas and “AI girlfriends/boyfriends.” Apps like Replika and Xiaoice are creating emotional bonds between humans and AI, blurring the lines between artificial and genuine connection.

The implications can vary dramatically:

1. Redefining Relationships: Will AI complement or replace human connections?

2. Cognitive Enhancement: Could conversing with AI make us smarter? You are who you spend your time with after all.

3. Cultural Shift: How will ubiquitous AI assistants change societal norms?

4. Philosophical Questions: As AI becomes indistinguishable from human conversation partners, how will it challenge our concepts of consciousness, intelligence, and even what it means to be human?

While the full realization of the Conversational Singularity may still be years away, its early stages are already here. The challenge now is to shape this future thoughtfully and ethically.

Finding Our Voice in the AI Chorus

As we stand on this precipice, one thing is crystal clear: the future of human-AI interaction will be profoundly conversational. We’re moving beyond prompt engineering into a world where our relationship with AI is defined by natural, voice-driven interaction.

This shift, as Microsoft CEO Satya Nadella astutely observes, is part of a larger digital transformation:

Digital technology, pervasively, is getting embedded in every place: every thing, every person, every walk of life is being fundamentally shaped by digital technology — it is happening in our homes, our work, our places of entertainment. It’s amazing to think of a world as a computer. I think that’s the right metaphor for us as we go forward.

Indeed, voice AI represents the next frontier in this digital evolution. Whether we end up with helpful but limited digital assistants or powerhouse AI agents capable of deep, meaningful dialogue and complex tasks remains to be seen. What’s certain is that this future is filled with immense potential, significant pitfalls, and more than a few surprises.

Are you ready to lend your voice to the future of AI? This isn’t just about adopting new technology; it’s about shaping the very nature of our interaction with artificial intelligence. The conversation is just beginning, and it promises to be one of the most crucial dialogues of our time.

Till next time.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI