The Dialogue Architects

Conversation Design Isn't Dead—It's More Important Than Ever with Rebecca Evanhoe

Episode Summary

How do you build enterprise voice AI that customers actually trust? Rebecca Evanhoe of Slang AI joins Lauren Goerz to discuss conversation design, LLM evaluation, hybrid AI systems, and what it takes to automate millions of restaurant phone calls without sacrificing reliability.

Episode Notes

In this episode of The Dialogue Architects, host Lauren Goerz sits down with Rebecca Evanhoe, Product Manager at Slang AI, to explore what it takes to build enterprise voice AI that works in the real world.

As organizations race to adopt generative AI, many are questioning whether traditional conversation design practices still matter. Rebecca argues the opposite. While new terms like prompt engineering and context engineering have emerged, the core principles of conversation design remain essential: understanding user goals, mapping conversation pathways, designing effective question flows, and building systems that reliably help people accomplish tasks.

Rebecca shares how Slang AI approaches voice automation for restaurants, handling millions of customer interactions each month. She explains why the company combines deterministic systems with carefully selected language models, using a hybrid architecture that balances reliability, speed, cost, and user experience. The discussion explores how teams can determine when generative AI is appropriate, when traditional workflows are still the better choice, and how rigorous evaluation helps prevent hallucinations and maintain trust.

Lauren and Rebecca also dive into modern AI evaluation strategies, including transcript reviews, funnel analysis, automated labeling, human scoring, and experimentation frameworks that help teams understand what users are actually experiencing. They discuss the unique challenges of voice AI, including speech recognition errors, latency, multilingual experiences, and the future of fully generative speech-to-speech interactions.

Whether you're a conversation designer, product manager, AI engineer, or enterprise leader, this episode offers practical lessons for building conversational AI systems that users can trust—and businesses can confidently deploy at scale.

Here's What You'll Hear

Why conversation design fundamentals still matter in the age of LLMs
How Slang AI handles millions of restaurant phone calls each month
The difference between deterministic and generative AI systems
When LLMs outperform traditional conversational workflows—and when they don't
Why evaluation is becoming the most important discipline in AI development
How transcript reviews uncover issues metrics often miss
The role of Braintrust, human evaluators, and automated scoring systems
Voice-specific challenges like ASR accuracy, latency, and interruption handling
The future of multilingual and speech-to-speech AI experiences
How conversational AI creates valuable operational insights for businesses

Timestamps

(00:00) Introduction to The Dialogue Architects

(00:39) Meet Rebecca Evanhoe of Slang AI

(01:30) A Day in Product Management & Voice AI

(02:25) Why Conversation Design Still Matters

(03:29) Why LLM Evaluation Is Critical

(05:38) Is Conversation Design Dead?

(06:19) Prompt Engineering vs Conversation Design

(07:07) What Conversation Designers Actually Do

(10:01) User Advocacy, Research & Design Workshops

(12:22) Hybrid AI vs Fully Generative Systems

(16:31) When Deterministic Systems Beat LLMs

(19:31) AI Costs, Latency & Model Selection

(21:07) How Slang AI Evaluates LLM Performance

(25:41) Transcript Reviews That Improve AI

(26:34) Finding Root Causes Through Metrics

(27:54) CSAT vs Sentiment: What Matters More?

(31:07) Choosing the Right North Star Metrics

(31:59) The Biggest Challenges in Voice AI Design

(36:23) Multilingual AI & The Future Product Roadmap

(40:31) Turning Customer Calls Into Business Insights

(43:12) How AI Is Changing Human Communication

(46:46) Rebecca Evanhoe's Advice for Conversation Designers

(47:51) Closing Thoughts & Where to Connect

About the Guest

Rebecca Evanhoe is a Product Manager at Slang AI and a longtime conversation design practitioner. After building her career designing conversational experiences, she transitioned into product management to help scale AI-powered voice systems for enterprise customers. At Slang AI, Rebecca helps shape the strategy behind voice assistants that handle millions of restaurant calls every month, combining conversation design expertise with rigorous evaluation methodologies to build reliable, trustworthy AI experiences.