Welcome to the Deep Dive. We're here to pull out the key bits of knowledge so you can get up to speed fast. Today we're looking at something, well, really transformative, I think. You know that AI agent you used, the text -based one? Super smart, super helpful. Well, what if he can actually talk back to you? Yeah, not like, you know, an old GPS voice. Exactly. Not clunky or robotic, but like, with a natural, engaging voice. A real personality. Something that sounds,
well, human. And the thing is, It's something you can genuinely build today, like right now, and without writing code. That's the amazing part, the accessibility. Right. So our mission for this deep dive is... So, OK, let's kick things off. Method one. We're calling it the AI voicemail system expert. This sounds well, it sounds clever, like a smart way to handle tasks that aren't instant. Right. This asynchronous idea. Exactly. Yeah. Think of it like a super intelligent. Voice
mailbox. You, the user, leave a voice message, maybe a complex question or instruction. And the AI doesn't rush. It takes its time. It listens. It thinks, processes everything, and then responds with its own audio message back to you. Oh, OK. So this asynchronous thing, it's perfect for jobs that need, you know, deeper thought, summarizing a really long document maybe or doing some complex research. Right. Not that instant back and forth,
but more considered. Precisely. It's about getting a thoughtful, rich response when you don't need it that very second. That makes a lot of sense. Like having a research assistant, you can brief and they come back later with the findings. So, OK, what's the actual journey from me hitting record to the AI talking back? Walk us through this voice message pipeline. Yeah, it's quite an elegant flow, actually, like a little assembly line. It starts when you send a voice message.
Let's say you send it to a telegram bot. Then N8N, acting as the orchestrator, grabs that audio file. downloads it, that raw audio, it goes straight to 11 Labs, their models transcribe it, turn it into text, and they do it brilliantly. That text then gets fed to the AI brain. Could be ChatGPT, could be Claude, whatever you're using. The AI does its thing, generates the response in text, and that text goes back to 11 Labs, this time for text -to -speech, turning it into
that natural -sounding audio. And then finally, N8N takes that new audio file and sends it back to you in Telegram. It's a full circle. Wow. Okay. Okay, that sounds incredibly capable. And the bit that really grabs me is the no -code part you mentioned. Like, for someone wanting to build this, what are the actual pieces, the Lego bricks in N8N? How do they, you know, snap together? Right, yeah, Lego bricks is a great way to put it. They really do feel like they're
made for each other. You're listening to Ghost. In any in -the -ghosted area, like the Telegram turn -out, you set it up to catch incoming audio files, specifically the voice message format. And crucially, it grabs the chatted... The return address. Exactly. It's the unique return address, so the AI knows where to send its response back. Next trick. The universal translator. That's an 11 labs node doing the speech -to -text... It takes the audio file and just grabs and transcribes
it. The beauty is NAN handles passing that audio data automatically. It's pretty seamless. Okay, no complex mapping needed there. Nope. And it totally changes the whole feel of the interaction. So the system prompt is really the soul of the agent. OK, then what? How does it get its voice back? Right. So after the AI thinks and writes its text response, you use another 11 Labs node. This one's for text to speech. You connect the AI's text output to this node. And this is where
you like. Cast your agent. You choose a voice from Eleven Labs library. They have tons. Or you can use a specific voice ID if you've cloned one or have a favorite. Got it. Choosing the voice actor. Pretty much. And the final step, delivering the message. That's another telegram node, a sender this time. You make sure it sends the audio file generated by Eleven Labs back to the original chat aid so the message goes right back to the person who sent the request.
And then the test flight. The moment of truth, as you called it. I bet it's satisfying seeing it all light up. Oh, it really is. Watching each node activate on the inning and canvas as your message flows through. Very cool. But, you know, things don't always work first time. What about when, Houston, we have a problem? Any quick debugging tips? Yeah, absolutely. The bugging and editing is usually pretty straightforward. The visual flow helps a lot. If a node doesn't light up
green, you know exactly where it's stopped. First thing, check the trigger. Does the Telegram node even receive the message? If not, maybe check your WhatsApp and Telegram as well. Then follow the noodles, those lines connecting the nodes. Is everything connected? Did you link an output to the right input? Easy mistake to make. But honestly, the most common failure point nine times out of ten is probably credentials. Ah,
yes, the classic. Triple check them. Your 11 labs key, your AI service key, one typo and the whole thing falls over. And maybe most importantly, read the error message. That's really solid advice. OK, so this AI voicemail bot is fantastic for those deeper asynchronous things. But what if you want that instant back and forth, the conversational feel? This is where it gets really interesting, I think. We're talking about building a fluid, real -time conversational AI, like talking to
a human assistant on the phone. This sounds like the main event, the showstopper. Yeah, this is where it feels truly interactive. So how do we achieve that, the proper AI assistant experience? Okay, so for this real -time system. you need a totally different setup, a different kind of partnership. I like to think of it using the NASA mission control analogy. Ooh, okay. Tell me more. Right. In this model, 11 Labs is mission
control. It's the sophisticated front end. It handles the direct chat with the user, the voice, the personality, managing the flow of the conversation. It's talking to the astronaut, basically. Got it. The voice on the comms. Exactly. Now, your NA workflow, that's the specialist team back in Houston. The engineers, the scientists, they don't talk directly to the user. Instead, they are a powerful tool that Mission Control 11 Labs calls upon when the user asks something complex.
thing needing research or a specific action performed. Ah, I see. So Eleven Labs handles the chat and ANN does the heavy lifting in the background when needed. Precisely. That's the key takeaway. Eleven Labs manages the conversation and ANN handles the actions or the deep information retrieval. That analogy makes it crystal clear. So if 11 Labs is mission control, how do we actually build it? How do we set up our agent in 11 Labs for these live calls? You actually do it right inside
your 11 Labs account. They have a section called Conversational AI. You go there, create a new agent, and this is where you play casting director again. Give it a name, something fitting. Choose a voice from their library that matches the personality you want. calm and professional, energetic and friendly. And then you craft that initial greeting, something simple to start, like, hello, how can I help you today? Okay, straightforward enough. But how do we connect mission control to the
specialist team, to NAN? That feels like the critical link. It absolutely is. So back in your 11 Labs agent settings, you scroll down to tools. Here, you add a custom webhook tool. This is literally the direct phone line to your NAN workflow. Okay. You need to give the AI a clear briefing about this tool. So you give it a name, like NAN Web Researcher, and a description, something really clear like, call this tool to search the web and find information about any topic. It
is an expert research assistant. So the AI knows what it's for. Exactly. You set the method to post and then you paste in the URL from an NAN webhook node that connects them. Got it. So the agent knows what the specialist team does, but it also needs to know when to call them, right? Yeah. That brings us to the prime directive, the system prompt for the 11Labs agent itself. How do we teach it to... Essentially, put the user on hold and call the NAN tool at the right
moment. Yeah, this prompt is, it's like the agent's constitution. It dictates everything. 11 Labs has a nice generate with AI feature to get you started with a base prompt. But you must manually add a really crucial instruction. You have to explicitly tell the agent. When the user asks a question that requires up -to -date information or web research, use the N8 Web Researcher tool. Tell the user you are searching and then wait for the tool's response before continuing. Ah,
okay, so it's a non -negotiable rule. Absolutely. Without that specific instruction, the agent might just try to answer from its general knowledge, which could be outdated, or it might just get confused about how to use the tool. This tells it exactly how to handle research requests. That's a really important detail. Okay, and what about building the specialist team itself, the NANN workflow that receives the call from 11 labs? What are the key parts there? So the NANN research
backend is pretty focused. It usually has three main parts. First, the researcher. The webhook node receives the query from 11 labs. You pass that query straight to, say, a perplexity node. You'd probably use one of their sonar models. They're designed for real -time web search, pulling back sourced info. Okay, so it gets the raw data. Right. But that raw data can be a lot. Maybe too much for a quick conversational answer. So
next comes the editor. You pass Perplexity's output to another AI agent node, its only job, to be a ruthless editor. You give it a sign prompt like, summarize the following information concisely, no more than three sentences. Nice. Keep it brief. Exactly. And finally, the report back. This is just a respond to webhook node in ANN. It takes that short, concise summary from the editor AI and sends it straight back to the 11 labs agent waiting on the line. Loop closed. This sounds
seriously powerful when put together. Let's make it real. Can you walk us through a quick test flight, a hypothetical conversation so we can hear how it flows? Yeah, sure. Imagine you call your agent. You could be using your browser, your phone. You start. Hello, I'd like to do some research on the company NVIDIA. The agent, 11 Labs, responds smoothly using the voice you chose. I can certainly help with that. Is there anything specific you'd like to know about NVIDIA?
Okay, nice and natural. Then you say. Yeah, let's look at their Q4 2025 forecast. Now, the agent recognizes this needs the specialist tool because of that prompt rule. So it says, understood. I'll search for NVIDIA's Q4 2025 forecast now. Please give me just a moment. Ah, putting you on hold politely. Exactly. And behind the scenes, bam, the N8N workflow fires up. Perplexity searches, the AI summarizes. The specialist team is working. Right. Then maybe five, 10 seconds later, the
11 labs agent comes back. Okay. I have found some information on NVIDIA's quarter four 2025 forecast. NVIDIA reported revenue of whatever the summary is. Wow. It's a seamless conversation, even with that complex lookup happening in the background. That's genuinely impressive. It completely changes the game for AI interaction. So thinking bigger. What does this mean for expanding? You mentioned a pro -level upgrade, the multi -specialist idea. Yeah, this is where it gets really powerful.
You're not limited to just one NANA specialist tool. You can create multiple NAN workflows, each starting with a webhook, each designed for a different task. Like what? Well, you could have your web researcher, but also maybe an internal database checker that looks at customer info. or a calendar scheduler, or even one that sends emails. Okay. You add each of these as separate webhook tools in the 11 Labs agent settings,
each with a clear name and description. Then your 11 Labs agent, Mission Control, becomes much smarter. Based on your conversation, it will figure out which specialist tool is the right one to call for that specific task. So it routes the request intelligently. Exactly. It transforms your agent from just a researcher into a truly versatile assistant. That's incredible, giving the AI a whole team. Okay, so for people wanting to actually deploy this, move beyond
just... Also, consider rate limiting on your webhook to prevent accidental or intentional overload. And keep a close eye on your API usage 11 labs, your AI model provider perplexity. Those costs can add up if you're not monitoring them. And technically, remember to switch your NAN workflow from the test URL to the production URL. Just remove the test part and make sure the workflow toggle is set to active. Little details, but crucial. Okay, beyond the text setup.
There's the experience itself, crafting the perfect conversational experience. This sounds like an art. Any tips on voice selection, keeping the flow smooth? It absolutely is an art. For voice selection, really think about the agent's role. Is it a formal research assistant? Maybe a clear, neutral voice? A creative brainstorming partner? Perhaps something more expressive? And definitely test voices with real content, not just, hello, a voice that sounds great for one sentence might
get grading over a longer explanation. For conversation flow, keep the agent's responses conversational but also concise. Aim for maybe two or three sentences for simple answers. Don't let it ramble. And crucially, plan for errors. What happens if the NAN workflow times out or fails? The agent shouldn't just hang up. It needs a graceful exit, like, I'm sorry, I seem to be having trouble connecting to my research tool right now. Could
you try again in a moment? Handling failures gracefully, yeah, that's key for a professional field. And this brings us back to the prompt, really. How do you ensure it behaves consistently? That's advanced prompt engineering for voice. The system prompt for your 11 Labs agent is its constitution. It needs explicit rules. Define the persona clearly. Set rules for conversation management, like always ask clarifying questions if a request is vague, keep information concise,
use natural language. And most importantly for this setup, explicit tool usage guidelines. Define exactly when to use each N8N tool, how to introduce it, and what to tell the user if a tool fails or takes too long. It sounds like the prompt is doing a lot of heavy lifting in managing the whole interaction. It really is. It's the brain governing the conversation flow and tool orchestration. So this blueprint, it really unlocks some serious superpowers. These advanced voice agent patterns.
We're talking multi -tool agents doing research, scheduling, emailing. Checking databases, analyzing sales data. Yeah. And context -aware conversations. Yeah. Connecting to memory systems like ZEP or Supabase so it remembers past chats. Exactly. So you can pick up where you left off or it can build knowledge over time. Makes it feel much more intelligent. And then my favorite, Star Trek mode. Voice -activated workflows. Using the voice agent as, like, a master controller
for other inane automations. Precisely. Imagine just saying, computer, run the morning sales report automation. Oh, man. For anyone who's manually pulled reports every morning, that sounds like pure magic. Have you actually seen teams implement that kind of thing? Does this save a lot of time? Oh, absolutely. E -efficiency gains can be huge, especially for repetitive tasks that can be triggered by a simple voice command. It frees people up for more complex
work. And the agent talks too much, doesn't know when to stop. That's almost always a prompt engineering issue. You need to go back to that system prompt and add stricter rules. Also, sales and lead qualification. An AI voice agent can handle initial outreach, answer basic questions, qualify leads, and then pass the really warm ones over to a
human salesperson. Very efficient. And internally, think voice -controlled tools for teams, allowing people to query databases, run reports, or trigger workflows completely hands -free while they're doing other things. The possibilities really do seem vast. Yeah. Okay, let's try and bring this all together. The bottom line seems pretty clear. The future is conversational. And think
about it. A few years back, the idea that a solo creator, maybe even just you listening now, could build an AI agent that holds a natural real -time voice chat, hooks into the Internet, performs tasks, that was pure sci -fi. Absolutely. Star Trek stuff. Right. And today. It's just another project you can build in AAN. You literally have the blueprint now to create AI systems that don't just crunch data, but actually engage, interact, connect on a much more human level. It democratizes
some seriously advanced capabilities. Totally. So here's your mission briefing, your takeaway challenge, how you use this blueprint. Are you going to cast your character? Maybe a witty, sarcastic J -A -R -V -I -S like Iron Man's AI? Or perhaps a calm, professional, endlessly patient Star Trek computer voice. The personality is half the fun. It really is. And think, what specific nagging, recurring... problem in your life, your work, your business could be totally transformed
by adding a voice. By batting one of these agents, imagine connecting it to your smart home, your office, literally saying, hey Jarvis, or whatever you call it, turn on the lights in the studio and start the coffee machine. The tech is there. It's ready. It really is. It's just waiting for your creativity. Time to get building. We really hope this deep dive has sparked some ideas, maybe given you some surprising insights. Keep exploring, keep building, and we'll see you on the next deep dive.