this conversation I speak with Tom Chapland of Canonical AI now this conversation was initially held inside of the Synthflow Academy and was a fireside chat so towards the end it did involve a bit of a screenshot so if you get lost towards the end I wanna see the outcome of that screen showing what chemical is all about from a UI perspective please head over to our YouTube channel and watch it there but hope you enjoyed this conversation with Tom
okay I guess we can we just get a rock and rolling if anyone's filtering after the fact we can so thanks everyone for joining I'm joined today by Tom Chaplin who's building canonical AI uh which is a mixed part of the voice AI agents you missed that intro before so I guess kick things off how you doing Tom and you wanna give us the quick one line intro who you are and what is canonical canonical they say yeah you did great you said it right uh yeah I'm I'm Tom Chaplin
I'm the CEO and co founder of canonical and and we give you visibility into what your voice AI agent is doing we also give your customers who you're building voice AI agents for visibility into what your voice AI agent is doing and what we're finding in building this product is that um there's some hesitation around the voice AI your voice AI customers to expand their usage of the bogus AIs that you build them um but once you give them more visibility into what's happening
and they can see that the agent is performing as expected that's what really unlocks growth and our mission is to help voice AI developers unlock the growth in voice AI um by by helping their customers feel more comfortable with putting their brand in their sales funnel on the line with the voice AI agent you're building them yeah I like that um it does seem a lot of the time it's a bit of a black box and people can say the voice I do certain things but can you back it up with the actual data
so this this course successful did it go through did it completely gone etcetera um you've been doing this for like a year it's kinda a year and a half August 2,023 if this is correct so you've been in the space for a little bit is this is this always been the same product no actually we started with something else so I'm gonna go a little bit further back in describing my background um I started an agriculture technology company back in 2014 and that company got into y Combinator um
it was a great company we we were building a sensor that help farmers figure out how much to irigate um is based on my PhD work and is really neat data feed is is a good company um and in the process of building that company I became more acquainted with nondeterministic neural network type models um it was uh when like the Tensor Flow API came out we're building cool computer vision stuff on top of our data feed um and really liked it and eventually I found a good home for that company
that company was called Tule and was acquired by Crop X they're great people um I want to start up journey again and of course um my co founder and I who was also the CTO of my last company um we want to work in LLMS and initially we're building a cache in there the idea of a cache in there is it's like a smart database every time a query came in um from a user asking an LLM something we'd look in that smart database to see if essentially the same question was answered asked before if it was
we could return the answer from the database rather than calling the slow lumbering LLM um and that LED us to reach out to a bunch of voice AI developers to see if they'd be interested this was about you know that year and a half Mark ago um and what we found was like yeah latency was a problem but well actually I kind of skip to step there the the thing I really wanna emphasize here is like we just fell in love with the voice face we started meeting these voice
AI developers and seeing all the cool stuff they're building and just seeing the incredible potential for voice AI that everyone here in this in the session with us believes and feels and as examples of seeing it um and we just fell in love with the voice AI space and we realized the cash wasn't the right thing to build it was too slow our growth was too slow on it I should say um but what our friends and voice guy kept telling us was like hey
like I've got these contracts with these big potential clients but they're only dipping their toe in the water and I need a way to make them more comfortable with my voice AI and I don't really blame them for not being comfortable with it because I'm like manually listening to a few calls I don't have any sort of visibility into what's happening can you build something that makes it easy to show people like here's what's happening in your data and everything's going as expected um
so I started building that about um I mean we shipped the dashboard in early October um and it's been it's been really fun it's been great ever since um I just love being a part of the Voice Act community and really glad to be here and and hopefully get to meet some of you that have joined and and get to make more friends in the space yeah that's cool was there one demo when uh you were kinda exploring options where you thought like
holy crap this is the space that I need to like apply my timer to ah yeah like what was the moment where I was like oh voice AI is so cool I think for me a lot of it is um like when I first logged into Sinflow and it must be similar for so many people like you first log in and build your first agent as like it's just a toy and it calls you and you're like oh my God I can talk to it and it just I feel like I still build like voice AI agents just for fun on the side
like little little toy demos just like for fun I built one that was like a baseball game for my son um I feel like I'm an artist with a palette in this really interesting palette that I get to work with with voice AI and I just think it's such a neat space to be in and also I see a lot of calls that um people make um voice agents make and they just it's amazing to see how well they can work you know like when they just nail it the customer skeptical and they're like
let me talk to a representative and then the voice guy just solves their problem at the end they're thanking the machine it's just like magical it's clearly the future yeah some of the times when you listen back to a call um so if you're testing an agent and I've done it a lot of times you're tweaking a prompt and suddenly you hear like a recording of a conversation and you're like damn that was actually really good so you're tweaking things like sentence lengths and like do say this
then say that and when when you get that conversational flow right yes yes my music to his yeah what was that for you Tom dude like what was the moment where you like oh voice the eyes the space I need to be in um so I've always been have like a had a community going for a few years called Shiny Object Social Club where it's just a bunch of us hanging out in a discord and playing around with any new tech that's interesting and then uh I think 11 lads came along when it was like synthetic voices
and you could actually just get into say stuff and then there's a bit of avatars coming around and then that got applied to phone calls with things like sin flow and battery at the time and uh I wasn't extreme an extremely technical person so sin flow is obviously the that the natural fit being a more no code solution and the the feeling it just gave that I could now have this power to create these really powerful uh workflows and tools and functionality was just yeah
it was just wild and then things like um connecting it with other alternations which we're gonna have a look at in a minute where you can not only extract information from the phone call but also have it trigger other workflows and suddenly and make internal calls and outbound calls and hooking together it's just just felt like everything is now on the table yeah yeah absolutely yeah it's cool so yeah and and it is is been kinda missing for a bit what we seen with chemical
and do you wanna give us the rundown we're gonna do a screenshot too so you can do that whenever you want um so basically this base if I'm right this gives you a really easy way to see how your agents are performing that's correct right yes that's the idea gives you and your customers an easy way to see what your voice AI assistant is doing and for you it helps you improve your agent and for your your client it helps you feel comfortable with what you're doing
so that they'll start pushing more call volume through it hmm got it and have you seen have you seen that the take of it is this meant for um like small regencies s and B's escaping to enterprise where where does where does it fit in enterprises are always the slowest right I think that um they they clearly aren't going to um use voice AI until they have a lot of comfort with the fact that the voice AI assistant is doing what you expect
so they're definitely gonna need people selling to enterprises building voicing for enterprises and enterprises are gonna need something like what we're building but it's not really my my initial focus just because it's so slow to get going with enterprises um our customer base really is two different categories um it's vertical SAS specialized vertical SAS voice AI companies are focused on something like building a voice AI assistant for um HVAC businesses um or they're the AI agencies um
and that's part of why I'm excited to be on this on this um in the session with you is like I think Sinflo does a great job with AI agencies and fitting the needs of the AI agencies um like with the white label product and that sort of stuff and the AI agencies you know the other type of our customer and we find with both of the those two types of customers the vertical SAS and our AI agencies they kept coming to us saying hey can this is cool
you built for us and it's helping us to hone our agent and refine it but can you also build this can you also expose it to our customers um can you give them a login or can you we embed what you built into our website so that um they can see how the what what's going on with the data um so that's kind of the direction we're going with things now um and making progress on that as well I'm sorry that's kind of maybe a rambling answer does that that's good yeah that's really good
yeah I've got a bunch with a bunch of Ad Sheerans and head so you getting the full white label route or is it more like semi semi embettable sort of scenario yeah so let me describe the current state of the world and then also where we're going um the current state of the world is you as the voice AI developer can uh send us calls um you send us recordings of the calls and a transcript I mean go through later how to do the integration with make and also the integration just with code
I'll points that as well um then you can log in and see all the calls for the different agents that you send us in addition you can set up you can work through with us to set up a login for your client and your client can only see their agents so we set it up in such a way that your client only sees you know the agents are supposed to see um that's the current state of the world then the next thing we're building is embedable components so if you have a dashboard
like you're using the white label solution for Vincent flow um Vincent flow you can take like the call flow diagram that I share with everyone shortly um and you can bed that in the the dashboard so you can show your client that um as well and then finally um this is in the works but I'm talking to the the cofounders of Sinflow about um doing a more native integration that just makes this whole thing easier for for you guys and that there's not really the step where you have to mess around
they'll just be one click and you can you can get this this stuff right there and sing flow got it so I guess you able to trigger different kind of queries depending on the outcome of the call like in real time or is it more of like a uploading batch and then analyze after say like a campaign it's a little bit both so let me let me describe like the the main wait so we can we can share the screen at any time so whenever whenever you wanna add context
so you took your own we can do that I'll first describe it um and then I'll show it and I'll summarize it again so the the what I wanna describe is the first thing we show you is um visualizations of what your calls are doing so most of your calls will likely go down this happy path that you designed um but some calls will go off that happy path and take a turn and those are the ones that you wanna look at and they'll help you as the voice AI developer to improve the the agent um
so and that requires that you log in and look at the data and this is also what um the voice AI developers customers also like they like being able to look in and kind of see the different paths that the calls are going down and get that kind of warm trusting feeling that they can believe in this this product and expand with it um the other thing we do is we have uh push notifications to slack around insights so like right after a call ends if it was supposed to be successful
if it was most likely to be successful given the fact that of how long it was and what path what call stages the call went to but it wasn't successful we push those to you um and make it easy for you to look at those and see if you need to do any damage control or um you know you wanna make any changes to your prompts or your your integrations um and then the last part is audio metric so we we have metrics like um the latency the number of interruptions and that sort of stuff
and that's really more for um helping you surface different types of issues and dig dig deeper into how the agents performing and also like give reassurance to your clients like look most of the time my latency is you know this many milliseconds and that's within the the human what's normal for human conversation and things are working as expected got it I guess that kinda gives you the opportunity to be a bit proactive as well so if you're already seeing
like a higher threshold of either foul calls or ones going off the happy path or highlight and see then you could kind of proactively get in touch with your clients just just to let you know this is happening and we're on it and we're looking at fix yeah so my last company we helped farmers make irrigation decisions and I think initially my first instincts especially as a former academic that built this technology and so close to me was like if something went wrong I would like
I didn't want to go get the farmer's attention and bother them and be like hey this went wrong um it just didn't feel right to me at the time but then later I realized that the key to our success was being very transparent with our customers so we were always talking to our customers when we saw something good and we wanted them to like notice how well this thing went and also when something went bad we reached out to them because it's better they hear it from us
and that they know that we're working on it that's what prevents churn um than if ended on their own it's all about you know maintaining and building that trust with your clients and um what what is the kind of state of monitoring and analytics tools especially when it comes to third party and so we do stuff in 10 minutes since low and but I haven't checked in too recently on all the available tools uh they what what's the existing looking at right now I think well I think it
what the ecosystem looks like is indicative of the state of voice AI right now voice AI is this emerging field and there's some people getting a lot of traction in it but not um most people are still kind of in the early stages of it so if you look at what other tools are out there for helping you like third party tools for helping you develop your your and understand how your assistant is doing there mostly on the testing side so the side before you deploy your agent into production
um it's for like running simulations of lots of different scenarios and making sure that um your your agent is trying to automate that testing that all of you are familiar with doing um the the problem with that approach is like once you test it it's great you can feel very confident in it but there's no way to really simulate human behavior like humans are the most unpredictable force in nature and once your agent out into the production and if you have agents in production
you know exactly what I'm talking about like people do weird stuff um and you're you're wouldn't expect what would happen there and you're going to really need to see what your agent is doing with real humans to figure out how to improve it and um yeah how to improve it so the in the pre production side there's companies out there like Vocera and Coval um in the post production there's us which is more of like this um low code way of doing these things um that
it's really amenable to working with AI agencies and your n customers um or you can use more higher code solutions like you can set up a whole like data dog pipeline and build your own visualizations uh and and data flows through with data dog but and and the the problem with that is it's not really centered around voice No. 1 it takes a lot of technical skills and second there's a lot of problems with voice that are different from just looking at like traces from you know text based lolms um
so I really think this area is kind of Mason and emerging and we're seeing things emerge like what we're building got it what's been like the most common thing that people have found after implanting your tool that they weren't aware of before oh interesting um well I think I once heard I was listening to a podcast recently of um by C Partners and they're talking about what builds a great what makes a great sass company uh that really can capture a lot of the market like
like a gusto is that isn't that they do one thing really well it's that they do a lot of little things um and they do all of those myriad little things well um and when I look at voice AI assistance I can always kind of tell when I'm looking at the data our customer sent us is like how how much care and time the developer has put into developing how much time it's been in production um because there's always little things along the way that it's just lots and lots of smooth
at rough edges that need to be smooth so um finally I'll give examples it's like a notorious one is um I should actually more generalize first there's all these workarounds you have to build given the current state of the voice AI technology with text to speech and speak to speech text and lolms um even with the audio only models and if you you're not careful about how to do those workarounds like how to get email addresses or um how to be careful about like repeating people's names um
then you're gonna run into problems um and it's really that sort of constant careful hearing and craftsmanship that makes a great voice AI agent I gave two examples there by mostly smoke high level let me know if you want me to dig in more no that's good no I like it um I think still very few people I think really digging into digging into analytics cause there's not too many doing a lot at scale but presume that's gonna ramp up now that they're getting more accurate
and they're getting easier to create and deploy so so I expect that to increase soon yeah so I'm looking at the chat and I'm seeing that people are eager to see screen shares and they're eager to figure out how to connect especially Nick I think Nick's very very keen to see a demo of screenshots let's dive in and we can keep on sharing my screen um let's see give me just a moment and any questions you go file them into file them into Q&A we can we can bring them up on screen
or you can even come up and ask yourself whichever whichever you fancy okay cool you can sit if you wanna follow along I'm gonna drop this in the um in the chat um so here's the sequence of what I mean go through I'm first gonna fairly quickly walk you through our platform and what we're doing and then from there I wanna spend more time focusing on how you can integrate with your sin flow agent with our platform we have a free tier so it's it's easy to get started I just put into the chat
also our blog post that really walks you through um how to integrate with make or with code um and that's what we're gonna cover now so so without further ado let me go back to our dashboard I'm on our demo account now um and I when you first log in you see the summary page the summary page will show you how many calls you've uploaded and the total duration of the calls you've uploaded and some high level stats um where things get interesting is when you scroll down
and you click on one of the agents that you've uploaded and then we can dig in and look at that that agent um so I'm gonna scroll down here and first gonna talk about how we process the data you send us for each agent you initially send us excuse me I should say it differently when you connect a new agent to us we take the first 15 calls that you send us and we use those first 15 calls to figure out the stages of the call what I mean by stages is
an agent will typically go through stages like reading appointment scheduling um you know answering a question about a product uh and then closing the call we figure out what those stages are then once we have those stages every time a new call comes in then we can take each turn in the conversation and assign it to the stage so when the user says hi we can assign that to greeting when the user says I wanna change my appointment we can change um we can assign that to appointment scheduling
what this enables us to do is to determine a path that each call goes down um that all path is helpful in and of itself but it gets really helpful when you can take that call path and look at the call paths of all your calls for a given day or week and really get the bigger picture of what your assistant is doing um we do that automatically you don't have to submit a core structure or anything you can just get that from the course well I'm really glad you asked that question
that was about to say is that we do it automatically because some people have so many agents they don't wanna go in there and edit each one but also if I were logged into my personal account right now um you could see a little edit button here and you can edit the stages to better match what you think they should be got it okay make sense we do the same thing with the call outcomes so we use the first 15 calls and some LOL magic to figure out what the typical outcomes are of the call
appointment not scheduled appointment scheduled or product um inquiry handled then we use those outcomes lolms are just like people in that they're more likely be accurate if you break things down into small steps for them um they're more likely to be able to solve a problem when the problem is broken into small steps um so we take those outcomes and then in another step we ask the LLM was this a successful call given the outcome or not a successful call um again
you can edit the outcome names in our dashboard you can't see that here but if you log in um you can see that uh can you still see my screen by the way uh I just opened your cash okay click on the sign up button here um you can sign up for free and you get uh I forget how many how many free minutes do you get you get 4 hours of calls per month for free um so you can click that sign up button um and once you sign up you can click the upload calls button we have a gooey for getting started
so you don't even have to integrate using make or um uh code initially you can just download calls from Sinflow and drop them in here and that'll bring you back to what I was showing you earlier where you have the uh will bring you back to our dashboard so this is it's really easy to get started and and try it out um you just go to uh I'll drop this in the the chat you just go here and click sign up um and you can try it out um and upload calls
okay so now that we've uploaded calls I wanted to um show you some of the things you get so this is the thing that my the Voice AI developers customers seem to like the most it's this call flow map um I think it just gives people reassurance that they can see like oh the voice the eye is doing what it's expecting like initially greets the people and then after greeting them some amount of calls go to objective not met like yeah people sign out um people hang up um
that's pretty normal um people hang up right away but even that's something you can optimize as a voice AI developer and work on different tricks um but then the calls are getting you know from their partition some are going to appointment scheduling some are going to product enquiry handling if we go to appointment scheduling we can see that some go straight from appointment scheduling to closing call we can click on either a node and see all the calls that went through appointment scheduling
or an edge between the nodes and that'll bring out a drawer where you can see the calls that went through this um stage of um greeting to appointment scheduling and what I'm looking at here is a call where um it started with greeting you can see the the stage and who was speaking is the assistant it's a greeting then the user says I'd like to schedule appointment that turn that conversation turn was assigned to appointment the appointment scheduling stage
and just gives you an easy way to kind of look at your call volume and see like oh I see what's happening um everything's going as expected or something's not going as expected this is gonna be an easy way for me to click in and look in and see what's happening and see what I need to change um hmm can we pause in there yeah that was that was really good that was a really good walk through so when let's say somebody's looking at this this report
and they see there's something which has gone off the happy path and they wanna dig in how would you kinda recommend someone like evaluate these calls just clicking in and read through the read through the transcripts and you can kinda tell where they've gone off off path yeah actually for that so this first screen is really something that's like for your quick view of understanding what calls doing if you want to figure out how to improve your call I actually like the call map more um
maybe that's a bias because I'm a little uh it was the first thing we built in it I think it's neat but um with the call map it's a similar thing where we're showing you the stages of the call but what's different about the call map is we show you will highlight in red where most of the calls are stopping with failure um it makes it easier to subset to just the calls that are going poorly the other thing you can do with the call map and and the demo accounts not as good as an example
but you can see you'll see something like most of your calls like 106 calls here 47 calls here 17 this is the happy path that most calls are going down but you might find like this weird branch and it's when you click on the branches that that go off in an unexpected direction that you find the insights you click on those and it will um and if you click on the transcript part it'll bring you to the part of the conversation where it's going off the the rails it will highlight in yellow
and it will give you ideas about what you need to improve about your assistant um we haven't one of the things we want to do is like start using diffusion models to turn this into like a like a comic book panel so it's quicker to read because reading transcripts is slow but people can ingest that else of quicker but we haven't built that yet yeah we we have like a the Ellen Musk judge to just like evaluates the call that summarizes it but again that's still uh
like a bit of a pain to read all this summaries especially be doing any sort of significant cool volumes this might be a bit like a out there question presuming might be possible and I don't like can I guess features etcetera but you can almost kind of extrapolate similar this out to self healing and self recommendations through call improvements and maybe if somebody ships like an incomplete assistant and maybe could use another couple of stages added on to it
you could you could probably even tell by and having a decent bunch of calls running through your system yeah that's exactly where we're going with this so with I'm gonna pop over back to my dashboard um it was a pretty scheduled comment below because I logged in it was this thing um I can't show it but we have a feature where you can integrate um with Slack and what we're doing at Slack in sizes were showing you like here's a call that went down a happy path
90% of your calls to go through all these stages the happy path they complete and have a successful outcome but here's a call that wasn't successful but it went down the happy path look at it um and where we wanna go from there is to then take those calls that should have been successful because we've we figured out how to subset just to the calls that um should have been uh that are problematic rather than like manually sampling um and we want to suggest improvements to people system prompts
um to to help them like you know figure out like oh well these calls are going poorly because when the voice AI repeats their name back spells their name back it speaks way too fast you gotta like prompt it to speak slower um things like that that we think we we wanna get to the vision that you describe Tom where we can help kind of build self healing systems with the human in the loop hmm yeah I'm wondering that you can really pull some really good insights out of this
and instead having a data served up this way like a visual for any decent sized company you have to report on the effectiveness of these things that's gonna be a really useful page to show them yep yeah um so another thing I'd like to show is um the custom metric so what a lot of people cause is in line with what you're just talking about Tom um you can ask custom metrics of call so the summary is great um but summaries one line summaries LLM is a judge
they don't capture kind of what happens within a call often they all think loss is over this is a common problem I say Isaac Gloss is over kind of the details that might happen in particular turns of the call um and one of the ways to approach that issue is to like look at call maps and it'll help you identify the different stages that you need to focus on and subset rather than random sampling um another thing you can do is you can define custom metrics like you may be interested in knowing um
your your voice I should be asking in the US for consent to record the call um and you might wanna ask like did the assistant ask for consent to record the call and will then for every call that comes in will ask that question and will surface you can click on the times it's false um and we'll show you just the calls where that was false you can look and be like why did the voice say I not ask for consent you can see what the issue was and try to figure out how to improve it
um so just one more way to uh improve your agent and it's also something that like your customers might be interested in too because they're like well how can I get my voice AI assistant to sell better um you know how to handle this objection did the voice AI handle the objection or something like that yeah nice there's a there's a couple of like interesting quirks sometimes come up in assistant calls one of them is sudden uh sudden bigger bigger lacency gaps in between questions
so say if you have a script everything's gonna read nicely and then suddenly for some reason a question comes up the user responds as normal but then there's a big lag and then the user has to repeat themselves and then maybe the assistant repeats themselves and then it kinda gets through is there a way to identify something like that in in your system yeah so you're this is like it's almost when I see this happen the most is around interruptions where like uh the the person the voice say
I will think the person has finished talking it'll start talking and then the person will like wait for them and it just like the question ends up getting repeated and things get confusing is that when it's like type of scenario you're talking about or maybe darkness some sometimes it can be like a like a rogue background sound which cuts the cuts the response out but there is uh now and again I don't know if this happens when you just uh maybe listening to voice articles all day
so you're more attuned to it and then an old person won't notice but sometimes you do find that now and again there comes like a little bug where there's like a nine out of the 10 questions that the assistant will use on the call be absolutely perfect like normally it's expected but then one of them for some random reason uh just has like a a double or triple the length of gap and it'd be good this is maybe like a more minor like fine tuning use case rather than a high level general one
but it would be good to be able to identify against photos and I I sit when I'm doing recordings since doing videos like cause I'm looking at the audio waves between the responses so I can see the legacy gap on each response sometimes they're most of the time they're even but then you can see one which is like double the length of the other one uh yeah I know what you're talking about so um we calculate on the back end we're calculating the latency
the for the human response and the latency for the AI response and we're calculating how much of the time the person is silent when it's their turn to speak which is indicative of confusion and all these other audio metrics we haven't surfaced them yet we just have a um there when you log in um this is something we're shipping hopefully today is something where you can just download all those metrics for your calls so you can see the latency for all your calls but eventually
we wanna surface that and put it in what we call a rain cloud plot where right now this is the call duration distribution and you can see like most of the calls this frequency history it's a frequency um distribution um you can see most of the calls are around here around 30 seconds um and then the individual calls are these dots underneath um and we wanna create a drop down here for latency so you'd be able to see like oh most of my calls have pretty low latency
but you could click on the ones with higher latency and see what was going on with those and see when they happen are they happening because of interruptions or is there some way you can might be able to improve the flow so that those those late and sees are less likely happen or like there's a long wait and see and maybe you wanna go tell your client like hey this thing happened um that's uh but we're working on it it sometimes happens it's okay most of the time your late and sees are over here
you know there's nothing to worry about yeah sometimes it can be random things like uh response length or sensitive length or if they're doing some sort of rag over a knowledge base which has like an awkward setup then there's just like a bubbling legacy there too if they're making like uh it's easier custom action which is like a laggy API and then they get a response back in time um yeah having that flag would be would be actually awesome especially at scale that'd be that'd be killer
yeah I mean what you just said speaks to the craftsmanship of building voice AI is like you can uh I think um you know you can get something up and running in in five minutes and that will get you 80% away but the next 15% is gonna take you a lot longer and a lot of skill um you know it's faster to get there if you have a lot of skill but it takes you know really understanding how these systems work and how to like avoid these latency traps and and other problems they can come up if that
you know if you're not designing your system well and carefully monitoring and analyzing it hundred percent there's a if anybody's got a new system flow we can just cover what we're talking about a bit more is if you're using something like custom actions or if you have your system connected to Knowledge Base if the user asked the question which isn't in the prompt then it can go and look up through a document or a series of documents Wikipedia and then bring that um
information back into the call which may take slightly longer than a pre determined script or another one which you can do in sin flows and called a custom action which is basically just like an API call which you can either trigger to happen before the call happens so uh actually an example I like to use is if you wanna get the weather so if you're doing anything with like an outside venue for instance you wanna know the weather to determine what you recommend to the
end customer or if it's like a car service or like a concert whatever it is then you can bring that weather forecast into the conversation you could probably do that before the call starts then you avoid the mid call conversation but if you're the mid call later see um the way moment to cover that if you're doing it mid call this could be something like looking up an order ID in a Shopify store to track where the shipment is for instance that could be the customer gives the ID
you can look up the database to see the states of the package and to cover that API call going through to to shopify you will set in your prompts or in your field in sin flow you can trigger it and say okay I'm just gonna look up your information now and while the this is the same that sentence is making me a guy called the same time so that then covers that look up hopefully that makes sense that was a that was a great intro to um knowledge bases and rag and tool calling uh
all all the right there very specific thanks not being at the most advanced custom action developer we've got you killers and simply does some amazing stuff but that's as simple as I ha ha I can understand anyway um story time how to integrate yeah that's right no that's it I was just gonna get good and I'll be I'll be awesome okay so one of the links I dropped in the chat was um uh to this blog post that tells you how to connect your sin flow agent using make um
and the place really just started it it walks you through all the steps um uh but you can also download the scenario like if you go into this blog post and scroll down you'll find the scenario where you can just download it and upload it into make so just backing up a moment um make is a platform for um doing server side automations um so it's like instead of writing the server code you can use make to um take the the the data from a call and do something with it um
like you know send an email with it or or whatever and we're using you can use make also to send data to our platform um so you log into make and the first step is to create a custom web hook and this walks you through creating that custom web hook and then you would then go into sin flow and go to your your agent that you want to um uh send the data from that the call data from that agent to our platform and you click on deployment and rest API
and you would paste in that web hook that you got for make um then the next step is I mean the easiest thing is to just download the the scenario from from our website right here and then you upload that into um into make and then the final changes they all you need to do to finish the integration is you end up going into here I'll go into my make and you have to um it'll look like this once you upload your your file into your scenario to make and you have to put your API key in for canonical
and then it's it's ready to go I mean I I blaze through that really fast but it's that's just the bigger idea you might be wondering where you get your API key to get your API key you go to our website and you click on the little icon and there's um the setup button and you can get your API key from in there um and I don't know what maybe I went through that too fast time should I slow down on part of that or does that makes does that look good I think
I think make make integrations are probably a bit more like on the on the niche technical side anyway and it probably won't be one of the like beginner use cases for for using make I think probably a lot of people are still at the either just trigger a call by a make with some flow or retrieving the details of the call by some society to actually produce some more education materials in this case maybe I can fire up the tutorial yeah and our tutorial actually you might find it
people here might find it as a good way to get started with me cause it's like a walks you through an example that you can get up and running um and and kind of acquaint yourself with the mix system um we all in the chat is a monitor synth play voice yeah and of course cool I'll let my drop this in the Academy too okay yeah that'd be great um and you can also we I also linked to code on how to programmatically to our docks excuse me so if you're if you're writing server code um
and you you want to integrate with with our code and send data via the send the calls via the API that's all in here there's a tab for sin flow for hitting our sin flow and point um so that's that's there for people that are kind of interested in integrating with code how granulate can you get with the make inspiration can you can you animate screen comprehensive when she passing Jason through can you I guess then you can filter filter by um if uh
there's like an unintended call which goes through depending on how you labelling it or a failed call and you could trigger a slight message too would that be right yeah one way if you pick up those dynamically created uh call stages from the retrieval so one way to do it I'm gonna start with the way that kind of better maps to my way of seeing the world first which is like just send all your calls use this make um scenario here to send all of your calls to our platform
and then use our platform to figure out the calls that you want to um be alerted about and you want to surface so you can set up the Slack integration for that um you could create a more sophisticated the other path is to create a more sophisticated make workflow where you're using some of the information that you can get from um the the end of the call the events um they get sent to the web the make web hook and using that to decide what to send to our platform
and what to do other things with it you know um along the way I think that's what you're getting at right Tom yeah yeah pretty much nice and one question I didn't wanna ask is when I guess this is gonna start becoming important when now everybody knows you can measure these calls and we have the Synthro analytics which are just Surf's level now we have this which set them more in depth what would we be able to at some point we like create a okay downloadable report with like a snapshot of the
the last the months worth of course to be able to send it to send it to a client or is this this is the one wait hold on let me get back you guys should just share this with the client right yeah so that's that's what I was about to say is that you can give a login we can set you up so that your client can login and only see their calls here and that way they get you know the interactive experience rather than the static report um and then as I mentioned earlier
we're working on I'm hoping that we'll have a native integration with with Sinflow and that will be these sorts of charts in the other analytics products I've shown you along the way like this stuff that that'll all natively be in in your white label account um and then like along the way if people want you know something that turns us into a PDF because they have some old school customers that want pdfs we can probably make that happen just FAX it to them and let us all yeah yeah nice
that's cool uh so then you got any questions of what you've seen any alternation folks and maybe directed someone like Scott who's very heavy on all the alternation side and like a high level is this something you could work into your stack for clients and then if there's any leftfield questions where you voice AI in general or questions to Tom about plans far far away and then we can spend a few minutes answering them yeah so while we wait for any questions come through
is there anything else that's interesting um that you're looking at Invoice AI that's kind of separate to this it could be um pure racing new models it could be languages it could be latency related things what what else is this space you interested in um you know it's always kind of tempting to talk about like the the latest demos that have come out that showcase the the cutting edge abilities of voice AI and multimodal AI um and a lot of ways I think that's a little bit
it's just it's not as um it's almost taking our eye off the ball the what's really happening in the world right now is we're in um we're used to this idea that you can do amazing things with lolms but some of you might have noticed this during the holidays that like the Thanksgiving holiday or you'll notice it at the upcoming holidays most people that you interact with maybe they have a chatty BT account and all they've said to it is hello they're like we're all living in the future
and we don't realize how much opportunity there is just to bring people that aren't in the future that we're living in right now up to where we are now and there's so much money and opportunity in helping the world go from where it is now just to the current state of technology um with these interactive voice assistance um and uh I think part of the reason we don't realize is I'm constantly talking to people like you Tom or the people in this in this chat
um are in the session that like are living in that future already but I think that we I think what's really exciting is just like the opportunity that's here in front of us with with Sinflow yeah true you do you kinda get uh wrapped up in everything that moves so fast and that curve has been so violent that you do realise that everyday semi news coming out but still people are still catching up so that's a that's a great point we have a question through through net does it does it email a report
say once a week or notify errors as they happen for real time alerts so we have the current notifications we have are the Slack um Slack alerts um and uh that's they'll tell you when a call was a long call that typically finishes in this success but failed or if it was a call that was on the happy path that normally finishes in success but fails um we haven't built uh any other sort of like email reporting or other types of alerts now um I'd love to hear what you'd like to have built Nick
um you know it's a fun stage of the company we're at uh where it's it's just my co founder and me at the stage and we're working closely with our customers on getting feedback and you know we love getting to hear and build for things build things for people I wonder if that be like a a good way I think it's pretty pleasant info too just saying if like a certain amount of cool volume then kind of falls outside of the norm in terms of computer core ratio or something
that she just pulls the assistant um trying to get like that feedback you uh yeah that would work really nicely for an outbound agent right when you have a little bit control over um you know you know it's okay it would be okay to turn it off for a little bit to make some changes yeah run just pumping out loads pumping out loads of calls and realize that 5,000 of them have gone crazy yeah yeah haha um chain just asked about booking a call with us
I just dropped my calendar in there for anyone that like to meet me and also I'd love to connect with people on LinkedIn I just like seeing people build right cool demos on LinkedIn of what they're building and it's like LinkedIn it went from being like this boring thing I hated being on Twitter when I was running my egg type company um so like it's kind of a trap for me now I spend way too much time on LinkedIn and Twitter just because I get to see like
the coolest stuff that people are building so I'd love if people um oops that's the wrong link uh would connect with me on LinkedIn Twitter so I can stay up to speed on what they're building amazing and uh any other thoughts just from you can like wrap up after this if anyone's got any more questions any more thoughts than just general like AI applications uh is there any no pretty much time for side projects but is there any anything like hobby related that you're playing around with
it could be video generation images language translators language learning know that with my kids could be spending a lot of time with stuff like Synthesis Tutor for math which is going really well so any of those types scenarios you earned I'm sorry I missed that are you asking me or are you asking the community should I answer at first I say to you I'm the Zike community I see if anyone's got any good ones um yeah so one of the things I'm really interested in now is
how do you get emails from voice AI they they they bubble it all the time and I have I have this API that um I just shipped yesterday but still uh last night actually but I haven't built the testing around it but it's basically a way to accurately get an email from a caller um and what we're doing with it is we're just doing a lot of data processing to um extract the the the proper noun the spoken letters um and hand that back I think um I don't know I find it to be a really neat problem
it sort of goes back to this idea that I think just voices and neat space with so many neat problems and like even this one's kind of a mundane problem like extracting an email from a call but like it's just been really fun to work on um I'm excited for getting to the point where I can start demoing it um and hopefully maybe even integrate it with something like sin Flow that would be useful because that is a current paying point that requires a lot of um creative prompting
but attention to detail and it's if you don't explain it well then a lot of people do slip up especially when they wanna capture customers emails and then they come through wrong so yeah plus 1 for that yeah yeah that's neat and the other stuff I just saw um a post by um Quinn and daily he's a friend of mine and he use Gemini's multimodal to um basically I think show us what the the future is gonna look like where you're talking to a computer and the computer is able to see what's on your screen
and it's going to respond to you in text and voice and video um and I think as humans we we find it easiest to speak um but we ingests information with our ears at a slower rate listening than we do for visually we're like visual creatures so I just think it we're gonna have this future relatively soon around the corner where people are talking to machines and machines are talking and back as well as showing us information back um and I I think that's a I think that's really neat
I mean I I start talking to my computer too with Whisper Flow and have really enjoyed yeah I use Whisper Flow all day every day yeah basically doubles the amount that doubles your speed of like output so any text box is there you can just reply twice as fast so like Slack replies email replies like DMS yeah it's incredible yeah so you're mostly using it for email and Slack is that the main thing you're using for um sometimes I do it for uh like notes so I wanna take a quick note
I use that um pretty much anywhere actually sometimes I will use it uh this weekend or just gone I was using it in a bolt dot new by stack blitz and also lovable and it was just building so while I was building I was doing something else and it comes back I'll prompt it again with with just my voice and then just go off so yeah I actually have to type so you're building software just with your voice now what what platforms are you doing that with so one is called bolt.new B O l t.net
oh yeah yeah uh huh yeah by staclets and the other one is called lovable L O v a B l e dot gov I haven't heard of lovable yeah I've been hearing lots of actually yeah and another one if anyone's interested in uh it's the one I keep you interested in NYC yeah there's another one that she which she may be more interested in she's open source works with Llama 3.3 centimetery called Cerebros Coda um I can share the link in the chat for everybody still here
this will build things in under 1 second which is kinda really good yeah the whole space is moving very fast and we got lots lots of stuff to play with so yeah voice AI is a super cool space to be in right now I came to said final comment uh agree on the problem one letter missing an email is a complete void so we focus only on the right phone number and then do post after post alternation getting emails right in a new flow we always focus on getting the phone right during the call
caller ID and confirmation by voice yeah yeah nice nice well I think it's pretty good place anything anything to add Tom anything you wanna at the end no I just wanna thank everyone for taking the time to to listen to me and hear listen to my opinions about what's happening in voice and and give give me the opportunity to tell them about what we're building uh really means a lot to me thank you of course I appreciate you taking the time to share with us
and maybe soon we'll see an inspiration inside the synth life as well yeah that'd be great looking forward to it hopefully we can figure that out nice thanks Tom thanks everybody and see you back inside the community so you posted straight after the call the replay and posting up on YouTube too if you wanna see thanks folks catch you on the next one