Diagnosed with cancer. I'm Chris Karragio, NBC News Radio, NBC News on CACAA Lomolinda sponsored by Teamsters Local nineteen thirty two, protecting the Future of working Families, Teamsters nineteen thirty two. Dot org. The information economy has a rod. The world is teeming with innovation as new business models reinvent every industry industry. Inside Analysis is your source of information and insight about how to make the most of this exciting new era. Learn more at inside analysis dot com,
insideanalysis dot com. And now here's your host, through Eric Kavanaugh. All Right, ladies and gentlemen, Hello and welcome back once again to the only coast to coast show all about the information economy. It's time for Inside Analysis. You're truly Eric Kavanaugh here with an old buddy of mine in the industry, Todd Mostak. As with Heavy AI these days, they've changed names a couple of times to be called Omniside. Now it's heavy dot Ai,
which is cool stuff. And they've got a new offering and folks, this is really some interesting stuff. So it's a database. It's at the GPU Accelerated database. Of course, you know about all the different database technologies. There's a real explosion of database technologies really about twelve fifteen years ago is when it started. And I remember I had a theory at the time I floated
by some of my friends. I said that I think a lot of this open source database stuff is they're really trying to hack away at Larry Ellison's corner of the market. Basically, they want to rest control from Oracle, and hey, let's be honest. You know, Oracle has some heavy handed practices over the years with lawyers and kind of messing with folks, and it became
painful and it caused a lot of bad customer experience. I would say, not that the tech didn't work, just that it was difficult to dealing with the licensure and all this stuff. And now you have this whole I mean explosion of database types, all these open source thingsdatabases, and one of them is heavy Heavy AI. It's a database and heavy IQ is a new offering. So we're going to talk to Todd about what this stuff is and what
it means and how you could use it. Frankly, so Tom want you to start us off with what is the platform itself, Heavy AI, and what are you announcing now soome Yeah, well, first off, thanks for having me Eric, It's great to get back. So yeah. Yeah, So you know Heavy AI, so Heavy AI as a platform, we are
a big data analytics platform. Our kind of claim to fame is that we are GPU accelerated, So we leverage the massive parallel processing power of GPUs, all those thousands of cores and terabytes of memory done with a second of memory down with that you have on these cards originally designed to render video games obviously now kind of part and parcel and core to the machine learning and AI revolution. So we use all that supercomputing bandwidth to good effect to actually process the
EAM much faster. In particular, run SEQL queries of billions of records in milliseconds without needing to downsample, to pre index, to pre aggregate your data. We can use kind of the GPU's power to do smart roof force across massive data sets and allowing you to get kind of interactive real time answers, you know, not waiting for tens of seconds, minutes, hours, depending on your database. You get questions, you get answers back immediately. We
can also visualize those SEQL results in line. We have our own front end called heavy immerse. So heavy dB is the database engine running on GPUs because one of the killer apps of having that kind of low latency and that kind
of speed interactivity was enabling interactive visualization. So imagine real time data, you know, millions or billions of call records, social media data, you know, whether it's Twitter now x or anything else, whether it's you know, oil and gas, whether it's it's cybersecurity, you know, logs on a server. We can process all that data, visualize it. We can actually use the gpu is to render that data. So if you want to display
that data, g you spatially put billions of points on the map. And so a lot of our customers say, you know, this is the core of it is fast database front end is very interactive. Some people have called it Tableau on steroids, but we have people using us both as a database
and kind of a full stack visual analytics platform. Well it's interesting because you attacked what is a fairly well populated space already but with something new, and you look at like what a cloud Darra and a Hortant works were doing, and initially, of course that was all hdfs, which is a filesystem it's not a database, it's a filesystem, which is why they ran into so
many problems. I think that was one of the the achilles heels, if you will, of that whole approach, and that whole movement was that HTFS is a filesystem. Many of the value propositions for that approach kind of dissipated, especially when cloud storage got so cheap, so there were some flaws in the architecture that. You know, they've done a really good job to pivot,
so they're doing much more interesting stuff now. Then you have all the historical players, like your terror data is your verticas then of course Oracle and IBM and all these guys. But those relational database structures are good in certain use cases, but they're not good in certain others. And a couple domains in particular cause them real trouble. One is time series and one is geospatial.
And if you bring in both of those things, you know, architecture is really important for being able to process that information, for being able to parallelize it, as you suggest, being able to get some meaningful insight out of it. And I think that's where your sweet spot is right is because you have this GPU foundation because you can parallelize a lot of this heavy lifting, if you will. Maybe that's where the name came from for heavy AI.
You're able to tackle some of these really dense challenges, you know, think life sciences. Like I say, I think the nexus of geospatial and time series, you're able to do that much better than some traditional relational database could do, right, correct, you know, absolutely, you know, I think relational databases tend to be grab bags of functionality, right, and and that's you know, SEQL is a very powerful versual language. We also
speak SQL. However, we've really focused on those use cases that we think need kind of that real time performance uh temporal and geospatial or space show temporal use cases kind of first and foremost among them, because like you know, we see that our customers and often you know, they're not working in the world of like old school tph Let me join my line line item to my orders table, to my customer's table, right, this is kind of real
time logs they're looking at. You know. For example, one use case we have with a customer illegal fishing, right like looking at on the federal side, looking at where you know, ships may be crossing and areas where they shouldn't be and actually doing real time analysis and figuring out, okay, these ships turned off their beacons, and so you're not only doing temporal now
where these ships are going, or doing spatial analysis. And to be able to run that over billions of records with like very fast, high velocity incoming feeds and looking at historical data, that's just something that even you know, even a performance a redshift or a big barrier or snowflake, these databases just
don't scale to. And so I think we've found kind of our core differentiator, and it really is real time data often with space shift, temporal and just today's databases aren't really built to handle those kind of workloads and scales right well, you know your snowflake. So the world certainly are designed for structured data, which I've always believed really means data in relational tables essentially, that's
what they mean by the term structured. And you know, data breaks is along the same lines, maybe a slightly different approach to getting there, but there's still similar engines. And what you've done is special because again you've you've targeted some of these much more complex use cases that would require tremendous compute power fromditional CPUs, so much so that it's just not really economically feasible, and it's not even really compute intensive feasible in a sense. Is that about riting?
Yeah, that's that's absolutely correct. And then the thing about CPUs, right, it's also about compute density in the sense of yes, you could pound four pound scale up a CPU cluster to have equal compute to a GPU cluster. By that time you're at hundreds of nodes, right, And then you got to think about interconnect You got to think about how you're sharding your data. There's so many inefficiencies of going to a massive scale out cluster versus
scaling up. And we can also get a multi noode, but first and foremost we're scaling up within even a single machine multigpus each with thousands of course, that gives us a lot of processing capability, and the bandwidth between these GPUs, if you're familiar with some of the nvidia's mv link capability can be measured in you know, many hundreds of gigabytes a second, going into the terabytes a second range, and so that's a whole different architecture, right,
where you can actually move data around very quickly Versus your two hundred nodes of impula on some commodity cluster. That gets very difficult to actually get that kind of performance just because you're spending most of your time shipping data between nodes and not actually doing kind of the core analytics operations. Yeah, that's pretty interesting. So the n v M e SSDs is that part of your vision?
Is that part of what makes the magic happen here? Yeah, we're getting more into that world, right, So you know, we are I call it in memory light in the sense that we do persist a disc. You can we can page in and out of disc. Obviously, our systems happiest when we can at least cash in CPU memory kind of the core data that
we're actively analyzing. But we're actually working with some partners on the storage side, and there's some innovations called GP directs and GP file system from the Nvidia side where we can read straight from the MVMME device and without with bypassing effectively bypassing the CPU and going straight to mv ME over the PCI PCI bus, and you can get some tremendous performance and in fact you can even have remote storage these days with you know, these flash blades and be pulling at hundreds
of gigabytes a second, which is kind of unimaginable, you know, from storage, and so all of a sudden, your storage can start looking at memory, except you have potentially petabytes on tap. Yeah. Well, the reason I'm bringing this up is because we have another company we're working with,
Ocient. I don't know if you've come up, Yeah, I notice it, But they really thought through what these NVM solid state DROGECT and in particularly the interface, which I think is what you're kind of speaking to now, the ability to just pull in massive amounts of data and what they call it is a is a compute adjacent storage is kind of how they're referring to it. It sounds very similar to what you're talking about, and it is.
It's orders of magnitude greater in terms of possible throughput. So it just changes the entire game in terms of how you think about what you're going to do
with this data and how you think about setting up the architecture. And that's where I see you guys doing something very similar in that you know you have specif typically addressed these heavy duty use cases, which is also what Ocean is doing, and you're conquering it really with the GPU accelerator, but also just a different architectural approach to solving the problem, right, Yeah, exactly, And I think you know, these new problems and new hardware architectures really do
the new thinking, right. Like you know, most common kind of data warehouses were built in an age where you know, interconnect was was very slow, where storage was very slow. You know, all these things are in CPUs, we're kind of getting faster at a very small fixed rate per year.
All of that's been kind of up ended with GPUs with fast you know, whether fast fabric connect right, whether it's internal, external to machine mv link, PC I four, pc I five, And so I think it demands different approaches and how you tackle this with software and in terms of deployment on prem also in the c what are the options for deployment. Yeah, So this is I think actually been one of the blessings. You know,
heavy AI we were mat D back in the day. You know, we came into a world where there was very little cloud CHAPU compute, and so that forces us to develop first and foremost, foremost on prim options. Obviously, now almost all our customers run the cloud, except we do have the ability to run fully hybrid, to run on prem, to run cloud where Docker rise. We can also run bare metal. We can run fully offline
and air gaps, which is important for some of our customers. And so we're pretty versatile in that respect versus being pure SaaS Micro service Play, which kind of hms you in a little bit in terms of your deployment options. So I'm one of these dorks who sits around and thinks about this stuff all the time. And you know, I've said for a long time now that on Prem's demise you know, has been exaggerated, or the rumors of on prem's demise have been exaggerated. It's going to be here for a while.
I think you're going to see a bit of a movement back for sensitive data for PII, but also for lms, you know, to be able to for big organizations to be able to do that stuff very effectively. You know, they're going to probably want to have a lot of that IP stuff in house and basically recast their on prem data center to be able to do new and different interesting things. But I just wonder about you know, the full cloud environment because as I think about it, all, you know, we've
gone from the monolith to the containers, which is a complete departure. It's a whole new way of doing things right, and there are trade offs in going fully containerized. I joked that we sacrificed what is it, we sacrifice state at the altar of scale with some of this stuff. And yeah, you can scale out and then scale back down and that's very useful, but you do have to to manage states some other way, and it is a bit of a it's a bit of an overkill in certain use cases, in
certain situations. But what do you think about all that? Yeah, you know, I think the you know, containeration I think is generally a good
thing. Most of our customers do run containerized. But I think the push into extreme microservices, I mean that can make sense if you're running at Facebook or metascale, right, I mean obviously there are a lot of advantage of that, but the amount of complexities, deployment, of development of often overhead just in terms of performance upgoing, you know, with the micro services approach of all these services talking with each other, redundantly calculating things, you know,
we've we're not quite a monolith, but by having our core databases one architecture very close to the metal, we think that's been a big part of our kind of performance wins versus parceling it up into you know, you know, tens or even hundreds of different micro services. So I think there's always, you know, there's always a pendulum, just like there is with you
know, cloud to on prem and then back to hybrid. I think the market finds equilibrium at some point, and obviously extremes, you know, often don't make sense. So I think we're at a people are a little bit more realistic and also realistic about the privacy piece of running on prem and also
sometimes the cost advantages. We've had customers starting the cloud, but then as they kind of deepen their investment and you know, heavy and big data analytics in general, you know, they want to run on prem just because the costs the economics makes sense, right, No, that makes complete sense, and you know, just real quick before the first break comes up. Here, there's also this concept of the modern data stack, which I saw spring
up I guess a couple of years ago. Now it kind of feeds into this that you know, again, you have sort of best of breed components that you string together in order to have a very robust solution. Well, one of my best friends in the industry, he passed away last year, unfortunately, but a super smart guy, and I remember talking to him a couple of years ago. He's like, yeah, it's modern data stack. It tends to be strung together with a couple of frail lines of code.
And I'm not a big fan of that because if any point along the way it collapses, you've got yourself a big problem. Right, And it seems to me that you've got enough heavy lifting again in the database and in your environment, and you can visualize in the same environment with the same technology that you're kind of circumventing some of those challenges that about, right, I would
say, so. I mean, I think that's one of the potential attractions for customers to our platform, but also something that we have to educate the market on. You know, everybody's thinking, okay, we need to have all this piece wise modern data stack, and we go in and say, hey, actually, you know, we don't want our place necessarily your data warehouse a record or your data leak, but for your last mile analytics. We can actually do the sequel, the visualization, even a lot of the
data science work all in one stack. And you know, you're seeing the data through a single pane of glass, and we can pull from your data bricks, we can pull from your Partka store. It's people say, it's much simpler to deploy and think about, and it's less headaches. Obviously if people are steeped in the notion that you have to you know, parcel out your data stack into all these different services. Obviously that can be a challenge
selling. But we found that, you know, customers get a lot of value out of having one kind of a single platform to choke if you will. But you know, having all their data in one place, with all the efficiencies both operationally and from a performance and analytics perspective that you gain from having data consolidation versus many different silos. Yeah. Well, and I'm curious to know In the next segment, we'll dive deeper into this. You know,
how you populate this thing. How much time does that take? You've talked about serving as a front end with a data lake or a data warehouse in the background. That's also pretty interesting because you know, I get that one environment for doing the processing. If you could parallelize very well, that's a very compelling storyline because you're not bouncing from system to system. You're in one environment and it's designed to handle all these different queries that you do and
give you the visualization all that stuff. So it's a very interesting, bold approach, I would say, but don't touch that, tot. Folks will be right back. You're listening to Inside Analysis. Welcome back to Inside Analysis. Here's your host, Eric Tabanaugh. All right, folks, back here at Inside Analysis talking to Todd Mastak with Heavy dot AI doing the heavy lifting
for you, folks. I always joke about the heavylifting and data warehousing and analytics and data science of course, and one of the cool things about heavy AIS they talk about really bridging the gap between those worlds because and I've always been fascinated by this. I'd like to get your thoughts on this, Todd. But from what I've seen, data science teams are very separate from data warehousing teams, and I have no idea why that would remain the case as
data science matures. I would think you'd want these people talking to each other and working together because the same goals. Why would you have a completely discrete team for your data warehouse versus your data science team. And it's just organizationally how things evolved. But I don't think it makes a lot of sense. So maybe talk about that for a minute and then we'll get into how you load this thing. Go ahead, Yeah, it is an interesting phenomenon,
and you can explain why it happened. But I do think in many cases, for many organizations, it's suboptimal to have these you know, silot orgs, you know, data science versus you know, your your database or your data engineering or your analytics teams, because they're often working on roughly the same thing across purposes, and they distrust each other, you know, and have different systems and different you know, it can be kind of a mess.
Some organizations handle it much better than others. You know. We we're not claiming to be the panacea here, but certainly we've had you know, one of our customers, a big hardware manufacturer out in California, has seen themn's success with actually having both kind of production operational teams hitting the visualizations, running BASER re COORSE and SQL, and having data science teams working on the same
data, cleaning the same data on the same instance. Since it's you know, and they say it's a huge it's a huge gain, and it actually these teams are working more closely together than they ever have before. They're sharing insights and even though they're very different personas, And some people are sitting there in a jupinter notebook which is attached to heavy via our Python connector, and some people are using Immerse and then other people might bull Tableau on top.
So yeah, that's the dream. It doesn't always happen that way, but you know, we like to think that we're helping the two sides of the aisle get a little closer together. Yeah. Well, and there's this jen Ai component too, So your new offering, heavy IQ is allowing your customers to leverage the Geni's capabilities. And we were talking before the show about RAG models, retrieval, logmented generation, which I think are going to be a
huge area of focus from here on out. Basically, I think they're going to be the mechanism by which very serious companies in the space do their data quality. Their data got even some security is going to get baked in there, which is very different because in a traditional data warehousing world, the data governance, the data quality that's baked in in different ways, and you have
different kinds of guardrails. But it's like, if the LM is the interaction point for all of your questions, well, now I can focus on building my guardrails inside the RAG model of the LLM. But what do you think about that change? Because the other thing I look at is the capacity of these engines, not just the text generative capability, but their pattern recognition capabilities for example, and the mechanism by which they choose what they choose is pretty
interesting. It's not an exact science yet. And you know a lot of people like talk to Joke, including people who are very experienced in this industry. I really don't know exactly how it works. It's like, well, you know, how's that going to work? But what do you think about all of that? I mean, are we really looking at upending many of the traditional ways we've done things over the past twenty thirty years? What do you think? I think it will upend things, and it's you know,
it's going to be faster and slower than anyone kind of imagines. I mean, I think faster in the sense of like wow. You know, two years ago or a year and a half ago, there was no shate GPT, you know, generative AI was being talked about. That it was people who were just kind of putting their feet in the water. And today everybody's thinking about everybody's using it, you know. I think it's the whole high cycle thing. Right. There's a lot of power you can get immediately,
there's a lot of problems. You know, the non determinism and the fuzziness is both a blessing and a curse of it, right. You know, a lot of the work that we've done in heavy IQ is how can we take these kind of you know, this neveroless power that seems to have fallen from the sky. It's almost like alien technology, and how can we like ballow it up into something that's predictable, that's reliable, that gives people the
answers they want, that doesn't hallucinate or almost never hallucinates. And how do we treat it like an engineering thing. We have unit tests around it,
and what can we put in the pipeline that's deterministic. I think sometimes people throw everything to the LEM when in fact you want to use LMS for what they're very good at, like pattern recognition, natural language processing, and you want to try to still you know, parts of our heavy IQ offering, which I guess we're kind of jumping ahead of here, but basically allows users to ask questions and get answers back the system. Heavy IQ's writing sequel for
them. It's writing natural answers, it's picking visualizations. Parts of it, you know, will actually correct the SQL queries and figure out, oh, the user got this little wrong. We'll do that through traditional software approaches, right, we won't necessarily trust the LM to do everything, but use the LM for the core what it's good for, which is, yeah, to take the user's natural language utterance and translate that into what they actually need or
what they want well. And I mean, I think that's going to be a very significant change in the usage of these tech analogies because when you can just ask questions and it's not new. I mean, I was joking with someone the other day. I remember progress Easy Ask, which was like I mean twenty years ago, I think or eighteen years ago, where you would type in a natural language query and it would build the sequel for you underneath, so you could see how it's doing that. That's not new, but
it's certainly much more sophisticated today than it was. And to me it's great because it opens up analysis to a vastly broader market of business users who don't know SQL, or don't know SQL very well, or don't want to sit
around and write sequel. Right, So that compoundent ended up itself as the front end is I think, in a revolutionize and greatly expand Do you see these technologies and the fact that you're doing some of the more traditional deterministic stuff under the covers, I think puts you in a good category because to your point, you can ask an LM all kinds of stuff and you'll get sometimes very good answers and sometimes very wrong answers, and you certainly don't want that,
right, So you I think you have a pretty clever fusion of old and new technologies in this stack. And of course heavy is the background, the dB, that's where a lot of the action takes place. You're using the LM on the front end to facilitate a conversation with that data, but then you're converting to SQL as part of your offering, right, so you are bringing together these two worlds to be able to enable very very rapid analysis
of very large data sets. Right, Yeah, that's that's correct, And I think like having the data engine, a very fast form of data engine
is actually dovetails with the use of the LM. For example, it's great to have the LM, but part of what we do to make it accurate is we actually mine often our customers have multi billion record tables, so part of it is that we're actually querying all that data to figure out what are the top string values, what's the time ranges for this data, what are you know, what are possible and so we're feeding all that information to the
l M, you know, deterministically so you can see it. So it's a great boon to have a very fast engine so we can pull all that data dynamically, so you can just look at tables that the system hasn't seen before and instantly start being able to ask questions. Also, like once you actually generate a query from the LM, you know, it's very nice to be able to have a fast system where you want this to be conversational analytics, not sit and wait analytics. So used to ask a question, you
know, it could be on a ten billion record data set. You run that SQL query, it takes another you know, half a second or something, and the user has an answer. I think. You know, I've seen some of these other systems and if they're divorced from the system of record, if you know, that's you have your l M service hitting one of
could be terr data, could be Redshift, could be snowflake. Yeah, they're not optimizing for their databases, and so it's it's much less seamless in terms of being able to get you know, time to answer, time to insight. Yeah, so that's interesting helps a lot. Well, it brings me to my next big question was how do you load this thing? So in order to use this, do you have to essentially etl in vast amounts of data? How are you doing that? Like, what's your mechanism for
populating the database to be able to get rolling? Yeah, what's cool. You know what's cool about iav IQ is you load your data into heavy dB and you can use our web interface heavy immerse if you want, you can do it, you can script it, you can do cover once it's there, it's automatically variable via natural language via heavy IQU. You have to do zero else the system. We have a kind of server running behind the scenes that once you start asking questions, it will know which metadata to pull.
But all of that happens kind of automatically for the user, and so there's no setup, no configuration, It just kind of works out of the box. The other nice thing is we're running on GPUs, right, and these lms run best when accelerated by GPUs, and so we can use that same infrastructure that we use to run our database to run the lms themselves, and so that's kind of cool as well. And then talk about how you would connect to like a data warehouse for example, a Snowflake or a Redshift or
something like that. You talked about being able to use heavy as the front end basically, so you would be pulling some data from these warehouses getting it into this environment. How does that work? And you know, can you do things like change data capture for when there are updates or different things of this nature. How do you how do you deal with that sort of threading of multiple information systems going into heavy. Yeah, that's a great question.
So we have something that we released a few years ago called heavy connect, which basically is it's almost a foreign data wrapper kind of interface for heavy. So that means we can sit on top of another store record, whether it's Snowflake, whether it's Terra Data Redshift or like a Arcade store or something like data Bricks, and it seamlessly. Any queries that you issue will get pushed down and data and then a data will get fetched from the underlying store.
We are working on things like change data capture. Right there's some notion of it's more built for a pin streams right now, it can pick up pin streams updates. You can do retriggers every night to like fetch the metadata changes. So we're getting more sophisticated with that. But for most of our customers, they're not necessarily putting this on top of their transactional store, and they're putting on top of another analytics store in which the underlying data it may be
appended to rapidly, but it's not changing. You know, you're not seeing a ton of updates. You may be seeing deletes, but overall it's a huge gain because you know, these systems like Snowflake is a very good system the separation of computing storage. It's economical in many ways for users, but in terms of providing real time analytics. You know, it's less strong and
so using heavy for what it's good at. They interactive visual analytics, they interactive quarium and then having that push down to snowflake is your store record, I think is really exciting for people. That's pretty cool. Yeah, and I on your site you talk about unprecedented context across location and time. That's kind of where I was going with earlier in the game. The time series
is so important for understanding what's happening in any kind of environment. You want to understand, you know, what what happened over a period of time, and what is the key period of time to understand is it a day, is it an hour, is it a week, Sometimes it's a month. And being able to bounce around in those different time frames to look at the
difference, look at the impact, or look at what's what's happening. That's a really powerful feature to enable analysis because if you have to reset, you know, restructure the query, wait ten minutes or something for it to repopulate or materialize views or something. Any any friction you put between the mind of the user and the information system being queried is going to slow you down. And it's going to cause you know, a hindrance and morale to go down,
et cetera. So I think that's why what you have is so interesting, because you are greasing the tracks for the analysts to figure out what is the appropriate time window I should be applying to this problem. What do you think about that? No, I always say, you know, people talk about real time analytics, but real time analytics is often not very powerful without
historical contexts. A good example of this is one of our longtime customers, Charter Charter Wireless, Charter Spectrum, you know, essentially uses us you know, as they see you know, they might see an issue, or they may get something on social media or people saying I'm having an outage or whatever. They can instantly go into our system with a bunch of historical data and say, okay, what are the piece of network equipment in that area and
we had faults with them historically? What's been the network performance this area? Is this actually an issue or is this a statistical anomaly? And you know, I don't think they'd be able to do that, at least in real time without without heavy We've seen that from a lot of our customers. They
say We used to have an outage or some issue in our system. We work with a lot of telcos, and we'd be sitting there, sitting on our hands waiting for an answer back from the data engineering team, Versus they
can actually do that themselves. Now they can dive into the data and often get to answers in order of magnitude faster, you know, minutes or tens of minutes and set of hours or you know a day or two days, which really has helped with kind of faster turnaround times, faster resolution to customer complaints. And it's all about drilling in on space and time and being able to look at the real time data as well as the historical contexts. Yeah,
I love it. I mean that's just really really good stuff. So folks, look at these folks up online. Heavy dot AI just like it sounds heavy like the weight and heavy IQ is the new but heavy dB is in the background. Todd Mastak, thank you so much for your time. Yeah, thanks so much. Or it's been great. Folks. Stand by. You're listening to Inside Analysis. Welcome back to Inside Analysis. Here's your host, Eric Tabanac. All right, folks, welcome back to Inside Analysis.
Your host here, Eric Kavanaugh, And next up we have Peter Voss from a company called the ig Ai that's a I G O dot ai and they have a conversational chat but a chat out with a brain. That's the tagline. So Peter, welcome to the show. When you say a chat out with a brain, what does that exactly mean? And how is your chatbot different? Yes, good question. So literally ours has a cognitive engine or a brain, and as far as we know, we are the only
company you're offering chatbot with a brain. So what the brain does is it has short term memory, long term memory, reasoning, abilowity, and deep understanding. So it's you know, it's obviously much more effective than a chatbot that just relies on a float chart, which is pretty much what everybody we also doing that you go down a bloadshot and you know that that's kind of
a decision tree. So we've developed this over many many years. This is actually like a third generation of our technology and allows us to really very effectively replace calls into agents instead of just augmenting or instead of just doing you know, the very simple task. Now, when you say long term memory and short term memory, I mean I understand persisting something either to memory or to
disc for example, but what are you really talking about? Is there a semantic layer to this, and then there's persistent data like concepts and historical context. Basically go into the details about when you say memory, what does that exactly mean and how is that memory then called and used to create context? Yes, absolutely, So the backbone of the system is a dynamic knowledge graph that basically has the knowledge about the company, the product, but also the
individual that you're dealing with. So their personalization is hyper personalization at the individual level. To complatize it. You know, one of our customers is one eight hundred Flowers Harry and David group of companies, and they use this obviously as a concercial level service to individual customers. By the way, just a few weeks ago in Valentine's Day, we actually replaced three thousand agents doing that and you know, very successfully automated service calls. So what it means is
that the system remembers what you said in previous conversations. For example, if you say I want to buy some chocolate for my niece of birthday, then that becomes part of the knowledge graph and that knowledge can then be used so later on in the conversation you established what the niece's name is and you know her address or whatever. That information is kept in the brain and it's available
for future conversations or to follow up conversations. So the long term memory is an integral part of the knowledge graph, and that is then used both for short term memory as appropriate and for context during the conversations. As you say,
it's a semantic level understanding. Well, that's interesting. So basically, every time there is a customer call of guessing, you're creating a new node and new edges on that knowledge graph, and it's ear marked as a node of this conversation I'm having with John Smith, for example, on this date. Is that correct? So every conversation. Yeah, And actually it's a little bit more sophisticated than that, because you really want to separate out individual
customers. You don't want to you know, combine information that you have any risk of, you know, you know, security, security risk. So what we do is actually there's a three level architecture of the knowledge graph. The the core level, the inner level is information shared by all of our
customers, so it's just you know, the generic information. The next level out is the unique information for the enterprise that we're working with, like their business rules, you know their product ontology and everything you need to know about the business. And then the third layer is unique for each individual customer. In the case of one eight hundred Flowers, it's twenty million customers that each
have their own part of the knowledge graph that is loaded. As soon as we identify with the customer is we load that additional part of the knowledge graph. And then when the conversation finishes with whatever has been added that is then you know it away. That's very interesting. So is the is phone number the key value to identify who's who? Or is it phone and some other
Oh? It really it really depends on what the customer provides and what channel you're using, because we you know, our system is truly omni channel, so you can really start a conversation from a website. You can go to chat, but then you could switch over to phone. So if you're coming through chat, might well be emailed address, or it might be the order number that you give to identify the customer, or it could be the phone number. Of course, there has to be validation to make sure you are
in fact talking to the right person. But the identification really, you know, can happen through different things. Now, if it's coming in through say Apple Business Chat, you already have validation of the customer. It's coming from a web website where the customer is already logged in, then you'll you'll actually have a customer ID. That's interesting and so hmm, so you're replacing call centers. Basically, are you able to do voice or is it only chat?
We do we do both voice and chat. Yeah, it's truly omni channel that you can. You can because it's the same brain whether you're switching your channel, your communication channel, whether that switches from you know, SMS to Apple Business Chat to voice, it's the same brain and you'll have the same context that available. So is it a computerized voice, then it's just a computer voice that's having it's correct for the lephany as computerized text to speech,
you know those does that become really quite good these days. And I think one of the important things is we believe it's important that you let the customer know that it isn't AI they're talking to I. Really you don't like it when enterprise want to full customers into you know, it's got to sound just like a human. I don't think that's a good idea at all.
We always give a well subject to the enterprise we're working with, but we very much want to design the system in a way that the customer can ask for a live operator at any time. Very simple. There's no you know, there should be no barrier to talking to a human if you want to. For whatever reason. You know, either the system doesn't have the knowledge
or somebody to simply simply prefers it. But you know, in the example I just gave them a Valentine's date, ninety percent of the interactions were handled entirely through automation by our system, and only you know, ten percent opted or needed to go to an operator. That's pretty interesting, So you do inform people. It's funny because that was going to be a question I asked you, Is that you telling people that this is a computerized chat butt and
you are have you seen have you seen much pushback on that? Or are most people okay with that? Oh? No, not not at all. You know, I was recently at a conference and and and somebody said to me, we're talking about the difference between chat and phone, And he's saying, you know, surely boomers, you know, they still want to talk by by phone. They don't want to use chat. And I looked at him and I said, well, do you prefer talking to somebody? Do
you prefer to use chat? He said, well chat? Well, you're a boomer on't you. So I think people are increasingly becoming comfortable with with smartphones. Everybody is, and people actually learning that it's that it's it's easier to text in many cases. And of course the younger generation doesn't even want to talk to to to a person. People just want to get stuff done.
That's really what we're finding. As long as your automation, whether it's voice or text, is moving forward, and you know that people know this thing is really helping me. You know, I'm getting things done. They don't want they don't want to talk to humans. I just want to get things done. That's that's really what we're finding. But you know, as soon as it becomes frustrating, you know, press one, please listen carefully. Options have recently changed. You know, all your business is important to
us. I mean, you know, you pull your hair off and you just press zero zero zero, get me. It's an operator. The society has to be useful, you know, and it has to get you to what you want to do quickly. Yeah, you know, we're going to do the podcast bonus seven here in just a moment, and I think we'll
dive into how you enrich or how you fuel the chatbot. I'm guessing you use corporate documentation, a history of the conversations that people have had, things of that nature, because you have to have a starting point, and then of course over time, you curate and you refine and you find in tune and you get these things to where they're pretty good at being able to handle whatever questions the person might ask. And that is the key, right because
customer service is very important. Everyone says customer experience. It's all about the experience. It's going to be mindful of all that while still leveraging modern technology. Well, folks were talking to Peter Voss of Igo dot Ai. Be right back for the podcast bonus segment. Standby, Well, okay, folks, time for the podcast bonus segment here on Inside Analysis talking to Peter Voss of Igo dot ai, and we're talking about your chat by your conversational chatbot
with a brain. I love that it's got long term memory, it's got a knowledge graph. I'm a big fan of knowledge graphs. I think they're going to be very helpful in the AI age because not just do they store information, but they store it in a way it's very easy to access in terms of relevance, because the relevance are on the edges, right, You have nodes and edges in a graph, nodes of the entities, edges of the characteristics basically, and so in a knowledge graph you can capture a lot
of information about individuals, their buying habits, the characteristics of who they are. Those are all the edges. First of all, is that true with your solution? And then second of all, tell us how you actually populate the knowledge graph in the first place to get rolling. What sort of information can you load into this graph? And in what format do you load it
to be able to get somewhere? Yeah, this is exactly right. And you know, an important benefit of the approach of a knowledge growth based system is that it's truly scrutable and it's auditable and as opposed to know the current trend where everybody's trying to do things with statistical models, large language models, which are you know not they black boxes, they can't be ordered it and
they hallucinate. You don't have that with us. Dappa actually calls this the third wave of AI, where you have cognitive AI, a cognitive system as opposed to the statistical system. So the way we populate the knowledge graph is, you know, as I mentioned in the previous section, there are three layers. The middle layer, the center layer, is the common knowledge that
is required for every conversation. You know about people, places, time, and you know, the common knowledge and that knowledge we curated over several years to act as the core of the intelligence of the conversational engine, and we keep adding to that. But then the second layer is basically where we import the customers ontologies for their different products that they have product categories and the various
business rules and so on. So that is then added to the knowledge graph and you know, just carefully curated to make sure that you have accurate information in that little lay. You also have connections via APIs to the back end system to obviously give you like you know, the current status of an order and to be able to get the basic information from the back end system.
And then the ult layer is the one that is unique to each individual end user customer, and that is built up dynamically through the conversation by basically expecting the semantics of the conversation and adding that to the knowledge graph as you go along. And you know, as you pointed out, the obviously getting all the business rules and interpretations and the ontologies and the specialized language that people use or how people might ask for help, all of that definitely requires tuning.
I mean, it doesn't matter how practiced you are at this art, you never you can never fully anticipate how people will actually use the system, So the ongoing tuning is definitely important. Also from the point of view that you know, products and business rules and demographics change all the time in a company, so you don't want the system to deteriorate as it gets out of sync
with whatever you know. The company and customer dynamic is. I guess that's my last question to you, which is, you know, the world changes, habits change, products change, but it's hard for a model to unlearn something. So in your graph, do you have the capacity to update offerings for example, like what the chatbot is saying. Obviously it's pulling from the knowledge graph to some extent and you can curate that and find tune it,
et cetera. But can you actually go in and overwrite. Can you go in and say, Okay, this old product is not in our service anymore. Here's a new product. And so how do you do that? In other words, how do you maintain the repository of information such that it's all current and you don't have all the information sitting around that is going to throw
people off, right absolutely. And the big difference with a cognitive AI approach with the knowledge croft approach that we have is you can literally retrain the system from scratch in minutes, and you know, and it costs next to nothing. It's not the one hundred million dollar type of model that you know, large language models, where you have to retrain the whole model and you really cannot change the information in the model. We don't have that problem at all.
So you can increment the change knowledge, or you can retrain the model if bigger changes happen from scratch, and literally it will train it in minutes. That's how effect that this kind of approach is. Because it's not working with ten trollium bits of information, right yeah, right, that's because it's a smaller model, right, I mean it's would you call it a small language model or it's just a knowledge graph basically the knowledge graph, I think
is a better way to describe it. The language capability are also part of the knowledge part part of the knowledge graph. But you know that part has been built of a long time. And I mean basic language understanding doesn't doesn't really change. You may add extra synonyms, you know, or new product names and so on, that they're part of the ontology. The core language understanding doesn't really doesn't really change. Gotcha, very very cool stuff. So
I go dot AI. You were conversational AI. So you work with one
hundred flowers. Basically anyone who has a call center, that's a good use case for you, right, yes, call center replacing call center agents really is the most obvious use case for us, but can also be used as, for example, a diabetes coach, you know, a hyper personalized diabetes coach that'll ask you and learn whether you love broccoli or you know you're vegan, or what you know, whether you when you want to do exercise, if you if you want to exercise, and what kind of thing well,
or an assistant for salespeople. So it has many different use cases. The core engine that we have, the cognitive AI engine. But our right now, our focus and our strength really is in replacing call center agents very effectively. Good stuff. Well, look these folks up online, Peter Voss, Theoss from Igo. That's a I T O dot AI. Thanks for your time and folks want to be in the show. Sending email in for at inside analysis dot com. You've been listening to Inside Analysis and now the voices
of GACAA for an exciting announcement. Want to hear NBC and News or KCAA anywhere you go, Well, now there's an app for that. CACAA is celebrating twenty five years in our silver anniversary with a brand new app. The new KCAA app is now available on your smart device, cell phone, in your car or any place. Just search KCAA on Google Play or in the Apple Store one touch and you can listen on your car radio, Bluetooth device, Android Auto or Apple Car Play, catch the KCAA buzz in your earbuds
or on the streets. Celebrating twenty five years of talk news and excellence with our new KCAA app. Just do it and download it. KCAA celebrating twenty five year to Heblt Club's original pure powdy Rco super TA helps build red corpuscles in the blood, which carry oxygen too our organs and cells. Our organs and cells need oxygen to regenerate themselves. The immune system needs oxygen to develop,
and cancer dies in oxygen. So the T is great for healthy people because it helps build the immune system, and it can truly be miraculous for someone fighting a potentially life threatening disease due to an infection, diabetes, or cancer. The T is also organic and naturally caffeine free. A one pound package of T is forty nine ninety five, which includes shipping. To order, please visit to Hebot club dot com. To hebo is spelled T like tom, a h ee b like boy oh. Then continue with the word
T and then the word club. The complete website is to Hebot Club dot com or call us at eight one eight sixty one zero eight zero eight eight Monday through Saturday nine am to five pm California time. That's eight one eight sixty one zero eight zero eight eight to Ebot club dot com. With sixty years of fascinating facts. This is the man from Yesterday and back in time we go to this time in nineteen sixty four, the Beatles have six songs
in the top thirty, including do You Want to Know a Secret. Another British band is breaking through. They're called the Searchers. That band got its name from a John Winn movie. Here is Sugar and Spice, Sugar and Spier gives a se Sugar Spies you know the Licus And from this time in nineteen seventy four, looks like NBC is adding The Rockford File starring James Garner,
from the world premiere movie shown just a few weeks ago. Also being added Little House on the Prairie starring Michael Landon from the recent high rated TV movie This is Jim Rockford. It don't leave your name and message. I'll get back to you. Hi, Jim, thanks for the dinner invitation.
I'd love to, but does it have to be the taco stand? And from this time in nineteen fifty seven, NBC TV says it is experimenting with the new videotape on some of its more important shows, among them The Steve Allen Show, Caesar's Hour starring Sid Caesar, and The Sunday Night Donna Shore Chevy Show Chevrolet in behalf of your local authorized Chevrolet Dealer Live from Mollywood. God Dinah Shore Heavy Show with more at Man from yesterday dot com. It's
that time of year again, No, not the holidays. Medicare open enrollment and if you have questions about Medicare, you should talk to the local experts. Paul Barrette and Associates, all of his agents are certified with plans that are accepted by most of the medical groups in our area. Call nine oh nine seven nine three oh three eight five. Their services free and after forty two years of the business, their agents are trained to help you pick the
plan that's right for you. KCAA Radio has openings for one hour talk shows. If you want to host a radio show, now is the time. Make KCIA your flangship station. Our rates are affordable and our services are second to none. We broadcast to a population of five million people plus. We
stream and podcast on all major online audio and video systems. If you've been thinking about broadcasting a week radio program on real radio plus the internet, contact our CEO at two eight one five nine nine ninety eight hundred two eight one five nine nine ninety eight hundred. You could skype your show from your home to our Redlands, California studio, where our live producers and engineers are ready to work with you personally. A radio program on KCIA is the perfect work
from home advocation in these stressful times. Just type KCA radio dot com into your browser to learn more about hosting a show on the best station in the nation, or call our CEO for details to eight one five nine nine ninety eight hundred. Ralph Waldo Emerson told of a dinner guest who went on and on about the virtue of honesty, offering his own life as a model of perfect rectitude. The louder he talked of his honors, said Emerson, the
faster we counted our spoons. That's my reaction to the cacophony of phony piety arising from Republican governors and legislators who are trying to enact more than two hundred and fifty new state laws to stop black, Latino, Asian, American, Indigenous, and other non Caucasian voters from casting ballots. Yet they proclaim we're
not racists, we're righteous crusaders protecting the sanctity of the vote. Really, so, why are they specifically targeting people of color with their repressive voting restrictions. For example, panicky Republican lawmakers in Georgia tried to outlaw any early voting on Sundays. Odd why it's a flagrantly racist attack on the black church. For years, a joyous civic tradition called Souls to the Poles has played out
in southern Black churches on Sundays prior to election Day. After the sermon and prayers, congruentts, ministers, musicians, and others in the church family travel in a caravan to early voting locations to cast ballots. It turns voting into a civic, spiritual, and fun experience. What kind of shriveled soul tries to kill that? Apparently the same shameful souls and the Georgia GOP who want to stop local groups from providing water and snacks to citizens forced to wait for
hours in line to vote. They're actually trying to make it a crime to give water to thirsty voters. Hey, Republicans, what would Jesus do? This is Jim Heitr Saying the goal and duty of every public official ought to be to maximize voter turnout. After all, the more Americans who vote, the stronger are democracy. But there's the ugly political truth. Republican officials no longer support democracy. KILLISTINAKCAA Lowell, Linda at one six point five FM,
K two ninety three, c F Marino Valley, NBC News Radio. I'm Chris Kragio. The US is rejecting claims by Russian President Vladimir Putin that Ukraine may have been involved in a deadly attack on a Moscow concert hall. In an ABC interview on Sunday, Vice President Kamala Harris said there is no evidence of that whatsoever. He added that we know isis k is actually responsible for what happened. Former Republican National Committee Chairerona McDaniel says she disagrees with Donald Trump's
promise to free individuals involved in the attack on the US Capitol. If you attacked our capital and you have been you and you've been convicted, then that should stay. During an interview on NBC's Meet the Press, McDaniel said the violence that happened on January sixth was unacceptable. She said those convicted should serve out their sentence. McDaniel emphasized, however, she doesn't hold the former president responsible for the riot. She stepped down as chairwoman of the RNC earlier this
month and will join NBC News as a political analyst. The interview took place prior to NBC News announcing McDaniel's hiring. Former Supreme Court Justice Stephen Bryer is speaking about the High Court's decision to overturn roeb Wade and what he and two other justices wrote in their dissent. One of the things we said is what we fear. They think this will be simpler the majority, because it will leave it all up to the states. We don't think it will be simpler.
We think that there will be a lot of more cases coming up, Bryar told NBC's Meet the Press the leak of the decision was unfortunate. He said he had a theory about who leaked the decision, but declined to share any names. The former justice added, however, he'd be amazed if it was a judge. Briar said it's possible the Supreme Court could one day overrule
the decision. La Dodger starshow heey Otani is set to address the media tomorrow for the first time since allegations of illegal gambling and theft involving his former interpreter's surface. Major League Baseball says it has started a formal investigation. The interpreter is accused of receiving more than four and a half million dollars from Otani to payoff gambling debts made with an illegal bookie. The Dodgers fired the interpreter after
The La Times reported on the case. I'm Chris Caragio, NBC News Radio, NBC News on CACAA Lomel sponsored by Teamsters Local nineteen thirty two Protecting the Future of Working Families, Teamsters nineteen thirty two dot org. Was your car involved in an accident or just need help with dents? All Magic Paint and by