I've got a question about structured versus unstructured data. What about the difference between the Azure AI versus Copilot from Microsoft? Well, we can talk directly to an AR architect at Microsoft as Nupi, Fishnum joins me on this bonus episode of the Business of Tech. With as many breaches and security concerns as I report in this show, it should be obvious that cybersecurity is not just about technology, but also the human expertise needed to interpret
and respond to complex threats. Huntress is focused on elevating SMBs and MSPs around the world. Huntress has a suite of fully-managed cybersecurity solutions powered by a 24 by 7 human-led SOC, dedicated to continuous monitoring, expert investigation, and rapid response. The proof is the execution. Huntress is the number one rated EDR for SMBs on G2. Want to know more about the platform? Visit huntress.com slash MSP Radio to learn more. Well, Naveen, thanks for joining me today.
Hey, I do. Thanks for inviting me for your focus. Well, I'm super excited to dive into this. We're all talking AI, right? And so I figured it would be incredibly useful to talk to somebody who's working at Microsoft Building Solutions right now. And what I started wanting to start with is give me a sense of where you see AI most fitting in to the kinds of solutions that partners are thinking about right now.
Yeah, sure. So I see a lot of spaces so far, and I'll start with retail, and maybe to some extent of financial services and to some extent of manufacturing, maybe, and automations. And these are the spaces where I see a lot of interest towards AI. And be it a new net new solution or be it announcing their existing solutions. Those are the places where I see a lot of interest coming from. And I have been doing this with a set of people
at offshore and I have been facing these challenges. And I'm not able to provide the thing what my customer is asking for. And could you please provide some solutions around AI? And this is the common question what I ask. And there is something else. There are a lot it depends on each and every customers and their use case and their need is at sometimes we may need to educate customers and they have to see like what what they are, what they
need is and how we can help them. So this type of things is what I see in a day today. Now I wouldn't think customers are necessarily saying I'd like to just buy some AI. I think what they're probably coming with a little bit more of a focus solution. Knowing that every customer is a little different and their needs are a little different. I know you've abstracted that to a few of common use cases. Can you give me a sense of what the common use cases
are the customers are asking about? The common use case I wouldn't say what customers are asking for. But in general what I am seeing in AI, what are the common use cases? What I see a bit in the common forums where I interact with my colleagues and my friends. What I see is a lot of interest in exploring the chat capabilities. Maybe the first thing first point where they start is the chat capabilities and how can I infuse support solutions
for my product. So that enhances real value. And other than this, what I see is sometimes people are interested about dynamic reporting. So they have been doing a canned reporting this long and they are I see a lot of interest in these types of reporting, dynamic reporting. I ask questions on the fly and just to manage to understand the natural language and converts to a query and run it against your backend and gives me the results in a nicely crived
manner. So that's what people expect people expectations of these days. And apart from that, I see a lot of interest these days around AI agents. So AI agents is getting a lot of sparks these days. Like agents is nothing but so far we have seen GPDs of just you ask question on the net response from your knowledge base. So this is the time how it can act on something. So I wanted to do this. Can you please help me? So can you please generate
a image and make sure that images this is this. So first in general, say image on the net sensor image to a GPT-4 model and then it kind of tells you what that image contains and how how it can reach this type of audience and all these details. Then again, it can sit it back to Dalai say asking for it. Can you make some enhancements to this and then give it back to me guys. So this is one sample. Agent there are other agent use cases.
What I see in part of the forum sort I read basically they are using it for their divorce pipeline maybe where they trigger a job and then they do a deployment at the deployment fails they know what and where the code is and they try to fix something and then do it deployment. So these types of things is where I see a lot of focus on this.
Now when I ask first about the reporting because it's interesting to me that as you know the AI and GPT's and particularly when we think about generative AI are proving to be useful one of the areas is exactly what you talked about is that idea of not being able to leverage
the canned reports and instead you know fighting against the idea of the custom report builder which has always been problematic because it asks customers to try and learn a reporting language when they really just want to ask a question about the data and it's interesting
to me to hear you talk about that use case because it implies in order for that to be useful you've actually have to have done some work on the data front to make sure the data is useful because we can't necessarily give the GPT all the data or it will without
understanding of say security implications or who gets access to why and so there has to be a step before applying this solution about making it structure like some level of clean and well structured yet at the same time the power of the AI is the fact that it can take piles of unstructured data tell me a little bit about what's required for effective data management to make an AI project successful.
Yeah so this is like it's not a very simple answer so it's there are different layers and different parts in this so first to talk about right as well there are two types of data what is unstructured data and other one is structured data I know by this time many
might have explored the ways of tearing those unstructured data like how we can make it into pieces and how it can answer better and a lot many things around that so let's focus on the structure so that's where the reports and things are getting into which
are right so how I can get rid of those canned reports and how can I be more sustainable with these reports with the ad hoc reports whatever customer asks and then it kind of generates so one thing what I can see is I have a blog on that so it's on medium AI with
an oven Christian so you can refer that blog so that blog is completely about converting your natural language to SQL queries and then running against database and then doing the results from so that in that I have covered from stop to bottom like if you are zero
and then if you want to understand completely and you can go read that blog so this is not a advertisement so this is just information for you so it to make your life easy so other thing is how do you prepare the data so as you said right so if you there are certain
situations where people have the database sitting there with the relational in the relational world and still those are like on 90s and 95s and 2k databases they don't have they have relations still but they don't be very they are not schema oriented and sometimes service
system cannot and may not be able to understand what their column names are and how this is so in that for those cases we can I have seen people creating some views out of it special views and then the views what they do is they filter out which data is needed and which
is not needed and then they combine that and then bring it as one single view of your complete database and they write on they do or messaging or on top of the view that's why they are getting a lot of they you avoid a lot of issues like SQL injections or running
some telet queries as based off UI right so that's one approach and other thing is there are certain God rails that are available where you can put in place and make sure that your data is secured right sometimes if I run a query if I ask the customer can you go if I ask the bot to tell
can you go and delete my database so it's not going to do that so we can restrict it to do us from doing any dm-al statement just to do only the ddl statement like it can do selects and things like that also you should not show that your complete table schema it's a give me the
structure of the table water putting for it we should be out of very careful in those things so you have to put a lot of prompts in there and then a lot of fine tuning and then validation and things like that will help you get rid of those problems and also the
user license like what that particular SQL user or the headless bot user can run query ahead of it so those can be controlled in your clouded by many ways by setting up access control and that will definitely help you solve those types of problems
now what about for data that I would turn it is a little bit more messy so when I think about like a law office right that's a lot of case management stuff that's a lot more text than it is they like what is the data preparation look like when it looks more like that
oh yeah sure for those types of for those types of data right so where there will be a combination of both in thus in those case what I would say is prepare a data pipeline so create a data pipeline and then source this data which is whatever you have and then make a pipeline
and then make it make kind of you have to do a ETL pipeline maybe or ELT pipeline whichever works based on your data and use cases run it through run your data through and then try to segregate all your text based like if I have a long text and I wanted to add some search on top
of the text then you have to bring it to kind of what you call it you have to invert those and then vectorize those and then save it in your Azure AI search so that that can be pulled based on the search that you can search against those types of columns and if it is keyword and hybrid
there are two types of searches where you have got keyword and then hybrid search you can go through and delete those articles around that AI search so this can help you get the data what you want so that's one that's one good capability what that product has got so make use of that
and make sure your data pipeline is not vast or something like that where you start with the very minimal solution and then start create a pipeline and then try to segregate all your data and then make sure this data goes there and what data goes to such a and what data goes to unstructured try
to segregate them all and then keep the pipeline established then you can add on top of it right that don't just dive deep dive into it and then create a wild big versatile pipeline and that kind of time to solve all this that's going to you will end up doing nothing at the end so I start with
a simple solution and then try to tackle those and then once you have that you will have we will get a confidence like what is needed and how to handle different types of data also there are a lot of forums and a lot of solutions available online we eat a lunch or be it semantic and all those are
the frameworks which are readily available make use of them those will give you a lot of if there's those will take a lot of jobs from you and you don't need to code everything so those frameworks are built for those purposes make use of that and then try to get best sort of this.
Now obviously somebody works at Microsoft you've got a couple of different tools that fit different problems give me a little bit of a sense of the way you think about the difference of where something like your AI should be applied to versus where something like co-pilot should be applied to because in theory they're in theory they're both working on data sets in similar manners how do I differentiate between kind of those two solutions and maybe any others that I
out of fact are in. Yeah sure very nice question so I hear this from a lot of a lot of my friends and a lot of my relatives from several I meet so yeah it's I wrote a blog on this as well so how do you differentiate and what are the different types of co-pilot so first let's talk about co-pilot
co-pilot is something but you're think that as your assistant which is already pre-built right and you can plug it into a different types of data source and then that can get the data from so that's all about co-pilot so it's readily available there are different types of co-pilot one is windows
co-pilot where you can see it in your database other one is office they saw you see it in your laptop other one is office 365 co-pilot where it kind of integrates all your office 365 components beat one drive share point it kind of pulls the details for example if you wanted if
you are if your employer has office 365 co-pilot and you wanted to search something about HR payroll or others of those can help getting those details if it is plugged into those readable data which is accessible by you right if you have got that access it can definitely go get you
and then it can also go through your emails and then try to so I sent that was a chat sometimes back to a manager I don't know his name I don't know what is about but the chat is all about this so can you go ahead and get these details from later it can do you will show you go so
it's chat and dig through those types of things will help you for sure and those are all co-pilots and there are several others as well I don't have time to cover all this there are security co-pilot there are dynamic 365 co-pilot and that so keep that aside and coming to AI right Azure AI
means it's it's nothing but where you want you have your own idea and you want something to do with that right so I have a I have a complex problem where I have where I'm getting my product is not a product is getting a lot of support calls and I wanted to reduce it with the self-service
assistant kind of thing so I which bot can I go so that that's when you will have to build yourself having all your project related documents for like FAQs, READMEs and other things fed that in and then try try to kind of develop that it's not going to take month long to develop those use
cases within like five to ten clicks with your Azure portal go to your Azure portal and then spin up your choose your model what you want and then tell your model that this is my data and that works based on that so there are all the tools what Azure has got so those as cutting port
vectorize and everything by default which takes care of a lot of your what I don't need to do much so with some clicks you will be able to get those quote custom co-pilot kind of thing out so which can be and what I didn't do in our front and keep it running so that's one thing what I would say
so what's the future look like if you think about the next say let's say only 24 months because we don't we know that like far out is going to be very very difficult what do you think is going to be happening over the next 24 months that you know partners that are trying to implement this
should be aware yeah so currently I see a trend around agents so and a lot of there were a lot of what you call problems a prior one say problems there are a lot of work rounds what I have been doing to tackle some of these things like my model as this limitations in terms of token it can
acts upon 28k tokens all right I run out of tokens every time I get what 29 and what do I do so that's why we have got prompt caching so in next at least six to eight months what I would say is instead of coming from big new you know patients happening around that I would say something
uh some they'll be focusing on addressing the problems what they have faced and the work rounds what we are implementing to cut them out and bring in a straightforward solutions where we can like prompt caching and then a lot of things which has got evolved in the recent open 8 leases if
you have here right so those things will definitely help you achieve those types of things what you can see and agent and multi multi model agent or agent tick framework so these types of words you may start hearing this very often the agents are the one as I already said agent are the ones
which are going to do the work of AI for you so for you have been listening to AI now AI starts listening to you and then you say it something and then it go to that job for you so those types of things will start evolving in next eight to twelve months is what I see that's that will
get definitely get picks and people will start developing their word agent like this is my sequel agent this is my documentation this is my that agent this agent so you can see agent library like what you see a new get package around that I do sit usually we'll see a start
at seeing agent library or somewhere where you can go plug that agent from month and start working with it so that's the innovation what I see around that and maybe the latest versions of GPT I don't know what what they're really come so I'm saved like you I don't have any other knowledge
apart from what you see in the market right now so yeah let's wait and watch be allowed here to watch Navin christan is an AI architect at microsoft without a decade of experience specializing in leveraging artificial intelligence and cloud technologies
to solve real world problems across various industries you mean thanks for joining me today thanks see you then are you ready to get your brand in front of the tech leaders shaping the future of managed services here at the business of tech we offer flexible sponsorship opportunities to
meet your needs whether it's live show sponsorship podcast advertising event promotion or custom webinars from affordable exposure options to exclusive sponsorships are offerings are designed to fit businesses and vendors of all sizes looking to make an impact prices start at just
five hundred dollars per month making our packages a fraction of typical event sponsorship costs be a part of the conversation that matters to IT service providers worldwide join us at msp radio and amplify your message where it counts visit mspradio dot com slash engage today to explore all the
ways we can help you grow the business of tech is written and produced by me Dave sobel under ethics guidelines posted at business of dot tech if you like the content please make sure to hit that like button and follow or subscribe it's free and easy and the best way to support the show
and help us grow you can also check out our patreon where you can join the business of tech community at patreon dot com slash msp radio or by our why do we care merge at business of dot tech finally if you're interested in advertising on the show visit mspradio dot com slash engage once again thanks for listening to me and i will talk to you again on our next episode of the business of tech park part of the msp radio network