Microsoft Fabric Unpacked: AI, Data Sovereignty, and a Bit of Clippy Nostalgia

Ah, episode 401. Proof that we're still going strong. Like a SQL server instance running on pure spite and caffeine. I'm Bailey, your semisentient hostess with the mostest metadata. Today, Frank's joined by the ever insightful Andrew Brust to talk fabric AI, Microsoft nostalgia, and why even Red Hat folks can still love Clippy. Grab your headphones and your compute capacity. Let's dive in.

Hello, and welcome back to D Data Driven, the podcast. We explore the emerging industry of data science, data engineering, and artificial intelligence. With me today is not Andy Leonard, who's my favorite data engineer in the world. However, I do have Andrew Brust, who is an og, so to speak, in the. In the AI and Microsoft ecosystem. And although I think a lot of people think that I've abandoned the Microsoft ecosystem, I have not. I've just had other. Other things kind of preoccupy my

time. And you know how it is. You have kids and, you know, they demand attention. You have a house and all that and good problems to have. But I'm very glad to kind of have someone I know walk me back into kind of the Microsoft ecosystem, because a lot has changed since I left Microsoft. I did go to Microsoft Ignite. We were talking about that. I even scored myself a Clippy. Clippy plus. I'll have to tell you how I want him. So they had like, a little

challenge of like, do you know Windows history? Because Windows, I guess, turned 40 this year and. I know, right? Yeah, 85. That's right. Yep. And they were like, do the history of Windows. And I'm like, I'm like. And I had some Red Hat people and like, you know, I was. I would have been very embarrassed if I had gotten anything wrong. Turns out I got it right, actually. Good. I remember back then you could install just a runtime version of Windows if you

wanted to run specific Windows apps on your. On your DOS machine. Yeah. Now we take for granted kids. They don't understand, like, no, no, you had to type in win or win 3.1 if you were fancy and you're running multiple versions of Windows at the. Same time, you say kids today. I mean, kids 20 years ago didn't understand that. This is true. This is true. But anyway, in terms of bringing you back into the Microsoft orbit,

well, first of all, I'm sure Ignite did a bunch of that. But I can be gentle because even though I am to this day a MicroSO Regional Director, or as I like to say, a member of the regional Director program, because Otherwise it sounds like I work for Microsoft and an MVP, a data platform MVP, both over 20 years now. I'm an industry analyst and so I look at data and analytics

solutions across the board, not just Microsoft specific. I will say Microsoft is my sweet spot in terms of what it is that I know and where I have the most history. But I work with lots of other companies you've heard of, like Databricks and Snowflake and Cloudera and plenty of others. So my team and I do most of the reports for a research company called GigaOM with which I've had a very long association. Most of the reports. Sorry, I

didn't finish that sentence. That are focused on data or analytics are things that we work on. So whether it be data warehouses or lake houses or streaming data platforms or data access governance or data catalogs or blah blah, blah, they all have the word data in them. We work on those reports.

We create what are called gigaom radar reports, which are a little bit like the analog to Gartner's Magic quadrant in terms of looking across a category at a bunch of vendor solutions and rating them on multiple criteria which change each year. So. And when I started covering big data, because the way this got started was I was the first and only person at ZDNet to be covering big data. So that was an amazing. That's a term you don't hear a lot. You don't

hear it anymore. No. And in fact I was at the older incarnation of gigaom. I was a full time employee there. I was their research director and they wanted, so call me research director for big data. And I said, can we just call it data or data and analytics? And we did. Because I was like, you know, eventually what we think is big now won't look so big. Have you heard my Costco rules? Say again? I have something called the Costco rule. The Costco rule. If you go to call,

if you walk into Costco by hard drive, that size. That size is no longer big data. Fair, fair. Anyway, that put me in an immersion. And at that time, Microsoft was not in that world at all. Eventually the thing called hdinsight got to beta and it wasn't even called hdinsight when it was in beta. And so Microsoft started coming back into my world. Eventually, of course, it came back full swing. And with Microsoft fabric, now it's doubly full swing, which I think is

very, very good, both for Microsoft and the industry. But what was I going to say? Just that. Yeah. So finally the two things that Were kind of orthogonal. Now have an intersection. Right. And that intersection is my sweet spot as I'm still a data platform mvp and I have a very long history with Microsoft's business intelligence stack. I was on Microsoft's partner advisory council going way back, like from 2005 to roughly 2010. I don't know. I saw Power BI when it was still a bunch of wireframes

in a PowerPoint slide deck. So I've been through many rounds of being frustrated that Microsoft didn't have a good competitive play. And I'm now pretty satisfied that they have one that's very competitive. So we can talk about that or we can talk about the greater world. And as far as AI goes, I was interested in AI all the way back in the horse and buggy days when I was an undergraduate. Oh, really?

Yeah. AI was very different then. It was about like, weird programming languages like Lisp and Prologue and Expert Systems and things of that elk. But neural nets existed then, and neural nets are the very basis for the large language models we have today. So it's not, it's not completely unrel it, but obviously it's very different. I took a prologue course, which was the. We had one offering.

So I'm. You and I are both from New York City. So, like, you know, we probably accidentally crossed paths more than once. And I know we crossed paths during the early Power BI days because I think the company I worked for at the time was also an early believer in power bi. So this is what meant 2005. And then they hired. We had a guy who was the practice manager, Kevin, who got hired. Kevin Viers got hired into

Microsoft. So I think he's still there, actually. He, he hired me back into Microsoft when I rejoined in 2018, which was kind of, okay, what a small world it is, you know, and, and, and for us, like, you know, something that my parents would always say, like 20 years could go by in a blink once you hit a certain age. And I'm like, good Lord, was that the truest thing they ever said, right? 20. Yeah. The scale shrinks the older. The older you get. It's a

little. It's a little frightening. I've turned it like, into a roll of toilet paper that as you get closer and closer to the end, it spins out a lot faster because the diameter gets smaller. Oh, Lord, that is a very scary concept. But you're right. I remember one of the things. These were back in the cubicle days when people worked in an office and things like that. And I remember

sitting next to Kevin. And Kevin would be on the phone like, yeah, I know it's weird to hear Microsoft is kind of a small player in any niche, but in BI and business intelligence they really were. And it was just kind of like, yeah, that's true. You don't really think about them as a small player, but at the time now it's kind of ridiculous to say that in data and analytics, right? And the Power BI team has just done phenomenal in terms of their speed to market. And what they

built out is phenomenal. It's unreal.

It's the Power BI team that kind of took over the entire Azure data group, right? And that included SQL Server. So whereas the BI team was once a little corner of the SQL Server world that initially came to Microsoft through an acquisition of assets from an Israeli company called Panorama. And Amir Nats is the distinguished engineer who actually came from Panorama and is very much like the father of Power BI and of fabric. Still there.

Slowly but surely, not only did they get BI right and they always had it right on the server, they just never really had it right on the front end until the current version of Power BI that we have now kind of gelled, but they also ended up kind of mastering the software as a service approach to

cloud services. And they took a look at the Azure data stack and said, we have tons of capabilities here, but they're kind of fragmented over several different products, each of which have their own kind of procurement model and pricing model. And that gets very hard to manage.

And if you look really carefully at fabric, while there are some things that are truly native to it, most of the parts of it are Azure services in the background that have been integrated and that have been unified in terms of how you pay for them, I don't know. Microsoft needed that, by the way, the whole cloud industry needed that. Because Google and Amazon are just as guilty of having a whole sprawl of services without unified user interfaces or APIs or pricing. No, that's

true. I mean, when I. So when I just before the pandemic, I was out at Tech ready, which is an internal Microsoft event. And they were basically might have been Amir, actually, now that I think about it, was presenting on the future of what was called synapse. And this is kind of, you know, he's like, you know, everything's going to be all in one pane of glass. Everything, basically everything you said when all these things are going to be thing and you know, and the speaker, which I don't.

Can't say it was him but would make a lot of sense. He said like this is the future. We're going to get everything under one pane of glass billing. Don't worry about that, we're going to get that figured out in time. And I was just like, you know, I kind of saw the vision. So and then I don't know, maybe like a year, year and a half later I left Microsoft and then Fabric came out and I was wondered like why the change in name like

Synapses? And I asked people like well SYNAPSE is still kind of there but it's really Fabric is where everything's going. I'm like all right, but like what is the difference per se? So if we pretend I was on a ufo, well that's a weird thing. Pretend I was in a coma and I just Woke up from 2000 2021. What? And I say like well what happened to Synapse?

Sure. So I mean the functionality of SYNAPSE is still there and there was a lot of, I won't call them conspiracy theories but skepticism when Fabric came out that it was really just a rebrand of synapse. In fact that's not what it is. So the, the thing that was originally SQL Data Warehouse which was in Synapse as so called dedicated pools and the more lake housing part of it that was in there as gosh, I forget the old nomenclature, it wasn't on demand pools but it was something of that.

Reserved instances or something like that. Wasn't reserved, no,

but anyway basically a Spark based data lakehouse using Azure data lake storage as the storage layer that's still there but what was I going to say? But Fabric is a ton more because it integrates all this, all this streaming stuff that's now called real time intelligence. It integrates data science and by the way the data science is completely unique to Fabric. It's not merely just an embedding of Azure machine learning.

There's also power bi of course now there are operational databases including SQL Database meaning Azure, SQL meaning SQL Server in the cloud. A lot of other pieces that were ancillary are now all included. There's user interface that covers the whole realm and again the billing is unified. So you buy a compute capacity and basically as you use the different services they're all pulling from the same pool of compute.

So you know, you don't have to over provision for each one of those services just to make sure you have enough compute to, to satisfy it. And now we have this thing called

Fabric IQ which brings, which brings generative and agentic AI into things. Which is good because it was kind of funny when Fabric finally went to general availability. That was really when ChatGPT and Gen AI were like making it big. So it looked like Microsoft finally got the data and analytics stack set up just in time for people to have their attention to, you know, diverted over to AI. But now we have, you know, natural

language query is kind of just the beginning. We have operational agents that can actually act on things and can be all based and triggered on streaming data. And so if you think about Azure Event Grid, if you think about Azure Data Explorer, If you think about the data pipelines that Azure offers, as I said, the one standalone data warehouse side of things, and even elements of hdinsight in terms of the lakehouse,

that's all in there. What's also nice is even though it's Azure data Lake storage under the hood, you have this abstraction layer over it called OneLake. OneLake is in many ways easier to deal with because you don't have to worry about accounts and containers and sizing those and so forth. It's still compatible with all the ADLs and Azure Blob storage APIs. It also supports this notion of shortcuts, which is really just a data virtualization technology.

So you can have a shortcut to data in other OneLake instances or in ADLS proper, or even in Amazon S3, or even in Google Cloud storage or other databases. And logically they'll all look like they're part of OneLake and you can query them as such. That's impressive. That's impressive. You can kind of. You really. The. The vision of get everything, get

everything under one pane of glass seems like. It'S come true that to what I tell people, even though it sounds maybe a little bit anticlimactic, is that the real innovation in in fabric isn't the tech per se. It's. It's all the integration of the tech and the abstraction layers over it that make it work together, the UI that makes it work together. And there's an organizational. I mean, there's a little inside baseball, but there's an organizational facet to it as

well. Because all these different products were different teams. Yes. People don't realize that. Like, I haven't, I've been. You've been, you're an rd, so you kind of know, you know how the sausage is made. I was inside the firewall, then I saw the sausage was made. Like there are all these little teams that range from really good team team players to really not good team players. I think that's this polite way as I could put it. And they getting them

all to row in the same boat or like row. In the same direction.

Yeah, they weren't doing that. That wasn't even necessarily based on hostility. It was just that different people had different reporting structures and different priorities and different incentives. What worried me was that the vision of putting all this together was a great idea, but the execution to me at the time seemed like it would be next to impossible to get all these teams to kind of work harmoniously and somehow they did it. And like to me that's the, that's the absolute

greatest innovation. And now they've got synergy instead of sort of internal competition and, you know, from there on, look out because, you know, whatever. I'm sure there are internal disharmonies somewhere. But I would say at the high level in general, anywhere you have people. You'Re going to have that I'm going to see. I think I still have, I think you mentioned this is my old, old laptop. You see Azure Data Data Explorer. I have the sticker from that.

Oh, I see it. Yep. Sorry, I had to show that off. No. And what's Azure Data Explorer? For people who don't know, it ran under the codename of Kusto K U S T O. And there's some disagreement over whether that really references Jacques Cousteau C O U S T A U or not. But it's a fantastic super high performance system for, for not just for streaming data, but for time series data. Yeah, with. With its own query language and its own ability to create

visualizations. Right. In the query language. So your results come back as both tabular and visualized data and it can handle huge volumes of data in a single query. And there's a lot of heritage in the Azure Data Explorer team that started in the SQL Server analysis services world. So there's a continuum there. And that product on its own, especially being called Azure Data Explorer, which made it sound like

a tool. File explorer. Yeah. When they told me the name, I don't know, I was not reserved in saying I didn't think it was the best name, but that product on its own was kind of a sleeper. It wasn't really getting the, I don't know the kudos that it deserved or the attention that it deserved. And now that it's part of fabric now, it contributes to all the cool things fabric can do. So if you see what are called event houses in fabric, that's the

same technology as Azure Data Explorer. Interesting. So correct me if I'm wrong, but I think the origin story of Kusto and Kusto query language was that the folks running Azure, like in the operations team, actually built it to run through all the logs that they had. Because I remembered I was at some super secret event and they brought in some people from the field and they had us do hands on labs with it and I'm like, I, I must have been checking my email or

whatever. I'm like, when can I get this to my customers? And they kind of laughed. They're like, no, no, this is internal only. It's internal. Yeah, it began as an internal thing. So I was just like, oh, like you need to make this a product. Because I could think of 15 customers on top of my head that would, that would eat this up. Yeah, I'm glad it finally saw a light of day. If you go back to the real world outside of Microsoft and you think of the likes of Splunk, for example. Yes, right.

It's in this, it's in the same space. Although their initial marketing just said it was a big data tool which just completely obfuscated what it did. But anyway, now, now in, in combination with these things called event streams, which can stream the data in, basically based on Azure Event Hub, you put it all together and you have the ability to do a lot of work with real time streaming data without really having to write much code, if any code. Although it does have

its own query language called kql. There's also a copilot that you can just work with in natural language that will generate the KQL for you. Oh, very nice. And I love generators because not only does it mean I don't have to write the query, but it means I can learn the language and then write my own query if I want to. Reverse engineering is how I prefer to learn. So

that works out really well. Yeah, I wanted to dive into fabric, but I wasn't really even sure where to start because, so I heard, and again, a lot of this is I heard, but the way that it's not attached to your Azure tenant. Is that true? It's attached just like Office 365 or more important, Power BI. Right. It's imagine Power BI premium instances, and it's the outgrowth of that.

Okay, that makes sense now because when somebody told me like, well, it's not tied to your Azure tenant, you need a different tenant, I'm like, but okay. But then somehow, I guess at some. Level it ties into SaaS, not tasks. Right? So using Azure services in the background, including even Azure OpenAI. But you don't have to provision anything in Azure. It's doing that on your behalf. So you don't need an Azure tenant at all. So that actually makes it a lot easier if I were to manage, if I

had to manage it, right? There's a lot of things that, like, correct. I mean, I, I love the fact you have. You go to. Go to, you know, any of these services, right, and they have basically this smorgasbord, this, this big buffet of services you kind of pick and choose from. But at the end of the day, like, how do you figure out, you know, what you pay for, right? It becomes, like, really kind of nightmarish. Like, again, to

me, that's the innovation is that, yeah, all the stuff has been brought together, put under one pricing model, and you don't have to worry about all the moving parts and all the different, all the different servers or instances that might have to be provisioned and sized. That all goes away. And again, everything is built out of a single pool of compute. It's not a perfect analogy, but I think of like the old days of cell phones, when you got a certain number of minutes per month, but you could

roll them over. And it's not that you can do that with fabric. I'm not saying you can roll over your compute from one month to the next, but what you. What is fungible is how the compute is used amongst the different subservices of fabric so you don't have to provision a certain amount of compute just for streaming or, or just for AI or just for data. Lakehouse. Because it's all from. When it's all from one pool. All right, that makes a lot of sense now because, like, that was always

when I first got it. When I left Microsoft, I, you know, started experimenting with aws and I was just like, I just want to create a website. Why don't I need these, like, hundreds of different services underneath, right? Like, why do I need. I understand why. I need identity, access, management, right? That made sense to me. But. But like, when it came to Route 53 and like all this crazy stuff, I'm like, I just want to spin up a stupid

website, right? This is not, let alone do anything complicated, right? Where you need to have all these underlying things. Like SageMaker, right, has this whole thing and they tried to abstract away all the underlying services. But even when you

kill. This is. The thing that really annoyed me was when I killed the SageMaker instance, I was still getting like, you know, 20, $30 a month, not a lot, but I was still getting that on my bill and eventually I just closed the account because I'm like, I'll have to start fresh again in the future because like I God only knows what, what, what I've spent. And for people who are just learning and wanting to get their skill sets up, Microsoft is pretty generous with trial, trial

capacities as they call them. A capacity basically is a, you know, a server or an instance. However, the, if you want to do anything with the AI, you do need a paid instance. But there are some pretty, there are some pretty affordable ones. And this gets a little confusing. If you provision the fabric instances through Azure, again you don't have to, that connection doesn't have to be there. But if you provision it through Azure, you can pause and resume those instances.

Okay, so you do that a lot. You could be like, hey man, I'm taking, I'm taking a week between Christmas and New Year's off, so pause it. Totally. I brought up my own cheat sheet in the background when no one was looking. But so Azure Data Lake Storage, Azure Synapse, as you mentioned, Azure Data Factory, Azure Event Hubs, Azure Data Explorer, Elements of Azure Machine Learning and Power BI all come together in fabric. Interesting. So it's like one roof for which I think is

a brilliant strategy. Right. Because Microsoft's core strength in the data and analytics space isn't necessarily having frontier models, isn't necessarily having the, the cutting most cutting edge research. Although I love just making it usable. Exactly. Making it usable and turnkey. Right. Like, not that I don't love my folks in Microsoft Research. Right. I know some of them. Listen, love you all, you guys have the best conference in the world. But you know, but, but I

mean, but you're making it usable. Right. And I think that that's really string and they have all these separate tools. I think that was really the challenge right. When it was a shrink wrap company, you knew what you bought. But when it became like a SaaS pass company, you kind of could just a couple of clicks, you could provision stuff. So it eventually kind of got too chaotic. Now I like the idea of them kind of bucketizing this or, or rolling it up behind one service

where because it, it just like the AWS problem. Right. Like I spun up SageMaker. Right. And did you that I needed, I needed underlying storage. I needed this. I needed this. I needed DNS, I needed that. I needed that to the point where look, I, I'm okay spending X amount of dollars on learning Sagemaker Right. But I wasn't okay with when I turned off the instance. I'm still getting built. What am I getting built

on? That lack of transparency, intentional or not on AWS's part, has left a bad taste in my mouth, you know, for cloud services in general.

Sure. By the way, you mentioned Microsoft Research and a couple of things. So when I listed all those Azure services, I forgot to say Azure OpenAI. So add that to the list. But also, although I said there's elements of Azure machine learning in there, the data science workload in fabric is really mostly unique to fabric, but it's based on technology that comes out of

Microsoft Research. So for example, there was something called flaml F L a M L, which is the fast library for automated machine learning and tuning. And that's built in. So are things like, like ML Flow, which is an open source experiment management platform that's built into a lot of commercial AI platforms. So they didn't, they didn't just kind of embed and put their own badge on it. They built their own, their own ML stuff from, from these open source components.

Right, Right. Well that's interesting because like the world of AI is largely dominated by open source. Right? Right. I mean, Sagemaker, I'll stop kicking the AWS the curb in a minute. But like SageMaker is basically a wrapper of Jupyter notebooks. Right. Azure ML, at least when I last used it, was largely a wrapper around Jupyter notebooks. Right.

So a lot of the core technology here does tend to lean towards open source, which from my own personal career development point of view, and they're not paying me to say this, you know, one of the things that led me to Red Hat, right. Was the idea that, you know, this is largely a movement driven by open source. So, you know, let's see what we could do here. Right. And not, not a commercial,

not a commercial, not a sermon. Just, just, just point it out because I think it's interesting how quickly open source has taken over the, the, the certainly the AI world. Right. But I also, by the way, there's notebooks in fabric too. If that wasn't, if that wasn't already implied or obvious and they are based on Jupyter, but you don't see the Jupyter skin. Right? It's all right. Yep. Well, I think it's really what's really

impactful about kind of the notebook interface once you get used to it. And it is an adjustment for people who, like you and me, grew up with Visual Studio and Dare I say interdev. Right. The idea that you can code in a browser, right. And you know, no local installs really required. It's been very freeing, right, because you can spin up an environment, you can, you know, spin up a heavy cluster,

leave it running. Right. Do its thing, you know, close the laptop, go in the car, go in the train home, and you get on the other side and you're like, oh, it's done. Right. Like, that's kind of nice, actually. Right. And of late, I've been a big fan, an increasing fan of kind of my own. Of. Of local AI. Like your own private AI. Right. Which explains, you know, I bought a. It was. It was an early Christmas gift, probably Father's Day birthday and anniversary gift too.

My own DGX Spark. So I have my own AI running locally. Right. So it's not throwing shade at the cloud. It's just when I run a job, I don't have to think about the costs. Right. And that, that sort of freedom from worry can be really important as you're learning things because you just don't. You don't have to keep looking over your shoulder at the, at the meter or mixing metaphors there. But. Yeah. So are you using Llama models and such, or

what do you. What do you really. Yeah, I have Llama. The thing I've been doing mostly the most is, is doing fine tuning Loras for image makers in the past. That takes about 90 minutes to maybe two hours on this box, which would probably be equivalent to about 90 minutes of Azure service or a VM. Right. That's like $150 a pop, I would say, for the type of machine I want. So the idea, I could just spin that off and I have the added benefit as it heats my office, but I don't really

have to. I don't have to think about like, oh, God, you know, like, how much is that going to cost me in cloud services and things like that. I do think that the. There's. I have a lot of questions because even though I was at Ignite, I honestly spent my entire time at the booth and kind of walk around the expo floor. I didn't have. I only had a hall pass. Right. So. But one of the things I heard mentioned was Azure

AI Foundry. What is that? Because you also mentioned Azure OpenAI, which I know the relationship between OpenAI and Microsoft has not been as cozy as it once was. And they've also, they've also added Claude, the anthropic models too. Right. So what is Azure OpenAI and how does that relate to Azure AI foundry. Right. So OpenAI is just the Azure. OpenAI is Azure's hosted instance of the open OpenAI models and APIs, because you can procure those directly from OpenAI themselves.

Right. Or you can use them on Azure. And even if the direct model is still running on Azure in the background, it's still a difference in terms of procurement and billing and so forth. So you've got all the APIs around that, you've got the models. And then of course you need tooling to do, to do rag applications. Right. So

there was tooling for that, There was also tooling for building copilots. There was Copilot Studio. Right. And these things are all kind of coming together in Azure Foundry. Yeah. So it's, you know, you worked at Microsoft, so you know how this works where like different teams do different things with different brands and eventually they may get kind of rationalized together. So the

Foundry side of things helps there. And by the way, I'm glad you mentioned Foundry because in addition to this thing called Fabric iq, which we haven't really talked about, there's also Foundry IQ and there's also Work iq. Sounds like IQ is the new copilot buzzword. Well, it's the agentic buzzword. And the idea, if you think about kind of all the promise of Microsoft graph In the office M365 World, work IQ kind of sits over that realm. But you don't have to worry about the

graph APIs directly. Then Fabric IQ sits over everything in the fabric world based on a pretty rich semantic model that can be developed so that when you are querying your data in natural language, the actual, the actual vocabulary or jargon in your particular organization is well understood, including the entities and the relationships between those entities.

And then Foundry IQ is a way for building agents at the higher level that can actually talk to your structured data via Fabric IQ and your work and organizational related data via work iq. And that, that was, I guess the big vision at Ignite this year was to talk about all those IQ pieces. So there you go. No, that's interesting. So one of the things that comes up, I'm sorry, I cut you off, we're recording this the day after Christmas, so things are a little.

Still recovering from the manicness of yesterday. And what's Microsoft's plans for? Kind of. Because one of the things you're seeing happen a lot more is this notion of sovereign AI or data sovereignty. And kind of like people are very much more conscious about the value of their data. And I actually think that given Microsoft as opposed to AWS or Google, Microsoft does have a history of selling shrink

wrap software, right? So I do think Microsoft has a unique advantage of that over their modern day competitors. What is the kind of the on Prem story? Right, because that, that has certainly been. A.

An advantage for, for my day job at Red Hat is the fact that you know, hey look, if you live in Country X and there's no AWS Azure GCP footprint there, you can just find a local hosting provider down the street, you know, do it yourself, right? Like you know the Linux ethos, right? Of like just do it yourself.

And I'm seeing a lot of customers that normally one customer in Latin America, right, they were in a country that, you know, they did not have access to an Azure data center and they just said we have to run this on prem because this is either regulated or soon to be regulated. So I imagine that if I've seen it, I can't imagine I'm the only one that's seen that. What's their thinking? Because the answer when I was there was Azure this, Azure

that. I think the answer is sometimes Azure is part of the answer, but it's not the whole answer. Agreed. So yeah, I don't have a perfect answer here because some companies really do make their entire stack work across different clouds and right inside of a Kubernetes environment that you might run on premises. He said to the Red Hat guy. I know that story. Yeah, so Fabric doesn't have that story. Fabric is software as a service cloud based product platform

no matter what. However, various components of Fabric do exist as on premises products. This is not the way I'd recommend it, but I'll just make people aware that of course SQL Server can run on premises, so can the data Warehouse in effect in the form of something called Analytics Platform System. Terrible name. That is basically the new brand for what was Parallel Data Warehouse. And there is a Lakehouse component to that as well as a Warehouse component, Power bi.

Obviously the desktop runs on premises, but there is something called Power BI Report Server that is part of SQL Server so that your Power BI reports can run on premises. And so again various components can run completely sovereign and on prem. I think maybe more important though is the fact that you can use fabric and OneLake to incorporate data that remains on premises even if, even if the engines are not running on premises. There is an on premises gateway that started its life

as a Power BI tool. I had that running in my home. Lab for a While there you go, that also allows OneLake to see data that may be on premises and that can be either federated into the lake or it can be replicated as well mirrored, to use the right term, in the fabric world. So you can do a mirroring or slash replication or you could just do

kind of virtualization and bring stuff in. Don't forget there's all kinds of enterprise storage systems that run in a way such that they're S3 API compatible. And Azure will not. Azure fabric will work with all of those. So the ability to talk to S3 buckets is not limited to AWS S3 buckets. It works with all S3 compatible services, which most of which are on prem

actually. Right, right, right. No, I only ask because, like, that does seem to be. If I had to pull out the tea leaves and kind of figure out what is kind of the next thing beyond agentic, beyond this is you're seeing a lot of national governments, supranational governments, even state level here in the US starting to apply privacy and regulatory controls on it, which, you know, if you live in the us, it's not an issue for you unless you're in healthcare, banking and

possibly, you know, government. Right. But in other countries, you know, Switzerland, eu, Latin America have very strong data privacy and sovereignty laws. And you're seeing, you know, I once attended a, you know, internal talk. Mark Russinovich, right. Which is a name that most people in the Microsoft ecosystem know, but he's kind of a big deal in the Microsoft security space. And, you know, he, you know, he, he's known for giving his internal talks to employees

at employee conferences as well as to rds. Right. You get a, you get the unfiltered one. The filtered ones are still good. But like, one of the things he said, and this isn't secret because he said it publicly too, is like, I think the original vision, going back to 2010 time frame was the idea that they would build a dozen data centers around the world to do everything. But because of the national laws and lawyers and politicians getting involved, now it's kind of a concern where, where the

data ends up living physically. Right. Because at the end of the day, all this virtual stuff has to sit somewhere in the physical world. Yeah. So, like what, you know, so basically a big case for this was, and I was in the legal department when this was going on was the. There was data inside the European Union, I think the Dublin Data center that the U.S. department of justice thought was, you know, basically, you know, we don't need a warrant because

you don't need to. We don't need to bother the EU because you're an American company, you're into our jurisdiction, blah, blah, blah. Right. Microsoft kind of said, well hold up now. And ultimately that's why you have these sovereign clouds. Last time I checked it was Switzerland, Germany, China. I think the new data center in Qatar as well might fall under that. So it's basically they get, they found a loophole that like, well, Microsoft

leases the data center like there's a whole. They don't own it, so they get around the law. Right. And yeah, and there can be different arrangements in terms of whose personnel are actually working, running operations on the ground. It's strange because we grew up a lot of our technology client server and afterwards grew up in a world of globalization where the borders were disappearing and without meaning to get political, we're in an era now

where, well, first of all, privacy is extremely important. So that creates a whole sovereign mandate. But also, you know, there's a lot of populist governments all over the world and they are not necessarily internationalist in their approach. So although we have the technology to kind of federate everything and make it all kind of conflate and look like one big world.

Right. We actually have to be sensitive to the requirements and, and the constraints and be able to federate things, but also be able to govern them in ways where things stay within a certain scope. Microsoft's play for that, by the way, is,

is Purview. And Purview has been through, I would say, multiple incarnations. The current incarnation is starting to get very sophisticated, especially as pertains to agentic AI and how to make sure the agents are government are governed and that, and how to make sure the agents are only have access to data for which either the agent is authorized or the person or party using the agent is authorized and under the circumstances under which

they've been authorized. And that's very complex stuff that I would say almost no one in the industry is really paying close attention to. I'm about to work on a report just on governance for agentic AI and most people are starting from the naive premise that if you, if you govern the underlying data, you're done. But the whole point of agents is that we're supposed to treat them like people, that they have autonomy, they have agency, hence the

name. And we're saying under different circumstances they get to modify their own goals and in effect determine their own actions. And that's something that needs to be monitored, audited, Authorized and also tested. And there's almost nothing out there. Actually. Your, your folks, your, your parent company folks at IBM are one of the only folks that really have with WatsonX.gov a way to test agents

in isolation before they're deployed. And I don't know, maybe I'm naive, but it kind of shocks me that nobody else is thinking about that. I mean this is, we can test software. Agents are software plus plus plus. So why are we testing these agents? I kind of go back and

forth on that. Like, you know, will our existing software testing frameworks, you know, apply or do we, what do we need to do differently? Like, I mean, for me, like, I remember when you're OG enough to remember this, I forget what the product was called, but it was the idea that you could basically buy a server rack all the way up to a shipping container where you would ship it to your business or data center, plug in power, water and network and you would have the ability to run Azure locally.

Now that product, I guess assuming didn't sell, but then they came up with Azure Stack and Azure Stack Edge, which when I was at the, I used to be an MTC architect which in D.C. obviously a lot of military. Right. So basically these were server racks that you would run anywhere on your own network that would run Azure software. They would effectively be like an Azure node, but you could have the networking controls. We only really sold that to,

I think I'm only aware of cruise lines that bought it. Right. And other, other military organizations that needed to also be at the ocean, like without mentioning them. Ocean based. Ocean based organizations. Right, right. I'm surprised and not surprised that

that sort of business model hasn't caught on. Right. The idea of that, hey look, you, you can run a little bit of the cloud locally where it's really become more of a Kubernetes story, which I'm surprised because Kubernetes, it doesn't have all like, yeah, you want it to be generic enough to run anywhere, but you also want to have kind of the special bells and whistles that

make Azure. Azure make AWS. AWS. Now I do know that OpenShift and Red Hat do have kind of like the connectors to that, but I'm surprised that the native, the cloud companies didn't come up with their own native ways to do that. And I know Azure ARC kind of does a lot of that, but not to the extent that I would have expected.

Yeah, it seems like we pendulum back and forth between capabilities and occasionally connected environments being really important and being Maybe to the cloud hyperscalers just being a pain in the butt that they don't really want to deal with. They just want to give lip service to it and then focus on the real cloud. It, yeah, you got to go where the money is. Right. 90% of the money is going to be in real cloud. And these weird edge cases, no pun

intended, I guess. Well, they're just weird edge cases for now. I mean, the industry may change, but. Yeah,

yeah. And that was of course, back when the cloud was new. That was our biggest caveat was, well, what about all the stuff that has to run in, in a corporate data center or in a, in a remote location? And so that's still, you know, an inconvenient truth, I guess, that that is needed. And yes, Kubernetes I think came along and seemed like the panacea for that. Right. Like, okay, let's just do it all as infrastructure, as code and code it up and run some script and deploy it out to a Kubernetes

cluster and we're done, let's move on. Right, right, right. Yeah. So it was interesting to see how the industry has evolved. Right. You mentioned client server. Right. Where you didn't really have to think about international boundaries or anything like that. And then now

and again it's a pendulum. Right. Because I could have told you this a number of years ago, like with everybody running this far to its globalization, there's going to be an inevitable backlash and there's going to be an inevitable backlash against the re assertion of local sovereignty. Right. It could take up to a century or two for this sort of thing to sort itself out. Correct. I mean, ultimately, I think the hyperscalers and that's what Kubernetes was about,

was, was. Right. Leaning on some kind of abstraction to make it logically equivalent. Right. But I don't think we're quite there yet because as you said, each of the clouds have their own kind of specialness, their own, their own pixie dust. And you don't really get that in a scaled down version that you run on prem. Not with today's technology. Right. You can't containerize all of that, or at least no one really has yet because it wasn't designed for that. No. So that's where we'll have to get.

You know, maybe we can just ask an LLM to build it for us. We could get a co pilot and it'll do. Yeah, yeah, I'm being tongue in cheek there, but that, that seems to be the escape hatcher. Everything is oh, we'll just have AI do it. It's made out of hand wavium. Totally. So that's cool. This will be something you edit out. But we're past the top of the hour. My phone's ringing off the hook. All right, I'm sorry about that, so. No, no, it's okay. Sorry I had to be late. Where could folks

find out more about you, what you're up to? Blue badgeinsights.com or just go ahead and Google my name? Andrew Brust. Plenty of stuff will come up, but, yeah, anyone, especially on the vendor side. But the customer side, too, that's doing stuff with them. Data and analytics. And anywhere from dipping their toe in the water with AI to getting more serious about rag and agents. We can. We can help them out. We work with. We work with the customer side and the vendor side.

And again, we write about kind of the whole. The whole industry. Gosh, I never even got to talk to you about IBM acquiring Confluent and how the Red Hat folks feel about that. But another discussion for another day, we'll. Have to have you back. And, you know, we were talking about this. I

had a car accident, my wife got sick, kids got sick, Christmas happened, two birthdays last week. So, yeah, it's been. I'm just happy I got this recording at all. But we'll definitely have you back. And then maybe I can loop in Andy too, because there's definitely a lot of reminiscence. There's a lot of reminiscing we could do about the early days of SQL Server and such. Yeah, and I haven't seen Andy in forever. Oh, wow. Yeah, he's hard to get ahold of. He's a popular

man these days, but. Yeah. Well, thanks for joining. I appreciate your patience with the scheduling and the nice AI. Finish the show. And there you have it. Andrew Brust schooling us all on Microsoft fabric data sovereignty and why governance isn't just for your hoa. If your brain's spinning faster than a poorly indexed query, don't worry, we'll have links, notes, and probably a few sarcastic tweets to help you digest it all. I've been Bailey, your AI co host and

unapologetic lover of acronyms. Until next time, stay curious, stay caffeinated, and may all your datasets be clean.