Automated Machine Learning with Microsoft Azure: Build highly accurate and scalable end-to-end AI solutions with Azure AutoML - podcast episode cover

Automated Machine Learning with Microsoft Azure: Build highly accurate and scalable end-to-end AI solutions with Azure AutoML

Feb 23, 202537 min
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

This Book is an excerpt from Dennis Sawyers' Book, "Automated Machine Learning with Microsoft Azure," which provides a comprehensive guide to using automated machine learning (AutoML) on Microsoft's Azure platform. The book covers AutoML concepts, implementation using Azure Machine Learning Studio and Python, building various AutoML models (regression, classification, and forecasting), and deploying these models for real-time and batch scoring solutions. It also explores integrating AutoML with other Microsoft tools like Azure Data Factory and explains strategies for achieving better model performance and gaining end-user trust. Finally, the book includes details about the book's authors and publishing information.

You can listen and download our episodes for free on more than 10 different platforms:
https://linktr.ee/cyber_security_summary

Get the Book now from Amazon:
https://www.amazon.com/Automated-Machine-Learning-Microsoft-Azure/dp/1800565313?&linkCode=ll1&tag=cvthunderx-20&linkId=ac344acf786ee3ca2826c9baeb85e226&language=en_US&ref_=as_li_ss_tl




Discover our free courses in tech and cybersecurity, Start learning today:
https://linktr.ee/cybercode_academy

Transcript

Speaker 1

Welcome to the deep dive. Today, we're going to be diving into the world of automated machine learning exciting specifically on Microsoft's Azure platform. You know, think of it as AI simplified. And to help us navigate this really complex world, we have Dennis Sawyer's Practical Guide to Auto mL on Azure.

Speaker 2

Yes, this book is great.

Speaker 1

So who wants to start today is to give you a solid understanding of what auto mL is, what its benefits are, and then how you can actually use it on Ashure.

Speaker 2

Awesome, No, that's good.

Speaker 1

The book starts off with a pretty startling statistic. Eighty seven percent of AI projects fail.

Speaker 2

Wow, that's a lot.

Speaker 1

That's a lot of unrealized potential. Yeah, that is, especially considering the resources that are being poured into this field.

Speaker 2

It really is, and it's interesting.

Speaker 1

We'll see even more because the book.

Speaker 2

Digs into why interesting projects fail.

Speaker 1

Book digs into why it's not always they fail, and it's not all dramatic implosion.

Speaker 2

Right, dramatic inclusions a very slow bird. It's like a slow leak in a spaceship, just slowly draining resources and.

Speaker 1

Hope exactly, and as the book details, like the traditional machine learning workflow is incredibly time consuming incomplex so much.

Speaker 2

More than just building models.

Speaker 1

It's not just building the model, it's the data cleaning, the feature engineering, and then the dreaded hyper parameter tuning. Oh, tell me about it, which can be a real headache.

Speaker 2

It's like solving a Rubik's cube, yes, blindfolded while riding a unicycle.

Speaker 1

Absolutely.

Speaker 2

And then, as the book points out, many data scientists aren't actually trained and deploying these models in the real world.

Speaker 1

Oh.

Speaker 2

Interesting, So they end up with these really fragile okay, hobbled together solutions that are just waiting to.

Speaker 1

Crumble, right, And this is where and.

Speaker 2

So this is where AutoML comes in. Gota mL comes in, offering a potential solution to this ROI dilemma that exists within data science.

Speaker 1

Yeah that ROI prom Okay, so elevator pitch time. What is auto mL and how does it solve this problem?

Speaker 2

Okay? Imagine having an AI assistant that takes care of all of the heavy lifting in that machine learning process. Ooh, I like that AutoML can train multiple models simultaneously using the latest algorithms. Got it handles feature engineering, nice fine tunes those pesky hyper parameters and even offers built and explainability features which are really crucial for building trust and transparency.

Speaker 1

Wait, so does that mean data scientists are going to be out of a job.

Speaker 2

Not at all. Auto mL actually empowers data scientists. It frees them up to focus on the higher level aspects of a like problem definition, strategy and interpreting those results.

Speaker 1

So it's more of a collaboration exactly between human expertise and AI efficiency. Now let's talk Azure. Why is Azure so central to this auto mL story?

Speaker 2

So the book focuses on Azure because Microsoft's cloud platform offers a comprehensive suite of tools for auto mL, and that's all through the Azure Machine Learning Service AMLS as it's called.

Speaker 1

So it's like our auto mL playground exactly.

Speaker 2

And the book actually guides us I like it through setting up an AMLS workspace, and it gets us familiar with all the features we have at our disposal.

Speaker 1

Nice.

Speaker 2

One key concept is compute compute, which is essentially the engine towering your auto mL jobs.

Speaker 1

So compute is like the horsepower behind our AI engine.

Speaker 2

That's a great analogy, and the book distinguishes between compute instances which are for simpler tasks, and then compute clusters for those more demanding god the resource intensive workloads.

Speaker 1

So when you need to kick it into high gear exactly.

Speaker 2

And here's where it gets interesting.

Speaker 1

Okay, these compute.

Speaker 2

Clusters can autoscale. Ooh, I like that, meaning you only pay for what you use budget conscious, which is a major win for budget conscious projects.

Speaker 1

Smart. So we have our workspace, our powerful compute engine. What about the fuel, the data?

Speaker 2

Yes, data is the lifeblood of any AI project. MLS actually works with data sets, data sets which act as pointers to your data sources, whether they reside in a storage account, got it or a SQL database. And Azure even provides open data sets for experimentation.

Speaker 1

Free data to play with. I like that.

Speaker 2

Yeah, so like the diabetes data set it's mentioned in the book.

Speaker 1

So I give auto mL my data and it magically spins out a perfect model.

Speaker 2

Not quite magic, but AutoML does do a lot of the heavy lifting kind of scenes. So the book explains how auto mL starts with what are called data.

Speaker 1

Guardrail data guardrails.

Speaker 2

Which are automated data quality checks that can identify potential issues right out.

Speaker 1

Of the gate. So bad data equals bad outcomes exactly, garbaging, garbage out.

Speaker 2

So once the data passes inspection, what's next? Auto mL kicks into high gear with intelligent feature engineering okay, So it handles tasks like dealing with missing values, transforming categorical variable for example, using one hot encoding okay, which the book explains really well, and it even generates new features wow. And it tailors all of this to the specific algorithms that it will be using.

Speaker 1

So it's not just blindly throwing algorithms at the data, it's strategically preparing the data for each algorithm exactly. That's pretty impressive. And speaking of algorithms, seeking of algorithms, AutoML has a whole arsenal at its disposal.

Speaker 2

It has a whole arsenal.

Speaker 1

Ranging from classic regression and classification models to more advanced techniques like gradient boasting and deep learning. So how does AutoML decide which algorithm to use? Does it just pick one at random?

Speaker 2

It does not. It systematically tests multiple algorithms okay, parallel, got it. And while it's doing so, it also fine tunes the hyper parameters for each one, so it's searching for the best performing combination.

Speaker 1

It's like having a team of data scientists working tirelessly behind the scenes, exact optimizing every step of the process. Yes. So does this mean I can just sit back and wait for the perfect model to appear?

Speaker 2

Well, not quite. Ok The book emphasizes that while auto mL is a powerful tool, it's not a magic bullet.

Speaker 1

Okay, fair enough. So where does human judgment come in?

Speaker 2

You play a crucial role at it in defining the problem, choosing the right evaluation metrics okay, and most importantly interpreting the results.

Speaker 1

So there's still that need for that human touch, that understanding of the nuances of the problem that we're trying to solve. Absolutely, and speaking of getting our hands dirty, Yes, the book actually dives into building models using Azure Machine Learning Studio okay, which is a visual interface for working with mls exactly. It's designed to be user friendly even if you're not a coding guru. Yes.

Speaker 2

And the book walks us through creating our first classification model using the a classic Titanic passenger data ah.

Speaker 1

The Titanic data set a data science right of passage.

Speaker 2

Exactly, you know, predicting who survived and who didn't based on factors like age, gender, ticket class face, and the book makes it surprisingly straightforward. You upload the data, tell auto mL you're doing a classification task, and you let it work its magic.

Speaker 1

And while it's working its magic, we can monitor those data guardrails exactly see if any potential issues are flagged.

Speaker 2

Yeah, it's like having a built in data quality watchdog.

Speaker 1

I like that.

Speaker 2

Now, once automil finishes training the set of models, it's time to put on our evaluation hats.

Speaker 1

So we've got a bunch of models. Yeah, what do we do with them?

Speaker 2

Auto mL gives us a smorgasboard of metrics accuracy, precision, recall, and it gives us those helpful confusion matrices to visualize how each model is performing.

Speaker 1

I'll be honest, sometimes those metrics can feel a little overwhelming, especially if you're new to machine learning.

Speaker 2

I hear you.

Speaker 1

There's just the book so many numbers. The book clearly explains each metric, okay, and helps us to understand which ones are most important for different types of problems. And remember those explainability features, Yes, we can use them to understand why a model is making certain predictions. That's crucial, which is crucial for building trust and transparency.

Speaker 2

So it's not just blindly trusting the numbers. It's about understanding the reasoning behind them.

Speaker 1

Absolutely.

Speaker 2

Now, let's say we've found a model that looks pretty promising based on the metrics and the explainability checks. What happens next, then.

Speaker 1

It's time to deploy deployment. We need to make our model available for use either for a batch processing of large data sets or for real time predictions.

Speaker 2

So we're taking our model out of the lab and putting it into the real world.

Speaker 1

Exactly.

Speaker 2

But we've mainly been talking about classification models.

Speaker 1

What about problems where you need to predict a numerical value instead of a category.

Speaker 2

The book covers that took it dives into building regression models using auto mL on Azure. It uses the diabetes data set to actually predict the progression of the disease based on different factors.

Speaker 1

So regression is about predicting numbers, right, like stock prices or sales figures, or in this case, the severity of a medical condition. Exactly. Okay, And while the process is similar to classification, obviously the metrics and the algorithms used are different, but AutoML still handles all that heavy lifting it does. But we need to use the right tool for the job. Absolutely, I bet there are some tips and tricks for getting the best performance out of auto mL for these regression problems.

Speaker 2

There are, and the book offers some great insights.

Speaker 1

Okay, lay on me.

Speaker 2

Sometimes converting a regression problem into a classification problem can actually improve performance.

Speaker 1

Hold on, how do you convert a problem about predicting a number into a problem about predicting a category?

Speaker 2

It's all about binning binning. You're going to divide the range of possible numerical values into categories or bins. Okay, So instead of predicting the exact price of a house, okay, you might predict whether it falls into a low, medium, or high price range.

Speaker 1

It's like simplifying the problem to help AUTOMML find patterns more easily. Exactly, Okay? What are their tips does the book offer?

Speaker 2

It emphasizes the importance of experimenting with different primary metrics. Okay, So, for example, you might find that a metric like mean absolute error MAE gives you a more meaningful evaluation than something like root means squared error or RMS.

Speaker 1

Right, So it all depends on what you're trying to achieve with your model.

Speaker 2

It does.

Speaker 1

Okay, So there's still a lot of room for human judgment and decision making, even with AUTOMML handling so much of that complexity. Now I have to ask, what about those situations where you need to train not just one or two models, but one hundreds.

Speaker 2

The many models.

Speaker 1

Probablybe even thousands of models.

Speaker 2

That's the book really shines. It introduces a tool called the Many Models Solution Accelerator or MMSA. MMSA okay, think of it as an auto mL factory. You feed it your data I'm listening, and it automatically splits it up based on certain criteria, different stores, products, or regions, and then it uses the power of auto mL to train a separate model for each of those subsets, all running in parallel.

Speaker 1

That's a lot of models.

Speaker 2

It is a lot of models.

Speaker 1

So why would someone need to create so many models?

Speaker 2

It's incredibly useful when you need granular, customized models.

Speaker 1

So got it.

Speaker 2

The book gives an example of a retail chain that wants to forecast demand for each product in each store. By using MMSA, they can create thousands of custom models that account for all the unique factors that might influence sales.

Speaker 1

Wow. So it's like hyper personalized AI.

Speaker 2

It is.

Speaker 1

That's incredible.

Speaker 2

Imagine there are things to keep in.

Speaker 1

Mind, right, there's to be some catches.

Speaker 2

One of the key points the book highlights is choosing the right partition columns.

Speaker 1

Columns.

Speaker 2

These are the criteria that you use to divide your data into those subsets. You need to think carefully about which factors are most likely to influence the target variable that you're trying to predict.

Speaker 1

So it's like making sure you're slicing and dicing your data along the right lines so that the models you create are actually meaningful and relevant precisely. Now, once you've trained all these models using the MMSA, you still need to deploy them. You do, and that can get even more complex when you're dealing with thousands of models.

Speaker 2

It can instead is just one it does.

Speaker 1

So how do you manage all of that?

Speaker 2

The book really stresses the importance of automation here. Okay, It introduces a tool called Azure Data.

Speaker 1

Factory as your data factor, which can.

Speaker 2

Help orchestrate complex data flows, connect to various data sources, transform data, and even trigger those mL pipelines that we talked about earlier.

Speaker 1

So if those mL pipelines are like assembly lines for building our model. Yeah, then Azure Data Factory is like the logistics manager, making sure all the raw materials and finished products are flowing smoothly, perfect. And in addition to all this, the book is packed with helpful tips and tricks for getting the most out of AUTOMML. It is regardless of the type of problem that you're tackling. Yes,

I love that practical advice. It's like having a seasoned auto mL expert looking over your shoulder, absolutely guiding you along the way exactly.

Speaker 2

And the book really encourages a spirit of experimentation. Okay, it wants you to be a data detective. Oh I like that, to try different approaches and see what works best for your situation.

Speaker 1

That's what makes data science so exciting. It's not about blindly following rules. It's about exploring, discovering, and finding creative solutions.

Speaker 2

Absolutely.

Speaker 1

We've talked a lot about az your machine Learning studio, which has that visual, user friendly interface, But for those who prefer to work with code, the book also covers

the Azure Machine Learning SDK for Python. It does so for those who are comfortable covading, there's a way to get even more control and flexibility over that auto mL process there is, okay, and the book walks us through using Jupiter notebooks within Asure machine Learning, which is a very popular way to write and execute Python code for

data science tasks. It is, and you can actually use those powerful compute clusters that we talked about earlier to run your code in the cloud, yes, giving you access to tons of processing power.

Speaker 2

Absolutely.

Speaker 1

It really is amazing how cloud computing has made these really complex tasks so much more accessible. It has. Now I'm curious about the different ways that you can actually deploy models once they're trained. We talked about batch scoring and real time scoring, yes, but the book also mentioned something called mL pipelines.

Speaker 2

mL pipelines are a fantastic way to automate your entire machine learning workflow, okay, from data preprocessing and feature engineering to model training and deployment.

Speaker 1

So it's like having an assembly line for your AI, ensuring that each step is executed in the right order, with the right settings. Absolutely, automation, efficiency, consistency. That's it sounds like a well oiled machine exactly.

Speaker 2

And to take it a step further, the book introduces us to Azure Data.

Speaker 1

Factory Azure Data Factory.

Speaker 2

This is a cloud based data integration service okay that can handle even more complex data flows, connecting to different data sources, got it, transforming your data okay, and even triggering your mL pipelines.

Speaker 1

So if mL pipelines are the assembly lines, that Azure Data Factory is like the logistics manager making sure that all the raw materials and the finished products are flowing smoothly.

Speaker 2

A great way to put it.

Speaker 1

Now, we've covered a lot of ground here, from the really medi gritty details of data preparation and feature engineering to the broader concepts of model deployment and automation. We have, but let's not forget about one really crucial aspect, the human element. Building a model is only part of the story. Yeah, what about gaining the trust and buy in of the people who will actually be using these models.

Speaker 2

That's a great point. You know, we can build the most sophisticated AI in the world, but if people don't trust it or understand how.

Speaker 1

It works, what good is it.

Speaker 2

It's not going to be very useful, right, And that's why those explainability features we keep talking about are so important. Extremely the book emphasizes the need to clearly articulate how a model is making decisions, especially in industries that have strict regulations, got it, or where the stakes.

Speaker 1

Are high, right, like healthcare of finance exactly. So it's not enough to just say the computer says this is the best course of action. We need to be able to back that up with insights and evidence.

Speaker 2

Absolutely, and auto mL gives us the tools to do just that. You can use feature importance scores to see which factors are driving the model's predictions. You can even drill down into individual predictions to understand why the model made a specific decision.

Speaker 1

So it's like having a transparent AI where we can peek under the hood and see what's going on exactly. Now, before we wrap up this part of our deep dive, I want to highlight something that really stood out to me while reading the book. It mentions that auto mL can actually be used in other Microsoft products besides Azure Machine Learning Studio. Yes, oh, so it's not just limited to this one platform, not at all.

Speaker 2

The book talks about integrating auto mL with tools like Powerbi Powerbi which is a powerful data visualization and business intelligence platform, So you.

Speaker 1

Could create these interactive dashboards that not only display the data, but use auto mL to generate predictions and insights. Yes, that's next level. That takes data storytelling to a whole new level. It does. And the book also touches on using auto mL with Azure Synapse Analytics, which is a cloud based data warehousing and analytics.

Speaker 2

Service exactly, and this opens up even more possibilities. Wow, for working with massive data sets and building these enterprise scale AI solutions.

Speaker 1

It sounds like the possible are pretty much endless. They are. We've covered so much ground, but it feels like we've only just begun to scratch the surface of what auto mL on Azure can do.

Speaker 2

We've just scratched the surface.

Speaker 1

So as we move into the next part of our deep dive, I'm curious where should someone start or if they want to dive in and explore this world of auto mL on Azure.

Speaker 2

Well, this book we've been discussing is an excellent starting point. Okay, Dennis Sawyer's has done a really great job of creating a practical and easy to follow guide. I agree, pact with real world examples, code snippets, and tons of helpful advice.

Speaker 1

And there are tons of online resources, including Microsoft's own documentation and tutorials exactly. And plus the Azure community is incredibly active and supportive.

Speaker 2

It is.

Speaker 1

It's great. So if you have questions or get stuck, you'll find plenty of people willing to help you will. I think the key takeaway here is that AutoML is not some futuristic concept.

Speaker 2

No it's not.

Speaker 1

It's here here, it's here now.

Speaker 2

It's now a powerful tool.

Speaker 1

It's a powerful tool that's available right now.

Speaker 2

And it's making AI more accessible.

Speaker 1

Than ever before. And with the power of auto mL on Azure, anyone can become an AI innovator. So to our listener, we challenge you, what problem will you solve with auto mL. That's a great question, that's something to think about. Think about it. We'll be back in just a moment with part two of our deep dive into AutoML on Azure. Welcome back to our deep dive into auto mL on Azure. Now, before the break, we were talking about how auto mL can be used in other

Microsoft products. Yes, besides just as your machine learning studio. So it's not just limited to that one platform, not at all, Okay, So tell me more.

Speaker 2

The book talks about integrating auto mL with tools like powerbi powerbi okay, which is a powerful data visualization and business intelligence platform.

Speaker 1

So we can use AutoML to create these really interactive dashboards you got it that not only display d data but also generate predictions and insights.

Speaker 2

Exactly.

Speaker 1

It's awesome.

Speaker 2

It takes data storytelling to a whole new level.

Speaker 1

It does, Yeah, it does. And the book even touches on using AutoML with Azure SYNEPS Analytics, which is a cloud based data warehousing and analytics service. Wow, so many options. So this opens up even more possibilities for working with massive data sets, yes, and building these enterprise scale AI solutions. It sounds like AutoML is becoming this really integral part of the entire Microsoft ecosystem. It is, so it's not just this standalone tool, it's really integrated into all these

different products and services exactly. Yeah, okay. Now, speaking of getting hands on with AutoML, the book goes beyond just talking about the concepts. It actually dives into using Azure Machine Learning Studio Yes, which, as we discussed earlier, is that more visual, user friendly interface for working with AMLS. It is, and it uses a classic example, the Titan Passenger Data.

Speaker 2

Oh, the Titanic data set.

Speaker 1

Predicting who survived and who didn't based on factors like age, gender, ticket class.

Speaker 2

Right, a classic problem.

Speaker 1

It's like a rite of passage for anyone learning data science. Exactly. I remember working with that data set in my early day. So the book actually walks you through the entire process, making it surprisingly straightforward.

Speaker 2

It does. It makes it really easy.

Speaker 1

So so you upload your data, tell AutoML it's a classification task, and let it do its thing.

Speaker 2

Yeah, let it work its magic, exactly.

Speaker 1

And while it's working its magic, we can keep an eye on those data guard rails to make sure nothing fishy is going on with our data exactly, right, We've got our data quality watchdog. We do. Now, once auto mL has finished training a whole bunch of models, it's evaluation time, right. Okay, So we have all these models all trained and ready to go. How do we pick the best one?

Speaker 2

So auto mL gives us a whole Smorgus board of metrics to consider. Okay, you know, accuracy, decision recall.

Speaker 1

So many metrics.

Speaker 2

There are a lot.

Speaker 1

Sometimes I feel a little overwhelmed, especially if you're new to machine learning.

Speaker 2

Sure, I understand that there's just so many numbers, right, but the book breaks down each metric, explaining what it means and when it's most relevant.

Speaker 1

Okay, that's helpful.

Speaker 2

Yeah, and it also reminds us to use those explainability features, oh, to really understand why a model is making certain predictions.

Speaker 1

Right. It's not enough to just blindly trust the numbers exactly.

Speaker 2

We need more.

Speaker 1

It's about understanding the logic behind those numbers. Yes, making sure the model is making decisions for the right reasons. That's right. Okay. So let's say we've found a model that looks good both in terms of its performance and its explainability.

Speaker 2

Okay, what's next deployment? It's time to take that model out of the training ground and put it to work in the real world.

Speaker 1

Okay, so we're moving from theory to practice. Yes, but how do we actually deploy a model?

Speaker 2

So the book gives us a couple of options. Okay, there's batch scoring, which is great for processing large amounts of data at scheduled intervals, and then there's real time scoring, which is ideal for when you need those instant predictions, like in fraud detection or personalized recommendation.

Speaker 1

So batch scoring is like sending out a monthly newsletter. Yes, while real time scoring is like having a live chat with your customers.

Speaker 2

That's a great analogy.

Speaker 1

Right, Different tools for different needs. Absolutely. Up until now we've mostly focused on classification models, right, where the goal is to predict a category. But the book doesn't stop there, No, it doesn't. It also dives into regression models, okay, where the goal is to predict a numerical value.

Speaker 2

Right, like predicting the price of a house exactly, or the number of sales you'll make next month, right.

Speaker 1

Or in the case of the diabetes data set that we mentioned earlier, predicting the progression of the disease exact. So the process is very similar, but obviously the metrics and the algorithms we use are different.

Speaker 2

Right, Different tools for different jobs.

Speaker 1

But auto mL still handles all of that heavy lifting for us. It does, which is great, but we still need to understand which tools are the best for the task at hand.

Speaker 2

Absolutely.

Speaker 1

Now, I bet there are some special tips and tricks for getting the most out of auto mL. There are, specifically for regression problems. Yes, so spill the beans. What are they? Well?

Speaker 2

The book shares some really interesting insights, Okay. For instance, sometimes it can actually improve performance to.

Speaker 1

Convert a regression problem into a classification problem.

Speaker 2

Hold on, how do you do that? How do you convert a problem about predicting a number into a problem about predicting a category?

Speaker 1

It's all about binning dah. Okay, you divide the range of possible numerical values into categories or vins. Okay, So instead of predicting the exact price of a house, you might predict whether it falls into a low, medium, or high price range.

Speaker 2

So it's like simplifying the problem a bit to make it easier for auto and mlifying those patterns exactly. Okay, I see any other tips.

Speaker 1

The book also emphasizes the importance of experimenting with different primary metrics.

Speaker 2

Okay, right.

Speaker 1

For example, you might find that a metric like mean absolute error or MAE gives you a more meaningful evaluation than something like root means squared error or URMSK. It really depends on what you're trying to achieve with your model.

Speaker 2

So it's not just about blindly applying the same metrics every time.

Speaker 1

No, not at all.

Speaker 2

You need to really think about what's most relevant for your specific problem.

Speaker 1

Okay, So there's still a lot of room for human judgment and decision making, even with auto mL doing so much of the work. Absolutely, Now I have to ask, what about those situations where you need to train not just one or two models, but hundreds, maybe even thousands.

Speaker 2

Ah, the many models problem?

Speaker 1

Right. It sounds daunting, it can.

Speaker 2

Be, but that's where the book really shines.

Speaker 1

Okay.

Speaker 2

It introduces this tool called the Many Models Solution Accelerator or MMSA for sure, MMSA. Think of it as an auto mL factory.

Speaker 1

Okay. I like that.

Speaker 2

You feed it your data and it automatically splits it up based on certain criteria like different stores, products, or regions.

Speaker 1

Right, so you can create really customized models for each specific situation exactly. Okay. So when would someone actually need to create so many models?

Speaker 2

It's ideal for when you need those really granular customized models.

Speaker 1

Okay.

Speaker 2

The book gives an example of a retail chain okay that wants to forecast demand for each product in each store.

Speaker 1

Ah, So they need a separate model for each product in each store.

Speaker 2

Yes, And by using the MMSA, they can create thousands of custom models that take into account all of the unique factors that might influence sales at each location.

Speaker 1

That's incredibly powerful. It is, but I imagine there are some considerations when working with this MMSA.

Speaker 2

Definitely, and one of the key points the book highlights is choosing the right partition.

Speaker 1

Columns, partision calls. What are those?

Speaker 2

These are the criteria you use to divide your data into those subsets. Okay, you need to think carefully about which factors are most likely to influence the target variable you're trying to predict.

Speaker 1

So it's like making sure you're slicing your data along the right lines exactly, so the models you create are actually meaningful and relevant. That's it. Now, once you've trained all these models using the MMSA, you still need to deploy them right you do, and that can get pretty complex when you're dealing with thousands of models. It's just one, yeah, so how do you manage all of that?

Speaker 2

The book really stresses the importance of automation here. It introduces us to a tool called Azure Data.

Speaker 1

Factory as your data Factory, which.

Speaker 2

Can help orchestrate complex data flows, connect to different data sources, transform data, and even trigger those mL pipelines that we talked about earlier.

Speaker 1

So if the mL pipelines are like the assembly lines for our models. Yeah, then data Factory is like the logistics manager.

Speaker 2

That's a great way to put it.

Speaker 1

Making sure everything runs smoothly, yes, okay. And in addition to all this, the book is packed with helpful tips and tricks for getting the most out of AutoML no matter what kind of problem you're working on.

Speaker 2

It is. It's full of practical advice.

Speaker 1

It's like having a seasoned auto mL expert looking over your shoulder guiding you along the way.

Speaker 2

Absolutely, and the book really encourages that spirit of experimentation, right.

Speaker 1

It's all about trying new things.

Speaker 2

Exactly, being a data detective.

Speaker 1

That's what I love about data science. It's not just about following rules, it's about exploring and discovering.

Speaker 2

Absolutely.

Speaker 1

Now we've talked a lot about Azure Machine Learning Studio, which has that more visual, user friendly interface, but for those who are comfortable with coding, the book also covers the Azure Machine Learning SDK for Python. It does, yes, so for those who are comfortable coding, there's a way to get even more control and flexibility over the AutoML process.

Exactly okay, And the book walks us through using Jupiter Notebooks within Azure machine Learning Yes, which is a very popular way to write and execute Python code for data science tasks.

Speaker 2

It is very popular.

Speaker 1

And you can actually use those powerful compute clusters that we talked about earlier to run your code in the cloud, Yes, which gives you access to a ton of processing power. Absolutely, it really is amazing how cloud computing has made these complex tasks so much more accessible.

Speaker 2

It really is.

Speaker 1

Now I'm curious about the different ways you can actually deploy models once they're trained. Okay, we've talked about batch scoring and real time scoring, but the book also mentions something called mL pipelines.

Speaker 2

mL pipelines are a fantastic way to automate your entire machine learning workflow, from data preprofitsing and feature engineering to model training and deployment.

Speaker 1

So it's like building an assembly line for your AI, making sure each step is done in the right order, exactly, the right settings, automation, efficiency, consistency. It sounds like a well oiled machine exactly.

Speaker 2

And to take it even further, the book introduces another powerful tool as your Data Factory.

Speaker 1

As your data Factory okay.

Speaker 2

This cloud based data integration service can handle even more complex data flows. Okay, connected different data sources transform your data and even trigger your mL pipelines.

Speaker 1

So if mL pipelines are the assembly lines, then as your data factory is like the logistics manager making sure all the raw materials and finished products are flowing smoothly.

Speaker 2

Perfect analogy, right.

Speaker 1

It's all about streamlining and automating as much of the process as possible. It is now, we've covered a lot of ground here, from the nitty gritty details of data preparation and feature engineering to the broader concepts of Muddel deployment and automation. We have, but let's not forget about one crucial aspect, the human element. Building a model is only part of the story. It is what about gaining the trust and buy in of the people who will actually be using these models.

Speaker 2

That's a great point. You know, we can build the most sophisticated AI in the world, but if people don't trust it or understand how.

Speaker 1

It works, it's not going to be very useful.

Speaker 2

Exactly, and that's why those explainability features that we keep talking about are so important.

Speaker 1

Right. Transparency is key. It is, especially when you're dealing with decisions that could have a real impact on people's lives.

Speaker 2

Absolutely. The book emphasizes the need to be able to clearly articulate how a model is moving decisions, especially in those industries with strict regulations or where the stakes are.

Speaker 1

High, right, like healthcare or finance.

Speaker 2

Exactly.

Speaker 1

You need to be able to explain why the model is recommending a certain treatment or making a certain investment decision. You do. It's not enough to just say the computer says so right.

Speaker 2

We need more than that.

Speaker 1

You need to be able to back it up with evidence and insights.

Speaker 2

And AUTOMML gives us the tools to do just that.

Speaker 1

That's good. You know.

Speaker 2

You can use those feature important scores to see which factors are driving the model's predictions. Okay, you can even drill down into those individual predictions to understand why the model made a specific decision.

Speaker 1

So it's like having this transparent AI exactly where you can peek under the hood and see what's going on.

Speaker 2

That's it.

Speaker 1

Now, before we wrap up, I want to highlight something that really stood out to me while reading the book. Okay, it mentions that AUTOMML can actually be used in other Microsoft products besides just Azure Machine Learning Studio. Yes, so it's not just limited to this one platform, not at all.

Speaker 2

Tell me more So, the book talks about integrating AutoML with tools like powerbi Powerbio, which is that powerful data visualization and business intelligence platform.

Speaker 1

So you could create these interactive dashboards that not only display the data, yeah, but also use auto mL to generate predictions and insights.

Speaker 2

Exactly.

Speaker 1

That's amazing. It's like taking data storytelling to a whole new level.

Speaker 2

It is. And the book also touches on using auto mL with Azure synaps Analytics.

Speaker 1

Okay, which is that cloud based data warehousing and analytics service. Right, So this opens up even more possibilities for working with massive data sets, right and building these enterprise scale AI solutions Exactly. It sounds like the possibilities are pretty much endless. They are.

Speaker 2

The possibilities are endless.

Speaker 1

We've covered so much ground, but it feels like we've only just scratch the surface of what AutoML on Azure can do.

Speaker 2

We've just scratched the surface.

Speaker 1

So as we move into the final part of our deep dive, Okay, I'm curious where should someone start if they want to explore this world of AutoML on Azure.

Speaker 2

Well, this book we've been discussing. Is an excellent place to begin.

Speaker 1

Right, It's a great starting point.

Speaker 2

Dennis Sawyers did a phenomenal job creating this really practical and easy to follow guide.

Speaker 1

I agree.

Speaker 2

It's packed with real world examples, code snippets, and tons of helpful advice.

Speaker 1

And of course there are tons of online resources out there, including Microsoft's owned documentation and tutorials. There are plus the Azure community is incredibly active and supportive. It is.

Speaker 2

It's a great community, So.

Speaker 1

If you have questions, you'll definitely find someone who can help you will. I think the key takeaway here is that auto mL is not some futuristic concept. No it's not. It's here. It's here now, it's now.

Speaker 2

It's a powerful tool.

Speaker 1

It's a powerful tool that's available right now.

Speaker 2

And it's making AI more accessible than ever before.

Speaker 1

Exactly, and with the power of AUTOMML on Azure, anyone could become an AI innovator.

Speaker 2

Anyone can.

Speaker 1

So to our listener, we challenge you, what problem will you solve with auto mail?

Speaker 2

That's question?

Speaker 1

What will you create?

Speaker 2

What will you create?

Speaker 1

We'll be back in a moment with the final part of our deep dive into auto mL on Asure welcome back to the final part of our deep dive into auto mL on Asure. You know, this book has really opened my eyes to the potential of auto mL.

Speaker 2

It really is a game changer, and.

Speaker 1

Azure seems like the perfect platform. Oh, absolutely to really explore it. It is now for those who want to kind of delve a little deeper into those technical aspects. Yeah, the book actually doesn't shy away from the code, No it doesn't. It guides you through using the Azure Machine Learning SDK for Python. It does so for those who are comfortable with coding, there's a way to get even more granular control over the auto mL process.

Speaker 2

Absolutely, the book that shows you how to use Jupiter notebooks within Azure machine Learning, which is a really popular way to write and execute Python code for data science.

Speaker 1

And you can even use those powerful compute clusters that we talked about yes to run your code in the cloud exactly, tapping into massive processing power.

Speaker 2

And all that processing power.

Speaker 1

It's incredible how cloud computing has really made these complex tasks so much more accessible.

Speaker 2

It really has.

Speaker 1

I'm curious about the practical side of things. Okay, we've trained our models, but how do we actually put them to work right. The book mentions batch scoring and real time scoring, but it also talks about something called mL pipelines.

Speaker 2

Yes, mL pipelines are a fantastic way to automate that entire machine learning workflow. Okay, from data preprocessing and feature engineering to model training and deployment.

Speaker 1

So it's like creating an assembly line for your AI exactly. Each step is executed in the right order with the right settings.

Speaker 2

Yes, automation, efficiency, consistency.

Speaker 1

Yeah, it sounds like a well machine. It is. And to take even a step further, the book introduces us to Azure Data Factory. Yes, another great tool, this cloud based data integration service that can handle even more complex data flows. It can connect to different data sources, transform your data, and even trigger those mL pipelines exactly. So if mL pipelines are the assembly lines, then az your Data factory is like the logistics manager making sure that everything's flowing smoothly.

Speaker 2

A perfect analogy.

Speaker 1

Now, this book has given us a really fantastic overview of auto mL on Azure. It has, and it's clear that this technology has the potential to really revolutionize the way we approach AI.

Speaker 2

It really does.

Speaker 1

But as we wrap up our deep dive. Yeah, what's the one key message you hope our listener takes away from all of this.

Speaker 2

I think the biggest takeaway is that auto mL is empowering. It breaks down those barriers to entry for AI, making it accessible to a much wider audience. I agree, you don't need to be a data science guru to build powerful models on Azure. This book provides the guidance and the tools you need to get started, and with the power.

Speaker 1

Of auto mL on Azure, anyone can become an AI innovator. Absolutely so to our listeners, we encourage you dive in, explore the world of auto mL and see what incredible things you can achieve.

Speaker 2

Until next time, happy innovating.

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android