Hello and welcome to the let's Talk. Azure podcast with your host Sam Foote and Ann Armstrong.
If you're new here, we're a pair of Azure and Microsoft 365 focused it security professionals. It's episode 36 of season five. Alan and I recently had a discussion around Azure web jobs. Azure Web Jobs are a feature of Azure app service that allows developers to run background tasks, scripts and long running processes alongside their web applications, enabling automated workflows and task scheduling in the cloud. Here are a few things that we covered. An overview of webshops and their use cases, the different types of web jobs that you can create, how you deploy and integrate them into your applications, and how you can monitor them and scale them alongside your applications. We've noticed that a large number of you aren't subscribed. If you do enjoy our podcast, please do consider subscribing. It would mean a lot to us for you to show your support to the show. It's a really great episode, so let's jump in. Hey Alan, how are you doing this week?
Hey Sam, not doing too bad. How are you? Yeah, good, thank you. Have you been combing through the session catalogue of Ignite yet? Have you seen anything that excites you? I've not had a chance to look it. To be fair, I started looking at it this evening briefly, but I can't, I don't think at the moment you can associate stuff with your account. No, it's just a big account.
It's just a big long list at the moment. And yeah, I count Boolean search string and then filter out AI. So there's definitely a lot of like partner sort of focused sessions. Obviously there's that. The combination of inspire is it, and ignite together. Is that right? That's right. To say it's like a combo event, isn't it? So yeah, lots around, I mean in the security space, lots around copilot usage, copilot for security. So yeah, it's interesting to see how much of their AI capabilities there sort of wrapping in to their different sessions. So yeah, yeah. And there is a lot of, I don't know, vendor specific sessions as well that will be interesting to see. I would, I would have thought how vendors are pitching sort of their wares integrated with Microsoft.
Yeah, yeah, exactly. So it's kind of how it, I think used to be before COVID kind of thing. The vendors used to be a bit of a bigger part of the event.
Yeah. And I'm hoping that like the partner ecosystem being more part of it, if that makes sense because obviously there was like there were vendors weren't there? You know, even post Covid. Right. Maybe not as many but also to now have the partner community there we should get some good sessions from that community and sort of driving different technologies and you know and approaches I suppose obviously selling their own wares and promoting their organizations but that might drive a bit more attendance from end customers potentially.
Yeah. I guess that gives some insight into the, the approaches around some of the technology as well about adopting and. Or adopting consuming and also. Yeah, the approach to I keep saying about adoption side of things. Yeah definitely. Anything else in the wider it sphere that you've been tracking this week?
No, nothing this week. Been head down, getting on with some work. Been pretty busy this week and I pretty much say it every week at the moment. But this week was definitely. Yeah. No time for checking out anything. What about yourself if you notice anything for tracking?
Nothing particularly. I, I'm I don't know, I'm getting the, there's gonna be a lot of I think, build up to ignite I think and any new because we generally tend to get things that maybe none of us even know about if that makes sense or proper. Sort of like real launches. So ignites usually a good one for our sort of area isn't it? You know and new releases. Pretty interesting to see the spat between WordPress foundation, automatic and also WP engine because obviously the WordPress. I still didn't realize that WordPress was so prevalent. Like you know they still, they still power a huge proportion of the Internet. So yeah there's been some legal disputes and trademark disputes around one of the WordPress hosting providers, WP engine and sort of a beef with a trademark licensing deal with the WordPress sort of foundation and that's been quite a public spat, people leaving and yeah, just I don't know, we can't call it Twitter anymore, but it's Twitter, just lots of public. But it does raise some sort of interesting concerns really around like organizations use of open source tooling and their trademarks because there are companies that you know their sole focus is you know providing you know managed, effectively managed and SaaS versions of open source tooling. And it's interesting to see in this scenario where things have fallen out what can be enacted if that makes sense and what sort of power can be wielded. So it's been quite scary to see in some respects.
Yeah I remember you telling me about that the other day. Yeah.
Every time I open like my feed it's still going like it's not stopping, it's not getting any better. So yeah, so that's, that's definitely interesting. One anecdote from this week, I found that Copilot has been a lot better for me this week. I don't know about you, but it's like saved my bacon three times this weekend. Its search and recall seems much better. Don't know whether I'm just getting better at prompting, but yeah, it's found like, yeah, like I covered an email today that I would have never have found. Oh, was that today or yesterday? No, it was this morning. I was in a call with somebody and I was like, ah, I bet you I've got that email somewhere. And it was like a random email from like four months ago and it was like a, I had just been, you know, copied into it. I wasn't mentioned or anything like that and I just wouldn't have found it, you know, in the sea of it probably just shows how bad, like outlook searches more than anything or my skills at Outlook search, but copilot is just such lazy searching. It's like copilot, go and find this wrench, this mention of this random thing in my mailbox, you know, and it's like, oh, here you go. And you click a link and it just takes you straight there. So I have seen the magic a few and that's just one example. I've seen the magic a few times this week and I'm like, yep, that license actually been worth it this month. So I'm using it more and more. I would say I went through a lot. I wasn't using it so much. But yeah, I definitely, definitely get some benefit of, yeah, definitely get some benefit of it.
Yeah, I think the integration with the, the office applications got better, especially within word kind of thing because it used to be that if you're trying to rewrite something and then you clicked away because, you know, you never stay on, on word sometimes and yet teams, messages, etc. It then disappear. So it would have generated you some, some content to maybe put in or to review. You have to redo it. But now it actually like the, the box stays there so you can at least sort of change or change focus at least with it. So. Yeah, small things like that.
Yeah, no, no, definitely. So, yeah, for, for recall searching, I don't think there's anything better than it. You know, maybe that's more of a criticism of the tools we've had so far, if that makes sense. But yeah, actually I think it's, yeah, it's really powerful. Yeah. Cool. Okay, so what's this week's episode on then? Sam?
Yeah, we're going to cover Azure web job support inside of app service. Little bit niche and probably not a huge really long episode. I'm sure we'll babble on for long enough, but yeah, I think it's quite a useful feature of app service and it's probably just worth calling out solo by itself. Cool. Yeah, I've never heard of it. So I guess let's start with what is or Azure web jobs and why are they important?
Okay. Yeah, so Azure web jobs are a feature of Azure app service. We did a previous episode on app service, I believe. I can't remember which episode that is, but I'll look that up in a moment. It allows developers to run background tasks or long running processes alongside their web applications. Now when you're building a web application, not everything is real time. And with a user, if you visit, I think everybody listening to this is probably pretty familiar with using a web application. We take the example of, let's say Google. You request Google's, you go to google.com comma, Google dot co dot UK comma and the web server will respond with that page. You're making sort of a, a request for a response. Now not everything can be done in the context of a user's request and session, if that makes sense. Let's say you need to do some sort of background processing. You want to be able to do that. But in the same vein, you might not want to build like a fully fledged separate system for that. You might want to share code with your applications, business logic, those types of things. So keeping them together does make a lot of sense and I'll talk about that in a bit more depth. So whenever you need to run code that doesn't require user interaction, this is the great thing. And it also allows you to offload time consuming work as well. So if you've got something as part of a process that is slow and you don't want to have to wait for the user to wait for it to complete, you could offload that task to a background processing job. So yeah, why are they important to just give you the ability to do background processing? When we're hosting applications in these PaaS type systems, we don't have the same level and access to the underlying infrastructure and operating system that we would have if we were running this, say, you know, in iis on a Windows server or any sort of other web application, Apache or Nginx, et cetera, because you could, you know, you could, you would have access to the operating system. You could build a cron job and run it from there so you can do this background processing so it won't impact the performance of the main application because you've usually got multiple cores there. This is like a multi, it's not multithreaded because it's separate process, but it's a separate process running so it can sort of load balance and move across those calls. Task scheduling, it gives you the ability to do task scheduling. So you can either trigger them manually on demand by another piece of code, or you can get the tasks, the web jobs to run on specific intervals using cron expressions. So you have different ways to manage these, these tasks. These web jobs run in the same environment as your app service means that they can share the resources that you've already provisioned for your application. It's got access to the configuration as well, the environment variables, any configuration that you pass down from Azure. And also it can benefit from the scaling plan that is applied to your app service. So you know, if you need extra grunt overnight to run some background tasks, maybe you run some database cleanup scripts, whatever that is, whatever you've got, you can run them in the middle of the night. If they take up too much processing power, you can use scaling for that. It's also cost efficient because you're using the, the resources that you've already provisioned. You're sort of not having to provision another resource to run this. You're already paying for that infrastructure. So as long as the infrastructure, well I say infrastructure is not IaaS, but it's Paas. As long as you're running that and you've got capacity there, then you can sort of, you could in theory run these background processes. It's not free of charge because you're paying for it, but in, included in your price, getting better value from your, from what you're paying for. Also it works, it works in conjunction with different languages and integrations. App service can run in multiple different containers and environments. There's Windows and there's Linux, and there's good support for the different languages and frameworks that app service supports for. Also web job supports. So you can do things like PHP jobs, you can do Python node, JS, Java.net, bash scripts, Powershell windows, sort of command and bat files. So there's a good flexibility there in what you can actually create as well.
Cool. I think you probably mentioned a couple of these, but what are the common use cases for these Azure web jobs?
Okay, yeah, if you've built web applications before. These will seem pretty common to you because they're always the things that you have to have some sort of like background processing application or something like that. So think about data processing for jobs. For web applications that are distributed in nature, they might use a messaging or queuing protocol system to communicate. For microservice transactions and queue based systems. It's code so it can interact with other Azure services like storage queues, service, bus, et cetera and actually process and react to those environments. Like I said, batch processing of data, overnight reconciliation jobs, data transfer jobs, backups x, y and z, lots of things there. One thing that you may have to do as part of your application, if somebody uploads a file, you might want to, if they upload an image, let's say you had a social media application, you would want to resize that image, maybe compress it, generate thumbnails, converting them to different formats. Maybe you take jpegs and you convert them to webp images, something like that. You don't want to submit it and do that processing whilst the user is waiting. You want to sort of offload that to a background task which you can do there. Things like sending emails, transactional emails. Somebody signs up for the first time actually sending the email, your welcome email, let's say as an example, you don't want that to be your web request to be dependent on sending that email. That email could take a few seconds, up to 30 seconds to send. It could fail to send, could have an issue. So you can batch those and use these to send that type of things. Reminder emails on a time scale. Let's say you send all your reminder emails at 06:00 in the morning. You could have a job that wakes up at 06:00 in the morning, goes all right, what email reminders do I need to send to people? Simple example, generating reports. That's a background task sort of data analysis. So yeah, we can generating PDF's, other report type systems and probably the other big one is sort of task automation and workflow orchestration. So I've talked about integration into other microservice type architectures, but also chaining tasks between different web applications. You have the ability to have a background task job that could chain your application into other applications and other jobs that you want to do. So you essentially cut out the need to build like a separate application and orchestration platform to do some of these key tasks. And some of the examples I've used there are sort of pertinent to every type of web application.
Yeah, no that's cool. It definitely sounds like there's a lot of things that would be very useful for them to be doing in the background. And as you said, you have to build this into your application for the user to wait or build a separate, potentially a separate service to do that. Automation. So what are the different types of web jobs that you can have and how do they work?
Okay, so there's two types of web jobs. The first is continuous web job. This has got essentially its own lifecycle. It starts as soon as the application, it starts as soon as the web job is created, essentially. And it's always running in the background, ready to respond to different events and just do background processing live. So you could have a web job that monitors a queue and processes messages when they've arrived. It could connect and listen that queue and pull that queue. A triggered web job is sort of a, it's either triggered manually on a, or on a given schedule. So you can run it manually from different ways. I'll go into that in a bit more depth, but so you can either trigger it manually on demand or you can schedule it so it'll wake up at a certain time every day. And it uses a, quite a common standard of cron scheduling in order to do that. So you could, as an example, create a triggered web job which could be scheduled to run every night, middle of the night, to clean up old database records without having to do it manually, but at a time that it might not interrupt any users of the system.
Okay, cool. That sounds really simple. In the sort of ones, like you said, you've got a scheduling or triggered one and then one that is just ready to go and. Yeah. Checking for actions to sort of take place. So probably going on to sort of the triggered web jobs. Yeah. How do you, how do you trigger them?
Yeah, so there's, there's quite a few different ways and, and this is why I, I wanted to talk about it because it is actually quite interesting, all the different sort of hooks they've got for triggering. So you can manually trigger from the portal. So if you did have like an administrative job that you wanted somebody to run manually, you could trigger it from the portal. So it's also almost sort of given you an administrative portal for free as well, if that makes sense, because it's just like a blade on the left hand side. You click on them and you can run them, you can run them from the Azure ClI. So that could be good for, for automation. There is also a rest API that can trigger them as well. We talked about scheduling a scheduled trigger using a cron interval. There's also event based triggering. This is essentially it's a type of continuous job that runs and waits, but it is triggered from an external source, but it triggers when specific events occur in Azure storage that can be really good for, you know, file drops into azure storage, then run a trigger this side. There is also a HP request and webhook to trigger the job as well. Kind of similar to the API, but I think that's more publicly exposed. Webhook on the way back, you can sometimes need web hooks for things like payment systems. You know, you send somebody off to pay via PayPal, your pass to PayPal, that there's a webhook to call back on. And you can use that, you can use this webhook ability to sort of call back on and trigger stuff this side. And you can also trigger from a CI CD pipeline, so you can trigger different actions. So you could say after I've deployed, run my database migrations as an example, those types of things. I'm not sure you'd want to do that in production, but you do have the ability to at least trigger your applications from your pipelines.
Yeah, okay. Yeah, loads of waste trigger and definitely an interesting one around the DevOps pipeline pipelines in general kind of thing. Yeah, definitely. Okay, so how did you create and deploy a web job? Yeah.
Okay. So it's worth, they are essentially, even though they live by your application, they are almost their own console applications that you can package separately. I do believe you can actually write them independently of your web applications, but still run them close. There's lots of different SDKs for different sort of languages. And so you essentially compile your applications down and you essentially deploy them alongside your application as like little separate bundles, essentially. There is also, there's FTP deployment you can deploy via the portal, and there's also git continuous integration, sort of CI CD publish, publish. You can connect it to a git repo and publish updates that way and it will build the jobs for you as well. So yeah, there's a few different ways to deploy them as well, because even though they do live in the same context of your application, they are sort of mini independent apps alongside it, if that makes sense.
Yeah, it makes sense. Did you, I don't know if you mention it, but was arm templates also an option, was it? Not necessarily. I suppose, like you probably go continuous evaluation if you're going to go through templates.
Yeah, it's more like you're deploying a app bundle like you would with app service, if that makes sense. So it's nothing, an actual resource because the resource itself is app service if that makes sense. You're just, you're slotting a web job into a web job slot basically. So yeah, I don't think it's, it could be, it could be terraformable, but I'm not sure you deploy it that way. You'd probably just connect it to a git repo and just have it continuously pushing updates in.
Yeah, yeah, absolutely. Okay, so you mentioned Azure job web jobs kind of integrating with Azure services. Can you sort of elaborate on that a bit more?
Yeah, because you've essentially got little mini applications that run in the context of your Azure app service. Right. So Sky's the sort of limit there really. It's no real different than any other application, but you, you are hosting in Azure in that app service resource. So you have access to the environment of that resource. So if you've got, let's say your application backs onto an Azure SQL database, you're going to have the connection strings you need. You're probably going to have the identity that you need to connect to it as well. So that's really good integration dependent on how you network and peer your applications. You're going to have access to that connectivity similar to your app service as well. And you've, because you're writing in.
You'Re.
Likely to be writing in the same language that you're developing your application in or a very similar application. Depends because that's not exactly true because you can containerize app service apps and there's a few little bit more nuance there. But by and large you're using, you know, a popular high level language that it is likely that there are SDKs that you can use to integrate with other Azure services. And the key there really is the identity because Azure app service is going to handle or can handle your identity management there. So it makes your authentication into other systems inside of Azure really easy because you can pull your Azure credentials essentially from the environment and just authenticate to whatever you need to in the context of your app service.
Yeah, yeah, it makes absolutely sense. At least it's really easy to have that integration side of things. So I think you've kind of mentioned some of these to be fair. But what are the benefits of using these web jobs alongside your as your app service?
Yeah, yeah, I think I have probably gone through this already, but really sharing that environment I think is the big biggest benefit. You have access to config you have access to identity, you have access to resources that your app service has. And I just think that leads to simpler management. Personally, there is a tendency in modern application stacks to split everything apart, microservice everything. I am on board with that at massive scale. But for most organizations they want a monolithic app structure and hosting structure generally because you need less different types of people to maintain it. There's less connection and crossover points to maintain. It's everybody's preference. I'm not going to force, I wouldn't force anybody to go either way, but if you do want one sort of app service with everything in it, like one logical grouping of all your different code and background tasks and application tasks, this can be a really good way to go because Microsoft Azure is managing those jobs for you. You don't have to worry about the lifecycle of those jobs running and I believe they're into, they're not connected to each other. So if you have an issue on one, you're not likely to have issue with your main web application or another web job there. They're handling all of that for you.
Yeah. And I guess it's also allowing you to optimize your computer. You may have purchased because you might be on a plan that maybe your app has to be on because production, but maybe it's not using all of the, the resources there. And I guess if you had it on, if you had it in other services, you'd be paying for that. So I guess you can kind of use some of the resources that are currently waiting to be used. Is that sort of something you think as well or.
Yeah, exactly. Yeah. You're, you're getting the best bang for buck out of your environment, right? Yeah. Okay, so probably slightly related to that sort of comment there, but also not, but when should you use Azure web jobs versus Azure functions?
Yeah, because there's a lot of crossover here really because Azure functions can give you a lot of the functionality that web jobs is giving. Azure functions primarily you're going to use them in a consumption based approach because you're going to get insanely low costs, especially if you're running things like background jobs that run once per day. As an example. An issue you can have there is if you are doing things like vnet peering, more advanced use cases of connecting to internal networks you can sometimes run into. I think we've fallen into this trap quite a bit where you use consumption based models but you need access to like v net peering and you can't always do it because it's shared infrastructure and it's serverless, or you have to go to a plan where it makes it prohibitively expensive. Now, if you're already bearing the cost of your app service plan to have those types of features, you may as well reuse what you've currently got because again, you're saving even more. I would say the deployment and developer experience of Azure functions is more structured. I think web jobs is more flexible, I think you can, but you also have to deploy them in a slightly different way. So Azure function has some really great benefits. And if you saw an application stack one with an app service and then Azure functions to do it sort of background worker jobs. But I do think if you do have that set up, you are going to be having multiple things to manage there so you can get the same sort of effect, get the same sort of functionality. But I do think you need to think about the way that your application is set up and how you want to integrate with that. If you're not using app service today, then running functional apps like you can build web applications in function apps, you know, end to end full, you know, consumption based web applications. So, you know, my conversation would then be, well, do you want to just go to a full service architecture instead of having like a traditional app service architecture? So very similar, I think, in what they can do. I just think the approach is different, but definitely something to consider. And that's a, that's why I think it's key to talk about it because they are similar. But I just think, yeah, if you're already using app service, web jobs is a really compelling option.
Cool. Okay. Is there anything else do you want to talk about around Azure web jobs?
No, I think that's it. It's just a call out. If you're not currently using, it can be beneficial to you. You are just building console apps and things like that. So it is pretty cross platform. I mean, the way it hooks in and the lifecycle of it in Azure is obviously azure specific. But yeah, you do have some good options there. And if you want predictable performance and you're happy with a more traditional pricing model of say, vcore or, or. Yeah, then yeah, definitely, definitely check them out.
Cool. Okay, well, thanks. Thanks. That, Sam, I think I found you some of your previous episodes. Oh, yeah, thanks. Yeah.
So season four, episode 21 Azure app service that might need a refresh at some point, but yeah, we have a wider conversation around app service and you've also brought in the Azure static web apps, which is probably a good one to listen to. As well because that can also link into Azure functions as well to build serverless web applications. So that's probably worth a listen as well if you're thinking about serverless type architectures and background jobs and things like that. Alan, what's our next episode?
Yeah, so our next episode I'm going to talk around Defender for cloud apps MDA. I'm going to talk about the app governance part of it. It kind of seems to be a section that maybe was forgotten about because I think we've always seen different cloud apps as being CAsB shadow it integrations with APIs to get sort of your user activity side of things. And I do recall app governance was sort of a extra license once upon a time, but is now included in e five. So I kind of want to go through that part about governing sort of applications in some of your Sass sas apps, but also in intro to give some visibility of what they're doing and things like that.
Nice. Yeah, that sounds great. So yeah. So did you enjoy this episode? If so, please do consider leaving us a review on Apple, Spotify or YouTube. This really helps us reach out to more people like yourself. If you have any specific feedback or suggestions, we have a link in our show notes to get in contact with us. Yeah, and if you've made it this far, thanks ever so much for listening and we'll catch you on the next one. Yep, thanks. All.