How'd you like to listen to dot net rocks with no ads? Easy? Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month. We'll get you that and a special dot net Rocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey guess what, it's dot net rocks. I'm Carl Franklin, an amateur cable We just had a conversation before SERGEI came online about phishing, and because I was watching the show
called Bizarre Foods America with Andrew Zimmern. It's an old show, right, it's been on many years, but I love watching reruns of things that are interesting. And he was in Vancouver. Yeah, yeah, and he goes to tojo like the guy who invented the californiyear old and I got you there, right, yes you did? Yeah, And Aerosmith walked in right yeah, right while we were there. That's right, you did that over the shoulder shot. Yeah. And my friend said, hey, who's that guy
talking to my aunt? Edna? Steven Tyler. Steven Tyler has aged interestingly dude looks like a grandma. Now. Anyway, that was a fun conversation, but that's not what we're here for. Let's get into better no framework and start this thing. All right, man, what do you got? All right? I found this list, a curated list of awesome open source projects on the web, and it's called awesome oss. It's a GitHub repo
and it's at Serene Blue. And I love these curated lists, and man, I wish we had these more in every category of life, like music for example. You know, used to have an uncle who had a record collection who you could just go to and say, hey, what's really cool right now? And they would say I love this yeah, and you listen to it and it's great. So developer tools is just one of the categories
in here. But the categories are there's only a few audio editors, audio players, documents, developer tools, email foundations, FTP clients, games, graphics, multimedia, networking, OS distributions, other password managers, payments, social text editors, video software, web browsers. I went looking in you know, non developer tool stuff, sure, and there's so many cool projects
that I had no idea. The curation he's done here where he's actually found the links for how you donate as well as how you contribute, Like, this is a ton of work to make this list. A lot of work. Yeah, yeah, no, I really appreciate that. Yeah. I didn't see anything where you could donate to him. No, no, not it all. But I think that's what this is about. I mean, in the end, I mean, they're still maintaining it. But this is
a properly configured open source project too, right. I appreciate that it has an MIT license. That's great. It's kind of funny. But yeah, they to do the work of saying here's the project, here's the source cover repositor. If you want to contribute code, here's how they take donations, if you want to contribute money. It's really awesome, that's really great. Yeah, so hats off to Serene Blue for curating that list. Who's talking
to us? Richard grabbed a comment off of show eighteen eighty, which was just a couple of weeks ago, right, yeah, right, which is why we're here actually, because when we were recording that show with Jimmy Bogart about migrating to dot net eight he called out hang fire. Yeah, and and right away I made a notice. I'm going to go look up what hangfires. I hadn't heard about it before, but one of our long time listeners, Dave Ackkroyd, also immediately commented on that show. He said,
this is a great episode. I'll be recommending my colleagues listen, though they should be listening to all of them anyway. Uh, and I'm not. That's not an editorial. I mean, that's what Dave said. That's why we love Dave. I had my ears prick up the moment when Jimmy mentioned hang fire, as that is a service I work with a lot on my current job. I would love to hear from them, especially as I'm currently
using it on dot Net framework and looking to the dot Net Core. Would open up some other hosting options besides an EC two instance running on Windows server or running Windows Server on EC two instance. That sounds right. I like that better. My ideal world would have no i as, and so knowing some more around options for platform as of service would be great to hear. Sorry, Dave, Yeah, we can't help you. No, Nope, not that they could do there. We're not going to go there, not
gonna do it and come, you know. Fortunately, in my initial research, and seeing the encouragement from David so Forth, I reached out to Sergey, who was very responsive to us, and so here we are just a few weeks later. Very cool. I know you got a copy of us iccode by already. I think you got all the things one way or the other. But pick me later and we'll chat. Thanks so much for your comment and a copy of music, cob By. It's on its way to
you. And if you'd like a copy of musicobe I write a comment on the website at Donna at rocks dot com or on the social media's like Facebook, publish every show there, and if you comment there and are reading the show, we'll send you copy of music kobay and you can follow us on Twitter if you want, or x or whatever they call it. I'm at Carl Franklin, he's at Rich Campbell, but the cool kids are hanging out. I'm Masterson. I'm at Carl Franklin at tech cub dot social, and
I'm Rich Campbell at Macedon dot social. Send us a tuote. We will definitely read it. That's all we're committing to. Yeah, that's all. Quietly, we might make fun of it, yeah, but publicly we always say nice things. We're nice guys, at least on a podcast. We are. Yeah, that's right, we play nice guys on a podcast. Okay, close enough, let's bring on. Odie Knockoff is a proud husband
and father of two kids, currently living in Thailand. He's been in offerare development for more than fifteen years, looking to find the exact reason why concurrent parallel distributed programming is so difficult. I hope he's hearing it enlighten us. I know he will. Welcome serge. Hi, everyone, how are you doing? Thank you for the invitation. I'm doing fine, but my bed is tired. Yeah. Yeah, this is a very much around the world
recording. You're in Thailand, I'm in BC Carls, Connecticut. Like we're covering all the time zones. Yep. This is amazing that we can all be here together on riverside. But let's talk about hang fire now. I'm just going to give the quick you know, read what the website says. It's very easy to understand. An easy way to perform background processing in dot net and dot net core applications. No Windows service or separate process required,
backed by persistent storage, open and free for commercial use. Wow, legit, that's and that's quite an undertaking. My friend and colleague Brian McKay, when I told him we were interviewing you, said, holy crap. We used to use hang fireback in the dot net framework days and he had no
idea that you could run it encore. Oh yes you can. So it centers around this idea of jobs, and you have just right on the homepage, you know, examples of what the different kinds of jobs are, fire and forget, delayed jobs executed only once, but not immediately after a certain time interval, recurring jobs like on the specified Crawn schedule. So wow, that's great. Continuations which are executed when its parent job has been finished.
And then the pro version has this thing about batches. We'll get into that a little bit later. But man, this looks great and so necessary and so easy. Thank you. How long you been doing this now more than ten years already, Guinness. It will be eleven birthday this year in August, as I remember, I started it in twenty thirteen. First commit it
was in another life. What was your inspiration? I mean, did you have like a problem that you said, you know, I need something like this so you wrote it for yourself or well that My problem was I found myself implemented the same beats from project to project on my jobs. So I waited for a long time that another one more experienced developer implement these. I was waiting for about four years, but I saw almost nothing. Most project or dead because the period, and after a year or two it became up
torment. So you have that healthy instinct of this is such a good idea, somebody else must have already done this. Yes, yes, yes, I was looking on other platforms like ururban rails, like Python. I saw Sidekik, Cillary and other interesting projects, and I was dreaming that someone will implement this because I really tired implementing the same and the same on dark Night platform. Yeah so you I mean you mentioned Celery and that's built for a
what platform is that for? It's Python. It's a Python, That's what I was thinking. Do you see that as an inspiration? Like that's just a give me one of those that please, just just just top level. I didn't go into the details. I was looking at API was dreaming, right, yeah, yeah, yeah, I just hoping that you know, you don't want to have to happily use somebody else's if you could have.
Yes, yes, sure, because the ground jobs today projects are more complex than in the early days of the internet, and almost every project I need some kind of ground processing. I was surprised to see that there is no movement in the network. I saw some projects. That's what that already as I already mentioned, because it's difficult to be to build a sustainable open source
projects. So that's but I think that's another story. No, but, and I when it's super relevant on the show that you know how you built this out of your own need, but you've got a huge raft of contributors too, so obviously with a need by other folks that they've helped you out or been a part of the process. Yeah, sure, because contributors published
a huge number of things. Because sometimes you don't have time to fix some rare features or very rare use cases, and contributors make a huge contributions help a lot you polish some advancity use cases. I noticed that you have an architectural diagram which is very helpful, but it has like the user and then the hang fired scheduler and then a server. But as you mentioned in the very first paragraph on your website. You don't have to go out of process.
This is just a you know, a logical server, if you will, right, logical server. Yeah, yeah, yeah, But you can move it out of process if you want to, right. Yes. You can start in the same process to keep things simple and later move it out
to another process or another server when needed. And the main things is that you're the top will not be complicated when you do this because in both casse, if you perform the process in the same process and in another process you're using the same configuration, you just plug in the storage and let hunfar do right. So you don't need any kind of inter process communication because it's all persisting in a database. Yeah. Yeah, that's very cool. Yeah.
And by database it looks like you mean just about anything the process I suspect I could probably drop the state information into a file and you could pick it back up and keep going. Not quite, because official is supported a skill server and already and in memory for simplified development, et cetera. But there are a lot of community by the storages that are implemented by other people, other contributors, so I don't think it's easy to use file lists but it
can be at least implemented. Sure. So when I was talking to Brian McKay, he wanted to know when is a good time to use hang Fire versus some sort of cloud service you know, Azure Functions or service Bus, because he says, we use that pattern a lot. I think it depends on architecture because there are companies that invest heavily in cloud infrastructure and I don't think I don't think there is need to move to Fire or other on promise
things for them, but it is still possible. Nowadays, company looks for ways to keep it possible to move to other vendors. Right. Platform independence, yes, yes, yes, And it's difficult if you invest a lot of time, a lot of development into one concrete platform to move switch to another one. So if a company wants to invest more in a platform independence, then it's a great opportunity to use Fire. I imagine that when it comes to scaling, right, your database might be the the you know,
the point at which you want to put your resources into. You know, it might be the bottleneck. Can you use just standard caching? Is that going to screw up the whole process? If we I mean, I mean it's really true. It's really about writing to the database. But there's obviously your services in the background are reading that database. And if you cash, is there a possibility that something will be missed? How do you feel about
that. It's possible to use different databases for applications, data and for and far and fire has a right heavy workloads, so it's difficult to catch anything there because everything is constantly changed, like the ground jobs that move from one state to another. So whattlelike is an interesting problem and I'm afraid there is no solution of them charging. Yeah. Yeah, you can use multiple storages
incanfi and create the ground jobs in one or another storage. So on the server aspect, you know, the thing that reads the database in fires off the jobs and all that stuff is there in memory. Cash with notification that the database has changed for you, so that you're not constantly polling the database. How does that work? It depends on the storage because, for example,
Alreadius has blocking commands. But actually that's why people love radis. Yeah yeah, but actually the most popular client for red this is stach exchange rate doesn't support blocking comments. So this is a kind of pops up magic punitions. And subscribe. I'm talking about a squall server. It has some motification about Historically it's complicated, right, you have to do triggers and things, right, dependency. I even forgot the name of it. There. Once
upon a time seql server had notification services. Yeah, yeah, yeah, yeah, yeah yeah. And to simplify things, I will I was trying to discover a more or less efficient way of pulling and let's versions. It has central bags, but it works, it works well practically, so very good. Yeah. And I noticed that you start with sql server two thousand and eight R two. It's like the base model, and that's the original
version for notification services, as I recall. Isn't that funny? Yeah, but they require additional loanments, additional database configuration, and right in the early days they didn't work well. Memory leaks are possible because I remember, yeah I recall living in the DBA land at that time, and we were usually
pretty hesitant to turn notification services on. So to have a dev come out and say, hey, we really want to do this, can you can we have notification services like we would push back, It's like that's another thing we need to monitor. It's a bit cranky so we're going to need some restart mechanisms like all of that kind. I remember in the old days, and I can't remember exactly when, but it was probably before two thousand and
six that we used to do triggers. Fernando Guerrero taught me how to do this, to write a trigger on insert that could notify your code somehow. And I can't it's been so long, I can't remember, but I remember actually showing this in a in a webinar, because as of SQL Server two thousand and five, you could make a dot net call out of a store procedure. Yeah, that's right, So you could have embedded in that on insert trigger on an external call like at gRPC or something like that. You're
you're interesting, fifteen years too early for GRP. No, but now you could, right. I mean they didn't take the runtime out of SQL service. No, No, it's still air. It's off by default, but it's always been off by default. Yeah, people abuse it. You know, you don't want to put an external call into an aggregate function. No,
bad, that would be bad, Serge. Were there any features that you added that came from your customers or you know, people who are using it I'm sure there were, But do you have any stories about that? That's a great question, because thank you. The whole story of Ganphar is everything is mixed together. Customer greary is my own thoughts. So it's I don't know how to say it in English, so yeah, no, I will really with everybody's input, Yeah, yeah, yeah, yeah, everybody
was. I was involved. I've tried hard to understand customers, to understand developers, and implement things as simple as it possible, because because you know, no one wants to spend time supporting difficult things. And what about crazy ideas? Does anybody come up to you like a crazy idea that after about minute thought you just said, no, what you talk? What's that stupid idea? I think the most craziest idea was my decision to add support of
both no Skill and Relational Skill databases in the early days. And no one told me don't do this. Trial by hang fire. Yeah, because it's really difficult to mix so different things in one a BI and to make them work. Because even now, when I'm trying to document storage API, every time, I understand I can do this right now because there there are so many words missing, and there are a lot of concepts missing when you need
to make friendship between locks, transactions and other things. Yeah, are you using on the back end you go all the way down the threads or are you trying to stick the tasks in a task parallel library in a sink away. That's that's another great question. Thanks. That's what I do, by the way, And I started by using threads. Then I switched to tasks,
but on running task that are threats under the surface. But then I even tried hard to build a synchronous API as synchronous processes asynchronous not a synchronous process us. Yeah. Yeah, I'm still learning. I was learning. I was learning English by writing documentation to him fire awesome. So I'm still trying. You know, you're doing great. You're doing great. I'm thinking
back with it. Stephen Taboo said, this is like, listen, there's only three people that really understand multi threaded programming, and two of them are lying the other one. I used to teach multi threaded programming as a concept, and what I did was I used Gary Kasparov actually as an example. I said, Gary Kasparoff is playing chess in the park with those big stone chess boards. You know, they are arranged in a circle like Stonehenge,
and he's got you know, twenty people. He's playing twenty games at once. He plays, he makes a move, he goes to the next one. He makes a move, and he goes to the next one. And so that's how he can play twenty games at the same time, but just make one move at a time. And that's kind of the idea of threads.
You know, down in the operating system that the OS gives attention to one thread at a time and makes a certain number of moves, a certain number of registers or whatever, makes a call, and then moves on to the next one. So it's kind of fortuitous that, you know, Gary casper off Man, he's a multi threaded man. Yeah, he's a multi threaded man, both inside and outside his brain. Yeah, yeah, I think we should take a break for just a moment. Yeah, let's do
that. We'll be right back after these important messages and we're back. You're listening to dot net Rocks. I'm Carl Franklin, and we're talking with Sergei Odie Knockoff about hang fire. We were just talking about whether or not you know, you started with threads then you went to tasks that use threads on the back end, and tasks men all the way. That's what I say. Yeah, life is better that way. I hope I never have to use threads. How about locking mechanisms? Do you just use basic lock objects
or do use semaphores? Or how you know? What's your what's your? Do your locking mechanism? We can't use a regular our logs or semaphos because they work only inside the same process, right, Yeah, you're right, and in far since we can move process into another machine to use something else.
There is an abstraction in storage API called distributed lock. This was one of the most difficult points to make them work and to make them work realiably because when talking about hand fire, we can expect unexpected shut down in every line. So not only distributed locks should work well, but the whole storage should work well, because we shouldn't leave abandoned logs because they should be eventually released, and we should somehow mix them with transactions to make out right to
succeed. And hand fire is prepared for expect to shut down every line, and it is really difficult to program with this in mind. It looks like reddis supports, or you could do distributed locks in reddist and maybe even something called zoo keeper. Have you looked at that? Zoo Keeper sounds complicated. This whole thing sounds complicated. Yeah, yeah, yeah, yeah, that's
why I'm trying to understand why it's so complicated. It's ready in near bio, so infant far Distributed logs are tied with transactions because radis supports transactions in a slightly different way than a relational databases, but they work anyway. So I read that you have a retry circuit breaker, number of retry attemps, you can customize your retry strategy, and then I wondered, hey, is he using polly and I didn't see that you are, So you're doing your
own retri logic. I tried not to introduce a lot of dependencies into hand fire because dependency eventually carted. Dependency help. Yeah, it is in the dot net framework. You can create an HDDP client factory with strategies or policies built in for retry. Is that what you're using? No, No, you're doing your own. They're not so complicated or so. Yeah, but you do allow the developer to specify and customize the retry strategy if it doesn't.
Yes, yea to some extent, because it depends on complexity, on the number of fissures tries in hand. Fire not required to be difficult, not required to be complicated. So they just work. Yeah, I JW technology, It just works. It works. I think we invented a new slogan and fire good. I noticed in the pricing model and it includes licensing that the open versions are LGPL three licenses. Can you talk about that choice?
Obviously this is probably decision you made ten years ago. Yes. First of all, it was based on Sidekick, right, I asked Mike Perhams my copy some things you told Yes. So there are a lot of things
when you start your project. You want to concentrate on features, on first implementations, and there are a lot of everything else in your project, like licensing, like side like some kind of marketing when you introduce your project, and you can do all the things together so you can copy least important things and concentrate on more important things for you. The same story was with lesser
GPL license. Some people who are asking to move to mt license on the course of years, but don't know how to say this, I mean licensing works and I'm afraid of touching. Yeah. No, changing a license is very challenging. Yes, yes, because it can lead to a lot of consequences you can predict and lesser. GPL works well for even for commercial use. It can people sometimes confuse it with a GP license and and scared that
they will need to open sources of their own applications. Well, and that's the perception of all GPR licenses, just not understanding that LGPL isn't like that. So I didn't know that. I just learned that just now I thought anything GPL meant. I if I were to say, implement poly around you know, my retry policies, I would have to then submit that back to the project. But it's difficult, it's complicated. It's complicated. Yes,
they allow commercial use and this phrase is highlighted on the site. Absolutely, and LGPL is it exists for how many years? A long time? More than twenty years? Yeah, yeah, so I think time. And actually there are a lot of sites that describe the differences, yes, of each license nowadays. So sometimes I get these questions, I answer, I answer
them. People I can use it, so I can I wonder if you don't have developers who have a limited amount of knowledge about over source licensing, but they've been told by their company as long as it's MIT, it's fine, and that's all they know and so, and that's why they're pushing on you. It is like, could I have MIT please, because my boss already told me that's okay. Yeah, these companies can buy a commercial license, and I thought that was so. As soon as I saw this page,
I thought this is brilliant. If you're not willing to learn the rules of LGPL, you can pay to make it go away. Yeah yeah, yeah, sure, so smart because if you learn the rules, I mean all LGPR well and correct me if I'm wrong. Here says is if you make improvements to this, if you make changes to it, contribute them back. Yeah. Yeah. One of the main points is LGBL license allows to
avoid competitions with ourselves. Right, Okay, when you have twenty fire like projects and you have to compute with your own code, right, it's strench for me. Okay. Can we talk about the batch idea that's in the pro version, and the provision is the paid version obviously, but batches and batch continuations. I mean, anybody who's figured out and listening by now could
probably figure out what we're talking about here. A batches a group of background jobs created automatically and considered as a single entity, and then a batch continuation is fired when all the background jobs and a parent batch finish. Is there also a way to know when each job in the batch has finished? Is that just a regular continuation continue job with In other words, can you do both of these together? If I have a batch continuation that fires when all
of them happen, can I also know when individual jobs fire? You can even create nested batches and nest that. That's crazy, you know, mister, and and and nested batch continuation that continue and other continuations. So yikes, it's possible to mix things and build a really complex work close. When I hear you describe that, I'm like, wow, it's your foot. You know what's scary out there? Somewhere there's somebody listening, say hey,
I could configure this with YAML. In the early days of Fire development, I was talking a one developer who advised me to use XML for configure and everything. But he was from Java world, so I understand him. It's yeah, Jason, all the way. Yeah. Oh man, so no,
thank you. Any piece of software I've ever met that's run for ten years has changed thessigence decisions along the way at some point that just that seems inevitable, the same way you started in threads and then moved up to tasks, Like, there's got to be other cases where you just find I have to change to do this. Well, now I'm trying to obstruct him far as much as possible, and the foundations like threads are not going anywhere,
so yeah, be something you can count on. Even the netflatform itself changed a lot, but yeah, fire looks the same, so yeah, your code's the same. What was there any difficulty for you switching implementations or having implications for both the framework and for core. The most difficult things was the early days of dot net core with lack of development tools support because everything was changing. I mean, god coverage tools called analysis tools and unit testing continuous.
Well, I mean you were through that era, coming right from twenty thirteen of the early days of dot net core. Yeah, but even the early days of how you do CICD pipelines like that's evolved a lot in the past ten years, but the concepts are the same. Foundations are the same, so I think I think not so much things changed added. A lot of things added, but not changed. Core concepts stay the same. Before we before we get off is can you talk about job filters. That seems
like a very powerful feature. Yeah, job builders, I think it's the most powerful feature from Fire. Every extension, every paid package depends on them, and they allowed to extend can fire so you can implement new things, more complex use cases, more integration with application without modifying hand far and without
rebuilding it or without using your own version of FAR. Extensibility was one of the main points during the initial initial development because it's not possible to build a solution that will suit everyone, and you have to make some extension points that are powerful enough to make everyone happy. Yeah, very cool. So it's kind of like, and it says this right here, they're kind of like they're similar to Aspen at NBC Action Filters. Yes, I've tried to copy
the concepts because their concept was already from sp developers. Is there any flavor of dot net or implementation on a particular system of dot net where certain features aren't available or the syntax is different or is it just so easy that it works everywhere on every flavor of dot net? Another good question. The most problematic one was the first version of dot net core. Oh yeah, nobody uses that anymore. Yeah yeah, yeah, but it's still supported by him
fire. Both users really appreciate that. Okay, you're welcome if you listen, if you listen to us. Okay, but the net standard, a lot of features are resurrected. It's a big kudos for the net developers because they did great job for making it run and even feel the same on different
platforms. In your list of features to implement, is there anything so crazy that you feel like you're going to have to take a week and go into a cabin and do a Ted Kazinski somewhere and just you know, is there anything really on your you know, on your implementation list that you're not looking
forward to. I think it relates again, different storage type support. It I dream implementing some kind of integration testing for that will work for different storages, so you don't have to write dedicated test suits for new storages, so you can just use it to test even new storages. Some kind of acceptance
tests. Yeah, have you ever considered taking the database out of the picture, except for maybe something to persist on the server, but you know, using a communication network like g r p C or something to go directly to the server thing, and then the server can just like keep a keep a local cache of that in case you know, it has to reset or reboot or something. Moreover, there are interesting projects available on GitHub like rocks dB.
How do you spell that, rocks dB, Rocks database, Rocks database, Rocks, rocks r O c k S. Yeah, yeah, like rocks but rocks I see rocks D. Caveman used to use these for for keeping track of things. It's it's an embedable database. You can add it
to your own application. So you can indeed build a gRPC protocol and use it in another high performance Okay, so in that case you would have you'd have to write an abstraction over the client and instead of hooking directly into the server process or whatever it is, the server, you would put a little messaging between that and then you would run the client and server, you know, logical server together wherever you're implementing that, I kind of get it.
Yeah, So at the bottom line here is it's wicked extensible. I mean, you've got the source for crying out loud. You're a programmer, figure it out. Not you, sir, I'm talking about the listener for using hang fire. You're a developer. That's what they pay you for. Okay, Okay, thirty years and I implement everything but no support regress for thirty years and everything will be done. So what's next for you? What are you going to be doing right after you get off this call besides going to
bed. We're going to hear tomorrow because one of the most difficult things for today was to make my children sleep. Uh huh yeah, I succeeded ten minutes before our show. Oh great, well, well done. Yeah, how do you get two and five? Yes? Okay, chaos monkeys, chaos monkeys? Yeah, all right, Well, enjoy the wonderful foods and smells of Thailand and we wish you success with hang fire, and thanks for talking to us. Thank you for the invitation and for the talk speaker.
You good questions, great, thanks, thank you, and we'll talk to you next time on dot net rocks. Dot net Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at d O T N E T R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives, going
back to show number one, recorded in September two thousand and two. It makes you check out our sponsors. They keep us in business. Now go write some code. See you next time. You got tad middle vans down home. Then