.NET Source Generators with Jason Bock - podcast episode cover

.NET Source Generators with Jason Bock

Feb 05, 202653 min
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

Why would you write code to generate code? Carl and Richard talk with Jason Bock about his experiences using modern .NET source generators to optimize certain aspects of applications. Jason talks about treading carefully - while .NET source generation has been part of .NET since 5.0 and Roslyn, it is a special case approach to problem solving. But with specialized implementations for regex and P/Invoke, there is some huge potential in these coding techniques that you can take advantage of!

Transcript

Speaker 1

How'd you like to listen to dot net rocks with no ads? Easy? Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month, we'll get you that and a special dot net Rocks patron mug. Sign up now at patreon dot dot NetRocks dot com. Hey, and welcome back to dot net rocks. I'm Carl Franklin, I'm Richard Campbell, and Jason Bacha is here with us. We're gonna be talking to him in

a minute. And of course, Jason, while we're doing the intro. You can feel free to jump in at any time. But we got a few things to do first. Should we talk about nineteen eighty eight and what happened?

Speaker 2

Do the history thing? Huh?

Speaker 1

The history thing?

Speaker 2

Okay, we'll try and keep it brief. But what do you got?

Speaker 1

Soviet Union began significant economic restructre churing you might have remembered Peristroika Peristroika and started to break apart and allowed some free travel.

Speaker 2

This is the beginning of the end, beginning of the end.

Speaker 1

YEP. George H. W. Bush was elected President of the United States defeating Michael Ducacas. I ran air flight six fifty five the USS Vincennes Vincennis mistakenly shot down. I ran air flight sixty five, six fifty five, killing all two hundred ninety people on board. At least that's what my report says. Did they determine that it was an accident?

Speaker 2

Yep, yeah, okay, it was yeah, you know operator air Ultimately, let's see.

Speaker 1

The lockerby bombing, of course, was a kind of historic thing that happened a lot of flight stuff. Here a hailstorm in Nepal at a soccer match and Katmandu caused a stampede, killing at least seventy people.

Speaker 2

Wow.

Speaker 1

We had the nineteen eighty eight Winter Olympics in Calgary, Canada.

Speaker 2

Yeah, interested the right of me.

Speaker 1

And this Olympics in Seoul, South Korea.

Speaker 3

That's back when they still had them in four year intervals.

Speaker 2

Yeah, all the same years. Yeah, I mean, then we figured out to offset them because you make more revenue that way.

Speaker 1

Andy Gibb died that year. And let's talk about some movies, right.

Speaker 2

M hm, So you like your movie collection in the eighties. This just shows our bias, I think.

Speaker 1

Yeah, die Hard you know, yeah, die Hard is on that list.

Speaker 2

Yeah. Best Christmas movie ever?

Speaker 1

Yes, kaym who framed Roger Rabbit was the number one grossing movie of nineteen eighty eight. Rain Man Coming to America.

Speaker 2

Oh man, I saw Eddie. I can't even tell you how many times I've seen that movie.

Speaker 1

Yeah, Yeah, Rambo, three big, three men in a baby beetlejuice, cocktail, and my favorite, a fish called Wanda. The car, sing It Sing It the Car.

Speaker 3

The best scene in that movie he was when he was going to get run over by a steamroller or whatever.

Speaker 2

That was right. Yeah, the whole trying to get the old lady and keep hitting the dogs. Like, there's so many funny bits in that movie. That's hilarious. And do you remember when they did the press tour and they all agreed, all of the big you know, Kleine and Clease and all these folks all agreed to say the same thing, which was, I carried this movie. The others are worthless, right, And so like every single interview, each one of them said exactly. They they just sort of blew up.

Speaker 3

I had no idea about that.

Speaker 2

That's great, It's just hysterical.

Speaker 1

So one of the iconic scenes in that movie is where Kevin Kleine is holding John Cleese by his legs, dangling him outside of a window to get him to apologize for calling him stupid.

Speaker 2

Apologize.

Speaker 1

And when Cleese was on the tour promoting movie, he was on Letterman and he said, he said, well, the whole apology thing is Britain's just, you know, they apologize for everything. You know, you're in America and you're at the dinner table. It's like a saw in Britain's are like, I'm sorry. So he was poking fun at BRIT's in their apologety ways.

Speaker 2

That's so funny. All right, what do you got, Richard? We'll start space. Of course, the return of the Shuttle after the Challenger disaster in January of eighty six ers now in nineteen eighty eight, so it's September twenty ninth. Discovery is return to flight, deploying the third of the TDR satellites. Use the satellites to allow the Shuttle be to continue his communication and that was the third and

last needed now though the member replacements in there. And then later that later Atlantis will fly one of the last military payloads to Shuttle. Well or Fly, which is one of the Lacross satellites, which should be its own show. So we're not gonna talk about that, okay.

Speaker 3

Because they had to take a two year break in between Challenger.

Speaker 1

Yeah.

Speaker 2

From Challengers Yeah and last show in eighty seven, I talked about the Rogers Commission and just the complete failure of NASA to be able to change their culture. They just fixed the tech stuff, but also just committed to the reality that they would never fly that many shuttles, and they never would again. Yeah, the shuttle was supposed to be an airliner. It simply wasn't, which that argument

is the reason we even have the geek outs. Carls, you told me we should record that, and I said that was stupid, and you said, but I was wrong, Clearly I was wrong.

Speaker 1

Well, I don't remember where we were, but we were having scotch after a conference somewhere.

Speaker 2

It was when Atlantis landed for the last time at twenty eleven. That's why I was raging.

Speaker 1

You were brain dropping. I mean, it was just like NonStop, and I'm like, you know, this would just be a great show.

Speaker 2

Yeah.

Speaker 3

The whole history behind the Space Shuttle program is just fascinating.

Speaker 2

Yeah, but also terrible. Yes, it cost a lot of lives and money. And I remember who was it, Dan, the old NASA administrator, saying, if we had not done shuttles, we'd stuck with Saturn. We would have done six missions a year, including two to the Moon every year the entire time, for the same amount of money. Yeah, presumably it would have improved things, but you know they always argument is it didn't spend enough on shuttle to make

it as good as it needed to be. They just limped it over the line and it limped the whole time. Speaking of shuttles, in nineteen eighty eight, we have the second flight of the Energy A booster by the Soviet Union, the only flight of the Burran the Russian shuttle the Soviet Union trying to keep up with the Americans, arguably a superior shuttle. It was flown unmanned. All of the engines were actually on the Energy A boosters, so it didn't have to carry that weight into space, did need

the external tank. I didn't do any of those things, but it was it was a very capable machine. It's terrible that they built three of them, flew one of them, and then they sat in hangars until those hangars clopped around them. So that's the end of that. And Energy it would never fly again. An Energy actually had a design again, this could be a whole show to have recoverable flyback boosters and things like. They really did pursue all the things that Shuttle could have had that never

actually happened. Other than that, you know, nineteen eighty is a quiet year for space. The Soviet Union's running out of money, so they don't fly. A whole bunch of the Americans are just getting back on their feet again. So that's all I got to say about that. But on the computing side, let's go.

Speaker 1

For a ride. Come some good stuff, some good.

Speaker 2

Stuff, right, I got to lead off with the next. So Steve Jobs, who've been forced out of Apple in nineteen eighty five, has now founded a company called Next. He builds the most outrageous computer ever, the Next Cube, which actually has three Motorola processes and an eight megs a RAM in nineteen eighty eight, eard of yeah, sixty five hundred dollars US in nine.

Speaker 1

I can't remember where I saw it, but I saw it and I used it, and I.

Speaker 2

Was well blown away. To be clear, this is the machine that Tim berners Lee will invent the World Wide Web with. Right, His next cube is insern behind glass and it's still got the sticker on it says do not turn off server. For the first web server and web browser were on that machine. It's great. Also, that's when Next licenses objective C. That will come back to haunt us all for a couple of decades.

Speaker 3

Thank you for expressing it that way.

Speaker 1

That haunt is right. What was a Rory Bleise's comment about objective C programm Stockholm syndrome?

Speaker 2

And of course we don't know it in nineteen eighty eight, but eight years later in ninety six, Apple will acquire Next and get Jobs back and so forth. Also related to Apple.

Speaker 3

There's one project that I did real quick on objective C and that was enough.

Speaker 2

Yeah, I oh man, terrible. Well, you know then it was always supposed to be a internal language, right, Jobs had never intended us to write directly to the iPhone at all. Those are all internal tools. But when we jail broke the phone, he had to do something, and that's when he came up with a store and released all those internal tools. Is why they were so terrible, you know, and admittedly like I'm not a big Swift fan, but objectively, no pun intended, Swift is a pretty damn

good language. That's what it should have been. Yeah. Apple, same year nineteen eighty eight files the copyright infringement lawsuit against Microsoft for the look and feel of Windows two against the Mac os. This is the beginning of that, and it won't be settled till the nineteen eighty five. The only thing that will take longer is Apple acquiring next, but not by much.

Speaker 1

I wrote a program that if Windows equals Mac call FTC.

Speaker 2

Ninety eight, the first release of Microsoft Office, the bundling of Word and Excel and PowerPoint and eventually Access and others.

Speaker 3

That was the first year nineteen eighty eight.

Speaker 2

Wow, first year was nineteen eighty eight, and it came on three and a half inch fluppies, how many and lots of of them? Yeah, twenty eight was always bad. Oh then you'd only find that out the hard way. Ask you you know that happened to me too.

Speaker 1

I don't know if it was twenty eight, but it was one of the one of the ones at the end.

Speaker 2

Yeah. Uh, sound Blaster Creative Labs, Yes, eleven Voice FM synthesizer, Texas Speech MIDI and a joystick port dun dun dum.

Speaker 3

I got a sound blaster card in college and that was Oh, that was so much fun.

Speaker 2

So it was such a big deal.

Speaker 1

Yeah, did you play any of the sample MIDI files?

Speaker 3

I think I did. I think it also had if I remember right, it had some kind of like a voice pitch, yeah, the parrot. But yeah, so you could you could change your voice. And of course in college that was so much fun.

Speaker 2

It's all you would do, doing well if you ran the MIDI files. I wrote those, Oh you did. I did well.

Speaker 1

Once the MIDI pack came out from Voyetra, which was in the nineties, and we'll get there. Yeah. Yeah, I did the MIDI files that came with that, but they would just work for hire, you know.

Speaker 2

That was my job, the musician with the text chops, so you make it work any got Yeah, MIDI was part of that. A couple other hardware pieces, the IBM AS four hundred sucked up a few years of my life. The Helid Packard desk jet printer, the first commercial inkjet printer. Wow, that's beginning the true demise of printing worldwide. Right, These days. They are essentially disposable. Right, it's cheap printed by Ink for it.

Speaker 1

Yeah, I was gonna say. And the onset of the Ink Subscription.

Speaker 2

Revolution Intel three eighty six s X released that year by Intel. Okay, let's talk Internet stuff. This is the release of the specifications for S and MP NTP right in the network Protocol and IRC Internet Relay Chat. I'll begin in nineteen eighty eight.

Speaker 1

Oh did you say NTP or n NTP NTP. Okay, that's different.

Speaker 2

And NTK comes a little later. Yeah, And finally the Morris Worm. So Robert Morris, twenty three year old father works for the NSA, gets access to the Internet, writes a non destructive propagating worm that is eventually found on six thousand of the sixty thousand hosts currently connected to the Internet nineteen eighty nikes ten percent of the Internet, and he was the first person ever committed under the Computer Fraud and Abuse sack. Wow, we're writing a propagator.

There you go, There you go. That's my story. I could mention Sega Genesis, but eh, you know video. Yeah, she's oh in the McChicken.

Speaker 1

Don't forget the McChicken, No kidding, I think that was the Larry Bird commercial, right, yeah, oh.

Speaker 2

In lunchables plunchables do.

Speaker 1

Oh wow, Well, the McChicken Larry Bird commercials are funny because he kind of looks like a chicken and his last name is bird Bird. Yeah, he was the perfect guy to advertise the McChicken sandwich. All right, let's roll the music for better note of framework.

Speaker 2

Awesome. All right, man, what do you got?

Speaker 1

Here's a trending repoll on GitHub. This is from Microsoft Data Science for beginners.

Speaker 2

We love it.

Speaker 1

A curriculum Azure Cloud advocates at Microsoft are pleased to offer a ten week twenty lesson curriculum all about data science. Each lesson includes pre lesson in post lesson, quizzes, written instructions to complete the lesson, a solution, and an assignment.

Speaker 2

Wow.

Speaker 1

Our project based pedagogy allows you to learn while building a proven way for new skills to stick some good names behind it too.

Speaker 2

It's good stuff, nice well, and you know, speaks to the fact that there's just so much training available out there if you want to learn, take the time. And the fact is that Microsoft knows that there are less and less people interested in programming and data science at least at youngins and so this is a good way

to you know, to invest in the future. Well then demand you know, I'm going into high schools a few times a year, and I keep pointing at like, this is an area that as interesting as you if you're into figuring stuff out and finding prout, you know, digging through data, like, you'll find interesting things that can be really valuable to companies. You can make a company a lot of money and make yourself a lot of money in the process. So, yes, still a fine area to follow.

And it's it's the non insane part of machine learning too, right, Yeah, cool.

Speaker 1

So that's what I got. I was talking to us today, Richard Campbell.

Speaker 2

I went to the way back machine. My friend will not literally, I just went out. You know, I don't even know this. We have quite a few episodes of dot net Rocks, but I know we were talking. We're going to talk about a little about so our generation. It made me think of Kathleen Dollar, who we need to get back. I haven't talked to her in ages too. And Kathleen did a show with us back in twenty fourteen.

This is episode nine thirty eight. It's only a thousand shows so called co generation and Rosalind because that's the year that Roslyn comes out. Yeah, and got a ton of comments. I mean this was actually one of the tour shows. We had Kathleen fly in to Saint Louis, right and we did we did it in person in front of hers, so we had a big audience, you know, lots of fun with that. And get this, this is

a comment from twelve years ago. This is Stephen Morowski, another past guest on the show and somebody who's frequented running as it was a big DevOps believer, So you'll love this. Comedy goes ah around the thirty seven minute mark when the conversation detoured from PowerShell to script cs. Your script cs. Oh yeah, yeah, I've used it to DevOps. My it was great stuff at the time. Was that was that? Nikhel?

Speaker 1

I think that rings a bell?

Speaker 2

Nick Heil, Yeah, Karthur Parthy, something like that. From PowerShell to script CS to DevOps my hope sword. I love that you talked about building a PowerShell API and possibly co generating some of the plumbing. Then Kathleen implied the DevOps is a job which is a person with a split skill sheet and you didn't correct her Richard, Yeah, I would have. I should have, because DevOps is not

a job. Meanwhile, one of the true intents of DevOps was never really talked about or identified, talking between the teams and sharing ideas and concepts, yes, developers and operators and stakeholders, and keeping everybody communication together. That being said, the rest of the show is pretty awesome, so I'll let that mischaracteris position of DevOps slide this time. Thanks Steven twelve years.

Speaker 1

Ago keeping us honest.

Speaker 2

I enjoyed the conversation around languages used by devs and ops coming together, but Kathleen hit on a key point. PowerShell is a shell first and foremost it could be used for programming, but it is a shell and offers shell semantics. Sea sharp is a programming language and script cs offers the ability to write scripts in it. But it's still not Oh and I cannot wait to see what Kathleen kicks out around ETW. That's event tracing for Windows. This is all about the telemetry back twelve years ago.

That might fill a huge gap in operations ability to tap into the ETW stream in a more straightforward manner. And I don't know that it ever became anything. Of course, these days that's not even how you think about. It's all open telemetry, right, you've changed the whole standard for that. Hey, Steven, nice flash from the past, man, Thanks so much for that. And a coffee of music Koba is on its way to you. And if you'd like a copy of music, go buy. I read a comment on the website at

donat Rocks dot com or on the facebooks. We publish every show there and if you comment there and I read on the show, we'll send you a copy of music Goba.

Speaker 1

You know, we did an episode with Glenn Block on script CS March tenth, twenty fifteen.

Speaker 2

Yeah, and you know what this one? That was the period where the roslins Sharp run time didn't have a license for you to use it freely, so you had to force anybody using it to download it themselves. Yeah, I could had all this weirdness around it because this is before open source and before that whole mindset. Although it's imminent, right this is January of twenty fourteen. We published this, like in April they will release rosalin As on GitHub as cross platform open So it.

Speaker 1

Was Glenn Block that started was CS so March twelve, twenty thirteen, we did episode eight fifty three with him and it's called Glenn Block goes open source with script cs.

Speaker 2

Right, so there you go. Sort of we're still battling with what that would even mean, but right, it's funny in the context you just bring it up because this is from January twenty fourteen. Four months later at the Bills conference, they'll announced it.

Speaker 1

Yeah, so there you go. And Richard mentioned music to code Buy still going strong and have downloads every day, twenty two tracks currently. You can get him in wave, flak or MP three format. All right, Jason, it's your turn. Jason Bock is a staff software engineer at Rocket Mortgage and a Microsoft MVP for c Sharp. You can find out more about him at jasonbok dot net. Welcome Jason, thanks for having me.

Speaker 3

I appreciate being here, looking forward to it.

Speaker 1

Yeah, so we're talking about source generation or source generators and kind of an open topic. So let's just start with the elevator pitch. What is a source generator and how does it work?

Speaker 3

So a source generator is a feature in the compiler. You know, Richard, you already mentioned Roz and that's the compiler underneath the scenes for c sharp and bend that way since twenty fifteen. And initially there were extension points in the compiler where you could write analyzers, code fixes, and refactorings, which is cool. That has a lot of interesting stuff and it plus the Holy API was there. You could build whatever tools you wanted on top of

that to do code analysis. So it really a lot of developer to do all sorts of wild interesting things and also beneficial for other developers. At some point they added in another extension point. I believe this was for c sharp nine, and it's called source generators. So effectively it allows you during the compilation process to look at code that currently exists in that compilation and say, hey, I want to generate some code, and there's a couple

of reasons why you want to do that. But you participate when you write, When you author a source generator, you participate in that compilation pipeline. And so you can see literally, like in an idea, as somebody's changing code, you can react to that and potentially change what you generated if the use case that you're trying to hit needs to do something like that. So it's native to the compiler, it's not like a tool that you do

cogeneration as a separate step. Though there's nothing wrong with that, but you don't have to have that separate tool. Do all that you were talking beforehand about tool or pre existing tools like T four T four templates is something that I think if you've been in dotnet for a while, you probably have heard of again, a nice fine tool, except that's something you have to manually run. So source generators are part and parcel of the Roslin compilation pipeline.

Speaker 2

Yeah, you know what. I found the blog post by Philip Carter, who was the PM at the time, Yeah, for the preview of source generators in DAT five.

Speaker 3

Yes, yep.

Speaker 1

And we had talked about them a couple of times. And one thing I can remember is the verb squirt you want to just squirt some code in there.

Speaker 3

I'm not going to be using that. I still use that term.

Speaker 1

Or I laughed at the time too, Richard. I laughed like Richard's doing right now.

Speaker 2

I'm offering at how uncomfortable Jason is.

Speaker 3

That's what's funny.

Speaker 1

You know, you just want to squirt a little code in there.

Speaker 2

There really no no code squirting.

Speaker 3

You can use that curl I probably.

Speaker 1

Won't actually picked it up from the Microsoft guests we were interviewing at the time. I don't remember who it was, but it wasn't my word. I'm just a messenger.

Speaker 2

But I mean, yeah, so now I can't even see code injection because it's ruined. Now you've read it, sorry, like about code injection.

Speaker 1

So I'm taking it that the answer to my question is now they don't use that term anymore.

Speaker 3

I haven't heard of that related to source generators. And maybe it was Von Ever and I don't think I ever would have used that as a way to describe source generators.

Speaker 1

Maybe it was Eva Lowe that might have been talking about it. I don't all right, I've I've derailed this long enough. Let's move on. So the source genera the source generator in the compiler. Then how do you decide when and what to uh, you know generate? Is it something that is just built into the pipeline that you configure, you know, if this is that, then put this code in or I don't understand.

Speaker 3

So there's I'll kind of break the answer up into two parts. There's the kind of the mechanism of what you do if you're authoring a source generator, and then maybe why you'd want to do this in the first place. So, okay, the the what or the how part of it is you essentially write a class library where you have a class that implements one interface I incremental generator. There's another one, but that's considered obsolete and I'm not even going to bring it up because I don't want anybody to even

use it anymore. That's the c sharp team really wants people. If you're offering a source generator, use I incremental generator, okay. And what you do in the implementation of that interface is you effectively want to trigger your source generation off of the existence of an attribute. So an attribute exists somewhere in code, and your source generator is looking for that attribute, and when it shows up, then you're going

to do whatever you want it to do. And we can cover some examples later on, but that's really the trigger point. You see that attribute there, and then you say, okay, that attribute is maybe on a class, or it's at the assembly level, or it's defined on a method. And so now what I'm going to do is I have all the information of the compilation, the syntax, the semantics, all that stuff, and so now I can say, well,

this attribute'es on a class. So maybe I want to look for all the properties on a class and I'll use the rosal API to find that stuff and potentially build another class. Or maybe the class is partial, so I want to extend that class with some other implementation so that all gets combined up during compilation. So that's what as an author of a source generator is going to do. You'll define an attribute, you'll write an implementation of this interface, and then you'll basically look for that

attribute to exist and generate some code. Now that's some mechanics, but I think it's also important to discuss why would you want to do this in the first place. Sure, what's the benefit of it. There's a couple of things that go at least for me that when I've written source generators that I try to keep in mind. One of them is does this it makes something that a developer has had to do and take it essentially out

of their hands because it's repetitive. It's something that you know is very mechanical, mechanistic, and you don't as a developer, can do it. It's just time consuming and I don't want to do it. So that's one place where source generators can be helpful because it takes that out of your hands. You can just put an attribute somewhere and the source generator does it all for you.

Speaker 1

So I think what you're getting at is something that you'd have to do a lot, like duplicate over a collection or something like that.

Speaker 3

Yeah, and not all the time, but that's definitely something that if you see you might go that might be a case to write a source generator so that I don't have to do exactly that all the time. Another one is, for example, if I am building an application

and I want to do ahead of time compilation. So if people know what that is, it's listening to this ahead of time compilation is essentially saying I've got my code, but instead of building it so that they're done to assemblies, I'm going to build it so that it's targeting a specific run time like Windows or MAC or something like that. And so it does some things in the AOT compilation step so that it's specific to that run time.

Speaker 1

But don't the pragma things that you know, the hashtag if this OS, then do this or if that OS. Isn't that kind of addressing the same problem.

Speaker 3

It can just for you know, depending on your case, that you're just trying to say, hey, if I'm running on a particular run time, I want to do this. If you're using some of the the types that are for Cindy instructions that you want to say, well, if my processor supports it, I need to make that check before I actually use them. This is saying I want

to have some optimizations done that I know. You know, I've written my code that's maybe more generalistic and I don't care about the runtime, but I wanted to run for these run times, specifically for potential performance benefits. The thing with source generators that we want to keep in mind with AOT compilation is that this is all code that's generated for compilation at compile time. If you've ever done things with reflection, that's all done at runtime, and

AOT in reflection is like oil and water. They don't mix. They really become problematic because AOT will see, well, you have this amount of code that's actually never being touched, So I'm going to just shake it out of the tree and get rid of it because I don't need it anymore. But then your code runs and goes, wait a minute, I expected this to be here for reflection reasons.

So that's another thing that is nice about source generation is that you can take those runtime reflection things that you're doing, make it a compile time process, and then they'll stick around with AOT as well.

Speaker 1

So it's very cool. Yeah, and all these things I'm trying to envision, you know, how you would do it without source generator. It's pretty obvious to see how something that's repetitive would be you know, impossible, especially if you know, don't know what those you know, what's in that list or how many there are, that kind of thing. But the other one I'm just still trying to wrap my head around.

Speaker 2

It's a very meta way to think, yes, yeah, I'm going to generate this code and with some parameters. You know. This is also it feels like the Java factory model too, right, that I beat a set of parameters and it spits out something for me. And this case is instead of just spitting out a class, is spitting out you know, potentially all kinds of things different.

Speaker 3

Yeah, and when you said the word meta, I was fortunate enough to be a co author in a book I came out like years ago. It's called metaprogramming in dot net and wrote it with Kevin Hazzard. Really good guy.

Speaker 2

Oh yeah, great guy.

Speaker 3

Yeah yeah, And that was one of the things he had the first chapter. We split the book in half and he got the first one, and he did a really good job at that first chapter, especially because he said, really everything with meta programming, people think about expression trees and reflection and aspector of programming and all these terms. And he said, beta programming is just treating your code as data.

Speaker 2

Right, that's great thinking. Yeah, it's just another kind of thing.

Speaker 3

And you're just looking at that data. Instead of looking at his code, you're looking at it probably as a tree, looking through the tree and saying, this happens here, So now I want to do something because of the existence of that data.

Speaker 2

Yeah. Well, and that's you know, that's what compilers do, right, They treat that code as data to turn into a set of actions in a machine. And we should take a break.

Speaker 1

Yeah, let's do that. We'll be right back after these very important messages. Stick around. Now we're back. It's dot net Rocks. I'm Carl Franklin. That's my buddy Richard Campbell either and that's Jason Bach you're listening to. We're talking about source generators, and I'm wondering, do you know if Microsoft is using source generators, you know, in their own code eating their own dog food?

Speaker 3

Yeah, I love I love that phrase. Yeah, eating eating your own dog food. They are. When source generators first came out, there wasn't anything in dot net proper, like a lot of developers like me and others were jumping on this because well it's so it's a cool thing, you know, but it was also helpful for certain circumstances. But it wasn't in as far as I remember, nothing in the dot need run time that did this. And

now at this point there there are implementations. So for example, I think one of the most prevalent ones is regular expressions. So I want to be very clear off the top, I am not a reg ex expert.

Speaker 1

To nobody is exactly said you where i'd have less respect for you.

Speaker 2

Yeah. No, there are noble to understand rejects, and one of them is lying, Yeah.

Speaker 3

There are things I know and there are things I don't know, and I have no problem talking about what I don't know and I don't know. If I want to look up how to parse something. I will go you know, now these days I'll just say, in copilot, how do I write it? Rejects for this?

Speaker 2

You know, yeah, you know, we say rejects is the ultimate cut and paste language. But but now it's clearly, you know, an ll M language.

Speaker 3

I'm very impressed by people who know it actually, who know it inside and out. It's it's it's kind of weird, but it's also very impressive because people can do some very cool things with it when when needed.

Speaker 2

Without a doubt. Yeah, it's like I feel like it's like PowerShell too. It's like one of those things. If you use it every day, it will stick, but if you don't use it every day, it fades fast. Yeah, yeah, it's not going to hang around.

Speaker 3

So with ready expressions in NET, they've always been something that's actually interpreted, compiled, and run time. So if you write your rejects, nothing happens until somebody wants to do a match or they want to do something with the rejects,

and that can lead to potentially some performance issues. Sure, So with rejects in C sharp, now what you can do is, I think the attribute is called generated rejects, but you put this attribute in your code, and you put the rejects string in the code or in the attribute I should say, and then there's a source generator that kicks in and will basically show you the implementation and see sharp of what that rejects will do. It's

actually wha wow. It's really cool because you can use tools like visual Studio or writer and you can see what that generated code looks like. So if you really want to know what it takes to implement a rejex, the source generator shows you. And I'll even give Microsoft

props on this that they went. I think that small but nice extra step in that when you have your rejects source generated to see sharp, they'll put in a whole bunch of XML comments and also just putting comments in the code, so if you really are interested in

what it does, you can follow it fairly well. And it's nice because it puts this header XML comment at the top of what it's doing, so when you hover over the attribute, you'll see you know this this rejects will do the following things, and they will tell you we're looking for characters zero to nine and then we're going to skip and do the end it's great, it's amazing. And again, all this now gets put in your compilation.

Speaker 1

Does it go back the other way? Does if you change the code, will it update the rejects? Yes?

Speaker 3

And that's also a great thing about source generators is that when you're writing one, if somebody's changing something underneath the scenes for you, you genera your code, that generator code is going to be triggered, and so then you can now say, well, they change rejects from this to this, So now I'm going to change my rejex implementation to what the new state is and you'll see that change as the thing you're targeting will change.

Speaker 1

That's very cool, n mind blown.

Speaker 2

So yeah, no, and super powerful. And you know, the guy, the person who wants to look at that code is really uncomfortable with source generators sort of nondeterministic effects. I want to see what this thing is doing. Yeah, And it occurs to me we're living with some really serious non deterministic code generators these days. We just call them large language models.

Speaker 3

Right yeah, and with source generators that they're actually very deterministic. If I put in this rejex, I'm going to get this output now every time, right now, the implementers of that source generator may or versions go hey, we can improve this, or we could change this, and so that generated code may change underneath the scenes. But me as the user, I don't care because I just want to essentially work and if.

Speaker 2

They need to make me as a developer, my care a bit, but not a whole lot. You should you expect the same results, Like if it doesn't work the same, that's a buck yes.

Speaker 3

Yes, exactly. But if it works the same, and maybe it doesn't allocate as much memory or it works a little faster because they made improvements, great, I don't. In fact, typically with source generators, by default they make files, but they're not really files that exist in your project. That may sound a little weird, but they're projects that are there are files that are part of the project. That's what gets compiled. If you really want to save them the disc and you want them to be part of

your repository, you can do that. But by default they're kind of these sort of a few you know, I can't even say the word right now, but these kind of ghost files, I'll use that term that they're there. They do exist, but they're not actually put to disc If that makes sense, so you're more if you're using something that as a source generation At least I feel this way as a developer, I just wanted to do what it needs to do it does it well? Does it match my coding style? Does it match any of

that stuff? I really don't care. I personally don't care. I wanted to actually just do what it needs to do and do it well. So regular expressions is one of them. In Microsoft, there's also places with p invokes, with serialization, logging configuration. You know that they've added in source generators for various advantageous scenarios.

Speaker 1

That's cool.

Speaker 2

I haven't heard P and voke in a while.

Speaker 3

Yeah, yeah, neither of I. I heard about it and I went, oh, that's great that they did that, and then it just went right out of my head because I haven't done a P and vulcan in decades.

Speaker 1

So can you give me a scenario where a developer working on a project is facing, you know, something that they have to do and a code a source generator would be the solution.

Speaker 3

It's a it's a good quest question because typically at least what and me as a developer day to day with the stuff that I do. It's not a common thing, and I think I want to stress that because while I think source generators are cool and if the opportunity arises,

people should use them. But I think that's a key part of this is that if the opportunity arises for a lot of things that we do as developers, I shouldn't say a lot, but there's typically even though there's things that we repeat, that doesn't necessarily mean we should go run off and do it as a source generator.

Speaker 2

Yeah, don't turn this into a hammer you whack everything.

Speaker 3

Well, yes, exactly. It's a great tool, but it's still a tool that you don't need to use that tool everywhere.

Speaker 2

It reminds me of aspect orated in general. Right, yeah, yeah, it's a way of thinking.

Speaker 3

It is it is, and it's also it's also again because it's part of that compilation pipeline, you have to be very aware of how fast it runs. If you're going to do like I've heard some people say, well, we generate this code because we read from a database. Nope, source generators don't do that enough, because you're going to kill performance as it looks in a tool like Visual Studio Writer. So don't do that at all. You can build a tool to do that. That's great. Don't use

a source generator for that. You really want to just look at things that are in code and then generate new code off of that. So a lot of the things that you see source generators written for are kind of these generalistic like patterns or things that people do, like serialization, like mocking. That's one of the things I wrote a source generator for, and funnily enough, cold rocks. So but that one uses source generation, you know. It's it's things that I think have a more general pattern

that you're going to write a source generator for. It's not maybe necessarily what you do specifically with things that are related to your business or whatever you're working on.

Speaker 2

Okay, fair enough.

Speaker 1

Yeah, if you do find yourself in that position where you know what, I think we need a source generator, here give us some tips for people when they're writing their first source generator.

Speaker 3

The first tip is the one I said before, which is considered not doing it. Yeah, and I know that I know it well, it's it sounds.

Speaker 2

You almost have to not care about performance, right like well.

Speaker 3

And it's also writing a source generator has kind of a I don't want to say it has a it's not an insurmountable cliff you got to come over. But there is something you do have to understand at least some parts of Rosmond. You don't have to be a compilation expert, but you have to understand what a sintexturee is, what a semantic model is, at least have somewhat of

a high level of that stuff. You have to understand how to build a source generator, which long story short, can be a little interesting, especially as you try to pasckage these things up into nu get packages. There's a couple of things over time that have been problematic that the community has kind of rallied around and figured out what's the best way to write these source generators. So

that's why I don't mean to discourage people. It's just if you think you want to write one, it might be a better approach just again, write a command line tool as a console app, or write a custom MS build task that might actually be easier.

Speaker 2

There's probably another way to do that. Some question is is it.

Speaker 1

Better maybe go find a package that's already written and that's popular, so that you can see how they did.

Speaker 2

It exactly be a good idea. I just said this horrible chill where it's like I invoke it. I use a source generator to call to an LLM to spin out code for me that I ain't started in my application. I mean, what could go wrong with that?

Speaker 3

A lot wrong with that? You know, Richard, go for it. That would be awesome, really cool, But.

Speaker 1

That concludes the anti pattern session that rock.

Speaker 2

Yeah the best.

Speaker 3

There's a lot of good resources out there now for people. There is some sites that actually list like all the known open source implementations for various source generators out there and the ones that Microsoft does in the box. There are some discord servers out there that people from the c sharp team are on, so if you have questions, you can go in there and if anybody knows, they know because they effectively built the features, so they'll they'll give you tips.

Speaker 2

I really like these, you know, source generators for Jacks, source generator for p and Vogue, like these rarely used technologies where sometimes it's a good solution to a problem and now I can introduce this additional mechanives that gives me more observability, more understanding of what's happening. Yes, that's exciting to me. That seems like one of my favorite cases here.

Speaker 3

Yeah. Yeah. But the other case that I briefly mentioned is the one with the mocking frameworks. With this one that I've created, I started it in twenty fifteen, and it was mostly just because, like I mentioned at the beginning, when Roslin first came out, it was analyzers co fixes for factories, and I wanted to sit there. I sat there and I went, what could I do with Roslin

That isn't one of these things? And so I looked at mocking frameworks, and I said, instead of using a way to generate types that run time with like castle Dynamic or Castle core and all that stuff, could I do that somewhat with just generating c sharp. So originally I just generated c sharp in this, and then I rosin was actually a very small part of it. It just compiled that c sharp.

Speaker 2

I had a lot about Castle project in ages a great point.

Speaker 3

But then when source generators came in as a feature, I went back and I said, okay, take all of what I did and move that into a source generator.

One other thing, well, one of the things with mentioning this that I think is also really important is that with a lot of the mocking frameworks that are out there, they use and I mentioned this before, the concept of an expression tree, which expression trees are these great little things in dot Net or I should say, She's sharp, but they really haven't been updated in quite a long time, and in fact, they're not going to be updated. That's actually been stale.

Speaker 2

Of course twenty seventeen. Dude, that's a long time ago, now, yeah.

Speaker 1

Wow.

Speaker 3

So there are features that have been added in C sharp that don't exist in an expression tree. And so the nice thing about writing a source generator is it's in C sharp. I can generate anything in C sharp, so there's no limitation as to what I can support. So for Rocks, one of the things, for example, that I can do is support restructs spans, which are becoming

more prevalent as time goes on. Other mocking frameworks can't because they're tied to expression trees and they won't be able to mock types that have these types within their definition. So that's another advantage of source generators. It's all c sharp. Anything you can do in C sharp, you can generate with a source generator.

Speaker 2

Right, Yeah, And it all comes back to Roslin this re entrant compiler. Essentially, this compiler is a service where you can create you can create stuff from that compiler that you can feed back to that compiler.

Speaker 3

Yeah, I mean I remember, you know, and you probably remember this too, But way back in PDC days, I started to hear that the you know, honors would be on stage with what's in new and C sharp and then he would hint right at the end this thing maybe called compilers a service, you know, and you would only talk about it for a couple of minutes. But I heard about that and I kind of got hooked. I was like, what is this, what is this new thing? And you know, as we found out, it was this

whole rewriting of the compiler. Yeah, and not just rewriting it in SEA sharp, but making it so they use a developer could use it. And I personally think it's one of the one of the best things or Microsoft ever did with dot.

Speaker 2

So that it was always an argument that they got to it too late, like it's supposed to be. The fun of the fundamental metrics of any language is that you can write the language, and the language they didn't get to it for a long time.

Speaker 1

I think it's funny that Anders chose to use like Columbo style to drop that one at the end.

Speaker 2

The one thing, well, and it's the story as I heard it, and let's face it, I'm the guy who's been collecking the story for a long time. Is they hit a point where you had the regular C sharp compiler and its features, and then you had the compiler that ran in visual studio to do all the intelligence for you, and they were having a tough time keeping those things in sync. It's like this should be the

same thing, Like why are we doing this twice? Yeah, And that was finally the impetus to spend the sheer amount of money it took to rewrite that.

Speaker 3

Yeah, I've heard numbers, and it does not sound like it was cheap, No.

Speaker 2

But it but it was going to get only more expensive. The other side of this was, and we did this on the show, was this sort of you know what really happened, all the C sharp developers, the folks that were building C sharp became C sharp developers because then they were C plus plus developers that's how they were writing C sharp And so suddenly, yeah, good point the shape of from C sharp six on this it changed because oh yeah, there was all these remarkable minds working

in c sharp every day and all of that. You know, I wish I could just I think on the Matt's targets you were talking to us, says this is a Cambrian explosion like this, and then look at what's happened, you know, eight versions later to c sharp. It's just a different beast now, and I would argue better in every way.

Speaker 3

Well with also it becoming open source, I think was a huge part of that too. You look at the pace and the evolution of c sharp before that and now after. It's funny because developers that I talked to sometimes or you'll hear in the community, it's almost like the pendulum has swung to the point of now they're complaining about it moving too fast. There's too many features, there's too many versions, and I can't keep up, and you know you're going to go, well, which one do

you want? But I think by them moving it to open source, that was also a big part of it, because now you know, people like you and me, we're you know, well maybe you're Richard, but you're not just going to go into rosalind clone it and say I want to add a new feature and submit a PR and it's going to go in. That doesn't happen.

Speaker 2

To be very clear, I am not going to do that. No, no, no, no, am I okay. And when we've actually had people on the show that became Microsoft employees because of their contributions into dot nowt Yes, but it's incredibly rare, like that is not a career path you should pursue, true kind of, you know.

Speaker 3

But there have been people that will, not not just with Roslin, but the Dune on time. They'll go in and maybe add in a new method to something, or maybe they'll do a little bit of an improvement. I think Ben Adams, if his name comes on my head, he did a lot of stuff underneath the scenes for

a while to help out. Those are just community members outside of Microsoft that just want to help this open source effort, and I think that's also really helped things evolve faster and also better over the years.

Speaker 2

Totally. Yeah, yeah, awesome stuff.

Speaker 1

So what's next for you?

Speaker 2

Man?

Speaker 1

What are you working on? For me?

Speaker 3

Yeah? Well, my day job, which you know this is all audio, but over my shoulder there I'm pointing. Now, that's that's my day job, and that's pretty much what I do. My focus is within Rocket and where I'm at and the team I'm on. We spend our time focused on building packages and libraries. You know, my focus is on dot net, but we also have people that do stuff with JavaScript. We've also we don't really evolve this, but there have been some stuff done in PHP that

we inherit. That's the fun of being in a corporation. Sometimes, sure that gets thrown.

Speaker 2

On you, but that code still makes the company money. Otherwise it wouldn't be there, exactly.

Speaker 1

Yeah.

Speaker 3

So, but most of my stuff is in dot net and just building stuff that other teams can use, other engineers can use to make things so they don't have to build that stuff.

Speaker 2

The definition of an enterprise architect, my friend, that's what it sounds like.

Speaker 1

Kinda yeah, so I take it some of that it has been source generators.

Speaker 3

Not yet, we haven't really found anything yet to do that internally, but I do keep that. I keep my eyes open to see if that's ever needed.

Speaker 2

This is a good answer to a particular problem the company has.

Speaker 3

Yeah, exactly. So that's you know, that's my day job. But I still go out. I still speak at conferences when I can and and if I can make that happen, I do that. Don't write books anymore for lots of reasons.

Speaker 1

I guess I write books anymore.

Speaker 3

Yeah, exactly.

Speaker 2

Writing one book is a good idea. You need to have that thing in your hand. It's an example of your effort. Writing more books means you're just not learning, right.

Speaker 3

I was about to say, like, I've been part of I think six or seven books in my life. Yea.

Speaker 2

Sometimes you have to learn these mistakes over and over again.

Speaker 3

Yeah. Yeah. The last one I wrote was in twenty fifteen. That was actually for something on Rosalind and I said, I'll do it if one of the stipulations is I don't have to pad the.

Speaker 2

Book, right, I just want to stick with the meat. It doesn't have to be eight hundred pages.

Speaker 3

Yeah, And so they were fine with that, and so it's like one hundred and fifty pages and like something like that. And that was great, and I sold tens of copies and I.

Speaker 2

Both guys, you bought it. Just awesome, Jason time.

Speaker 3

I did have somebody come up to me at a conference once and say to me, I really want to thank you for writing that book. Because I was doing my master's thesis on compilers and I was using C sharp and I was stuck in your book actually helped me and I was.

Speaker 2

Like, that's awesome, that's cool, that's great, that's cool here. You only get so many of those ever. Yeah, yeah, once in a while you can affect some people like that's a good day. That's why you really make that stuff.

Speaker 3

Yeah.

Speaker 2

I thought about this a lot. I would hope you don't have to. Here's an approach.

Speaker 1

Ye.

Speaker 3

So, but that's the last time I did it, and that's probably the last time I'll ever do it.

Speaker 2

Yeah. Yeah, the world's moved on. Yeah, very good.

Speaker 1

Well, the website is Jasonbock dot net. You can read all about him there. And Jason, thank you very much. This has been very informative.

Speaker 3

Thank you for having me. I appreciate it.

Speaker 1

Thanks you bet all right, We'll talk to you next time on dot net rocks. Dot net rocks is brought to you by Franklin's Net use by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two.

Speaker 2

Thousand and two.

Speaker 1

And make sure you check out our sponsors. They keep us in business. Now go write some code, See you next time you got javans am

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android