How'd you like to listen to dot net rocks with no ads?
Easy?
Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month, we'll get you that and a special dot net Rocks patron mug. Sign up now at patreon dot dot NetRocks dot com. Hey, welcome back to dot net rocks. I'm Carl Franklin, an amateur Campbell and this is episode in nineteen forty six.
Richard Yay, end of the war.
Yeah, we're inching closer, aren't we. Let's talk about what
happened in nineteen forty six. The of course, the post World War II world saw the establishment of the United Nations after the dissolution of the League of Nations, the Philippines gained independence, Italy abolished its monarchy, the US Atomic Energy Commission was established, the Soviet Union rejected a US proposal for an international agency to control nuclear energy production and research, and the ENIAC, the first electronic digital computer,
was dedicated. You got anything else to add to that list?
Well, I mean INIAC was electronic. It had eighteen thousand vacuum tubes. Yeah, so you know, you can you can debate how electronic it necessarily was. I was trying to figure out what the processing rate of it was, because it could do something like five thousand additions per second and it did have one hundred kilohertz clock. Wow that there was one table I found this at It runs at about five hundred flops or floating floating boat operations per second.
So I just remember Uniblab from the Jetsons. Do you remember that?
Yeah, a year in a blab And it was of course originally built to do artillery table calculations, right, that was three or four years earlier. But by the time they actually get it built, most protect stuff solved, and they've shrunk down artillery table calculators enough that they're now being put inside of battleships. The very first program that it ran was a Monti Carlo simulation for calculating neutron propagation in thermonuclear weapons.
How does Monty Carlo and thermonuclear weapons go together? In the same sentence, I'm glad you asked.
A Monty Carlo simulation is an idea that we don't know all of the variables. We just know some behavior of some elements. In this particular case, you're talking about neutron propagations of how neutrons move through certain materials, and so you're able to run multiple simultaneous calculations. It's too hard to do by hand. You needed a fast computer, and at that time, ENIAC was the fast computer that sort of proved that a Monte Carlo simulation is even possible.
So they call it a Monti Carlos simulation because it's related to gambling of some kind.
Yeah, that was partly at one of the scientists that have a degenerate uncle that was prone to gambling. They needed a code name, so it started out as kind of the code name. Okay, but it is a a probabilistic heroism calculator, which is kind of the way a lot of gambling numbers work anyway, So it all kinds
of goes together. There's a randomness factor to this, But I would point out it did demonstrate that beryllium was an effective reflector of neutrons at a certain rate, so they knew how thick to make the linings to be able to do the second stage explosions like it worked at bend the path that would take several years for them to actually build a working thermonucular device and go from kilotons to megatons. But this was the beginning of it.
An ediac was key to it. Wow, it's good to know history, kids.
Yeah, just listen to Uncle Richard and we do geek outs on these things occasionally once a year around the end of the year. So if you just search geek out on our air rocks dot com set, you can find a whole host of topics. All right, well, let's get started with better no framework roll the music. So what I have is a little tool that I wrote for Blazer. It's a Blazer component called AVN Audio AVN for app v next that records audio and provides buffers
in real time. So, knowing the topic that we're going to talk about with l S today, I pulled this up. This is not audio and video in dot net. This is Blazer. So it's web based audio. We're actually using a web based tool right now to record audio and video in real time. And the reason that it works is because the browsers now support these standards that make this stuff possible and finally obtainable for your average JavaScript developer.
And it's really remarkable. Like we're looking at our video in real time, we're listening to each other in almost real time. You know, there's a little delay, but you don't notice it.
It's close enough, right, that's the key.
It's not perfect, right, And it's uploading our audio in the background as we're recording it. And then by the time we're done, you know, maybe five ten seconds later, it's there. It's all there in the cloud. So the technology has really gotten good. And I use this opportunity to create a component that we'll start recording audio and then call you back in you know in Blazer and see sharp were the buffer and you can do whatever
you want with that buffer. You can compress it and stream it, you can write it to disc all that kind of stuff. So I thought that would be a fun little way to start just and it uses well, it used to use ffm peg, but there's one demo that does use it. So ffm peg is a command line tool for doing all kinds of things with audio and video and it works great and c sharp if you're using standard in Standard out. Just a great little tool.
Use it all the time anyway, That's what I got Richard, and who's talking to us today.
We don't do audio video too often, so I jumped into the wayback machine. I went back to twenty thirteenth episode eight point fifty four, when Mark Heath talked about audio and Windows. Yeah, specifically the project he was working on, which was an audio in CodePlex code plex. I love in audio and got it. We got a ton of comments on the show. I just grabbed one give mely twelve years ago. This from Jph says, Hey, Carl, what
a great geek out show. I'm sure some of the discussion took a lot of the listeners, including myself, back to the SoundBlaster days. Oh yeah, I remember getting my sound blaster back in the early nineties. I was thinking how magical it was to record and play wave files a computer, producing real time sounds. Magic. Yeah, And obviously we've come a long way since then, and things usually work pretty well on Windows when you just need a
standard audio playback. There's one thing though, when we all went from XP or Vista to Win seven and most people with built in sound cards lost the capability record exactly what you hear as standard out of the output back, which it can awesome be useful. There's also an option to you in the Windows Sound Mixer, but for some reason we've lost that and it was you know, it was in there somewhere, but they were messing like how many different flipping control panels do you need? Right, It's crazy.
Discussion about doing real time audio processing and showing wave forms of meters during recording reminded me of something that has in fact only a tangent but appealed to me as a dev when I was tickering with MIDI controllers. Hey, and it may be helpful people who don't want to shallow a lot of cash for MIDI hardware. Touch osc is an application for touch devices, iOS and Android admittedly
is in twenty thirteen. Right I moved on that lets you define a rich MIDI control surface with buttons, rotary knobs, faders and even xypads and other controls, then shows up as a regular MIDI controller on computer via Wi Fi. Had great fun hooking up different controls to minifunctions using your form designer and using it on an iPad as a control ser geeky audio stuff and computers. I love it.
And you know, I have a little history with a sound blaster because at the time the MIDI pack for the sound Blaster came out company. I worked for Voyetra Technologies. They had the deal with the sound Blaster to provide that hardware and the sequencer sequencer plus software, and I was tasked with making some sample MIDI files that could be shipped with it. And so when if you got the sound Blaster MIDI add on you look at those mini files. Yeah, I wrote those.
Nice.
Also, I remember going out to dinner with mister sim who is the inventor of the sound Blaster, and seeing actually him at PCXPO I think it was at the time the Javit Center, and he wanted me to record re record the files for the Talking Parrot. You remember the talking parrot Elias right, yeah, yeah, And I never did.
I was such a jerk.
I could have done that too, because he wanted, you know, funnier things than he had responded with the parrot. Basically, you talk to the parrot and it gives you these crazy, wacky, high pitched responses.
Don't touch me.
Yeah yeah, die q, I think that was one of mine. Anyway, didn't do it, but it was fun, fun times, fun times.
So thanks JP for your comment and a copy of music co buy is on it's way to you, and if you'd like copy music, go buy. I write a comment on the website at dot at rocks dot com or on the facebooks. We publish every show there and if you comment there, and every ready on the show was at your copy music, go by.
And music to code by going strong. We've got track twenty two now and you can go to music toocode by dot net, which is a URL rerouter. So HDTPS isn't going to work. But if you just say music tocode by dot net, that'll bring you there and you can get just track twenty two if you want, or the entire collection in MP three wave or flack format. So with that, let's bring back to dot net Rocks. Elias Purinin He is an early innovator in digital event
experiences and the founder of Tractus Events. As both a programmer and producer, he leverages av over, IP and NDI technology, which I also love to deliver broadcast quality conferences, trade shows, and workshops on any budget. Elias personally develops much of the software his team uses, ensuring every aspect of production
is tightly controlled. His high standards for quality have led to award winning events, including a collaboration with University of Waterloost Speed that garnered over fourteen thousand views and earned a Case Circle of Excellence Award. He is also the author of Memorable, Profitable Virtual How to run virtual Events that create meaning, value and lasting connections. Finally, for the
nerd crads. Elias first started with VB dot net one point one back in two thousand and five, but has since crossed over the dark side and writes almost everything in C sharp these days. He used C to write games for the Game Boy Advance. He programmed a game for the Atari twenty six hundred in six five h two assembly in twenty four hours, and loves writing a great select Star from a sequel server database.
How to make your DBAs angry. That's right, select Star.
Welcome back, Elias, Thank you gentlemen.
It's awesome to be back. I cannot wait to talk P and voke and NDI and.
Every day day over ip P and voke Man, we were talking about one of the earliest components or ds that I wrote for dot net was in vbnet and I used P and voke to do to simplify access to.
MIDI right and so like back then, we didn't have any audio, we didn't have any of those other tools, Like you had to go into what was it the MMAPI, right, MULTIMEDIAPI.
To do that MULTIMEDIAPI.
Yeah, yeah, that was. But what's stunning is how little the p and voke APIs have changed in dot Net, Like how you actually write that code. Yeah, and how relevant it is like twenty plus years later.
And just to back up, p invoke stands for platform invoke. So this is a way that you can use whatever managed language that you want to make API calls to the unmanaged platform APIs that you're running on.
Right, So imagine you've got a unmanaged DLL of some sort and it does something for you, and you want to use the result of a function call from that DLL in you want to as a result of that in C sharp or just dot net in general. You
could do that with VBNT two. So you would write a special function wrapper, you know, especially defined function like public extern void, whatever, the signature name is, put in the right parameters, decorate it with the correct attribute, and maam, now you can call into As long as that DLL can be found by your process, you can call into that DLL.
And your parameter types are absolutely accurate. If they're not all things, all sorts of weird things will happen.
Oh boy, how do you well? Lots of weird things happen.
Hey, your compiler won't necessarily know about it ahead of time, will it?
No, not necessarily. And sometimes the runtime errors that you'll get, you know, you'll just crash out. You won't even get an exception, and things will just fall over that said the compiler, even though the compiler may not catch it. There have been instances, and we'll talk about that. We'll talk about specifics later, especially around union types where at runtime you'll get very specific pacific errors to say, hey,
your structs aren't defined properly. Yeah, but that's we'll talk about kind of the nuts and bolts there later.
Can we talk about the wisdom of using a garbage collected language in a real time operation like an event?
Yeah? For sure.
Shouldn't this all be written in C plus plus? Like? Why are you in the p and voke land? Why aren't you staying closer to the metal?
So I thought about this. I thought about this a lot because the background behind what I've been doing for the last probably two years has been real time audio and video, specifically using a technology called NDI. NDI short for Network Device Interface. Long and short of it is, it's a way to send real time audio and video over a LAMB. And when I'm talking real time, we're talking latency of a frame or less.
In some cases it can be Yeah, you have to have a very fast network. But there's oh boy, are there a lot of variables on that one. So it's the network, The knicks have to be configured, good drivers, you know, there's a lot that goes into it.
And I've done a lot of research on it. But the upshot is the lowest theoretical latency that I've read is down in the number of scan lines. Wow, so less than a frame of latency you get down to scan lines.
That's great. Well, especially when you're talking about four K where you've got what twenty one sixty scan lines per frame.
Yeah, so it's I mean there's a lot of asterisks to achieve that low latency, right, But I mean, if.
You're doing four K over a one hundred megabit network is not going to get You're not do net ain't going to work.
Ivan gig is humming at the limit at that point.
Yeah, yeah, depending on your encoder, depending on the device. Like you can cram about four full bandwidth NDI sources onto a gig connection. Probably don't want to do that, but you can.
You can. Yeah, yeah, because network storms are still a thing. We usually don't hit them because we have enough bandwidth we don't think about it. But once you get upwards at eighty percent or so and start getting collisions, it gets way where it falls off a clip fast.
Oh yeah.
So, and is a spec that is created by a company, right, and they give you the NDI tools that you can use, right, Can you talk a little bit about that, yeah?
Sure. So NDI was originally created by Newtech who created the tricasters and going even back even further the Amiga video toasters right, right, And how NDI evolved was they wanted to find a way to bring in remote camera sources or remote screen share sources as video sources over a network. I think the original term for it was ivgay and then it's kind of grown from there. Like behind me, I've got a whole bunch of cameras. They're
all NDII cameras. There's the NDI tools, which is just software tools that you can use to have a virtual webcam to send out test patterns to preview your NDI sources on your network. And then there's the actual hardware itself, right like cameras, there's some audio gear that's coming out, converters that will take an NDII signal and turn it into SDI or HDMI many three and four letter acronyms
around this world. So the idea behind that is instead of running physical copper cables, SDI or HDMI cables that carry audio and video, and then you have another cable for doing control and then maybe another cable for power. If you've say, got an NDI camera and it's power over Ethernet, I'm now sending audio, video, control, and power over one cable.
Yeah right, and it's an Ethernet cable.
It's an Ethernet cable. So like if I destroy a fifty foot Ethernet cable, okay, so what I go to Amazon and buy a new one twenty bucks?
Yeah yeah, yeah, if.
I destroy a fifty foot SDI cable, someone's getting in trouble.
Everybody's sad. Yeah, but I mean tricasting the day was the device. The market's blown wide open now with atems roads stuff and so forth. Although I think the real thing that I noticed was one day everyone said, you know, maybe it's okay to drop a frame now again, Like SDI was so strict you needed those really good and expensive cables and they never dropped a frame Like that was the whole claim to fame, and HDMI was just
nowhere near that rigorous. And I remember when the first ATM mini pros came along in the minis and you're like, yeah, no, occasionally we'll throw a frameway to maintain SINC and live with it. It's like, but we're going over the internet. It's not that reliable anyway, what do we care?
The TriCaster still is like the Mac daddy of all things, right, it's essentially a Windows PC.
It's a with a with a console on it.
With the TriCaster software yeah, with the console yeah, and all sorts of consoles that you can control it with. And it's expensive. Yeah, you know, let's face it.
Yeah, it's a it's an amazing product. It's expensive. The same compan so the company that made it, New Tech, they eventually got acquired that's now vis RT. VISRT split this NDI project off into its own entity. So NDI exists as a company as well, and so you can you as a developer, can go out and get the basic SDK, integrate that into your application, and then you can also get the Advanced SDK, which gives you a lot lower level control over how the whole how your NDI application behaved.
Now there's a lower cost alternative to the TriCaster that I know you use and because we talked about it before we started, and I use it as well that I want to talk about. And that's v mix, Yes, vmix dot com. And here's what I know about VMX. It's a WPF app written in vb net. Yes, how cool is that? Yeah, it's it's super cool.
Like to be clear, they've got unmanaged stuff going on as well, like obviously just to interact, but it is written in vb net. It is super super cool.
Yeah, and rock solid. Like these guys are in Australia and you can have I don't know they have these callers right, and you can buy it based on how many input callers you can have remotely and the caller API think that max is out at eight?
I think right, yeah, v mix call I think it's eight, yeah, v.
Mix Call and so you can give out these you know, URLs to everybody who you want to call, and then you have this dashboard where you can you know, select the It's very much kind of it's kind of like if you've used you know, what's the OBS, right, OBS studio, but on steroids. That's the way I look at it.
It's kind of like OBS went to college. Yeah, nice, Yeah, it's and like it's for us. We also use v mix to do like when we're doing our hybrid shows, like we'll have we'll have one single v mix rig that's pushing out the local feed, like what's going to the local projectors. We'll have one that's pushing out the
virtual feed to English listeners. We'll have one that's pushing out to French listeners, and like all from one application, right yeah, And this was the sort of stuff that you used to have to get you know, the big broadcast consoles to do, right sure, And it's like a fifty dollars license and you can do the same thing that you know, cost hundreds of thousands of dollars back in the day, and it works.
And as a developer, it's even cooler because they've exposed an API that you can access just through a low URL. So if you can write a c sharp app that you know uses an HDP client with a local URL, you form that URL with the commands and data that you want, and you can pretty much do everything that the app does programmatically from your own interface.
Yeah, about eighty percent of it, but yeah, it's pretty pretty close. We use it. We use it a fair We use a fair amount.
The trick, of course, is getting all the data into your machine. But if you're going all Ethernet, if it's all POE and stuff, then you don't need a lot of hardware on your machine to make this work. Yeah, avast of computer.
I've done multiple events with v mix and I've done I did one from my home, and I did one down in Brooklyn at a at an event down there that was just amazing. And that was the probably the hardest event I ever had to do because there was so many cues and so many buttons and so many things to think about. It can get overwhelming. And I think that's why if you get to that level, people I know would move up to the TriCaster.
Or you're loading your desk up with a whole whack of stream decks and using bit Focus Companion.
There you go.
That's talk about that.
So I've got on my desk, I've got three stream Deck XLS plus along with A plus and those are all being used to control you know, which presenter goes into which box on screen, which layout we've got, you know, transitioning doing takes muting music on muting like it does our whole show automation.
So stream Deck is a hardware device with a bunch of buttons on it that are actually you can load up with images so in text, so you customize it for what you want to do on your particular event or stream or whatever. And as Elia said, you just pushed the buttons at the right time.
This is the Elgado hardware. I Gato.
This is the Algato one, but you can pair it with a piece of software called Companion. It's made by Bitfocus and it's like if you thought the Algato software was awesome. The bit Focused Companion also allows you to do stuff like I'm going to dynamically update the text on a bunch of my buttons depending on if I'm recording.
Most common use case I have for that is if vmex is not recording, I've got a button that is blinking red at me, not recording, not recording, not recording MM. That has saved me more than on so many.
Times, because, yeah, the companion also is basically just a macro programmer. Whatever you want that button to do in virtually any other app, it'll do it.
Yeah exactly.
Yeah.
You know, the definition of a genius in the twenty twenties is, right, someone who can push the right button at the right time. Oh, yes, to think about it all comes down to that.
It's doing your thinking ahead of time.
Yeah, exactly.
So when we're talking about audio and video and programming that with dot net, are you talking about NDI with C sharp? Is there an NDIAPI for c sharp?
So originally there was just the CAPI for NDI, but with the magic in vogue again, if you get the signatures right, you can basically take your header, your cheader, translate that into you know, your public extern, void calls or public extern whatever your struck name is right, and call into that c API. And as long as you do everything, as long as you decorate everything correctly, it all just works. And a lot of times even if you don't, the marshaler will be there to kind of
save your bake and do things correctly for you. Strings especially is where things get really weird.
So you know a questions coming next, Yes, have you ALISP created a dot Net rapper for the NDI API.
So I created a dot Net rapper for the Advanced SDK, and I improved on the rapper that NDI themselves had originally released to make that portable too. They originally released it for just the dot net framework, and I ported that to dot net at the time dot net core and now just dot Net seven plus. Yeah.
Wow cool.
But here's the magic of that rapper though, And this is where p and voke really starts get powerful in the modern era of dot net, because when you think about it, dot Net originally was Windows only, right, And then, you know, Richard, I would be remiss if I didn't mention that the only reason that we have the portable CLR today, and we have a lot of what we have today is because of all the effort they made to make dot Net run in SQL server, right, right,
there was that huge effort. What was that like two thousand and five?
Yeah, yeah, well delivered in two thousand and five, but it had been working on since two thousand and three Net two.
Right, that was dot Net two O and all the magic that we got during dot Net two O. So that work. So now that we have this cross platform dot Net library, you think about we have all of these native libraries across all these different platforms. NDI itself it has an SDK available for Windows, for Mac, for Linux, and then on top of that, for Linux we have X eighty six sixty four and we have ARMS sixty four, and then on Mac we also have eighty six to
sixty four and ARM sixty four as well. Dot Net also targets.
All those platforms, right, That's right.
So I can take my dot Net app, I can compile it on all these platforms and it's going to run and not just that, but it's going to run more or less the same way across all the platforms because the same source code.
I think you just made your argument for why I'm writing this is cea Sharp and said at C plus plus.
Well, so philosophically, I was thinking about this before we got on. I was like, Okay, well, why are we if we're going to be calling into native libraries anyway, why are we even thinking about C sharp? Why is that even entering our thought process. When you think about it, there's only a small percentage of the operations that I'm doing that I really really need that low level raw speed.
And I would even argue that whatever the dot net compiler does in the background, especially for talking about ahead of time compilation, is probably going to write faster, more optimized, and better code than you can by hand, unless you're really talking about assembly. In that case, that's a whole other special world.
Who's going to do that?
Then you got chromium twit tweezers out and you're stacking your own electrons and that's it.
Oh yeah, exactly that. I mean you start a button. Yeah, yeah exactly, So like you can you can go do so we could write the whole application and say CE or C plus plus. But it's like I don't need that low level control all the time.
And you don't want to pay the price of using that low level language all the time.
Yeah, and there's a big price involved. Yeah, you know, I don't get automatic garbage collection. I don't get you know, I have to think about reference counting. I have to think about okay this because like, yes, there's type checking, but there is nothing stopping me from casting one pointer type to another in de referencing garbage.
Sure, you know. At the same time, it's like, does a GC bonk here client? Like, I haven't thought about a bonked client by our garbage collection in a decade, easy, right, Like because things have been so fast and it just stopped being an issue, like noticing the garbage that the guar coelector was running is a two thousand and five thing, not a twenty twenty five thing.
Sure, and there's ways to mitigate that too, if you're smart about memory management, right, you know you don't You don't have to worry so much about that these days.
Yeah, rule number one have more memory.
Yes, So I was thinking about this. You know, Let's say we have written our p and voke stuff for Windows and we want to run it on the Mac. If you're doing a what is it, you know, a MAUI app that can run on the Mac. Right, Those P and voke C sharp extern method declarations from what I remember, include the name of the library that you need to call into, like by name Windows there dot DLL. But what are they on the Mac? Do you have to change those on each platform?
So if the marsh the p and VOKE libraries. They provide a mechanism that instructions dot net how to look up that DLL or how to look out that dynamic library. So on Windows, Mac and Linux, you can just generically say we're dealing with that dynamic link library. The file name is different, but the concept is the same. This is a library. It contains some sort of unmanaged code, and I want to call into it.
Great, So you don't you have one codebase and it literally goes from platform to platform without modification. You don't have any pragmant statements for the platform that you're on.
See that's and that's the other thing is like if you think, I think about doing the code portability stuff back in the WPF silver Light days. So like back when I was in university, one of my one of my co op placements was to write was with a broadcaster and I had to write a timeline control for a non linear editor like you would see in Adobe
Premiere or after Effects or something like that. Right, But the use case was this needed to be cross platform for both WPF and silver Light and had to be the same code base.
And so to make the portable class libraries at the time.
Yeah, this was Yeah, that was PCL, right, so which they moved away from, right, like, oh yeahcls have been retired. Yeah, PCL. Because we did PCL, that gave us a baby step to dot nets standard once we kind of split, and then dot net Standard was kind of the way for a little while and then we realized, okay, let's just open source dot net and we're going to go, and that's good his future.
Well, and because the PCL approachment that every time you added a platform and it added complexy to everyone else, like it was an N over N minus one problem and it just got bad. Yeah as the count went up.
Yeah, and it's and you also ran into hey, this is supported on this platform. So you still needed to do some checks, right, Yeah, And to a degree, you still need to do certain checks depending on what you're doing and how you're using p and voke and whatnot, Like there are certain educating conditions that do matter that
you need to check. But back in those days when I was doing WPF and silver Light, I was doing a lot of if deaf and then checking for certain compiler flags right right when I'm doing my you know, where should I find the NDI library. What's it called on this platform? That is a native if statement in c sharp. Nice, So that is compiled into each version and that check is being done at runtime.
Okay, all right, which he also gives you room for new versions and new configurations. It should just work.
Yeah, right, all right, this is a good time. We're going to take a break, I guess and about halfway through the show stick around after these very important messages, we'll be right back with more from Elias Purinin. You know, dot net six has officially reached the end of support, and now is the time to upgrade. Dot Net eight is well supported on AWS. Learn more at aws dot Amazon dot com, slash dot net. And we're back. It's dot net Rocks. I'm Carl Franklin, that's Richard Campbell, hey,
and that's Elias Purinin. We're talking p and voke and audio and video stuff that you can do in dot net and c sharp. Now you've written a rapper for NDI, but you've also written rappers for other native you know, libraries and things. What's what's another cool one you want to talk about?
So for my NDI multiview, which kind of which is where all this p and voke. Goodness came from for me and where I saw the light of pnvoke and dot net in general. I as part of that, I needed to write against the NDI Advanced SDK because I wanted to be able to send out multiview outputs in a technology called NDI HX. So NDI has two major flavors. There's HX, which is compressed either H two sixty four H two sixty five or the full bandwidth version, which is used as something called speed HQ.
Because I want to make my Ethernet cables glow yes white.
Speed HQ maybe a misnomer, let's see.
Yeah, there's a lot of data in this stuff HQ.
They're big frames, but yet still compressed. So that's that's the thing. One thing a lossless compression, A lossless compression exactly.
Yeah, where two sixty four five are both loss e compressions.
Yeah. So one of the but one of the reasons I want to do the HX output is because there's two decode H two sixty four two sixty five. Pretty much anything on the planet can do that real time now, and there's a lot of devices coming out now that if they do NDID code they only do HX, so for so for me, I wanted to be able to send out an NDI feed as HX so that we could send it to those devices. Opens up a couple other categories two and so I went and took a look,
and there's there were a bunch of N Video. There were some nvideo wrappers out there already for c SHARP because what every if you've got an N video graphics card, you have access to something called the end video encoder or en VANC. And so this is the hardware on your card based H two sixty four, H two sixty five and depending on new your card is av one.
Isn't this Kuda under the hood.
So it can use Kuda under the hood. There's also ways to access it via DirectX and with OpenGL context I believe, which gives you a little bit of more platform.
See look at this using video cards for video not AI. I'm so excited.
It's a cool X it's your foot, isn't it cool? We can actually use GPUs for video again, Yeah, what a concept. It's amazing. To give you an idea of how fast it encodes though, I can put a ten EIGHTP sixty or I can put a ten ADP nineteen twenty by ten eighty video frame into endvanc and get an encoded frame out in about two to three milliseconds.
Wow, Jayvers Miles and marsh Miles. And you're not talking at fifty ninety here, you're talking about like a ten eighty.
Yeah, a ten eighty. Like you know, I've got a machine here that's got like a forty sixty, forty seventy. It is like two three milliseconds. It's nothing, it's nothing.
Yeah, now you'll be able to keep up. You could do a bunch of those, like, yeah, sixty frames per second? How many can you do? Five off that card real time?
So again depends on the card. That card I think is just hardware is limited by the drivers to five. But like if you go with the RTXA four thousands or like the really professional grade.
Cards, which is what they're bill for. Yeah, the work those workstation class ones are designed for multiple parallel threads, which is not great for gaming, but for this application, that's brilliant.
Oh, they are absolutely beefy cards. They're super cool.
Yeah.
Anybody who knows what I do knows that I create videos and things, and I'm a big fan of Adobe Premiere. But I remember in the days when Adobe Premiere Pro didn't have access to the GPU for rendering, and it was just using the CPU and it was slow, and it's like, oh yeah, you got to go turn on this thing. And then it was it was faster, it was much faster. Now now I can take an our video and render it on my I nine, you know, with my Nvidia card in maybe five minutes.
Yeah.
Yeah, it's that's that's really cool. That's that's the beauty of the modern GPU.
Yeah.
So, and because I've got access to that, you know, I've got access to those two GPU n code and decode. Like all of a sudden, I can sort of get around licensing around h two sixty four and two sixty five because that's a whole weird legal beast that I'm not even going to touch.
So you created wrap around the end Video encoery encoder, so INVEC.
Yeah, So ENVEC gives you if you go on end video site, you can download the CSDK for ENDVEC and I call it end bank.
I don't know why, but I think you're right.
Yeah, that's that's how I'll refer to it. I'm used to that one. So you download that, you get a c API. But what's interesting in the if you look at the and we've provided links. I've provided a bunch of show links for this. You can find it if you look uptractice dot encoders on GitHub.
We'll put them on the main page too.
I have yeah, so we'll be on the main page.
Two.
So if you look up, if you look up that library, you'll notice that I actually only have one direct P and voke function call into envank, and that is to initialize the library and get back astruct. Now what does
that struct contain. It contains a bunch of unmanaged function pointers to whatever version of the API I'm referencing, so for backwards and forwards compatibility, like if i'm I can say I'm on version twelve, I've compiled against version twelve of the nvank SDK, So that gives me access to anybody that's running n Video Drivers five twenty and above, which is striking a nice balance between supporting some legacy stuff like that goes back I think to about twenty nineteen,
twenty twenty, but not being so new that like you need a fifty ninety and all the latest drivers to run it right.
Yeah, and well, let's face it, the fifty ninety is crazy overcow for virtually everything pretty much. Yeah, except we're just just taking money off you. You know, I'm realizing there's Envy Inc and Envy Deck, so I can't I don't think he can say n bank because you can't say env deck. Ah.
True, that would be the decoder version.
Yeah, there's an encoder and a decoder. This is legit, you know, we get this. Yeah, so I think if I'm going to go for a naming consisting, it's going to be Envy inc, an Envy Deck, envanc en v deck. I like that. They're still awkward, just bank deck like modem because n bank is easier to say no toys about them. But there's nothing nothing you do about the decoder. It's glunky.
Yeah. Yeah, at least it's not WPF.
They'll save me.
Who was who was it that was ranting about that way back in the day we can't have actros w No, it was it.
Was Brad Abrams who said, listen, if we give the project a good code name, it gets a tear product name. Yeah. So that's why the code name for silver Light was WPF slash E because that's the worst code name ever. So the product name was awesome.
I was just waiting for Windows Transaction Framework, which would be WTF.
He said, where did my data go?
So, but getting back to the getting back to the n bank wrapper library. So there's only one direct p in voke function call in, and that gives you a struck back and that struct has nothing but function pointers, right, and that's how you actually interact with the API. That's how they're maintaining that backwards compatibility. So even if I'm running the.
This is really not that horrible, right. Once you've got connected, it's really smart. Then you're just feeding it data. It's just just dream.
And then throughout that library, I then have a bunch of Marshal delegate to function pointer. I forget the exact get delegate for function pointer calls, so it's Marshal dot debt get delegate for function pointer. I pass it in the end pointer, you know, native end pointer for each of those function each of those function pointers. And I now have a first class c sharp delegate that I can call. Great, so now I can take my whatever memory I've got allocated, that's got my raw video frame.
I can now say, okay, here's a point of that memory end bank. Here you go, give me a frame back, and it all just works. And I'm doing all of these calls in c sharp.
And if it changes, you don't care because maybe the memory changes, maybe the pointer changes, but that's.
Fine, exactly. So I'm I'm shielded from all that stuff. I get the advantages of native, of the speed of the native code, but I'm also able to and i can access all these native libraries. But I've also got a lot of the protections that c sharp gives me out of the box.
Yep, You've got another layer of abstraction there, which is always a good.
Thing, exactly. And as long as I've got all of my strucks set up the right way, and I've got you know, and I've been very careful about how I've instructed dot net about how these structs are laid out, everything just works. That's a big.
Asterisk, though, I jw technology.
It just works. Technology it works works.
Yeah, what's the asterisk?
So in some cases you do have to be very specific about what the unmanaged code is expecting in terms of how is this struct laid out? How should this If I'm p invoking how should this function be called?
Because there's the there's standard call. In the unmanaged world, we have standard call, which is the way that we usually call into like a WIN thirty two API for example, and standard call one of the versions it's the caller is responsible for cleaning up the stack once the function returns, and in the other way, the call e is responsible
for cleaning up the stack. It's the collie that's responsible for cleaning up the stack in a standard call call, whereas with a with a C decal call, the caller is responsible for cleaning up the stack. Which is why and C you're able to do like a print aff with your string, your format string, and then you can put as many parameters after that as you want because
it's an also for the standard call. That's why if you ever crack open you know, a C plus plus library in a tool like GEDRA, reverse engineering tool like GEDRA, you'll see these really awful mangled names like you know, get function pointer at three and that at three tells you this version this overload is expecting three parameters, no more, no less, okay, and if you call it with more things could happen.
Self documenting sort kind of sort of sort of you don't know what they are, but that's why you have to see sure, you.
Know, you could just you know, like anything else, like sometimes you just got to poke the function calls and see what happens. Just like let's call it with five. Let's see if it works.
Now, you had a specific need to create this wrapper, but what might other people need to use it for?
So for Endvank, it's part of my philosophy of creating some of these rappers was like was to was to show that, look, if you want to get this high performance real time audio and video, you can totally use a higher level language like c sharp because when you need to drop down and get that crazy speed, you can. M h. So let's talk about crazy speed. Let's talk about unsafe code, and let's talk pointers.
Yeah, so love unsafe code reminds me of my teenage days driving a car at one hundred miles an hour through the town. Chickens flying everywhere, mothers pulling their children off the street.
Go ahead, that sounds like an interesting hobby. I just tried.
I had to pick up the feathers afterwards, though, I had.
Oh man MESSI my biggest adventure of driving a car was knocking over construction fences.
There you go, all right, well that could be unsafe.
Yeah. So if you compile your C sharp code with you know this, with slash unsafe for you marketing your project file that you want unsafe code, then you can use what's called Then you can use the unsafe keyword. Now what does unsafe do? This allows you to do stuff like pin objects in memory and get raw function point and get raw pointers. So you would think, okay, So a common misconception is that C sharp doesn't have pointers. Oh yes it does. Let me tell you about them.
So there is if you're working in an unsafe context, there is nothing stopping you from calling Marshall dot alec h Global and saying I want a big chunk of memory and it'll go cool. Here's a pointer that memory. You can then cast that to whatever struck type you want and bam, all of a sudden, I have pointer de referencing arrows in my C sharp code.
But you said the magic word though, which is pinning. Yeah, if you don't pin it, it could move around on you and you go to access it and you're writing over something and security.
We have a.
Problem over here.
Bad things happen.
Oh yeah, you could. You could get into big problems real fast.
So but pinning is the key. It basically says, don't move from this spot right here.
Yeah.
Now, from my experience, I when I've been working in modern dot net, I found that I've had to do pinning a lot less than I used to. Yeah, Like, it seems like there's been a lot of work going behind the scenes. And it feels like that is because of dot net running on mobile platforms and running on mac os. Like that work seems to have you know, the native interrupt story is way different from like the
dot net one one two oh days. But you can, but you can totally allocate just a block of memory, and I actually do this, like for my advanced SDK rapper and for some of the encoding that we're doing behind the scenes. What NDI expects when I send a compressed frame is that it wants in memory physically laid out. It wants the struct header that contains information about resolution, frame rate, data length, and then after that in memory
should follow the h two sixty four bitstream. So the actual stuff that comes out of the encoder, right, that's a little bit hard to do in see you can
do that. You could just allocate a bunch of memory for whatever memory you want for your frame size, plus you know, say forty four bites for your for the struct header, and you new memcopy in C it's a in C sharp, it's a little bit more difficult to do that, but you can still do that because you can allocate memory marshall allocate global that's equivalent to your to your mallek and sea land. You can then cast that.
You then cast that to a pointer to whatever struct type you're dealing with, and in your struct declaration you can add attributes that say hey, this struct layout is sequential, so don't move stuff. So dot net compiler, don't try to get clever with moving my different fields around to do memory alignment or you know, save space or do
anything clever. Just like how I've laid this out, Like if I if my first field is an ind to, my second field is a long, and my third field is a short, like I want it laid out exactly like that in.
Memory Yeah, sequentially exact.
Yeah.
So if you do that, you have your own safe code, your global maloc and you pin it is it really unsafe?
So you can still dereference, you know, something outside of the bounds of the memory that you've allocated. So in terms of unsafe, it's kind of like saying, you know what, if you want to dip into doing you know, memory management and pointers and all that stuff yourself, you can
totally do that. It's your foot, It's exactly c Sharp's like a big Swiss army knife, right, It's got all these different tools and abilities, and then you pull out the final one and it's a big friggin' you know, precision rifle, perfect for your foot.
Nice.
It's got everything, so it's not gonna it's not going to move on you, which is one of the problems and one of the reasons we have managed you know, frameworks in the first place, like that net. But you can still overcopy, under copy, like you said, de reference to the wrong thing. You can still do all those stupid things, and the compiler is probably not going to help you all that much.
But it also depends to how you're interacting with that data that you are or those or the pieces of memory that you're working without a low level because you can also take a byte pointer or a void pointer whatever you want, like say I've allocated a big chunk of memory, and I want to pass that chunk of memory to a managed function that expects a spam like
a SPAN of byte I can create. I can instantiate a SPAN byte, pass in that pointer to memory and say, hey, this is you know, two thousand and forty eight four thousand, ninety six bytes long. I now have a SPAN rapper, like a first SPAN wrapper around this chunk of memory that is quote unquote unsafe, and I can do all of my dot net goodie things to it.
Right.
Yes, SPAN is a wonderful, wonderful construct.
And like I use that, for example to as part of encoding that ndiachex frame. I need to tell NDI here's how long the actual frame data is and all of the ancillary data, so like when this frame should be presented, that's how long this is. I have a parser at h two sixty four bitstream parser that is written in C sharp. It's all managed code that uses SPAN to go through and see, oh, here's where this part starts, here's where this part ends, here's what needs
to be copied. So I'm doing all of that checking and it's I've got all the type safety and I've got all the bounds checking right, Yeah, so I'm not concerned. So I'm not as concerned about you know, overrunning underrunning, like as long as I'm as long as I'm instantiating with SPAN correctly. I've got all the type safety of C sharp.
And that's SPAN. We've talked about that that span before, about span, and just remind us what it is from what I remember, it's sort of like a fixed position array of stuff that isn't going to move around under you, right.
And it wraps it without doing memory copies.
Yeah.
So and then if you want, say you want to get you know, you've got a SPAN that's two that wraps a two megabyte bite array or a two megabyte bite pointer, and you want to get the data at offset five hundred and twelve K to oneenty twenty four K. But you don't want to copy that. You just want a you just want a slice of a slice of that to view it basically or to do something else with it, but you don't want to make a copy
of it. You can say span dot slice I believe, and that will give you a SPAN without doing any memory copies that starts at that offset that you specify cool, so then you can do whatever you want to that nice, Yeah.
Very nice.
I remember talking to Mark Mark Heath of an i udio fame about spans when I first heard about them, and I think he implemented them behind the scenes in and audio and made it more performance and safer.
Oh yeah, and it's it. What is really awesome about is you get you also get access to all the first class stuff like for each four loops. Yeah, and you're not having to do that manual You're not having to do the manual pointer arithmetic as well.
Yeah.
Again you can in unsafe mode. You can totally do that and it will, it behaves exactly. But you have to really think about it.
Yeah, more than I want to think about plumbing.
Yeah, but you think about it once and it's done.
That's why we like the library builders too. Absolutely may make our lives easier for this stuff. I might include a link to an audio now on GitHub. I know I referred to it in the comment as on CodePlex, but they moved. Well.
Remember once upon a time stuff used to it was CodePlex. Stuff also lived on code project for a long time.
Yeah, code project. Yeah.
Number of libraries that would just be like, hey, here's the DLL. Go have fun.
Then one day, I mean GitHub has won. Right, It's almost not a conversation anymore, pretty much just the way it is. Well, remember their original tagline was social coding, and we realize how important that is, Like I look to the issue log to understand the health of a project, Like the way people talk about that code matters a lot.
Yeah, oh yeah, intrinsics. Okay, what's that about.
So not entirely related to p and voke, but it's a big deal when it comes to performance and dot net. And this is kind of one of the arguments about, you know, should we write our high performance code in C sharp or should we write our high performance code in a lower level language like C. And one of the items is I want to use hardware accelerated vector
operations or single instruction multiple data SI MD instructions. Where this really comes into play is you know, think about like you're doing matrix transformations way outside my pay grade. I'm not even going to pretend that I'm good at that stuff. There are instructions that many modern CPUs support that allow you to say, you know, do a bunch of additions all at the same time, or like, you know, do four data rights at the same time instead of having to do you know a four each a four
loop where it's like I'm going to process. Well, where I do it the most is transforming filling black for example. Sure, so big operation in my multiview is after every frame I have an I have four or five frame buffer setup that I draw to. And so after I'm done drawing to, before I draw the next frame, the first operation I need to do is fill black. So just clear, just blank out that frame for all rows and then
for all columns exactly exactly. So if I do so, if I do the purely naive version where I'm just I'm literally writing black, you know, a black byte pixel across all the pixels in that frame buffer. On one of my target systems, that takes a millisecond. And now you think, okay, a millisecond. So what if I'm running at sixty frames per second, I get sixteen milliseconds in
which to realize a frame. Yeah, that's not a lot of time, and so a full one to sixteenth of that time is now spent just filling the thing with black. If I go a little bit smarter and I say, okay, I know that I'm essentially writing a constant, So I'm just going to write a thirty two bit int, you know, and then do skip four bytes out of time. So
I'll write four bytes skip, write four byte skip. That time goes down to about two hundred and seventy one or zero point two seven one milliseconds or two hundred and seventy one microseconds. Right. Nice, So we've already saved. So just doing that hack has saved us some time. If I do sixty four bits, so I do longs, I bring that time down to one hundred and ninety microseconds.
Nice.
If I use avx two, which is a single instruction multiple data extension for the X eighty six platform, that code is super crazy, and I will fully admit that chat, GPT and lot helped me write that and make it work at all.
Ah excellent.
Yes, but the original naive version took one millisecond one forty three microseconds. The SIMD version took seventy two microseconds.
Well, jeez, that's significant.
Think about this. Yeah, significant, not only significant, but it's in C sharp and I'm accessing and I'm accessing, say I'm accessing raw hardware essentially raw hardware instructions. So instead of having to write that performance critical code like normally if you wanted to access AVX or SIMB instructions, you would be going into C Like that's just that's just a done deal, dude.
You can now write the trippiest screensaver ever, Yes, in C sharp. Yeah, I knew you would come to the killer app and sooner or later we can find it's finally here.
We can finally do all the plasma moost scene effects we've always wanted.
Absolutely in hardware, but so which you can then ingest into a v mix thing and you know, while you wait, you could just you know, be rendering that stuff.
Well yeah, exactly. Another place the SAMD is important for too, is say I'm doing pixel format conversions where I know, I you know, I'm i've NDI likes YUV four two two, so I have. And the way the yav pixels work is basically the human eye is more sensitive to brightness than it is to changes in color. So what we're what we can do on most modern displays is say, hey,
I'm going to encode my luminance. My brightness is going to be encoded at full resolution so I have nineteen twenty by ten eighty resolution for luma, and then for my color data. I can take advantage of the fact that the human eye kind of sucks, and I can do I can encode a macropixel and say, for every one bit of you and V data my chroma data, that will be decoded into two pixels worth of color data. So if I combine the luma and the chroma together, I get reproduction of color.
Why am I thinking of Gaussian splatting right now? Do you know what that is?
I don't.
All right, I'm not even gonna we're gonna take that out Brandy, because it's only good if he's up on it. But I'll tell you while we're in an edit point. Okay, it's basically a way that instead of doing regular rendering, you can, oh, you know, like Gaussian blurring. He basically create these splats of starting with low resolution detail and then getting higher resolution over them, and for some way you can do like sixty frames per second rendering of
a model in real time. Wow, And it's really amazing. So yeah, go look that Upusian splatting, Gaussian slat latti NG. There's some great demos out there, but it's coming. It's a technology that's coming.
That sounded like something I needed a towel for.
Absolutely, you will you will need a towel when you did this stuff. All right, So what haven't we talked about before you before we wrap up here, anything else?
I the philosophical side of YP and VOC is so important, Like because I took a little bit of time after doing after making all these wrapper libraries you know, around the around NDI, around n Bank, even starting on doing rapper libraries around Apple's Video Toolbox, which that's their hardware and code. By the way, the Apple silicon for doing hardware encode and decode of H two sixty four, two sixty five and AVY one is like worth the price of admission of having the burden of voting a Mac.
It's just those chips are insanely good.
Mm hmm, even on a MacBook.
Yeah, on even on a base M one, yea, even on a base M one MacBook Air. It's unbelievable the amount of horsepower you get out of that machine. Yeah, yeah, it's it's just unbelievable. It's just unfortunate that it's a Mac.
Yeah, but any of those all those m chips are extraordinary. Yeah they are.
Yeah, But the philosophical importance of why P and voke is such an awesome technology, why it's underrated is because say I'm in Java land and I wanted to call into a unmanaged library. Their story has gotten a little bit better, but you are still needing to write an unmanaged wrapper that your Java library can call into. So think about the process. I want to call the Nvidia API from Java, I need to write an intermediary layer in C that I can then call from Java in order to do my P and voke.
Yeah.
Whereas with dot net, I, as long as I get the function signature is right, and as long as I annotate everything correctly, the compiler in the back end figures all that stuff out for me.
Mm hmm.
So what you to take, Like you could literally take, and I've done this with a couple of APIs. Take the C header file for you know, an SDK of some sort, put it into chat GPT. Say I need P and vote calls for this.
I was gonna suggest that, yep, And it's exactly what I was gonna do.
Yeah, and bam, You've now got a complete wrapper for any of this stuff chat.
GPT is great for transforming something into something else.
Yeah.
Yeah.
But the reason this is really cool is think about what we've got here. Think about the original design goal of the C language. We wanted to have portable code because we had all of these different architectures, they all operated differently. We wanted we wanted to have encapsuley essentially our logic into a higher level language that human beings can understand, because let's face it, you know, looking at a line that says tax, what the hell does that do?
Right?
But if you look at you know, say, uh, you know, frame rate equals sixty. Oh okay, I have a general idea that I'm setting a frame rate variable that represents the frame rate of some operation, and that is equal to the value sixty. I, as a human can understand that.
The problem with the problem, I would say with C, maybe not even the problem, but just the limitation with C is because it is such a low level language, and the design goals of it were to balance out the raw hardware speed with a bit of human creature comfort. You don't have a whole lot of safeties there. We try to rectify that a little bit with C plus plus but that is a whole mess that I don't even want to talk about. I've done enough C plus
plus CLI that it makes me hurt. I still have nightmares from those days.
It hurts. It hurts my brain just thinking about it.
Oh.
I think Kate Gregory is the only one that actually enjoys writing C plus plus Kate.
Kate Gregory is a champion. She she's like, the fact that she can make that stuff work is just I've infinite respect just yea too. I can listen to her speak for days on that stuff.
But you've also found a way to get the kind of farms we're talking that we need here to handle multiple streams and not need to live in C plus plus line.
Yeah, or or just live in seaaland when you need to for those for them, it's the application, because like do we need to write every last I'll give you an example for my multiview. I've got a web api h gdp api that you can use to make to change everything about the multiview. If you want to assign sources, if you want to move boxes on the screen around you know, why do I need to write that in C plus plus right? Like, there's no good reason for.
It, no reason, and plenty of reasons not.
To, Yeah, exactly, not least of which are security, right, Like how it's just.
The fragility, the maintainability, like it's all those things.
Yeah, So nothing else today, kids, you've learned that p and voke is way more powerful than you thought it was ever going to be back when you were calling the Windows API from vb net or c sharp, you know, one point zero. Because now the platform have expanded and we still have p and vogue, so we can take our dot net code and go anywhere and get to
the native hardware. That's that's the philosophy that Elias is talking about here, that we have to get out of Windows mode in dot net, yes, and into cross platform mode. And this is just one example of that.
Well here's one last point. I just thought of this, and this is like a penny drop moment for me.
Imagine you've got some legacy code and I'm talking like the author, and I've been in the scenario where it's like literally the author of that original code has passed on, so we can't even go drag them out of drag them out of retirement and ask them like what does this do maybe we've lost the source code, but we've got this, you know, but we've got this wrapped into a library of some sort, or we can get close enough, right, And we don't want to rewrite this old piece of
logic that lives in a c library, lives in an unmanaged library, but we need to use it for some reason. Okay, cool, we can write a if you can write C sharp, you can call into that existing DLL and that can just live on as a rapper. And because we can target X eighty six thirty two, X eighty six sixty four, ARM thirty two, ARM sixty four whatever platforms dot Net
will compile down to. This is now a means of preserving and carrying forward that old legacy code, you know, in a modern language with all the modern creature comforts.
And if you want to bring cameras into your software, use POE cameras. Your life will be simpler. Oh yes, please do no custom hardware. It's just on the network. It has an IP address. You'll be fine.
Mm hmm, all right, And NDI is your uncle Elias. Thank you so much. This has been a great talk for me personally, and I hope everyone else that's interested in audio and videos.
Thank you, it's been an absolute pleasure. I think I'm going to go off and see if I can adjust my sound last settings.
Yeah, playjazz dot mid while you're at playjazz dot mid. All right, We'll see you next time on dot dot net. Rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of
course in the cloud online at pwop dot com. Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show number one, recorded in September two.
Thousand and two.
And make sure you check out our sponsors. They keep us in business. Now go write some code. See you next time. You got Jack Dully to see A summer time that means is home than my Texas a lie fresival