welcome back to Tricky Bits with Rob and pj. So when I was a kid, I had a Commodore 64. I'd been inspired by the movie Tron. I'd be getting the Commodore magazine in the mail, and it gave you all these great programs for doing these really simple graphics things. So these very brutal raster graphics for doing these simple fireworks. And it was a huge amount of my imagination that then continued on to the Amiga for doing cool graphics.
That was actually one of the biggest things that got me into computers in the first place. But all of this, Rob starts to culminate into this beautiful set of amazing APIs that were developed alongside the hardware. We've got Iris gl, we've got Open gl, we've got DirectX I, I feel like we've got a beautiful story that's to be told around the graphics APIs that you know.
We're really that, that core kernel that blow up and transform into what we've got today in terms of like how do we go from these simple, very brutal 2D raster graphics on the Commodore 64 to realtime ray tracing and it all starts in one place, doesn't it, Rob?
It does, but you mentioned the Commodore 64, and that's midway through the story in some extent. I mean,
That's true.
and those graphics were harsh and NTSC PAL quality, graphics goes way, way back from there, 20 years earlier. 1963, Ivan Sutherland makes Sketchpad app, and this is his PhD thesis. It's a light pen driven piece of software that's running on a mainframe, driving in oscilloscope vector type display. It's revolutionary. You can draw triangles, squares.
It's the basis of early CAD and it's do trivial 3D what we take for granted today is the four way split screen of top front side view with the perspective in the corner that came from there. Ivan Sutherland is, and David Evans with him too. And obviously those names go together as Evans and Sutherland. That's the company they made after all this and through the sixties and seventies into the eighties.
That set the groundwork for lots of other researchers to dive in and start to solve the problems. How do you draw a line? How do you draw a line on the Rasta display? Obviously Bresenham's line algorithm
Mm-hmm.
All of the big names you think of of Ed Catmull and Jim Blinn and Lu and everybody who did all the early research in computer graphics was in the sixties to seventies and all that. But while we owe them pretty much everything, like how do you ize the triangle in, how do
Yeah.
do perspective correction? All of this we all take fully for granted today.
Let's not forget NYIT as well.
But that's not what we're talking about today. What we're talking about today is how we got to where we are today, the APIs, the hardware, and the future. I want to jump forward to a very important paper in 1982. This paper was by James Clark. James Clark worked at a company, you may have heard of Silicon Graphics
I mean, I've heard of it of certainly a lot of our listeners might not have because it's not really around anymore, but it was one of the giants.
yeah, but the IRIS workstations and all of that and lot of what was Silicon Graphics became Nvidia. So they are still around in spirit, even if they're not around as a company anymore.
Yeah,
anyway, 1982, they made a, a geometry engine. It was the first hardware transform engine, and it worked very much like Open GL works. It had a stack of matrices and it took in. vertices, given a primitive type. So we knew about lines, triangles, even knew about curves to some extent. And it had this stack of matrices, which looks very much like you'd think of open GL having in its original form.
Yeah. Classic open gl. Yeah, yeah, yeah,
it consumed primitives, it did matrix multiplies on them. It had a clipping system, which would clip to the typical view for stream as defined by what we call today, like a, standardized four D homogenous view for stream type, system.
Your typical pinhole camera.
So this normalized all the it could output view space coordinates. So we'd output coordinates in the current view, given a vertex input. The whole thing was done in hardware and obviously this hardware was. Kind of important 'cause it's kind of fixed function and we couldn't do hardware until we knew what we had to solve. This wasn't programmable hardware, it wasn't a CPU, it did these exact jobs, transform, clip, and scale is all it did.
And until we knew we had to do those operations, which is what all the sixes and seventies and all the pioneers of graphics came up with at this point we go, okay, now we can do this in hardware because we know the operations we need to do. So Silicon Graphics made this and they integrated it into a, a bigger system that could do a. Raster Imaging, they called it the Integrated Raster Imaging System, which is where Iris comes from. That's literally what it stands for.
the early machine was, I believe, a 68,000. It had, I dunno, half a mega gram or something. And it had some hardware attached to it. One of them was the geometry subsystem, which was this thing. And then it had this Iris's custom Rasta subsystem, which actually drew the triangles and would apply simple textures and things like that. So this was the first hardware pipeline. It was multiple pieces of hardware, but it was the first true hardware pipeline to program this.
Silicon Graphics, wrote a thing called IRIS gl
Just to check Iris gl, Iris Graphics Library, right.
yeah, obviously IRIS Graphics Library. That's what it literally meant. This was about, I don't know, 1983, I think IRIS GL one came out. ran on, Unix. It was still very workstation based and Silicon Graphics was always workstation based. And until the very end, they were very Unix peeps. at the end, they switched to windows based machines with custom hardware and ultimately switched to based machines with commercial hardware. And that was kind of the end of them.
But back in 1983, Silicon Graphics were ramping up to be the kings of Graphics and IRIS gl. One came out in, I think it was 19 82, 19 83, slightly after this hardware was first made. And it progressed through IRIS gl, I think version five. I think it got to by the mid nineties. It was about the early nineties when Silicon Graphics kind of opened it up and made this thing called rather than IRIS gl, they called it Open gl.
'cause it was open API Wise, it functionally looked and worked just like IRIS GL did. It was the same kind of matrix operations, the same primitives, the same everything. But this was for most people still software for, obviously if you run open GL on Iris Hardware, it would be in hardware just like Iris GL was. But being open, this ran on Windows. It ran on pretty much every machine there was, but there was no consumer grade hardware for a Windows machine.
what was the, the reason for opening it up was, was there a business strategy that said, Hey, if we open this up, you'll get to have it run slowly on a Windows or a Mac machine, maybe, or other Unix machines, but it'll be blazing fast if you pull it over to our hardware and you don't have to change the code.
not a hundred percent sure as to how it all came about, but control of the market this point in time was all
Hmm.
on control of the API. if you opened it owned it so you could basically
Got it. Okay.
it. It wasn't owned by Kronos at this point. still Silicon Graphics. controlled it, they set the standard and everything, but anyone else was free to implement it, kind of give them all the power because their hardware, like you said, was fantastic. This is the early nineties. Silicon Graphics at it, at its peak, they have the, uh, the giant workstations there. lots and
Yeah.
were made on
Was that the, was that the Onyx at that point in time or was that the Onyx later? Yeah. Okay.
came out. I think the later, yeah, we, we are heading down that path. But Silicon Graphics at this point had crazy powerful hardware compared to software
Yeah.
a, a 1995 Windows machine, for It's the, the power difference was insane. So, but yeah, I think you're correct. If by owning the API and by having a common API that you could mess with on Windows, you could always take it back to Silicon
Yeah.
and make it run incredibly fast. So I think that was the thing. Over time, the openness became more open and they handed it off to Kronos Group and all that. Mostly as they're starting to fail and realize that they can't control this anymore. where the Kronos group originally took control of OpenGL.
so obviously that gives us the first on the PC and the common pipeline for multiple machines as to where our origin story for today's hardware really comes from the early stuff was just to give context. Open gel is kind of where it all started on. On a home pc.
So at this point in time, we've got a standardized set of, of APIs. It's still a fixed function pipeline. And I wanna hammer on that for a second because I think it's important because at that point in time, in the mid nineties, we start to see a bevy of companies start to come up that begin to challenge the, at least from a commodity standpoint, graphics, hardware.
And the three that really come top of mind to me, and maybe there's others, would be match a TI and Nvidia as getting their start in that, that mid nineties standpoint.
most important one from the mid nineties. That's through the fx.
Oh shoot. Are you right? I've, I totally forgot about them. Um, you're right. 3D FX.
So there was a big war. Everyone knows, well, maybe not everyone knows 'cause I'm old and people listening may not be. Uh, there was a huge war started in the mid nineties between Open GL and Direct X. At this point, we're still early nineties. Direct X isn't out yet.
Right,
isn't out
right, right. right.
GL is early. And there was a, I think about 95 Windows NT got open gl the mess of 3D project started around this time too. It's really, really old. I mean it, everyone uses it today. But the open source mess of 3D started, I believe mid nineties. It may have been like 94, 95, something like
yeah.
the, uh, the glut open GL utility cut, uh, toolkit, which was. this day is the principle of a lot of GL applications, Vulcan applications today.
Yeah.
the also started around 95, 96, something like that. And beyond about GL 1.1, maybe 1.2
Right.
95 still isn't out. It's coming out around this timeframe. We're in the mid nineties and direct X is still a, a pipe dream. It doesn't come out for another couple of years, especially direct 3D. so we have this, we start to have hardware that can start to do primitive 3D graphics. Uh, you mentioned all the, all the players. Nvidia really wasn't a player at this point. They, they had the NV one. The NV one was kind of a disaster 'cause it rendered curves.
It didn't render triangles and all of these, graphics chips as well, they add. BL 2D Sprite support lots of DOS games, took advantage of this SEN, T-S-E-N-G. a big one for the 2D was space matrix were big and all these people competed on these 2D graphics performance. And the 2D performance was important because obviously Windows three point ones windows were just bl rectangles. it was important for these things. And they had drivers for early versions of, windows.
But DOS games just took straight advantage of the hardware and didn't use the operating system at all. And
Well at this point, I mean, we still don't, I mean, most of this stuff is still being done in software. We're still rendering in software and then splitting it to the screen. Any sort of like faux 3D like doom or whatnot, um, or Wolfenstein before that is faking it using 2D tricks.
well, it's not, I mean, some of the. DOS games worth true 3D, they just rasterized it in
Oh, sure. Yeah.
I mean, I worked on lots of software rasterize and we took lots and lots of tricks. We didn't implement the whole pipeline. was, did exactly what it needed to do. And 3D games were out at this point and they were, people were playing them. I mean, your story was the 64, my story was Elite. And on the BBC micro, where I saw 3D graphics. It was like blown away. And I was too young to really comprehend what it took.
But then the, on the Acorn EDUs, I wrote an, they had this vector, uh, graphic system where you could like rectangles and triangles and things like that. and I extended it so it would do 3D so you could start to do 3D graphics, very crude and no depth buffer, no hidden line removal, things like that. But it was that kind of one of the first commercial projects that I did was that software izer to be able to do, Polygons in 3D and I wrote a logo program, which was kind of like logo for 3D
Oh wow. Okay.
so you could kind of draw cubes instead of join squares and you, it was kind of a cool little project. I think that was my, my A level project. I can't remember, but I still have it somewhere. mid nineties.
The software rested games for, for PCs, there's lots of hardware to do 2D graphics that people are taken advantage of everyone's writing their own drivers it's a fun time for a programmer, but it's a mess of things to support and open gel wasn't used in games too much a lot of the hardware at this point was starting to solve 3D graphics, but it's still very software driven.
The hardware could do things like if you gave it the correct gradients for the ends of a scan line, it could render once scan line. to draw a triangle it would still be a lot of software. They solved the Rasta side first, and then came back and solved the side much later. So all the geometry was still done on the CPU. It would be into screen space. And then in screen space, they'd figure out the, the gradients for walking the, edges of the triangles. And the, this was all done in the driver
I think it's, it's important to say all these things were being marketed at the time as graphic, uh, accelerators. They didn't, as you say, do the whole pipeline on the card yet.
the whole
Yeah.
This, was, this worked like this for a long, long time, and so these were marketed as 3D uh, accelerators and they were significantly faster than, you doing it yourself. The hardest part of software izing is literally the
Yeah.
So if you could take that away and just step the edges of the polygons, work out some gradients and pass that to the hardware. You could get it to do the stepping of the pixels across a given scan line, and it was significantly faster. then they could add textures and they could add perspective. Correction came out at some point and not, not really much changed. It was just how you step these gradients across the triangle. But it was all still done in the driver lots and lots of CPU overhead.
so that's how it was for a couple years that 3D FX came out and they did the whole triangle in hardware. You could just it screen space coordinates a triangle and some perspective factors and it would render the whole polygon in hardware. So all you had to do now was the transformations do a bit of screen space projection and a little bit of math 30 effects'. Glide API took care of most of that, and that was a game changer. 30 FX was like the king of the hill for a hell of a long time.
around this time as well is when Direct X came out. And initially Direct X only, had direct draw.
Right.
it didn't do 3D it was just a kind of a blitz engine. Kind of took all those 2D Blitzers that we used in those early games, by Windows for GDI kind of made it into an accessible API called direct draw so anybody could split Sprite really quick. So 2D games suddenly got, easier to ride. And at this point we're in Windows 95. So abusing the hardware directly wasn't as simple as it was in the DOS days and it go into Windows NT it was gonna get a lot more
Hmm., Just to double check, I mean, right now. The primary players in this market, or the primary player in this market really are the Windows boxes, because by this point in time, in the mid nineties, I mean, Silicon Graphics never converted into something where it was gonna be a mass market consumable thing.
Commodore basically had failed at this particular point in time, even though I, my first encounter with 3D Graphics was on the amga, and then Max seemingly, you know, was in, its, its doldrums at this point in time. So,
mid, late nineties. This is Mac barely existed.
So right now, really windows is, and, and PCs are really the only thing on the scene that are gonna be taking advantage of these things. Everything else is either defunct, uh, in its doldrums or so commercially unviable for a mass market.
extent. The PlayStations are out. The PlayStation one is out, but that wasn't perspective, but it was hardware
Yeah.
are all the platforms. Doing 3D and the N 64. Uh, there was a whole bunch of them speaking of Silicon Graphics. That was the N 64 too. And Oh, I didn't know that. they designed a, a little mini IRIS chip for the N 64. So there was all the players, but the big players was windows. Um, windows is really the defining story of graphics as to how we got to where we are today. Linux didn't really exist. It was a kind of little project in Linus bedroom at this point. so Direct X came out, unified.
The 2D made graphics possible in an operating system without abusing the hardware. And Direct X two didn't exist. There is no direct X two. Direct X three came out and it, this thing called Direct 3D, that was kind of where it all changed. It was a very inefficient API to start with. It still had all these software layers, very CPU heavy to get anything out of it. Very clunky to program. I actually have DirectX three's original install C Dig, and I believe it still works on
Wow.
One day we'll do a video and we'll, we'll run these early apps. As far as I know, direct Draw still works for sure. I think early Direct 3D works too, which is insane as to how compatible Windows is. They could have dropped Direct three, direct X three
I've seen enough machines where it's these medical devices that, you know, they were built 20 years ago, but they cost like a million dollars. You can't build 'em again anymore. You can't replace 'em. And they're all connected to some like Windows 95 or Windows NT or XP Pro. So I can imagine there's actually a significant demand for why these old graphics APIs need to hold around.
X three
Sure.
it's fine. If
Yeah.
Windows
Maybe they hope to upgrade those some days. Yeah.
97, 98, something like that. it had these early graphics and it was clunky to program.. Ironically, in some ways though, direct X three had these execute buffers, which was kind of
Oh, interesting.
which build. And it was kind of like you'd program them today, but they weren't buffers. They were just like action command buffers. And would pass them in software and do its thing, and driver would do it to ize. And if it had to. This is kind of where the four bath walls started.
gave up on the NV one, the spline quadratic spline rendering hardware that they had, and went all in on triangles for Direct X. The, reaver, the TNT and things like that started to come out, which were very much direct X calls. And of these had GL drivers too. So GL was still there in the background, Microsoft was pushing Direct X really, really hard for 3D fx. really wasn't a decent direct X driver.
They wanted to be the API owner just access the format was back then if you owned the API, you owned the market. So basically, 3D FX did their own thing and never really played a part in the Direct X
Might be one of the reasons why I forgot them. But I mean, are you saying Rob, that there was a third API besides OpenGL and DirectX that came out? Glide. Okay,
was three df X's API, and again, it was completely different to, it wasn't completely different to direct to OpenGL the ideas weren't completely different to any API, but it was a different API you wrote
got it.
or you wrote games for And so they kind of got into this format war with, Direct X and obviously Microsoft won in the end. So speaking of format war, John Carmack started a huge one in the, the Quake, which probably been the late nineties,
we're in the mid to late nineties at that point in time. Yeah.
And Carmack wrote all of Quake to use OpenGL and basically wrote a blog post as to how OpenGL was way better than Direct X. And at that point, it's not wrong, but it started a whole API war, a Cultural War, which still kind of exists to this day. Like people will still that OpenGL is better than Direct X 12. And that's obviously far, far from the truth. But it's almost started a religious war as to how and why these things work.
And this point, we're still talking CPU overhead to get the graphics to do something. So if you rendered the same scene in Direct X or the same scene in Open gl, the open GL one would be faster on the GPU and on the CPU as to how much work the CPU
Right.
to feed the graphics to get open jail was just generally better, but it's, state management has to have had this global state machine, which time is what killed it. Like GL to this day isn't practical because of that global
Right,
There's lots and lots of problems with gl, but back then none of these were issues and GL was a better way to go, but it didn't evolve, it didn't keep up with the times. over time, DirectX changed the API, like OpenGL 1.0 still works to this day. changed the API
right.
single version, which really annoyed There was 3, 5, 6, 7, 8, 9, 10, 11, and 12 are all completely different APIs. So over 20 years there, at this point, I. There's been a lot of versions of DirectX and each one was modeled on the API, the modeled the API on the
Right.
at the time. can see as the hardware changes, you can see the API changing to take, advantage of that into expose
I think this is actually though a good example where, I mean, there's different governance that's occurring across OpenGL versus Direct X. OpenGL has basically this consortium, which is Kronos, which has got a lot of different players at the time, you know, hardware people. I think Microsoft is even a part of it as well.
And so there is this consensus basis of just trying to say, how do we be nice to be backwards compatible and then you get into this whole sort of GL extensions where it's like, Hey, what is, what do I wanna check on to see if there's an a, an available bit of functionality here on the hardware? So, uh, getting stuff into mainline OpenGL would take a long time. Direct X was ruled by a czar, which is Microsoft.
you are, you're picking
I am a little bit, I am a little bit,
do DirectX feature
uh, okay.
thing ever they would have this bit mask of things that you supported, but you could lie, the drivers would lie all the time. They'd say, yeah, we support this, but
Oh,
it in
Okay,
it's like it was supported as a checkbox on the box for marketing reasons, but it wasn't really usable. It was done in software or it would break something else, made programming it really difficult. 'cause you'd start jumping through hoops going, yeah, it's an A TI card. I know it's this model and that bit is set, but I have to ignore it for this partic particular card.
So you had this crazy amount of logic on the CPU side, which was dealing with feature flags and drivers that lied and Direct X fixed all of this, and 10, 11, and 12 have these big blocks of functionality you have to support, otherwise you're not certified. but going back to this timeframe, you're absolutely right. It was like Kronos at this point had OpenGL and they had to support a wide range of hardware, very different hardware.
Don't figure, it some extent Vulcan to this day has this same problem. Like Vulcan has the concept of a render pass, which makes no sense for a forward. Rasterize like hardware, but makes total sense for a tile based renderer such as a power vr, even Apple or a morally type GPU. because obviously they have to load and flush the tiles before you can render to a frame of it. even today, Kronos still fights these same problems.
The Vulcan extension system is identically the same as the GL extension system, which is a mess. but it does allow you to support anything you want to support, which back to gl, they had GL one,
Right.
three, and had, and there's obviously GLES where they
Yep.
the
yeah, yeah,
mobile we forget about that. just a mini version of gl. And they did add things. So at this point, we're in the late nineties and the big thing that came out in the late nineties was the GForce one.
Yeah.
GForce 2 56, this was the first consumer hardware which had a full, not only a full triangle restaurant, but a full hardware
Yeah.
feeding it. basically give it the, going back to that seminal 1982 silicon graphics paper, it had a full transform engine in hardware. So you could give it object space coordinate, and it would do the object to world, transform the world to camera and the camera to perspective
Yep.
AKA clip space. It would do all that in hardware. It would clip it, it would make clip triangles if it needed to. Then it would rasterize them in hardware. So this was the first consumer one, which had no CPU minimal
Is it is it fair to say that, I mean we're, we're looking at a timescale between 82 and that paper came out to maybe what, like 98, 99. So
is
17 years basically before we went from, you know, a commercial industrial form of this in hardware to something that's now available for the consumers.
a while, but you've gotta remember at this point, Nvidia had a lot of Silicon Graphics employees. Dave who was the architect of the GForce one and all the early GForce, was a part of
Hmm.
team. knew where it was going and he knew how to do it. It was just being able to make it in a consumer price point, had to wait for fabrication costs to come down and things like that. ultimately, that's what they did. And they were the first, and then a TI followed. at this point there were still more players than just Nvidia and a TI, uh, matrix was still in the game. And like that, but they all kind of quickly dropped away and it ended up being just the two that we have today.
Uh, Intel wasn't really a player in the 3D space at this point. They show up much, much,
Sure.
later on, and then shortly after the 2 56, some of the early rally ons came out, which also had a hardware pipeline. So once they had this, the next step was, well, we can DMA drive this. So now it's just, you can send entire list
Yeah. Yeah.
the hardware and it will and do it with minimal CPU overhead. So all the time, the amount of work the GPU is
yes.
increasing and the amount of work the CPU has to do to, to drive the GPU is decreasing. So the overhead of the CPU gets better and better, and this is where the APIs start to struggle, Because they're not doing much or not supposed to do much or don't have to do much. If they do do any significant amount of work, you start to see it as a huge problem. And early Direct X would do things like it had these token streams, it had the direct X command buffer.
so as you called the API, it would make these fake tokens kind of like the equivalent Direct X three did it, but you did it yourself in Direct X three. This was the execute buffer. That concept stayed around for a long time. It was just that the API would generate that execute like buffer, the command stream. And this wasn't a hardware command stream. It was this virtualized direct X command stream.
the API would generate this virtual command stream and it would pass the whole thing to the driver, and the driver would then have to pass it and basically Interpret it to see what it meant. So there was always this massive amount of overhead on Direct X in OpenGL. It was done very differently. The vendor wrote the API themselves. There really was no driver, like Nvidia would implement a DLL that would
Right.
directly or at least with their driver and implement the API. So it was easy to have bugs or have bugs between A and B, like a, a Nvidia on this hardware and this hardware. There's a bug in this, API call because they own the whole stack,
Sure.
efficient. a direct tech stayed like this for a long time. finally got away from it in the the 9, 10, 11, 12 framework. But it's recent, more recent. I mean, we say more recent, direct Tech nine is probably when we did that come out. Direct Tech nine
Yeah, I was gonna say it's like 20 years old at this point in time.
It's a long way back at this point in time.
I remember at this time,, I'd had programs where it's like you could choose which engine you wanted. It was direct X versus OpenGL. Like I had Ari True Space where you know, you can basically engage with the scene before rendering it. I remember there being market differences depending upon what machine, what computer, and what graphics card I had at the time.
a lot of it came back down to how these features were exposed. Like, because GL had the extensions and vendors could add extensions themselves, it was quicker than getting it into direct text. Direct text. You might have to wait for the
Right?
release to get that feature at all. So, GL for a long time was the place where a lot of it happened. And over time, GL added. Features like GL two and three started to say like, now all hardware based. The traditional pipeline goes away. Things like that. it wasn't like GL stayed this old API and
Sure.
it all through extensions. fold a lot of the required common extensions into the new versions
They had tiered extensions, I think. Didn't they have like where it was like vendor specific and then.
same system kind of today, whether it's a KHR extension or an NV extension of who added it. And is it common is all kind of the same. If it is as it is for Vulcan to this day, you're gonna forget of this, we're talking about side, the geometry side, at the same time. pixel side's getting more complicated. Initially it was single texture, then it was single texture perspective. Correct. Then it was, oh, now you can blend two textures. You can do multi texture, but you only get these simple blend
I remember those.
the two and, then I think the GForce added these, you could do like six or eight stages and you could do simple ops between them. It was still very much a blender. It was kind of programmable, it was very much
Yeah.
ops. But became things like dock product. And so you could start to do your own lighting per pixel, light lighting, you could do bump maps and things like that. And was lots of people that were abused it to do various other things. But that's kind of where the programmable pixel came from. It's at this point, it's a fixed function pipeline and. a semi programmable pixel with
Right,
blend states. still
Now, at that time, did, had we gotten the depth in stencil buffers yet, or was that still yet to go?
around from day one basically. And so this is the time we're at the late nineties now. This is when I'm making the original Xbox and we're talking to Nvidia and lots of other people. And obviously Nvidia won that, battle in the end. this is kind of early PCS game consoles, and when we first made the Xbox, we got laughed at because we were making a console
Right.
hardware wasn't this esoteric stuff that was good at certain things, but terrible at other things. And wasn't until, I guess the PlayStation three would've been the last of the esoteric pieces of hardware. And then
Yeah.
was. PC based. So from the 2010, 1214, whenever the PS four came out, that was the first Sony hardware that was PC based, and now everything's that way. So we kind of set the standard, but back at the time, it was revolutionary to use PC
And
all the bits came together. It's like hardware could do graphics, hardware could do audio, leaving the CPU for the games. So it was just made sense to do it that way. obviously over those 20 odd years, power of the GPU is becoming in credible and the flexibility is become in credible. But at this moment in time, we are not there.
moment, we're actually on the cusp of something. That's, that I remember being big. That was changing traditionally, you know, I'll pick on OpenGL. It was a fixed function pipeline. I remember, and at first this was only, I think on Nvidia cards. Like we start to get like, hey, now we can do vertex shaders. Now we can do what was called pixel shaders later. Fragment shaders.
early on these were all assembled. This is around the, the uh, the GForce time too. This is kind of when
Yeah.
to change. So geometry pipes and you go, okay, I need to transform my vertices to clip
Right,
before I can render them, compute, uh, texture coordinates. But then what if I want to animate the texture coordinates? What if I want to transform the light? What if I want to transform something else? What if I want two textures to use the same texture coordinates? And the third one to use a different texture coordinate? How do you communicate that to a fixed pipeline?
it started to get very difficult to manage this fixed pipeline when options came in So again, early on, Nvidia, with the lead in force in all of this too, is you've got these vertex shaders. They wrote an assembler there was a,, pixel shader assembler and a vertex shader assembler. The pixel shader assembler was, it looked like pics, like he was doing operations, but it was really just this initial, like says the GForce had this blend stack and it was really there.
the vertex shader was basically, you did your own transforms. It was very good at doing dot products. So you would, pack a vertex with your data and you'd transform whatever you wanted to do. At this
Yeah.
programmable so I want to transform my positions, which is still the same to this day, but then I want to do something weird with a texture coordinate, which you could never do, in a. Programmable shader. You could do things like, I want to transform from tangent space so I could do it per pixel lighting. I want to actually encode tangents as colors so I can get them into the shader. It opened up this massive world of other things you could do.
You could manipulate geometry on the fly because it was all being done into software. All al be a very primitive software and this was the start of the programmable shaders. And I remember talking to Dave Kirk around this time telling him one day, Dave, you're gonna be writing compilers and just be running full apps on your, on your GPUs. And it's like, nah, get out of here. It's never gonna happen.
But that compiler happened fairly early. I mean, cg, was that, I mean, initial language that came out from Nvidia, that was CG slash hlsl, right?
Well CG was an HLSL? No, it was a competitor to it. It was. It was, we're probably at the GForce two or something at this 0.3, but it's
Yeah,
early GForce. again, you have to wait for the direct X machine to cycle through. It takes ages. So by this point, Nvidia have got fairly decent programmable hardware, but you're still programming them. And Assembler on DirectX and Nvidia released this CG thing, this is probably around 2004 or
think it was like, it
well, CG on the PlayStation. That's kind of, again, it was Nvidia hardware and we used the CG compiler core to do everything on on it. it was ultimately built into custom tools, but it was, the compiler was itself cg. made this tool because they wanted a high level way to program. They realized assembler was not
Yeah, yeah. Yeah.
the in this language called cg and it was kind of a bastardized version of C had VE falls and things like that. And it was useful. I mean, it did it needed to do, and you could write them in these higher level languages and the compiler could then compile down to different architectures have the same high level code. Where for the assembler versions, that wasn't true. You have to, like most assembler, you have to hand write them for each machine.
DirectX helped a little bit, but generally it was problematic because this machine,
I do think it's actually really interesting though in that this is an early instance and we've, I guess we've seen this with other companies, but here with Nvidia, where CG originally only ran on NVIDIA cards. it meant that if I was making my game or making my thing to make it as cool as possible and I was using cg, it would require the user to have an NVIDIA card. And the reason I point this out is that there's a very, uh, analogous situation today where we talk about this with Cuda. Yeah.
came from cg, so they are
Yeah.
to each other. so CG was this thing that Nvidia did this, they did a compiler and it was a crish compiler to, to start with, and couldn't do loops. It couldn't do conditional uh, branches. It had minimal flow control because the hardware had very minimal flow control At this point as well, if we're probably getting into the direct X nine, eight had the programmable shader, uh, assemble shaders.
I remember those. Yeah.
that's the one that had the assemble shaders and direct X nine is when HLSL first came out. And HNSL initially was also fairly crude. All of these tools would just generate
Right.
by code that the assembler generated, and then the driver would take it from there. There was no real optimizations inside the driver. CG would just spit out direct X by code you at least could write in these high level languages. And then over time they started to add more functionality to the shaders. At this point, they were still very dis different between vertex and pixel This was most evident in, the PS three, which had, disjoint.
Architectures, it still had the old school Vertex shaders book, slightly newer school
I see.
shaders. gone at this point around. This was the, uh, Xbox 360 time a TI had gone to unified shaders, so it was the same cause running
Oh, I see.
instructions, whether or for pixels. And the hardware would just allocate resources as it needed to. three was an NV 48
Yep.
Seavers Nvidia. Uh, so NV 40 long time ago, had this weird disjoint pipe where it was two different blocks of hardware doing two different operations, and they had slightly different instruction sets and slightly different things that they could do, which kind of made it annoying. And obviously ultimately Nvidia in the 50 series, GPUs did move on to unified architecture. So compute and everything else, it's all the
Now, j just to double click on something you just, you had said, I mean, when you say there was no loop and no conditional branching, uh, do you mean that that wasn't in the assembly because the languages supported? A lot of that stuff, but it, I felt like they were all just unrolled in the assembly or had to use bit masking for the, the actual branch, uh, resolution.
So dynamic execution and shaders has been a problem since day one. Uh, it's still
Sure.
in the ass to this day. uh, the problem ultimately is you have all these pixels doing the same thing, running the same programmer. But now what if two pixels want to go different directions? What do you do? And. Ultimately today we've run both sides and some pixels take a and some pixels take B for the
Right.
whatever it may be. but you, you do a lot more work. If all the pixels go one way or they all go the other way, then you can do half the work. 'cause you don't have to do both sides of the if, literally how it works to this day. back then there wasn't really any flow control. And then they started to do kind of static flow control, where you had these static branch instructions you could set basically balls outside of
Right.
the shader. You couldn't dynamic data, but you could say, this is gonna go this way, this is what's gonna go this way. So you could start to write uber shaders that could do functionally multiple different things and you could static branch around the bit you wanted to go around. so it started there. They started to add, conditional. Execution.
So pixels, it's kind of like early on where you could do a compare, then you could conditionally execute the next few instructions based on the condition of that compare. But it always executed them time-wise. You didn't, you didn't skip anything, but it meant you could get some conditional execution within the shader and the high level languages.
Would you just write it as an F and it would kind of, the compiler would kind of figure out what it was doing based on the, uh, instructions in the target architecture. So CG obviously did this first 'cause they had the best control of their own hardware. And was a long time before true dynamic execution came around where you could literally do a compare on a, on a live register branch on it. once you could do that, you could start to do.
True programs, these machines became to incomplete at this
Sure.
So they could do all sorts of flow control op operations, they could do loops and which ultimately, initially it was a problem because you could loop for so long, you'd lock up the GPU, which is where the timeouts today come from. Like if you take more than a second, it'll kill your shader.
if you want to take longer than that, you have to tell it in advance that this is gonna be a slow one, uh, and you lock up the GPU and 'cause it wasn't multitasking or like that back then, it would just take out a whole machine. So it became possible to kind of lock up windows by just having a bad shader. but this was a huge step in the right direction. 'cause now you could do loops in code, and the compiler wouldn't have to unroll the loop. If you do a loop of a 10, it could unroll it 10 times.
But if you do a loop of N where N's not known. compiler would
Yeah,
the early compilers because he N comparison. But now he could. So now we could do Y loops repeats, four loops. Although she still had a hard time calling functions. Function calls were still not a thing. Uh, maybe there was a stack, but it was only
sure.
level deep. call a function, but you couldn't call a function from a function. but as you can see, we're gradually getting towards general purpose computing. And the early attempts at this was G-P-G-P-U, like GPU program. And then added a lot of this. We do physics sims by packing data into textures and then loading numbers, frame buffers and textures and applying floating point math where we could and writing out the result.
And it was basically this multi-path system, but with floating point textures or sometimes it was in input and floating point outputs and things like that. But there was this whole
Yeah, I remember that. I, that was huge.
to solve obvious problems on the GPU Navi, Stokes fluid sims and things like this became kind of the look well I can do on, uh, the GPU and could start to solve more complex problems. You could start to do things like, well, I could probably sell up some rays and do some collisions and things like that and get the results in a texture, then get that texture back to the CPU so I could, use those results.
So you started to get this unholy integration of GPU and CPU, So that's what G-P-U-G-P-G-P-U was, but it kind of got normalized. NVIDIA took CGMA, Coda Coda 1.0, which was kind of this nice interface again for Nvidia hardware. point, no one had an idea what it would become, and it kind of normalized this compute programming and over time, direct X 10 added compute skaters. So it was a normalized way of doing compute across different pieces of hardware and gl.
I also added all of these things too. It also had open cl, the open compute language, which was kind of a cuda to, for doing compute on generic GPUs, not just Nvidia and it, and no, it didn't really go anywhere because they added compute. To GL itself, and it still exists to this
Well, it's, it's fascinating because I.
in uh, in some of the a MD open source compute platforms. But, but ultimately this opened the floodgates for compute operations on the GPU and also changed the APIs because up until now you had to have a graphics context to do
Yeah, yeah, of course. Yeah.
At became its own thing where you didn't even need a graphics app. You could just fire off this work to the GPU the driver would just take care of it without you having to have a a compute framework within your application.
So what's what's really interesting as we're we're talking through this, I mean, OpenGL is fading in the background at this particular point in time as a graphics API, direct X is taking charge, NVIDIA's taking charge. We don't really see a lot of innovation. From my perspective, coming out of a TI, I mean they built really solid hardware, but it seems like they are increasingly seeding at this point in time, any of the interface side of this stuff.
Yeah, a TI obviously became a MD they were very much in the Direct X today in the Vulcan Camp Butler, they do some good work. They have some
Yeah.
fast GPUs. I mean, but they, they lost out on the kind of the compute front. They didn't have a coda. And by fluke, Coda became the defacto language for heavy purpose. computing. that became used for AI as well as physics sims and weather sims and supercomputers. became the language of choice for data mining, machine vision, machine learning, all things like that. So that's how cute it got to early to this day.
And A A MD didn't really have an equivalent and really got on the board in the early days of making a coup a compatible thing. 'cause it was an Nvidia thing and it was, it'll go the same way went, do a text will kill it, blah, blah, blah. But it never happened. So kind of kudos are kind of the one that escaped and became a giant thing by itself. But as far as graphics go, a MD is still up there performance wise with
Yeah.
and in some ways they're than Nvidia. 'cause today, Nvidia have all these bullshit specs, which are, oh, that frame was AI generated and. Like what are the specs? Just rendering it straight. Forget all the things you could do to make it faster. What's the true speed of the GPU? And that's an apple to apples comparison. And in those worlds, a MD hold its own pretty Uh, well, where they start to fall apart is when they start doing these AI processing of frames.
they don't really have the hardware to be able to do it where Nvidia do. But it doesn't mean the stats are any less bullshit.
just to to stay on the G-P-G-P-U thing for a second, the, this an interesting I analogy in my mind. I'd, I'd love to kind of talk through a bit with you, and it goes back to the cell processor because, you know, with the cell we had an underpowered, you know, PPU, and then we had these awesome stream processors and obviously the NVIDIA chip was on there as well.
And it really occurs to me as we're talking about the graphics card being used for gp, uh, GPU stuff, I think about those stream processors as just being those real like, awesome, like, Hey, I can execute shit incredibly fast, you know, pack it together, vectorize it, like make stuff happen, is the GPU basically became the commodity version of that.
Like the cell processor never really took off as an architecture of these kind of different styles of compute because the GPU was effectively able to occupy that space and it was already in the commodity market. Like, is that a fair like way to think about this?
That, you know, a lot of the stuff that we're talking about solving Navier Stokes about trying to do interesting collision stuff, like I would've thought we would've put that on like an SPU on the cell, but now we're, but we're now putting it on A GPU because it's, you know, fairly similar analogous hardware but it's out in the open and I can use it on basically anyone's machine.
depends where you're solving
Yeah,
If you are solving that problem to get a be Avi your Stokes Sim, and that's the entire goal of your software, then today you will do that on the
yeah. Okay.
if you are doing Navi Stokes in a game, A, it's probably not state of the heart because it just needs to look good. B, the GPU is already very,
Yeah. Yeah.
So in a game, it comes down to execute. Solve a problem where it's best solved for the whole performance envelope? Not necessarily. I could do it faster on the GPU. If it takes two milliseconds on the SPU, takes one millisecond on the GPU, then in a, your performance head would say, well do it on the GPU.
But if that's taken a, a millisecond of GPU time away, that's not doing something visual and the SPU sitting idle, then it's a net loss and architect that was kind of the problem of program in the PS three of we have this spare time here. It's very powerful. It's a pain in the ass, the program, but if we do it we'll get all this free time and we don't have to hassle the GPU. And to some extent, that's still true to this day.
games tend not to do heavy computation on the GPU, even though things like physics and, lots of other. Libraries can solve on the GPU. They also solve on the CPU because games typically need all the GPU time they can get it's getting less true. I mean, things like on real five with lumen mega lights, things like that do a lot of general compute on the GPU, but it's all towards graphics.
little, although it is possible traditional, there's very little of like, go solve this completely arbitrary problem and do it on the GPU. If they're solving non pixel problems on the GPU, it's generally something like related to caches or something like that so they can reapply it to the graphics later. And a lot of this comes down to availability of data.
If you are solving a. A problem that's typically CPU and the data is accessible to the CPU, it may not be accessible to the GPU in an easy way. getting that data, the data management, you, GPU can see it and you can transfer it easily. So there is a lot of marshing of data to get the GPU to do things. And if, again, if you're setting yourself up to solve these problems in compute on the GPU, then you'll think of this from day one. But if you.
If you are writing a game, then a lot of this data is not readily accessible to the GPU and getting it there is just a problem in itself. going back to the, the, PS three, though, you mentioned the cell, those SPU originally were made for graphics. were, before we had the NVIDIA chip. In the PS three, we had the, the original rs, the Sony rs, which is, was just a pixel engine. It was very much going back to the early iris architecture where geometry was done here.
And the, GPU was just a pixel engine. And then the do all the geometry transforms and fully programmable whatever you wanted to do. And they, DMA, the results is screen space triangles to the rs. There was this thing called The L-D-P-C-U in the middle, which to this day, I'm not quite sure how it worked. Uh, but anyway, it was some sort of asynchronous gatekeeper. So you could submit geometry in order to the spu, they could complete out of order, then it would render in order at the pixel side.
And it was basically a big scoreboard gatekeeper thing. And it was a, it was a nightmare piece of hardware and I'm glad it went away. But anyway, that was the original architecture. So the PS three would've been the, just the crappy ppu, the power PC cores core, two threads on a core, doing all of
Hmm.
And then the would've been used for geometry, could have been used for other things like they were, but PRI primarily would've been doing the transforms and
see,
graphics. And then the RS would've been doing the pixel side. then when NVIDIA showed up and we put the Nvidia chip in it, the RS became the RSX The S PS no longer had a job, so they were now used for general purpose compute, helping out the PPU, which was ultimately the lifesaver of the PS three.
I see. So, this heterogeneous architecture on the CPU side is more of a happy accident and sounds like because Nvidia showed up. And can occupy that graphic space entirely.
We didn't need to do the transforms for the vertices On the SPU
Right.
effectively. 'cause the GPU could do it by itself and it could fetch you DMA it by itself.
Okay.
suddenly had no job.
Well, but you are able to give it great jobs after that,
jobs, but first party did a very good job at doing that, where others didn't. they were very hard to program and it was kind of. It was hard because of the memory footprints and things like that. It's the same problem the GPU has today. I mean, they're just big vector processes, anyway, so that's how we got to compute shaders and then at this point, things like became a problem because it wasn't integrated with anything like direct X compute shaders.
Now all HLSL and HLSL is a, term in complete language. it's a very nice language. I like it a lot and it. It's the same language for of the shader types that are inside a GP today, which is not just, there is whole shaders domain shaders and geometry, shaders, ver shaders, fragment shaders, compute
Yep.
blah, blah. blah. All of this is becomes, one language and they all interact. So this buffer is visible to a compute, but it's also a texture. And so it becomes it for games and, visualization becomes very nice.
Where Coda is, it's not, not possible, but it has, there's a definitely an interop stage you have to go through where it's like, this is call a few random APIs both sides, so you can see your coda buffer as a direct X texture, or you can see your coda buffer as a Vulcan texture or something like that. There is a, uh, A definite interrupt where you have to share that resource. Where in DirectX you typically don't have to do that. So, but so ca Coda is at a disadvantage there, but not by much.
They, they did solve the problem where steel never really did solve the problem. So it became this, if you need to do pure compute, you could do it, but you can't do anything with the result. There was lots of copying back and forth, which takes us back to the original prompt of the efficiency of these APIs.
direct X got really good in the direct X, 10, 11, 12 days of to make very efficient APIs that very closely matched the hardware to the point where Direct X got so big where they'd make a new spec and then the hardware would match what the, uh, what Direct X did. So Direct X drove the industry for the last or 15 years for sure. And. things like Ray tracing, nobody else added, and then
Right.
it and then it shows the paper anywhere else. So the API we haven't talked about much yet is Vulcan.
Before we get there, 'cause I, I, I actually wanted to backup a little bit to, to OpenGL again because OpenGL is still hanging around at this point in time. It's mainly being used on Mac. We start to see it on iOS, but it is flagging, I mean it's used all the way up until about seven years ago and it's actually officially deprecated. But OpenGL is struggling at this point in time to grow beyond, in some sense its fixed function, britches.
'cause I remember like, uh, especially I was dealing with OpenGL, ES three maybe, maybe four, I can't remember, uh, where it was like, hey, you have, you still had vertex shaders. You still had, pixel shaders or fragment shaders. Um, but geometry shaders hadn't quite shown up. They were in the main line, but at this point in time, it feels like OpenGL is really getting long in the tooth. It's still there, it's still supported.
Mac is really its main place, but it's effectively gone from Windows at that point in time.
Mac too. I mean, it works on a Mac but it hasn't been updated for years
Well, it was deprecated back in 2018, but I mean is like in this interim period we're talking about in the 2010s, it's still
it's still used, it's still developed. Uh, it's easy to use. it's the nice thing about it,
Sure,
use a, if you make an app at Vulcan of direct X 12 today, there's thousands of lines of boilerplate code that you have to write. we'll get to Vulcan and direct X 12 in a, in a
yeah. Yeah,
Butler. They easy to use. So if I was writing a quick AP to like visualize
sure.
I'd write it in uh, Is it used for big apps? Yeah, for some, yeah. It's still like a lot of the CAD packages use gl. A lot of 'em are moving to Vulcan, but gls still are a, a definite solid backup. it's reliable. It works. It made a bit the fastest thing, but working
Sure it.
is more reliable than is more important than performance?
So, I mean at this point in time, OpenGL, it is still, it, it largely hasn't made the move to something vastly more dynamic like DirectX has. If you look at the code, it's still kind of reminds you of that fixed function pipeline from
if
25 years ago at that point.
state machine. The
That's state machine,
for years and years and years Uh. The, it's the, it's the state machine of the modern paradigms that you can't do in gl. And if you modify GL enough to accommodate these modern paradigms, have a new API, which is
Balkan.
where Vulcan came from. So, so Vulcan originally came from, I think called Mantle, which was an A MD project. it was a basically a console esque way to program A GPU. It was like, we'll build the actual hardware command buffer, we'll build it in user mode. The APIs will just map basically one to one to out command buffer tokens, they, uh, will take this whole final command, buffer the whole frame or the whole whatever, and we'll kick it to the GPU all at once.
And so that defined kind of a new kernel mode, mode, split. The kernel mode just deals with. The things that need to be dealt with. Memory mapping executing buffers like you pass to it and handling exceptions, the things you have to do in kernel mode, in two ups, et cetera.
And then if the kernel mode sets up that you could fill with data and then you can pass 'em to the GPU to execute, everything else can be done in user mode, was a huge step in the right direction for rendering to get rid of things like that Direct X token stream. now the APIs, as you call them, they just spit out hardware command buffer registers, and that's what gets passed the hardware.
But Mantle did this first and it was a MD only, and ultimately people played around with it and really liked it. And then a MD kind of opened it up to the Kronos group to be like, this is a better way of doing graphics programming. And that ultimately became the initial versions of Vulcan. it was modified because they have to support so many pieces of hardware all this, but, and this is. The direct X 12 framework. So Direct X 11 was still a traditional API as you think of it. very nice.
I still like direct X 11. It's again, it's like gl, it's very easy to use, minimal amount of code to get it working. Whereas direct X 12 and mantle, which became Vulcan, which became, all it is today. Both of those two direct X 12 and Vulcan have this thing of, took all the work out. The driver, the driver pretty much does nothing these days.
All the things of like synchronization, I rendered to a texture and then used that texture, if I ran into a frame buffer as, which is a texture, then I use that texture to run it to a different frame buffer. Then who does the synchronization? And it used to be in the older APIs, we would just implicitly synchronize for you. But the driver didn't always know best, it didn't know what he was trying to, uh, do. So indirect X 12 and in Vulcan, all that went away.
It was basically there's a compute engine, engine of some sort. There's a buffer and there's some synchronization primitives. Go for it. And you do it yourself. So Vulcan and direct X 12 are very easy to break, very easy to get flicker and graphics. have to allocate memory yourself. You have to basically deal with the upload and move in from GPU to VAM, uh, from system memory to a VAM and back again. All of that, you have to manage yourself.
So that's where these thousands of lines of boilerplate code come from, because to do anything, you have to set all of this up. There is no quick way, and you get a lot of power, a lot of flexibility. It starts to look more like a game console. A lot of the tricks we did on the, the consoles, the Xbox, and the PS three, which were both ps, PCs, uh, lots of tricks we did to get performance. All that can now be done on, Volcan, like on direct takes 11.
You can't have a buffer that's visible to the CPU and the GP at the same time. You have to lock it, unlock it, use it, and then if you want to access to it again, you gotta lock it and unlock it again. Direct X 12 doesn't give a shit. if you wanna modify a buffer on the CPU while the GPU's reading it, do it. you can't synchronize it, you'll get bugs and you'll get flickering,
Sure.
But if you can figure out how to synchronize it and make it work, then it'll work as you expect. Exactly the sort of thing we used to do on the consults and that's where we are today. Very low overhead on the, CPU. Very thread friendly. So you can build multiple command buffers at the same time. Stitch 'em together as you wish. And all these command buffers become hardware format, command buffers, and they just get submitted to the GPU in the order.
You say sub uh, submit them in and if it doesn't work, it's probably 'cause you didn't synchronize it properly. And in that, in that way, GL. Has no, has no place in the modern world. It, it, it could never be modified enough to work this way, again, is why everything now is Vulcan I love Vulcan. It's a great API and I like DirectX 12 too.
So this actually brings it me to an interesting question then, right now, you know, we started with, with OpenGL and DirectX as being these state machines. They were fixed function pipelines that then got broken out. There were, you know, very much focused on the nuts and bolts of putting pixels on the screen. Nuts and bolts of transforming the vertices correctly izing.
To what extent now with these GPUs being general compute devices like are direct X 12 and Vulcan really graphics APIs at this point in time? Or are they really just a thin layer over a very generalizable fast vector compute engine? Uh,
That one. Uh, they are graphics APIs. They still have graphics concepts
sure.
in them. but as we gradually move more and more things to compute, we are gradually heading back to a world of software shaders. today we'll do things like mesh shaders, which is a vertex shader in a compute shader. shaders have very restrictive inputs and outputs and very one-to-one index. You can't reference other vertices things like that, where in a, if you did your verex process in, in a compute shader, you can do anything you want. You can compress the data on compressor
Yeah.
access, whatever you want, where you want generate data on the fly. You can read one vertex output, many read many output one, do anything you want. So people started to do this, they'd run a compute data output, a bunch of vertices, and then. Ask them back through a null vertex shader to get them back into the GPU.
Now, over time, they've had this thing called mesh shaders, which is basically a, compute shader that feeds directly into the pixel You don't need that dummy vertex shader to feed them back in again. so that's got rid of the vertex shader to some extent. And then you've got things like nan night, which is rendering in a compute shade software rendering pixels from a
Yeah.
And in some ways it's a lot faster, not always, but in a lot of ways, small triangles, it's a lot faster than the GPU of doing it itself. So I can see A GPU evolving into what exactly what you say. It's just a huge compute engine and you do it all in software, but it's more and more and more overhead to get a software renderer to world. You've gotta know how to software render but it does take a lot of hardware blocks out of the GPU, which aren't necessarily needed.
It's gonna be a compute engine and a few dedicated pieces of hardware to, to handle buffers, be. them I can see is everything going down this path? 'cause there's advantages to doing all the stages of a pipeline in a compute,
So I'm gonna poke the bear now a little bit on this one. Uh, because again, back in the day, I could just open up OpenGL. I guess I could still do it today. I could not open it up, but I mean, I could just use it out of the box. I could very quickly like assemble some particular scene and it was easy and I, I got a pretty picture at the end. Given the amount of knowledge required to correctly develop a software rendering engine, is the new graphics API at this point in time?
Uh, really unreal in unity,
maybe. Yeah, I mean, that's what I'll say today. If I was gonna write an app to visualize something, I'd use Direct X 11 or I would use OpenGL 'cause it's quick and easy
right? Right.
don't care about performance. DX 11 is actually very fast. But for gl, if it was a Linux machine or something like that, I would, uh, I would use gl. Uh, but yeah, you're absolutely right. If, if you have to write and loads of shade of code and know how to software asta and know how to manage buffers and you know you have to, you, then you have to deal with the APIs for buffer management and, moving things around and all that. It basically becomes an entire job just to get that to work.
And all I want to do is render one triangle. It
Right.
the PS two was hard to render a triangle. This is order of magnitude more difficult. So yeah, in the grand scheme of things, maybe they are the, the future.
I mean, it's sort of fascinating to see how it's like we went from a world where, you know, this was just basic acceleration for a very bespoke thing. Again, putting pixels on the screen, to this full compute engine that we have today that is, has grown beyond, I mean, it's hard to call it a GPU anymore. Um, it, because it's now use, I mean, we compute cryptocurrency stuff on it or, or whatever blockchain stuff, uh, on the GPU, it's.
math engine is
Yeah, we use it
processor, massively parallel and that's it. But that's the world we are in today. And that's kind of the graphic story. It's, it's progress and that's where we are. And be new winners and losers as we go forward. But all goes back to that 1963 Ivan Sutherland of making sketchpad
making sketch pad.
to today, 50, 67, 65 years We are in a totally different world, but graphics are still fascinating
I think it's
it's not gonna be any less fascinating tomorrow.
I think it's actually super cool that we, you know what started off really as a thing where I want to create something visual. I wanna make something cool on the screen leads to this, you know, domino effect where all of a sudden, like we do AI on these GPUs today and, and, and never. Gonna necessarily produce a particular pixel at all on these things, but we have all these other things that we're able to do.
But it all started off with I wanna make a cool game and I wanna make some fun graphics. And that was it.
Games and graphics are, have a lot of fingers in all the pipe. People tend to, FFR upon games is kind of like this use of com of computers. Although it's a multi, multi-billion dollar industry, it drives a lot of other tech in the, in the PC space. And without games in the PC world, we would not have had any of this cool tech.