From Concept to Screen: Virtual Production in Film and Television

Hi everyone, I'm Kali Bateman here for Mixing Light. And today I've got Bryn Morrow here, who's the virtual production supervisor for Steel Bridge Studios. It's really exciting to have you here to have a chat today about volume work. We met grading some car commercials together, which actually I didn't know were shot on a volume until about halfway through the session. And I turned off a resize and went, oh, look, there's the edge of the universe. And it was really impressive stuff.

Once you started to look for it, you could start to say, oh yeah, maybe that's something to do with the volume, but it was pretty seamless stuff. And we got to talking about how it all works. And I realized how little I knew about how volume production works. So very excited to have you here to shed a bit of light on that today. Thanks so much for coming. - Oh, thank you. I'm very happy to be here. So thanks for the nice introduction, Kali. - Such a pleasure.

So can you just tell me, basically from the very beginning, you need to have a image to go on these volumes. So just to try to explain what a volume is, and you'll probably do a far better job than me, but a large LED wall or walls

that serve as a background for motion pictures. So cameras in front of the volume and usually actors or props, some kind of integration of physical sets in between the volume and the camera. And then a whole bunch of data is recorded and the backgrounds are happening in real time through Unreal Engine. Have I kind of got the basics down there? - Yeah, actually, I think you did very well there.

I mean, there's obviously variations to how virtual production is done, but that is certainly one way of doing it. And in the project that you were grading for us, that was essentially how it was done.

Like the backgrounds were projected onto an LED panel, like a large sort of screen, which is a combination of a bunch of very small panels as well that we combine together and that particular one, from if memory serves me right, that was about, I think it was about 15 meters by four and a half meters high for the main wall. But yeah, so I mean, essentially a volume is a combination of panels that are sort of stacked together to make a wall.

And then whether you have a ceiling LED panel in there and a side, a couple of sides or a reflection sort of panels as well that can combine to create, as we call it a volume. And you don't necessarily have to have digital assets in there. So that particular example, yes, those were all virtually created from our virtual art department.

And so those were full 3D environments, but we've done many projects where we have live action plates that we filmed and then those are projected onto the panels. But essentially that's what's happening. We're projecting the digital environment onto the back, onto the LED panels. And we have sort of a 3D space recognition that we understand where everything should be in the volume in terms of its 3D positioning.

And the idea is for us to translate that to the real world set that we've created as well. And then that needs to exist in our digital environment. So we have specific distances that correlate to distances to the panels. And essentially you're able to then get parallax purely because the camera that we have, which is obviously a production camera and can go into all the different sorts of cameras that we would use for virtual production, but that camera is being tracked in real time.

So we have a bunch of motion capture cameras that we create our own little volume for. And those are constantly tracking the physical production camera in space. And then that is translated into our digital environment. And we have a digital replica camera in the digital environment. And those two essentially are matched together. So that's how when the practical camera moves, then the background is moved in relation to that. And it's field of view changes based on lensing.

Well, once we adjust those, and so just the parallax shifting in the background, that's where that all sort of happens. But that would happen more for digital environments. But if it's a live action shoot that we've captured plates, it doesn't exist the same way. Of course, that then is not, it's not a digital environment anymore. So the parallax is still sort of there in some senses because we do a sort of a two and a half D sort of workflow.

But in that sense, it's not quite as 3D as the 3D virtual ones. - Wow, okay. I think for me, it kind of comes together when you tell us how the camera itself and the motion capture in the camera is linked to a 3D camera. I think that's the bit that I didn't quite understand before our discussion, because it all seems quite magical to me. So the process is that you would get those environments digitized prior to the shoot, right?

So if it's a fully 3D environment, that's quite a bit of work that's gone in prior to actually cameras rolling. Can you tell me a little bit about how you take an image? And it can be a 3D one or it can be a,

something that you've sourced in the real world. How do you then get that into the volume? - Yeah, I mean, if we could start from sort of the 3D side of it, our virtual art department, obviously we've been getting a brief as to what that environment wants to be and how realistic it also needs to be.

Because of course, it's subjective, working in an artistic sort of realm, there could be multiple sort of environments that you can make, but in certain, I mean, in most cases, it is realistic as we can make it. And then of course, we very much create the environments, sort of we like to Hollywood the environment.

So in that sense, we build a full 360 environment, but we do Hollywood it out to a specific field of view, at least for a range of motion for the camera that we're expecting, so that we don't create too huge an environment because-- - What does that mean, Hollywood it out? I've never heard that. - It's the term, you know, like back in the day, in production, you know, you're building a set to, you don't have to build in full environment, you know, the whole world if you're building sets.

And so we consider it just Hollywooding it to the environment, I mean, to the camera. I don't know, it's a term that I picked up way back. So I don't know. - I love it.

- So anyway, we ultimately build the environment to the camera's, you know, like the range of motion that we're expecting it to sort of go along and we're able to obviously deduce that by doing previs or techvis and also obviously, you know, creating boards and we have a, you know, we get quite far into our previs prior to, you know, projecting it up onto the screen. So we already sort of have a pretty clear idea of how much of the environment's gonna be seen during the production.

I mean, that doesn't mean that we're not giving the director as much opportunity to change the camera as, you know, they generally would like and also the DP for that matter, obviously. So, but we certainly give them a bit of a constraint and we've already built it to sort of, you know, we're expecting, for instance, you know, we wanna see behind us, we're gonna be shooting reverses, we're gonna be shooting sort of POVs and sort of wide vistas or whatever.

And so we'll build it to those constraints. But again, there's many sort of factors with regards to optimization and things like that, which the reason why we don't build everything out is purely because of optimization and being able to run at the required frame rate that we're gonna wanna roll at. And so again - go down the rabbit hole. We could sort of get into that later if you want to, but-- - Yeah, sure, I'd love to.

- But ideally, you know, the idea would be once we've pre-vised out those environments and, you know, we can have multiple environments in a day that we wanna shoot, we'll then have a pre-light at least with the camera and we'll project those onto the wall.

Now, when I say project those onto the wall, we, within Unreal Engine, for instance, we have something called nDisplay, which is a plugin, which essentially is a projection plugin where you create planers within the scene which represent the panels, represent the position of the panels. And that nDisplay node essentially is what we call it, that we then have the environment all around and then with the camera, it projects onto a very specific position of where the panels should exist in space.

And that becomes, that's kind of where the trick really lies in understanding the real distance that things relate to the real world, from the digital to the real world, which is why we tend to LIDAR scan our sets.

So we will go ahead and do a full LIDAR scan of the set environment, whether it's just in a studio, where the panels in position, where they are in position, where our camera is expecting, we create something called a home base so we know where the camera wants to always be at the beginning of a shoot. And so we then do a LIDAR scan and from that point, we filter that back into Unreal Engine. So we actually have real world scale positioning of everything.

And once we've done that, we'll then align our nDisplay node in Unreal to where the panels are, and then where that set exists. And then we can work out distances from practical positioning. Like obviously, for instance, if we had Artabar and bump in some foreground elements or whatever, those elements need to exist in the space digitally as well. It's what'll end up happening as things will overlap on the screen.

And then something on the background is supposed to be in front, et cetera, et cetera. And so you need to be very clear about where the opening is and how you align that. And it's all about calculation, real world calculation. And then that needs to be translated into the digital space and vice versa. - Is that why it's called the volume? Because it's all about the space, the actual depth and distance between things? Is that something that gets its name or is that some other?

- For me, I think volume because it's a volume of light. So when you have multiple panels and generally it sort of encompasses you as the environment and you're creating essentially an exterior environment inside an interior environment, that becomes a volume of light, its own environment. And so that's how I... That's where I'm getting it from. I still like what you're saying though. I don't mind that. I can think of it that way.

- I've been trying to just, as I've thought about speaking with you about this, trying to imagine where the term came from. So I'm looking for it everywhere. It sounds so futuristic, but that is fascinating. So you've kind of got these projected elements as well as physical elements. And that was certainly the case in what we worked on together. There was a foreground that was physically bumped into the space.

And then on top of those two things just existing together, you've also done a LiDAR scan of those elements in the space and you fed them back into the digital simulation

of the environment. - Yes, that's great. - That must be the trickiest thing is getting that scene between the digital and the real kind of working seamlessly. - Yeah, that's definitely the trick. And we tend to build a lot of, in our digital world, and depending on what sort of practical set that we have available to us, we tend to build into our digital set little tricks.

Like, for instance, in that Sydney scene, I know it's hard to talk about it when you can't see it, but we built in a pavement, an edge, a pavement edge where we're expecting the edge of the panels to exist. And so we will build in where that edge of the panel wants, sorry, the edge of the pavement, which happened to be in the digital set. And then we decided, okay, we'll replace that pavement with the real pavement edge, so that becomes practical.

And then we align that up to our digital pavement and we delete our digital pavement, and then that becomes the new pavement position. And so when the camera moves, it should stick in a line. I mean, I think there's just, in those instances, you're then locked into a distance to the panels.

And so I think one of the main advantages, I think, in volume and in sort of quick shooting, if you wanna do that, is to be able to cheat the distance backwards and forwards and move things around digitally quite quickly, and to be able to, so in other words, if you build something into your set and say, now that is now physically connected to the set, and it's very heavy, for instance, when you start spinning the environment around, and I guess we haven't really kind of gone into this,

but for instance, if we wanted to do a reverse, there's no panels behind us, so we spin the environment around digitally, and then suddenly that's the reverse of the shot and keep the camera in the same position, so that you're kind of cheating it in that sense. But if you've now built a practical floor which wants to connect to your panels, that's incorrect, right?

So then that would need to move too, so you'd have to flip that around, or you don't always wanna be doing wide shots that need to obviously align with each other in the cut and things like that, so you just need to be quite, you don't wanna build too much infrastructure into your foreground, which doesn't have the ability to move, I guess is what I'm saying. - Yeah, unless you have it on a turntable. - That's right, yeah.

And a lot of that is the case, and people do tend to do that when it comes to much larger virtual production shoots, which is a massive advantage, and if there's that sort of budget thrown at it, then I will say yes and put my hand up and say let's do it every single time, because it just helps out, and helps the production designer out too, and of course, we also then, every time there's new props or anything like that, we try to scan those as quickly as we can, but those are just sort of

photogrammetry style scans, not full LiDAR scans, and then we'll try and, because we would, like I said, when we spin the environment around, there could be elements that were created as the bump in practical set, those then now suddenly exist in the digital world, you'll actually see them through the screens, and if you haven't created those exact props, then they won't be able to be extended out into the digital world, so that's why we also scan and recreate all those,

and match them if we can. - That is absolutely full on, like the level of precision that you would need.

I know that in the color grading world, we're typically not working with that level of precision in terms of getting everything measured and accurate, it's sort of the approach is more about getting a feeling, and instead of doing like really tight shapes, we're typically doing softer shapes, and sort of being a little bit more gentle in our approach to manipulating the image and trying to do a bit less, but this is sort of the opposite where you're doing everything,

you're generating everything. I imagine you get into the uncanny valley pretty easily doing that sort of thing.

- Yeah, I mean, and look, and having sort of broad power windows, I mean, that stuff is great, and it certainly lends itself to what it is that you guys are doing, when all you have is a 2D image to manipulate, but I think for us, it's just, it can very easily go wrong, and so we're just always trying to stay physically accurate as much as we can before the cheating, and even though it is a full cheat anyway, the only way for that to be a reality is for us to be very strict and constrained

to the physical aspects of what's going on, and so that needs to sort of loop back the whole time. And yeah, I mean, once you create a pipeline for what it is that you're doing, these sorts of things don't have to always be thought about as intricately. I mean, ultimately, I will be the one that needs to keep an eye on that sort of stuff, and if it goes awry, because everybody within our brain bar, which I haven't really sort of discussed, but they have individual jobs.

- Yeah, but it has now become a universal term for some reason, so everybody uses it as the brain bar. They call it the brain bar. - That's cool. So I suppose I'll just do a little bit of background about the actual studios themselves. So Steelbridge are quite new facility in Brisbane,

and it's been really popular, and Australia's actually been quite a large adopter of virtual production worldwide. So we've got some facilities in Melbourne as well at, jeez, I can't remember the name, but-- - Nance Studios. - That's the one. - Nance Studios in the Docklands, yeah. - That's right, yes.

And VCA have put in some virtual production facilities there in Melbourne as well, but Brisbane has Steelbridge, and then there's some volumes available on the Gold Coast in the studios there as well. But Steelbridge are probably the most active in the region in terms of doing a lot of work. They've got their pipeline pretty well down. They've done quite a lot of commercials.

I'm not so sure about Longform, but I'll have to ask about that, what you guys have been involved in, if anything, in that space. But definitely in terms of commercials, they would be the ones that you would go to in our area. And some of the advantages of it in the work that I've seen is we worked on a car commercial together, and the environments were quite varied because our landscape in Australia is quite varied.

So they were able to do sort of red desert dirt and city driving and coastal driving, all of these environments in one day. So obviously the practicality of that is great. You only have to take the cars into one space. You don't have to freight them around, and you can go to a lot of places quite quickly. So, although you're doing quite a lot more in pre-production, I think in terms of production, everything's a bit more sewn together.

But yeah, so in Brisbane, they've got this great facility and the pipeline's pretty well sorted out now and some pretty seasoned staff on board, including the friend who we're talking to right now. - Yes. Yeah, look, I mean, Steelbridge has been certainly pushing really hard, and I think it's been, I've been very fortunate to be able to work with them. I mean, initially, it was a bit of a brainchild between myself and Colin, who's the founder of AltVFX and owner.

And, you know-- - That's Colin Renshaw. - That's Colin Renshaw, yeah. And essentially, obviously the new tech was coming in. This was quite a few years ago now, but it was just sort of something that we knew it was gonna become a thing. And at the same time for Alt Studios AltVFX, sorry, they were pushing their animation, which is obviously quite integral to the work that they do, a lot of creature work that they do.

And so motion capture was certainly a thing that they used, and they used a lot of XN suits. - They were very good at motion. Very, very good at that. I remember doing some grades, and occasionally I'd see people walking past in those suits. - Yeah. - So those suits are sort of like a, almost like a wetsuit with dots all over them. - That's right, yeah. - And then as the character moves around, those points are captured, and then they can be transposed onto a 3D model.

Like you could see a dancing polar bear or something, but it's just, I love seeing that behind the scenes thing of the person doing the silly dance, and then the polar bear does it behind them. I think that's always a cool party trick. - Exactly, yeah.

And so I think that Alt just wanted to push their motion capture facility, and actually buy and look into the Vicon cameras, which are the very high end motion capture cameras for, to again, push their pipeline, their animation pipeline, but at the same time, find a permanent place for that as well. And I think that's where this sort of started growing from there, and of course, with having the motion capture facility, where you don't, it's not an X-Sense one, but it's more a camera driven one.

You can have multiple people wearing suits at the same time interacting. So, you know, there's certainly a lot of fun things going on here, there, and we do a lot of that here at the moment as well. So it's fantastic to be able to push both parts of it, and it's quite integral in the visual effects side of things. Of course, I'm a visual effects supervisor too, so it's a very important aspect of visual effects too.

So that in itself drove the volume, motion capture volume, and in that sense, you know, we're tracking a camera at the same time. And so we decided that, you know, with that technology working out and, you know, moving into the real time sort of software like Unreal, those obviously go hand in hand. And so we just decided to push onto the virtual production side of things as well.

And while both those streams continue the motion capture, and then also the virtual production, and then it's developed into where Steelbridge, you know, creates a lot of virtual production commercials. And yes, there has been some long form inquiries, and we talk about a lot of long form projects with producers. And I think, and again, look, it's always gonna be something that producers and or production people, you know, it's a learning process for everybody.

And I think it's hard sometimes for people to commit to a project, a virtual production project, especially if it's a long form project, and it's just a really big part of what they're doing. And there's so many things that can go wrong that they're not in control of anymore. And I think it's certainly hard to get people on board, but I don't think that it's far off. And we're talking to a lot

of people is all I can say. - Do you think that's why, you know, the kinds of things that have been shot in the long form space in volumes, like Mandalorian and the Batman and things like that, that have used them quite a lot, they've had really big visual effects studios behind them that are actually responsible for the production.

So in the case of the Mandalorian, it would be people who intimately understand the technology, who are kind of funding and controlling the decision making process, as opposed to more traditional productions, going out on that limb and having to have that leap of faith and trust in someone else's, like obviously amazing abilities, but just, it can be difficult when you don't understand what's involved, right? - That's very true.

And also, and this is certainly something that Colin mentions a lot, and it's very true is that if you have a post-production facility that's behind a virtual production facility, you have the ability to fall back on sort of visual effects to fix sort of things. And that sort of happens a lot in virtual production. I mean, as much as we're all trying to get 100% in camera out of the virtual production techniques, I mean, it's most probably more like, you know, it could be up to 60 to 70.

If it's a long form project, the problem is that so many changes can happen further down the pipeline. You know, it's very hard to make all the decisions at the beginning of the project, and that is essentially what is happening in virtual production.

So I think you kind of then look at it as, what percentage are you gonna actually get in camera that you're happy with, and it's not gonna have changed when you've suddenly decided you're changing your character or you're changing your monster, whatever it is, later on in the process, then of course everything that you shot gets thrown out the window. - But it's the same in regular production, isn't it?

Like it's exactly the same, you know, possibly more visual effects required for things that are shot photographically in the real world, right? Like nearly every shot in a long form show can have a hidden visual effect in it that the audience would never know about. - That's true, but the problem is, is that when you don't wanna pay twice, you know?

And so the issue in that sense is that if you're shooting it, that's fine, but then you might have a visual effects budget on top of that and that works out. But then if you're kind of, you know, already doing a virtual production budget, which in some cases can be higher than just going out and shooting on location, then you're sort of, and then you then need to do the visual effects process back on top of that, then that can become a second, like a double cost.

I think that there certainly is some sort of, you know, you have to think about that and make sure that everybody's on board with what you're gonna get and the fact that you probably have to build a contingency plan into the budget that there will be shots that need to be fixed or shots that need to change or extended, you know, and that tends to always be the case. - So can you talk to me a little bit about how you do that pre-visualization

and how you might digitally scout a location and how do you get that world built digitally before you go ahead and, you know, storyboard and, yeah, how do you take the director through these environments? Because you're not physically location scouting and looking for physical props here. - Well, I mean, you say that, but sometimes that is the case and certainly in the proof that we worked on together, all those locations were essentially sort of located in the real world locations.

And then we went and, you know, sometimes went and LIDAR scan those sets and converted them to digital sort of sets. Or we just took heaps and heaps of reference photos from that and then just digitally replicated those places because those needed to be iconic locations that people needed to recognize.

But again, in say, you know, we've done a lot of interiors and things like that, you know, in that sense, we will actually initially talk to the production designer and or the client, depending on the size of the project and, you know, come up with the locations with, you know, treatments, directors boards, all the rest of it that you generally come up with and send that to our VAD team who will start building it out and will do sort of large renders as we're moving along in the process

and showing those to the client and director initially for them to sort of kind of enjoy what they're looking at.

And then once we know we're on board with that environment we're building to the right, or at least the correct creative on that, then we will go in and we'll do sort of digital scouting, which is, you know, you can either do that with the virtual headset, like a VR headset at least, or, you know, we'll just, you know, get a director in, you know, sit over one of the artists shoulders or whoever it is, and then we'll actually lens it up for them, you know, with the proper camera and sensor sizes

and then we can sort of look at finding those right frames that they like. And then those can become part of the storyboards and then we'll go ahead and either have digital storyboards or we'll kind of draw back over top of them and add people in, or we could do that in Unreal as well. I mean, that's also sort of quite a few methods in that sense.

And then after we've sort of built these environments that I think the director's also sort of enjoying and liking and we know now what to build, we'll then do something called techfers where we create the panel layout, which we've already sort of worked out. Well, it's kind of like a mutual sort of decision there because here at Steelbridge, we tend to build a volume out based on the needs of the shoot. It's not always a general LED volume layout that you would have.

You know, we wouldn't always have a floating wall that can be maneuvered if we don't have a car

or sort of some sort of object that needs to be reflected on all the time. So there's different ways to build the volume. We won't always have a ceiling in. We probably won't always build a curve to our wall. It might be a flat wall. I mean, there's multiple sort of constraints and also reasons why we would build them in a certain way.

So then in that sense, we would then work out what size volume we need based on those that tech scout essentially that we've done, or sorry, that virtual scout that we've done. And then we'll add in the panels that we've built digitally into those environments. And you can actually then work out the distances prior to actually, you know, your your art department comes in or anything like that

when you're starting to build out the sets. And you can then work out the scale inside the studio and what your best focal length could be or what the constraints are on the the lensing that you need, you know, and you can sort of then inform the the DOP as to sort of what, you know, we then want to talk and engage the DOP obviously early on as well.

And also with the director and what sort of shots he wants, but and what the constraints are within the studio or whether we need to go to another studio and build it out over there. And then sort of all those decisions are informed by this tech that you end up doing after sort of building the environments. Yeah, that's cool.

And then the DP, like, do you find that there are certain DPs who are becoming specialised in shooting in these environments or can sort of anybody with an understanding of how to shoot in the physical world shoot in a digital environment? Or is it a bit of a specialised thing? No, I mean, I think that, you know, for us, we love getting new DPs in to certainly to train them up.

And also when I say train them up, we don't obviously have to train them up on this craft because, you know, it's all very much translatable, like everything's very much translatable. It's more just about understanding the process where he can relate it to what he's used to. And then he just essentially needs to know who he needs to talk to, to be able to fix something that he needs to fix. And at the same time, so it's still his eye. It's still what he wants to do.

There certainly are some constraints on lensing that that will be purely based on the framing and the boards and and then also, you know, the aperture like, you know, we really need to always be on that fine line of wide open if we can, purely because of the scale of our studio and also the the distance that the panels are from from our our main actors or whatever. So the idea is is that we can't allow those panels to come into focus because, of course, we have this moire issue that can happen.

And so we always need to have quite a shallow depth of field. Like we don't want to have too much too much depth of field. We need to limit that a little. And so unfortunately, that's sort of something that not all DPs want to be working with. And it does give a certain look. But we we always try to work right on that edge of moire because more focus in the background, more realism creeps back into it. You know, if it's everything feels a bit sort of under watery, it's just not looking right.

You know, so it's always it's a very that's why it becomes quite mathematical. And the lenses need to be very we have to profile the lenses and really test the lenses to make sure that we're not suddenly on the edge of moire. And in that project that you and I worked on, which was super, super crazy, the lenses had a weird pincushion scenario where the outer edges were more in focus than the inner edges. And so it was it was very strange. And, you know, these were sort of old school lenses.

And we really didn't do our homework on those lenses. And so it kind of, you know, essentially meant that we had to push everything further away from the screens even more, which meant that we then were limited by the framing that we wanted to achieve. So, yeah, so it's not something that I typically think about is like the amount of focus across the lens.

I you know, usually you're thinking about depth of field and, you know, your F stop and all of that and how much light you've got in the scene. But, yeah, I mean, I suppose when you're really on the edge there, if it's a bit sharper in one area of the lens and a bit softer in the other, you don't want that sharp part of the lens to start to see the actual LED points, right? Because that's what creates the moire pattern.

Yes, that you've got these, you know, grid essentially of points that will strobe. Exactly. Yeah. And that's yeah. I mean, that's sort of the we're always working to make sure we don't see that. Yeah. Yeah. Wow. There's so many considerations there. And on top of that, you're actually creating either a 3D or a two and a half day environment, and I take it

that there's actually quite a bit of like color pipeline that has to happen between the start and the end of that, because you're actually projecting an image that is quite final at that point. That's right. Yeah. How do you how do you design the pipe there to get the colors accurate? Well, I mean, for us, you know, the most important thing is to make sure that this and anything on the screen just looks realistic, you know, and natural.

So if you're looking at the screen with your eye, it needs to sort of feel real as well. But and I know it sounds sort of, you know, you think that's what you would expect, you know, but the problem is, is that when you're creating digital environments, you can very easily get illegal colors, as we'd like to call them, which are not really real life kind of colors. And that all is dependent on the renderer if you're doing digital content. And that can be kind of quite tricky.

So we do employ an ACES pipeline. And for us, we've got, you know, OCIO library, which we output from Unreal, an ACES color. And, you know, whether we then re-project that out onto the panels as an ACES profile, and then that gets converted with a look up on the processor before it hits the panel. That's sort of something that we sometimes shift between whether we actually do the conversion on in Unreal and then onto the screen.

But the thing is, is that obviously the panels are all slightly different in some instances, like as in different brands have got like different color with different looks, you know. And then the problem with the LEDs is that they, you know, of course, it's an RGB light, essentially, each one of those little pixels on there.

And you can get something called color shift very easily at certain angles that you look at a panel and, you know, you get these kind of weird magenta reds or you get the greens if you're looking at opposite angles. And so that that's kind of got multiple issues with that. Is that how it translates onto skin tones as well? That can be a real problem.

And if you start sort of shifting color, overall color, for instance, on a panel and what that actually does to skin tone is it's quite remarkable, actually, in some instances, you just sort of hope that it just works. Everybody's skin tones are slightly different and the LED panels can really just just warm it up or just make things look red. Some skins just goes red under panels. And it's very, very tricky. And some skin tones just do not. So I'm just look fine, you know, and so you have to.

It's quite a tricky sort of science to get that right. So our main aim is to make sure everything's sort of linear as it comes out of unreal and into the processor. So we want to try and make sure that there's not too much done. Like we don't want to do much color management prior to it going to the Brompton processor, which is the processors that we use to drive a signal to the panels.

And on the Brompton processor, there is quite a bit of color management that we we tend to adjust and and use to help merge, you know, merge the foreground like the practical and the background together. And then, of course, you really only want the camera to be driving the color after that fact, you know, with its color temperature and then the lights that then change accordingly. So, you know, just like all lights, panels can be warmed up and cool down. And most LEDs run quite cool.

They're very sort of like as a default, they're quite a cool sort of temperature. So you can warm them up to to try and balance it out. But in doing so, you know, it's very easy to break things. And so that's why, like I said, we just try and make sure that everything coming out of unreal is very linear. And so like, again, we transcode everything that as in textures and or footage that we're using into the ACES color workflow.

And then we will then convert it out onto the panels through the Brompton. So it's kind of and then we could possibly look at it in any other look up after that. But we just want to have as much color gamut as we can have, you know, and make sure that those panels are working correctly. And so, oh, sorry, you go.

Oh, I was just going to ask - So about that linear signal, I mean, my understanding of the usefulness of linear for visual effects is that you can do more realistic things because that's the way that light kind of functions in the in the real world is that it does work like linear. Not like logarithmic curves. You don't you know, you don't end up with those soft roll offs that we then put on once we're starting to work in the log space afterwards.

Is that why the linear pipe works so well for this or is it just because it's such a robust pipeline for visual effects working in ACES and OCIO? Well, I mean, I think just obviously keeping it linear just allows us not to add any sort of look ups on top so that you can kind of keep the color information there for as long as possible and then try and only, you know, convert it later down the pipe.

And again, like you said, if we have some sort of roll off that's happening prior to it hitting the panels, then it's very hard to invert that, you know, the other way around. So keeping the signal as linear as possible all the way through in terms of what you were saying in digitally, it makes total sense that we like to linearize everything so that when our final look up happens in the renderer, we actually then convert it so we can look

at it. Essentially, we'll look at it in, you know, Rec 7- Rec 2020 or whatever it is that we tend to look at it in. But we will still be working linearly the whole way through. And it's also because of the renderers that we use.

So our renderer is the way that they actually render, you know, whether you're using like you know, it depends on if it's an unbiased renderer, which means essentially it just shoots out light rays just like you would have in reality, you know, with photons, you know, essentially, your materials are like balanced and real world balanced as well.

So they just work far better in terms of a linear workflow, because essentially computers work linearly initially, and you don't want to bake into the the any of the color or anything like that within your digital, the digital component of your renderer. If you start introducing some sort of look up, it just breaks and then you have to invert it somehow down the line. So we stay linear as much as possible all the way up until the end.

And then, of course, as you know, like, you know, the camera itself will be adding to that as well. And so, you know, you just have to mitigate too many things that can change all the way up until that point. Yes, yeah, because the camera itself is going to have, you know, its own ISO natively and it's going to have its own range of stops that it can render onto the actual image and capture. Wow, so much can go wrong,

so much needs looking at. So do you essentially color the material in that process? I know that you're saying that you are trying not to do very much to it. But like when I looked at the raw ungraded footage for the car commercial that we looked at together, there was nothing like there weren't any parts of the image that were wildly different from others. It did feel as though somebody with an eye had integrated and composited a world together that was kind of harmonious and worked color wise.

Choices had been made, I felt, on the way to the volume. Is there somebody like a compositor or a colorist or a visual effects artist who is kind of responsible for that or is it something that just happens? No, I mean, so I mean, to answer your question, that's probably me that's doing that.

But I think, you know, again, like if we've got a realistic environment in the background, the aim is to make sure that that looks like a realistic environment with your eye looking at it and so that it needs to actually, you know, it needs to react correctly to light in digitally, digital light, at least. And then that light needs to sort of feel realistic, you know, as you're looking at

it. So that's the first part of it. And then secondly, when you start integrating your foreground elements into that, you know, the there's only so much light wrapped, for instance, and this depends on the size of your volume that is encompassing your person or objects that you've got that are being lit as well. And so essentially, a lot of that can be matched, you know, from the lighting

that's coming from the volume. But again, like, for instance, if you consider that project, we have the floor, you know, that the, you know, if we look at the the outback scene where, you know, it was like a reddish, the reddish color never sort of we never quite got that one working quite right. So we can essentially, we're trying to match to the foreground environment, texture and color to our digital environment needs to match to that. So there is a level of that happening in real time.

And in our brain bar of people that we have, they'll be making those adjustments, looking, looking at the screen, looking through the split to make sure that that sort of feels more in line, especially if it's like a wide global, a global adjustment to a particular material, for instance, not a global color adjustment that you do to a 2D image, but to a specific piece, you know, like the sand will change the color of the sand to match to the sand that we have in the foreground.

Those are things that we can sort of do on the day, and we'll try to get those as close as possible. And that generally would happen during the pre-light day, which is you'd hope to be in

that position. And then afterwards, there's sort of subtle adjustments in color, overall color of the panels that want to then match with the, I would be working directly with the DOP to understand when he exposes in a certain way what we want to do to our highlights in our scene, for instance, to make those work and sort of make them feel

correct as well. And then we have something else, which is called, well, ICV effects, which is in-camera visual effects, but essentially it's a new projection, kind of like a, you know, power windows that you can add to the background panel, and you can actually isolate areas and draw mats around certain areas and grade those independently to try and fix that scene where you might see the integration of the foreground elements hitting the panel at the background, and so that those can be

graded independently. But we try to stay away from that for as long as possible until there's just like a scene that we need to fix because those, you know, because they actually exist there all the time, like they're stuck to that piece of the panel. And so when you start moving your camera around, you can imagine that that doesn't necessarily relate in volume, per se. And so like, as in like volume tracking, so it might then stay in a position and just not be in the right position after that.

So we try to avoid that if we can, but sometimes you have to actually use that. And also, you know, this ICV effects option also allows us to add like, you know, hotspot lights that we want to use for reflections and added reflections and animate those that you have, kind of like what you would do with sort of like a certain LED lighting that the gaffers use to be able to actually have some running lights over cars and things like that, which is quite cool. That's really cool. Yeah,

that's fun. That's really cool. I mean, it does seem like there's an opportunity there for a colourist on the set to be part of that process. And I know that visual effects artists and compositors are extremely good colourists, because that's sort of essentially what a compositor does is matching elements and making them integrate seamlessly. Can you tell me about the brain bar and who's on that and what's happening there? Yeah, I mean, before I do that, I

mean, you're 100% right. And you know, having like a colourist eye is probably, you know, would be amazing to have. And I think that, you know, you know, the DITs, after it's coming out of the camera, I think is a better place for that currently the way I see it. And that's purely because, you know, we want the foreground to

stay married to it. I think what ends up and can end up happening is that when, if the background panel is being colour adjusted to match to what initially is being seen, the problem with that is that the balancing of reality can kind of really shift, you know, in that sense. So when our background becomes sort of separated, I mean, yeah, there could be very

easy adjustments that we can make. But when you start making adjustments before it hits the panels, you're then in this world of changing the environment to match prior to doing it after the fact when you have your integrated environment in there. So it's just been hard to then work out why certain things aren't working. You know, if we have a grade that's happening prior to it, then going, we've tried this and then it going into the Brompton and then getting

projected up on. And so when something is not working, it can't all be fixed in grade before it goes through, because some things, like I said, for instance, we need to adjust the ground colour, for instance, specifically on a material, like a 3D material for that to work, or we need to adjust the position, the sun position and what that then ends up changing in the environment by just changing the lights on everything and how then the grade fights against that opposite, you know,

because we need to be quite fluid, essentially. But I think just maybe before that would be okay, but like, there's so many shots that we're doing that sort of have to happen. I mean, I think there's certainly, you know, heel and toe and levels that need to, you know, mids that need to be moved around every sort of shot. And that would be amazing, you know, almost like a one light feel that would be great, you know, for that to sort of happen. And that is sort of something that is

happening every time just before we roll. But a DIT approach after the fact where we have the foreground in would just be amazing, you know, and if that was sort of set with the director and the the DOP to have a look at what's more the DOP, I suppose, at that point, I think that would be, you know, amazing, you know, because what that does is gives the client even more you know, happiness when they see that. Yeah, it's perfect. It's great. This is exactly what we want.

And then we sort of designing a look and automatically on the day, and then suddenly, you know, just like you'd normally have in a long form project, it just becomes perfect, you know, in that I say perfect, but you know what I mean? And then the clients just even more into it, you know, and then they feel like and they, you know, you just they just feel like a way to lift it off them, because sometimes it might not feel exactly like or they're not used to that process.

And it just, you know, sets them at ease a little bit more. That's really interesting. So in terms of the skills that you think that that colorist who who could be part of the virtual production pipeline, they would be more of a DIT to you than somebody who is skilled in Unreal. Yeah, I mean, I think so again, skilled in Unreal, if a colorist is skilled in Unreal,

I think that that's an amazing thing to have. I think that that would be and integrating those that person would be would be sort of a priority. I'd love to have somebody like that in the team to be able to do that, because I think and I know there's a lot of tools that are being developed now where there is more control in grade that happens in Unreal and as a, you know, as a wrapper that goes to the next stage. And I think that that

that's sort of quite important. I think one thing to keep in mind, especially in Unreal Engine, if we're talking about digital sort of creation here, the renderer is not a real, it's not a real world balanced renderer like you would get in say some new visual effects style renderers or not, sorry, not new old renderers that have been around

for a long time. So there's a lot of cheats that happen in Unreal Engine to be able to get that renderer to, you know, work correctly for optimization for shadows for bounces and all these sorts of things. And the problem that ends up happening is is that you need to actually understand what the limitations are of the renderer in order to look at an image as well and go, this is what's

missing. Because in reality, when you're looking at something, you just expect things to look real and when you see it on the wall, you're expecting things to look real. But the reality is is that there's a lot of cheats that are happening. And you need to be aware of those. And if you make color adjustments to fix things to make it feel visually appealing, the problem is is that these cheats that are actually happening are just exacerbated, they keep getting worse, you know.

And so I think you're not starting, you have to keep in mind that you're not starting from, you know, something that looks real. You need to be looking at something and understand the fact that what you're cheating already to get it to look real is happening, you know. And I think that that's a broader thing to think about.

You know, it's like, oh, we've got one light that's on and it's casting some beautiful light on something, but we've turned off shadows for some objects here and there because it's costing too much money. Sorry, too much in processor and it can't render correctly on the wall because it's starting to get stepy. And so then we go, well, there's a shadow missing over

there. We've done that on purpose so that we can actually render it out in real time, you know, and that needs to be hidden and translated a certain way. And we're doing a lot of these sorts of things all the time. And when you integrate sort of somebody who, well, people need to be aware of all of those sorts of aspects of what's actually

happening. And then on top of that, add two rather than just change in a 2D because of course, as a grader, you most of the time, everything that you're grading is realistic already is a real environment is real, you know, and so you just need

to keep that in mind. I think. Oh, look, that's so true. I mean, in terms of my experience grading, I mentioned at the start that I didn't realise that it was a 3D background. I didn't realise that it was virtual production until halfway through the grade because you're 100% right. You expect to see reality and you're very forgiving of what you're seeing because why wouldn't it be real? But then once you start

looking, you start to see things. And I'm not saying that there was anything glaring or major, but I just started, you know, OK, so this is constructed. I would like to see some evidence of that. So I start looking around and then I find little things. And then, you know, you start to get in there surgically and shape and reduce

anything that might not look real. And the goal is always realism as opposed to some kind of stylised treatment or just like you say, just changing sometimes as a colourist, you go, well, we don't want it to look like the offline did, so we better do something. But, you know, that might not be the right approach to this. Yeah, because I could break it. I could just break it like immediately. And you're like, oh, actually, now I can see why it's digital now.

Yeah. So like a light touch and not doing something unless you have to would be probably the best way forward as a colourist in that scenario is not to do anything unless you have to. But again, because like I do, you guys add this massive add to so I'm not saying sure, but, you know, we we couldn't we can always do the grade afterwards. I'm saying what what do you do during the production? You know, how do you add to that process? Because you can

always grade it afterwards. And I think there's like, you know, every time I've ever graded something that has been virtually produced, whether it's a virtual production or a 3D animation or something, you know, it always adds to it because it gives it something cinematic and it gives it something filmic that it just needs to give it coherence and to make it feel correct to to be part of our screen language. But it's slightly different to the job of creating

the thing to begin with. It's 2D. It's about it's just a different approach, right? Vignettes and exposure and, you know, tones in different parts of the curve and, you know, generating that graceful curve. You're just thinking about a whole different bunch of considerations as a colourist.

Exactly. Yeah. So what happens in the brain bar? Well, we've got we've got so we've got a operations controller, I guess, is what we call him head of operations, Johnny. And he sort of so I guess to start from the beginning, I think Unreal has a multi-user approach where you have one scene that is the environment, for instance, and then you have a server that's sort of hosting that environment essentially. And then you have these multiple machines that are all connected to that

environment or that scene. And those multi-users then can be identified as separate machines that exist working on the same scene at the same time. So it's kind of slightly different to how we would do visual effects projects in that sense, because things, you know, when something changes on a scene that changes and then, you know, you have to publish that change down the line. So in this instance, you kind of have a bunch of collaborators were working on one scene at the same

time. And that's part of Unreal Engine's architecture. And it's something that's sort of a game driven architecture, but it works really, really well in this instance. So we have someone who's head of operations, and he would be he or she would be integral in hosting this scene. And as the server, it would also be where all of our other sort of peripherals attach to. So like, for

instance, the virtual, sorry, the motion capture Vicon system that then is plugged into its own suite, and that has another computer running that. And then that is then streamed into Unreal. So that gives us the real world position of the camera. So then again, in the scene, the operations managed would then have this virtual camera that

we sort of liken to the real world camera. And that has these the crown that I don't know if you've seen it, but obviously a little, you know, crown that exists on the top of the camera, and has little dots on top. And when that moves, the Vicon system is sort of plugged into Unreal and streamed into Unreal, and it tells it where that position is in space. And then Unreal, in that Unreal scene, we can move that camera around in the scene to reposition where we want

it to be existing. So then those are sort of two roles there at that point. So you've got the the Vicon operator, and who's also like looking at the cameras because they overheat, over time, you know, we've got things in the scene that are occluding that are in front of those cameras that can cause the tracking to mismatch, there's quite a few things to keep in mind over there. And so the tracking of the cameras

is super, super important. And so that Vicon system, we're constantly, you know, it's harsh, because, you know, certainly for the gaffer and the grips, you know, where they want to put up flags, and they want to put up, you know, like big bounce slides, or they want to put a bunch of stuff up to also help with the scene. When those get in the way of the Vicon cameras, as well, then that so we have to work together to make sure that

that doesn't become an issue. So anyway, so we have the Vicon controller as well, and they operate independently. And then that plugs into the main

scene with the operations manager running that. And then we have two other, generally, it depends on the size of the project, but then we have two other positions there where we have the head of the virtual art department, so the VAD supervisor. And then anytime there's an update to the scene, which were at least two objects and color and things like that, that we just talked about by updating the sand or updating the plants and

repositioning plants here and there. And we want to maybe say, let's move an object in the scene quite quickly. That will be taken care of by the VAD supervisor, who then repositions all those. And then we'll have an extra person that would be essentially working independently in another environment, and then that might be publishing new objects into the scene, like brand new assets.

And so that would then be published into the VAD supervisor's scene, and then he would make a decision of when that gets published before we change the setup. But of course, you can imagine that when you've started shooting something and then you add something new to it, and it can affect everything. In fact, the optimization, the speed, the lighting, absolutely everything. So we have to be very careful about that sort of stuff. But generally, that's sort of the brain bar and those

guys. And then I would be in direct contact with the operations manager in general and also probably

the VAD supervisor. And then especially if we do lens changes or we make any sort of changes like that, or if the motion capture volume is not capturing correctly, all these sorts of things, then I would then translate all that information back to the first AD and/or the DOP and director so that we can be sure that before we do a take, we've got that take. That take is sort of

working. And so there's a few steps that we need to make sure that we're locked and ready to go before we can actually roll on something as well. Oh, cool. Wow. That's a pretty big team without even thinking about the productions team. Yeah. Well, I mean, digitally, yes, you're right. It is a biggish team. I mean, also we record every single camera as well. So every single camera that we're sort of filming, we do an internal recording of the 3D positioning of that in case we need to

re-render things later on down the line. And that needs to be time coded. We need to have things genlocked so that the frame rate is running optimally with our motion capture cameras, with our actual, sorry, production camera, and also that the screens are all locked together at the same time. So we have quite a few things that can go wrong and people that are in place to keep an eye on those particulars. Oh, wow. I've

got a couple more questions for you. I know I've kind of taken up already an hour of your time, which I'm very grateful for. 11.08. Okay.

I would love to know a little bit about your background. Okay. What brought you to the position that you're in to be somebody who can understand how this emerging technology can be implemented in this way? What's your history? My history? Okay. Well, I was a painter, a traditional

animation artist. Then I went to digital sculpting or traditional sculpting first, then digital sculpting, became a character artist way back, studied at the Vancouver Film School quite a long time ago now, and then was in sort of pre-rendered game cinematics back then, doing characters and sort of lighting and texture work, and then moved into the film side of things, and then became more of a lighter, so a 3D lighter and lighting scenes for visual effects integration into live action

plates, and then continued doing that for a little while and became a CG supervisor. So I ran the floor of the 3D artists and implemented new

techniques. So back in the day, I was into fur and feathers and hair and all these sorts of things, and there weren't pipelines in place in those days to be able to roll those out that were like plugins to make those work, so we had to use multiple bits of software to make them work, and then I transitioned from that role as a CG supervisor, and then I just wanted to, I was just

more into practical sort of filming as well. So I went to AFTRS and studied cinematography, and I was already sort of a visual effects supervisor at

that stage as well. Then I went to AFTRS and studied cinematography in Sydney, and I was working on film, still work with film back then, which was great, and then transitioned to digital cameras, like the red one, and you know sort of early days with sort of digital cameras back then, and then sort of pushed cinematography for a little bit, went to a few different studios, it's like a combination cinematographer and heading up their 3D departments as well, and got into

sort of virtual cinematography, said doing sort of full 3D environments, but you know working with the camera because I was quite skilled at that point in practical cinematography, and then sort of started directing a little bit more, and then sort of started my own production sort of company as a visual effects production supervisor, where I sort of worked mainly on long-form projects, doing second unit directing, and just pretty much being on set all the time working in visual effects

in long-form, and pushing motion capture as well, and then sort of got back into working in Steelbridge when we sort of pushed this tech, and I had a LiDAR scanning company as well, because I've always been into the LiDAR, photogrammetry, drone photography as well, and all those sorts of things, so I'm creating digital assets, just all sort of work together, and it all just sort of put me in the right space to be able to sort of do this kind of job, which is pretty much all the things you need

to know sometimes. Yeah, so you've done a very very good mix of kind of physical work and digital equivalents along the way with the physical painting and sculpting to the digital sculpting, and physical lighting to digital lighting, physical camera to digital camera, so also working in sort of emergent technologies as you go. It does seem like a pretty pretty special combination that can allow you to understand so many things that need to come together.

I mean when you were doing the onset work in visual effects, when you're talking about second unit VFX shooting, with that you know you've got a shark movie and they've shot the principal photography, and then they've got a really important scene with a shark that needs to come up underwater or something, and you're there with the sort of gray shark head and capturing everything you need to create the 3D shark. Is that

the sort of thing we're talking about? Yeah, exactly, yeah, and I still do that, and that's sort of kind of a big part of what I still do, so but you know I'd have wranglers that I would get to go out and you know do that sort of stuff.

I wouldn't be the one holding the shark head, but I would certainly be the one that decides on how the methodology to be able to accomplish the shot, and that's why doing sort of long form, you know I might sort of you know break down scripts and we'll look at what needs to be visual effects, traditional visual effects, you know and integration in that sense, and then also what may require virtual production, and so I'm sort of in a good position to also sort of break

down scripts and look at them because there's you know it's yeah there's no as virtual production is kind of there's only certain circumstances where it really is required, you know in certain cases where it's just it just works, you know, but in other senses you just need locations, you need movement, you need much larger expenses to be able to do that, and sometimes you know shooting on green screen is the best approach for something, and sometimes shooting in a location

is the best approach for something, and so you know there's quite a few reasons why you would and wouldn't want to do that, and so that's sort of what I'm still doing, so I'd be you know I work with big studios as well, and we'll talk about whether VPP is an option, but you know inevitably I think it's not as mainstream, it is sort of in some studios overseas for some instances, but you know there's certainly type cases for for why you

would want to use virtual production, and I think for commercials it actually lends itself really well because you can just you know bump out heaps and heaps of environments within a day, and there's just you know the cost works out far better, you know, and so you know the quantity, the scale economy works just sort of well in that sense too, so in that respect it works, but I think you know long form again like I said I'm a production visual effects supervisor, so I look at those sorts

of options as well and make those decisions.

And so you would, if you're on set you would be looking at okay you've shot these lenses, we need to profile them, and you've shot under these lights, so we need to get reflection passes and have the balls go out and all of that, and yeah do the LiDAR scanning, and I can really see how that works right in with doing virtual production, but also like it's really great that you know that it's not right for everything as well, I think that would be, I don't know if

you were the studio trying to sign off on something you don't want the hard sell, you just want to know what's going to work the best for the scene that

you're trying to shoot. Exactly yeah and no one really wants to you know put their hand up to decide to do virtual production and then it just breaks and doesn't work you know either, so I think just you know if you're trying to sell a technology you know you better be in it in the middle of it to really know whether it can be done as well and what the obvious problems are but also the things that the gotchas that can happen and they can happen you know very quickly and technology is you know

is reliable but it's not always reliable, that's all I can say, so you have to have contingency plans and everything yeah exactly. And so thank you for that, the final thing that I wanted to

ask you about was as a colourist what do you see as being the most useful things a colourist can bring to a virtual production, whether that's if a colourist wanted to get involved during the production side or colourists who are working on something that's already been shot and edited. Oh I mean that's a, I know that's a tricky

question. Yeah I mean I think for me I think like I was I kind of alluded to earlier on I think just what LED panels do to skin tone is just is really really tricky and is certainly something that needs to be you know given its you know just the time in fixing and looking at and if you as a colourist are and this is now posts now at the end of and you know you're sort of grading a final spot or whatever those are the things you need to sort of kind of

look at and understand I think so understanding maybe the limitations or the things that can cause issues within in a virtual production environment are things that would be so if you have this sort of knowledge and the things to look at in terms of trying to fix those are really those would be super advantageous for for for you as a colourist so that would be you know in my opinion is you know that there's colour shift that happens too so if you look around certain areas and you'll

notice that some things are looking greener and redder in some areas again those are things that aren't real they shouldn't be there and those are things you want to sort of grade out so I think just understanding what can happen in a virtual virtual sort of environment and then looking at the the pictures and trying to be critical you know not overly critical but I mean critical in that you know those are things that if you just automatically fix those and hit those sort of bits

you're in a really good place to then do your your creative grading on top of that you know so it's almost like the one line which is almost like to fix to fix the led issues that can happen on the day you know and and for me that would be great you know like for a colourist to actually understand what could be problematic you know and then I think you know moving ahead and and sort of being sort of an earlier part of the pipeline I think and I've worked with a few colourists that

have been really interested and just want to be there you know and go can we can I just be there while you guys are doing some testing and sort of maybe we can get some panels you know like some you know balls out and see if we can like plug into that and just see what we can do with the image and and try and understand it I think all of those are invaluable for us as well and and would be the kind of experience that would really help you even afterwards so like post you know in

the post process just being there on those shoots and understanding what's happening is actually a really good place to be and then help you down the line fixing sort of stuff yeah yeah of course I mean when it comes to the skin tones we're looking at the light that's reflected from the panels that then hits characters here in the space right so instead of being hit with like the full spectrum of colours in the light there would be certain colours that would be enhanced and

others other parts of the tonal range that might not even be there right so you might get like a a flatter skin tone instead of like all of the variation that you would get under natural light is that the kind of issues yeah that and also there's certainly a bit of colour shift that can happen and there's sort of you know you mentioned red skin tones yeah for some people and some will just go greenish or not have any change and then they've got two people standing side by side like

for instance in the one job that we were on like you had those two people on the beach and the guy he kept going too pink and then she had a different tone like more melanin in her skin and hers sort of almost went green and there was like a shift that you needed almost do between the two of them and it's when they walk through underneath the top panel you if you just move your hand and go along there and along there the colour is slightly different over top of the person all the way

through which is problematic very problematic you know and we're always chasing that sort of stuff as well so that's certainly something to so keep an eye on um i would say and and also as a as a as a colourist realize that you know what's happening is not just an a natural artifact it is a mistake well it is something that shouldn't be happening if you know what i mean and not just accept things that are happening and just try and fix them because ultimately you know

when i'm in the grade or if uh you know Colin or the director is in the grade if it's Colin it's different but if it's maybe a director that's not used virtual production before they might miss those sorts of things you know and so then it needs to have say the virtual production supervisor or somebody that already has an idea of what they should be expecting in those instances and try and correct for those you know i think right so it's just being being aware that things

aren't natural to begin with i suppose you know i'm thinking back to a um a story that a friend of mine told about checking renders for quite a large 3d film that was very popular and saying that the hardest thing about checking those renders was knowing that if a character had been removed for a scene their matte might still be in there so they might be like a little ghost of a character like lingering and and anything could happen anywhere um in the shot so i suppose you've

got to be in that mindset a little bit when you're grading virtual production if somebody moves through light normally as a as a colorist i would say well they're moving from you know daylight into tungsten we want to see some we want to see those because obviously that's been done on purpose if it's in front of the camera it's meant to be there but you're saying no we need to be a bit more critical here we need to say just because it's there doesn't mean that it was meant

to be we need to really interrogate if somebody's skin tone changes as they walk from one side of the screen to the other that's right is that because they're walking past a window or is it because the panel was outputting a slightly different shade of white there yes exactly and that happens on the floor too you know if the floor is in the practical floors in shot and then the edges might have slightly varying color adjustments on them

and it's not something we can fix on the day um but again look you know panel technology is getting a lot better they're introducing white pixels into the leds now which really really helps a lot especially for overall exposure but also the the the angle of the so the the field of view essentially of these lights is broader and that actually helps with a lot of the the color shift as well and the tighter pixel pitch actually helps with color shift too but then you run into sort of

issues with brightness levels so i mean the the panel technology is getting a lot better and so and so that does help with that but i would always be on the on the side of trying to keep an an eye on that and also you know the panels themselves i mean there's a lot of light pollution that can that can exist and hit the panels you know and so you have a you know you have this coating on the the the lights on the leds and the better the coating and they've really

improved their technology in the coating the better the coating the less light pollution from practical sources or even from other panels that you've got in the scene are not lighting the the the panels themselves that is then losing contrast and when you start losing the contrast in your panels the blacks are getting lifted up even more and obviously the idea is to retain the blacks as black as possible and then when you get that light pollution on there we can get shadows

you know like there's we've added a practical light in in the shot and it actually then starts showing off the edges of the panelists you know because of the the reflected quality of the panels themselves the physical kind of panel so you know those are the things that you can actually end up seeing in the screen but not really realize it so you just also need to be aware of those sorts of things and look at them and go oh actually the blacks are just looking a little

too milky here for some reason they weren't like that in this particular shot just before um and it could have been that we added a practical light in that instance and that

is also something to keep a real keep an eye out for yeah i suppose um the colorist then can ask as well because you do have so many elements that can be separated potentially you've got some mattes for the foreground and the background and if the backgrounds are starting to look a little milkier than the black point in the foreground you know it'd be much more simple to request a matte and pop that in to adjust the background rather than try to spend your whole session

you know doing some complicated animated shape that is a nightmare for everyone to sit through yeah well do you often have a lot of mattes well actually and that's funny you say there's this new tech and unfortunately it's only the red camera that can actually do subframe and actually maybe you know in maybe there's more cameras that can do it but the subframe technology is something that we're looking into quite heavily at the moment which means that your camera can pick up

a subframe and the Brompton processor can add inject a subframe into the panel so you wouldn't you can't perceive it with your eyes but essentially you're adding an extra frame and they have green screen in the background and your camera is picking up subframes so so you say this for instance is at 25 frames per second that you're rolling at it's essentially picking out 50 frames a second but still interpreting it as a one over 50th sort of shutter angle for a 25 frames per second sort of

film so it's not ultra sharp no it still retains its motion blur across the the frames but then you can have separate outputs coming into your split where you can see straight green screen on the panels and that action is happening and then this on the other output that's coming out is your beauty which is what the panels are looking like at the same time which is amazing you know and then you can then use that to create your mattes from and inject that into your beauty after so you

always then have a mat running at the same time yeah it's amazing i can't see any reason not to do that that just sounds so useful i know it does it sounds good the problem is is signal and so you know we wouldn't have to do it from unreal so we're spitting out 25 frames per second which is fine or we'd probably go to 50 and then we just roll at 25 but then when you inject an extra frame you're actually adding so we've got say a 4k image that's being projected onto the screen you're

then running it at 50 hertz or 50 frames a second on there so you need to have that much more signal going through the panels and so you have these bronton processes can only output a signal of 4k across these panels and then it's all about math to work out you know what the signal ratio is through and you want backup and you want you know like so it's the power ratio as well so we have the ability to have 50 frames per second on 4k which is good but if you were trying

to shoot at a different frame rate and then it goes up over a certain frame rate then the panels aren't able to run that way so you then need another processor which is fine it just implicates the cost goes up every time we have more of these processes which are very expensive pieces of kit so nothing comes for free in terms of options no nothing comes for free exactly yeah yeah so yeah that's super interesting well as as useful as it would be perhaps it can still be somebody's job

to clip around those elements in roto yeah i mean look i think it's actually super super helpful and i'm sure there's going to be projects where we just go we just need it and and that's we'll just you know pay for that extra processor and just you know make sure we've got the power for it and i think we'll be good you know don't take down brisbane's grid yeah well we've got at steel bridge here we've just had like another we've got like three 32 phase outs or ins or whatever so we've

really like you know bumped it up a little bit more so for a small studio so also a good place yes also a good place to uh to go and charge a tesla if you need to well yeah let's not tell too many people none of us are driving tesla's out here we should do that it's a great idea we'll take that on be a very quick charge yeah look

thank you so much bryn for your time i really appreciate it i think it's a very complicated bit of subject matter and i'm sure we've only just scratched the surface on understanding how it all works but definitely know a lot more now having spoken to you about it than i did beforehand because it's a totally foreign different universe to me so i can't thank you enough for explaining some of these concepts and i really look forward to seeing how people can integrate the the world of color

grading a little bit more into virtual production as as the technology begins to become more mainstream and you know more widely adopted yeah well it was a pleasure chatting to you Kali so you know it was good and i'm glad i mean you know like i said you know the more people that are on board especially you know colorist as well i think it'll just make the the final results so much better and i think that's what we should all be striving for so happy with that for

sure absolutely well thank you so much for Mixing Light this is Kali Bateman see you next time yeah