Virtual Production Takes Hold of Hollywood with Epic Games' Unreal Engine

creator of the global phenomenon Fortnite. Joining me today from Epic Games. From business development for film and television is Miles Perkins and the head of its l A lab, Connie Kennedy. It's all coming up today on this episode of Strictly Business. Welcome back to Strictly Business, where my guests are Miles Perkins and Connie Kennedy of Epic Games.

We're gonna talk in a bit about Epics Unreal Engine, the software that is the secret sauce powering a lot of the virtual production that has been put to use for TV shows like HBO's Westworld, His Dark Materials and the film John Wick three. But before we get to Unreal I want my guests to help me break down for the listeners and as jargon free a manner as possible, just what virtual production is, because as I've studied it, it's it's not any one thing, and it has many

implications for transforming the traditional production process. So Miles get us started. Can you sort of lay out a uh an idea of what virtual production is? Sore? Yeah, I think that the techniques that have been behind virtual production have been around for a long time. Whether we're talking about pre viz um, animal attics, UM, we have some pitch biz kind of things, all of these things where how do you visualize how do you very quickly get to a visualization of an idea, but you kind of

start to fast forward. And what what essentially happened is between hardware and software, things started lining up in such a way where you no longer had to wait for things to render. So we've gotten to a point where we can have and and this is kind of what I say virtual production is. It's a bridge between what is physical and what is virtual. The two things can live in the same space. You can start in a virtual and transitioned into the physical in the way that

you would use your normal filmic vocabulary. UM and you don't have to switch all of these modes all the time. So that to me, that's virtual pretendent and it encompasses pre vis. There's of course pitch biz there, stunt viz, um, there's the in camera visual effects that are there. Um, there are people who are doing post viz or final pixel.

All of this is a part of virtual production. And then, as you pointed out, the amazing ability to do these things remotely yet feel like you're in the same space as if you're actually on a physical set, but you're actually operating in a virtual set. That that's what virtual production is. So let's ground this in in some actual examples. Let's talk about a production that happened I don't know, a month ago, six months ago, Uh, that employed some

part of virtual production tools. Yeah. Absolutely. So one of the things that happens is you start to shift where instead of everybody in in the past, the crew would come, they get to the set and they would shoot, for example, green screen, and the DP has no idea what the lighting is going to be, like, nothing has really been decided about this shot, so they light it flat and they move forward. Work well, in a virtual production scenario, what they're gonna do is they're going to create the

content that they would have created in post. They're going to create that in pree. They're going to create that right up top. Just as they're setting uh there, you know, designing the stage, they're going to be also designing the virtual environment so that when they get onto the stage, the director of photographer fee and the director are actually interfacing with the actor as if they're actually they're in

a physical space. Yet if they're on an LED wall or even if they're doing a live composite, they're actually

seeing what this is going to ultimately be. UM. So you know, in in the case of I use a great example of some of the stunt biz people UM and their ability to before they even get to the stage, they know exactly the measurements, they know exactly the lights that they're going to need, they know what they're going to do before they her got there, even to the point of they're able to lend things so that when they get onto the set they're just executing at that point,

or they're really starting to a little bit more explore the performance rather than having to work about on all the technical issues. So this is happening today, and and and also happening in remote scenarios, you know, where you have people who are in their own homes, yet they're acting as if they're actually on the set yet on their computer, so when they do get to the set, they feel like they've already been there before. So this sounds like a quantum leap in the ability of production

to do all sorts of things that nothing. They couldn't do it before, but it's just so much more efficient. It sounds like it is a quantum leap. It is. This is I think, you know, to be honest with you, I early in my career, I worked at I l M. And I walked into the C Theater at the time when we were doing the Jurassic Park test, the first test that showed that we could use computer graphics. Um uh, Spaz Williams, Steve Williams and Mark to Pay and West Takahashi,

all these guys. Um they had kind of gone rogue and they put this up. And when I walked in and I saw it, you know, your knees kind of buckled because you were like, Okay, this is gonna change things forever, And in fact it did. I when I won the Mandalorian. When I walked onto that set, I had the same feeling. I had the feeling that this just changed the entire industry. This is a quantum uh

leap ahead. And and ultimately what it did is it got rid of all the byproducts that came from c G I, all the waiting, all the notes that get past, all the five dimensional chess that you have to play in order to to to get the ultimate in image, and not even being able to to rate on things as a whole rather in parts. So if we get rid of all that, now you're just able to interface with the image or with the story in the same way that you would on set. That is just a

mind blowing idea. Yeah, and I would add to that that I think one of the most important things is that we no longer have to complete mentalize the process. We're able to put everybody in the same room. Now you used to have to wait for months in order to find out what was happening in post whether that was going to work. Um, it was something that almost was like two separate movies. Um. Now everything's you're able to iterate in real time, collaborate in real time with

everybody on the set at one time. And that's because well, we have the tools now, the technology to and the software to bring everything together onto the stage. And that's what's making this really exciting is that we can do this in real time. Connie, you run the l A lab. What is that? Uh? And and talk about what it's like nowadays in terms of evangelizing this kind of production at a time due to COVID that so many people

are seeing the usual practice is disrupted. I think what this is done is it's you know, by by bringing people together virtually, which all of us are doing now, um, We've it's given us time for reflection. It's given us time to learn and to start to reassess the way in which we're working together. And I think that there you know, it used to be that people would think that this kind of technology was replacing something. It's not. Instead,

it's adding something to what we're doing. It's it's um giving us the opportunity, say, for example, if you do want go on location, and certainly as we get back into production there's going to be some hesitancy with that. You might do some of your location work while you're able to determine that it's it's safe and it's going to work. Um. And then you could take that onto an i CB effects stage and you could do all

your pickups. You could, you could do everything that you might have to go back to that location to get. There's all kinds of things that we're adding now to the tool kit to be able to create um, television and film and other kinds of media in ways that are providing an efficiency, cost effectiveness and just logistical solutions that are helping everyone. I want to add on to one thing that she just said, because I just saw something just the other day. Um, it was a pickup.

It was a pickup that they needed for editorial to make the story work, and they just needed to pick up a few lines of someone who was in a car driving. And the image that I saw with someone sitting in a folding chair with the steering wheel of whatever the model was that they were actually driving. They had a door which was like from a you know, maybe they got it from the wrecking yard or something

like that. And then they had two head rests on c stands and they were able to get that shot and and just have the and they could book it at any time any time. The actor could make it, sat down in the chair, picked up those couple of lines, You're not going to know the difference between whether he was actually in the car or not. And that's just like to me, that's phenomenal. That's that's incredible that that we can do. And exactly as as Connie was saying,

it's just it's changing the paradigm. But when you change the paradigm, it sounds and I think Connie alluded to this that there's gonna be some disruption that people who are used to doing their jobs one way or companies who have vested interest in things going a certain way could have issues with that. So are we going to see as we often see in Hollywood when new technology hits uh, that you know it's gonna take some smoothing

over to get this to where it needs to be. Well, I'm eternally the optimists, So just so let me let me preface it by saying that, um, what I actually think is going on is you do have some people that like to paint by numbers, and the paint by numbers is going to be a challenge. You have those other people that like puzzles and they like to figure things out and you know, that reminds me a lot of kind of the you know, eighties, nineties and things like that as we're trying to figure out what this

can really be. So for some people, you're right, this is going to be a bit of a challenge. And um that you know, there's some people the creative process, some people can't make the decision in the moment, and

so deferring it is better for them. Now that said, I want to say that being able to have a DP actually come and really light something, not have to just light it flat like they actually get to contribute to real lighting in the previous stage and as pages are coming off of you know, from the writer, they're actually able to start that visualization. And you know, the the other part of this is, um, the barrier to entry is extremely low for this. The Unreal Engine is free,

so anyone can download it. Um there are assets that can use to to start to do kind of previous I'm hearing of dps right now that are taking this up so that they can kind of visualize shots for free. All they need is their gaming computer to do it right. And and so what what this means though, is that

I'm seeing people in other departments pick this up. So set design, I'm seeing some set design people that you know have traditionally been collecting things that are in the physical space also having a person or two on in unreal creating that same thing that they have physically so that they can have but when they come to the stage, they're bringing both things to the stage and contributing in the virtual space just as they do in the physical space,

and bringing that talent that they have because I mean they have years and years of expertise and how to set up a stage and how to block something out. They're able to bring that talent now into the virtual space as well. And then the same thing for those people who are have been typically on the in the post production side, they're coming forward and now all of a sudden they're able to be on set in some situation. It is a change, but I'm I am really optimistic

at the change that's coming. It's not more technical, it's it's less technical, so that we can have the same conversation that we've been having for a hundred years in how to make film and how to block out chopped And I think what's really important now is that we're kind of breaking down these barriers and and demystifying the process because there's so so much about the tools that is uh making it possible for people who have been separated in production and post and they're now coming together

and we're blending the two worlds. And what this means is that, just like Miles is saying, say, a production designer has to design sets now that will blend from physical to virtual and that's something well, you need to understand how these assets are being built to a certain extent in order to know how to create the physical set so that that it is a seamless process. And I think what that's doing is just adding a level

of creativity to the whole process. It's attracting people that that may have assumed that they will not able to be in production and now they have a place in production. And it's really exciting that we're finally breaking down that barrier between posts and production. We need to take a quick break, but we back more with our guests Miles Perkins and Connie Kennedy of Epic Games. And we're back

with Miles Perkins and Connie Kennedy of Epic Games. UH. They referenced earlier work on the Mandalorian and back on this podcast in October, director Jon Favreau came on to talk about his work on that Disney Plus series and Epics role in it. Let's have a quick listen. Part of what were exploring is is using game engine real time rendering, because with all the compositing that we're doing, we could either do it in camera using video walls.

Were were the first production to actually take advantage of innovations that have been taking place. In this case, we're working with I l M and with Epic Unreal Engine, with Magnopus, a lot of people that I've worked with before on other projects, and we try to create a new way to use this real time rendering where you can actually do in camera effects on video walls with camera tracking and parallax miles. How did Epic come to

get involved in this? I mean, did the company understand from the very beginning in terms of its gaming engine what the implications were and and saw Hollywood coming from the second it was invented? Or how did we get here? Well, there's a there's a little secret here. Do tell So the first part of this is the Unreal Engine was used actually, um when I was back at I l M on AI back in ninety one, so you know that's the Steven Spielberg movie exactly. Yeah, UM, are AI

artificial intelligence? Yes, And it was just to kind of figure out what was going to ultimately be built on the in the model shop, UM to kind of visualize where he how he wanted to block out his shots in Rouge City. So there's always been this awareness within the community that hey, wait a minute, this game engine stuff. Not only are you able to render fast, but game engines also have this if then kind of programming. If I moved this chair, that flock of birds is going

to fly away. So that's kind of been in the background now. The CTO of our company, Kim Liberry, is also someone who I've worked with for about fifteen or so years. He was a visual effects supervisor at Industrial Light and Magic. He's actually one of the people UM on the Matrix who the original matric came up with Bullet Time UM and has worked on numerous other films.

Working with Kim has always been something else because Kim is always five ten years ahead of everybody else and and in some ways kind of frustrated in that, like I want to I want to see it right now, And so he landed at at at Epic Games as the CTO at this time, where he could take the game engine and make it look photo real. So naturally, um as Kim is, you know, having conversations with Colin Wilson and and the other members of the team. UM there at Lucas film this idea. You know, why can't

we do it this way? And Dave Felony also, you know, why can't we do this? So you know, the test got put together real quick, and the idea really is not just as some sort of translate or something like that, but that it's actually wrapping around So so all of the especially on a shiny object, all of the flections are accurate, the lighting is accurate, all of this stuff. So, um, that's kind of how it started. And the engine and

the hardware and everything was up to the task. And now we find ourselves here with you know, at that time there was one such stage and now we're tracking about a hundred and thirty five hundred forty stages throughout the globe. You know, it's it's just it's been amazing to see it take off. But that was really the germination of this and the other. The other thing I'll say about John Favreau in general, Um, John Favreau is one of those people who is willing, He trusts his

the people that he works with. He's willing to take risks. Not only that, but he knows that it's going to be hard work and he knows how to get you there. And that's a rare uh quality there. There are a

few filmmakers that have that. But the really neat thing about this is that right now it's not just relegated to those people that are at the top of their field, like the John Favreau or the um or the Cameron or you know, George or No, it's not actually right now because of where we stand and because of epics approach to this, we are really trying to democratize as much as possible access to these tools. You know, our our founder truly believes that you really should not be

setting brought barriers to creativity that should not exist. You should allow people to create and be the best that they could be because the world is going to be a better place for that, and then there's opportunities to

share in the success of great creations. And I think that extends to education, of course, and that's something that we've tried to add to services through EPIC is to offer things a fellowship program that we started during the pandemic that had a huge response, and I think that's because people are really ready for this. They can see the opportunity and they can see it working. They can see projects that are amazing like the Mandalorian, and they're

excited about trying to put it into practice. So we've combined the free access to the software with support and education and UM. We're creating a curriculum that we can start to teach all around the world to try and help people get going as soon as possible. And now you're launching a new production hub in El Segundo. Tell

us a bit about that, Connie. We will have a demo stage for EPIC and UM and that will have a production stage, and a demo stage is where we'd like to bring in clients to look at the at the tools that are used to explore the pipeline and to be able to have a hands on experience with this new technology, so which will help people to to determine the type of production that they like to to

UM implement depending on the project. They are. All of these tools are specific to different needs, so it takes a while to determine which ones are most appropriate depending on the story and depending on the type of production. So I think getting the opportunity to be hands on is crucial to being able to make those decisions. And recent announcement that came out about this campus, um so Next Studios has been someone who's been on the scene

that runs studios and everything. They're they're independent of us, but they're also an independent stage provider, which you know we we um not only them, but there are others that are that are out there that the other parts.

They were bringing this sense of a campus, a place where virtual production there could kind of be a hub in Los Angeles, and so what we decided we wanted to collocate our our our labs there, and the reason being is you know that that is their business to run and everything like that, but being able to be

close uh to UH production stage. And then also as Connie was saying, you know, being able to have our own demo stage where we can bring filmmakers by, we can do training, you know, can really be a hub of where the latest and greatest and virtual production, not to mention the fact that since this is a campus, it's also attracting other people who you know, virtual art

departments or previous companies. There are all these different people that are interested in being this area, and you know, for me, it's exciting to see when you have these independent companies coming together to kind of colocate, to create a space um that is really where the best ideas

are kind of coming out. And I think that when that happens, the entire industry benefits from it because you know, we get to go and we get to work on the next latest and greatest, and we know that it's going to be production ready, so that when we're working with the studios that they know when they have a tent pole film that they can rely on the technology that's coming to them. And I think what's at the core of that concept is that Epic is interested in

helping everyone across the board. This isn't something that we're focused on anyone company, and the opportunity to help create a community around that notion is really implotant, like Miles of saying, for the entire industry. So I want to understand, though, from a business perspective, what this means to epic are are you out there are licensing the software and making revenue? Is this material business? I'm sure it's not quite making as much as Fortnite if it is, but I'm trying

to understand that. You know, the fact of the matter is you you you kind of hit on something there is that we're fortunate enough to not only have Fortnite, but Fortnite is built on the Unreal Engine, and so as we make the Unreal Engine better for not only UM for Fortnite, but also for other Triple A games. You know, there are many other games that are built

on the Unreal Engine. UM. But I think what we we we see this future um and and we're starting to see it play out in small part right now where you know, if I look at my son, my son doesn't necessarily pick up a remote control to consume his media. He actually uses his gaming console whether he's watching linear content where he's playing with his friends, and

then he's also communicating with them. So I think what we know is that you know, right now, there's games over here, and there's linear content here, and there's experiences over here. Or I think the promise of the Unreal Engine is that you'll be able to create a world that can have implications in linear content and experiences and

in gaming using the same assets. And so for us with what what we're looking at is, you know, some of the greatest ideas come out of film and television and and happen in the Los Angeles areas and in other areas as well, Don't get me wrong. I mean, you know, we're seeing amazing stories coming out of some of these other countries. UM. And what we're trying to do is really create these tools whereby stories can be told in all of these mediums. And we are fortunate

enough to have Fortnite, which is helping this. We're also a private company so that we can look at the long game and what this is. UM. And you know, I think again back to what I was saying before, UM, we just want more and more ple to be able to be creative. And we believe that when people are successful in their creativity, everybody is going to be successful. But are you making money on this yet? I want an answer there, huh, So I would say that we're

not making money anywhere near what we're making. So actually I'll answer more directly. Our business model is with gaming. If you use that n Real engine. The U list is that you'll pay five. In the film and television space, we get rid of that, and there are no marketing rights, so literally anybody can basically use the engine. We don't charge very much right now, So to answer your question, no, we're not trying to make a lot of money in

this area. We believe that the future is going to be successful for everyone, and that that the future will be that you're going to see some of these filmmakers and storytellers not only tell linear stories, but they're also to be in the game space, and and we will continue to grow that the future is also Unreal Engine five. This next iteration of the secret sauce, Uh, what what is that going to bring to the table? Is that gonna When does that hit? When does that? What does

that change? I can't contain myself. It's so exciting. This next year is going to bring some of the most amazing advancements. Um kind of across the board. Um so um what is it? Unreal five? What does it really mean? It means that you're going to be able to do film quality assets and moving around in envirnus. You're going to be able to filmmakers will be able to in essence play god um and do anything in the engine, and you're not worried about the processing power or anything

like that in in some of these. So if you saw the demo, you can see how close you can get in on these assets and then it looks like a feature film yet it's running and basically uh ps five in that case. So it's it's gonna be fund It's gonna fundamentally shift the way um people are making content and how iterative they're going to be able to be on a near final image and um miles. You also recently completed work on a second edition of the Virtual Production Guide What what does that do? And what

is new? Well, so when I kind of first came into Epic, one of the things I noticed a lot of the colleagues who I had worked with for a long time, who are out there you know, the third floor,

and and Halon and some of these other folks. We had brought together a group, and I noticed that the conversation they were talking about the same thing when they were talking about virtual production, but they were using different words, they were using different ideas, And so it struck me that we needed something that could kind of have people tell their stories, tell their variance is but we could start to standardize on some of the ideas out there.

I also always knew that we're in the beginning, and my experience when you're in the beginning of something, anything that you printed last year is not going to be accurate anymore about it next year. So, UM, this new version, if that first one was, you know, roughly nineties some odd pages, we're up in the hundred and fifty seventy page range. Um, and a lot of a lot more stories from a lot more people who have actually had experience with it, and we get into much more detail

about how you employ some of the virtual production techniques. Connie, I want you to look back at which of course was shadowed greatly by the pandemic. Do you think the arc of unreal engines work in virtual production would have been different in a more normal year or is it

something that really drove the technology along. I think it's really driven the technology along more so than um it may have otherwise in some ways, because I think the ability for all of us to trust virtual solutions has changed. I think the fact that we've all gone to work from home when that was looked at as possibly something that was detrimental to the productivity of many companies, has

been proven quite the opposite. And I think also UM you know, techniques of filmmaking have UM maybe been a little resistant to some of these tools, and now I think they're embracing them because it's a solution that's going to get them back into production sooner. So I've noticed a lot of people preparing their projects differently in order to UM two increase LEWD, different aspects of virtual production

that they may not have decided to use otherwise. And I think that has UM created a different kind of community because everybody needs to support one another in this process.

There everyone's had different experiences that they can bring to the table, and I think there's been a lot more sharing of ideas and collaboration as a result, and I think UM I think we're going to see some really interesting stages set up in the next few months where people are willing to take this risk because they can see that there are so many opportunities to what it has to offer terrific. Well, I'm really looking forward to seeing the progress you guys make this year, and thank

you both for coming and talking about it. Thank you for having it. This has been another episode of Strictly Business. Tune in next week for another helping of scintillating conversation with media movers and shakers, and please make sure you subscribe to the podcast to hear future episodes. Also, leave a review in Apple Podcasts and let us know how we're doing