Hello and welcome to Think Like A Game Designer. I'm your host Justin Gary. In this podcast, I'll be having conversations with brilliant game designers from across the industry with a goal of finding universal principles that anyone can apply in their creative life. You could find episodes and more at ThinkLikeAGameDesigner.com
In today's episode, I speak with Ethan Mollick. Ethan Mollick is an associate professor at the Wharton University in Pennsylvania, where he studies and teaches innovation and entrepreneurship, and also examines the effects of artificial intelligence on work and education.
His papers have been published in top journals, and his book on AI Co-Intelligence is a New York Times Best Cellar. His research is highly cited by other academics, has been covered by CNN, The Wall Street Journal, and other leading publications. In addition to his research and teaching, Ethan is the co-director of the Generative AI Lab at Wharton, which builds prototypes and conducts research to discover how AI can help humans thrive while mitigating risks.
Ethan is one of the most thoughtful, intelligent, and fun people that I get to talk to. He and I actually got to work together through multiple projects, including the Breakthrough Game, which helps to improve innovation and creativity in groups, which we've run for not just students at the Wharton School of Business, but also companies like Google, Zillow, Twitter, and others.
We've also worked on an entrepreneurship simulation game, which helps teach the principle of entrepreneurship. Ethan was actually going to be one of my first guests on this very podcast. He was going to be an episode number three, but our audio recording did not make it, so we had to re-record now five years later.
I'm really glad that we got the opportunity to do this, because AI is everywhere. If you have paid any attention at all to what is going on in the world, AI is changing the face of work, the face of creativity, the face of what it means to be a human.
Ethan is the foremost expert on this that has a very thoughtful, very reasoned way that he approaches things. He is regularly in touch with the CEOs and the leaders at companies like Google and OpenAI, and in Thropic, who are building the Frontier models of AI. He has a front row seat. He's been doing a lot of deep research into how to interact with AI, and we get into the details.
We talk about the ethical implications of AI. We talk about the changing nature of work and how it impacts you as a creative, whether you're an artist, a writer, a game designer, an entrepreneur. We talk about how to incorporate it into businesses. We talk about how to think about the next six months versus the next 12 months and the next three years. We really dive into how you can use it as a great game master, and what types of game designs are possible that weren't possible before.
There is so much incredible useful information here to help deal with a very fast-moving, very impactful technology that I am ultimately very optimistic for, even though there are a lot of concerns and a lot of challenges that we have to face. I hope you find this as inspiring and useful and practical as I did. Without any further ado, here is Ethan Malik.
Hello and welcome. I am here with Ethan Malik. Ethan, it is a long time overdue to have you on the podcast. I am so glad to be here. Thanks for having me. Yeah, we had. We had. Are you actually my third guest on the podcast now, but five years ago, but the audio ended up being unusable, just where I learned the lesson to have backup recordings, which is thankfully you're not doing as well. I am glad to have you back with me. Thanks for changing a lot since we had that conversation.
Yeah, for both of us, lots of the world and technology and games. Yeah, so I think we have so many areas of interesting overlap. The projects that we have worked on together both the breakthrough game and how the things about creativity and teams working together, innovation generally.
But I have to start with kind of the proverbial elephant in the room, the thing that you are now most known for, which is co intelligence and the sort of understanding of how AI is affecting our lives in our modern world. First of all, let's just I'll have already given some of your bio at the beginning of this, but briefly, like, how did you come to be a guru of AI and just walk us through a little bit why, why this has become an interesting topic for you.
So I was AI adjacent for a long time and nobody cared, you know, so I at the media lab at MIT in the early 2000s, I worked with Marvin Minsky, and it was one of the founders of AI. And I was like the AI whisperer. So like I was the technical guy, but I helped explain the technical stuff to other people and have conversations.
It was another one of the AI winters right before the latest AI boom. And so we kind of kind of kept my eyes on, you know, AI systems as we were building training and teaching games, but I'm not really a machine learning kind of expert, but turns out large language models are very different kind of beast and just having been aware of them and using those kind of tools for education for a while, put me in this really interesting spot.
Yeah, and so I first I'm going to, you know, I'm going to encourage everybody, you know, to read your book, co intelligence, it's a fantastic and very clear description that helps people understand not just like where we are in AI current development, how best to interact with and use AI as well as kind of, you know, predicting as best we can, you know, kind of plausible paths forward.
But you know, in my industry, man, I have heard everything from I'm sure this is true across the board, but that you know, this is going to be the greatest thing of all time to this is the end of the world. Generally speaking, very few people take the kind of reason middle position that it seems like you do.
If you're speaking just generally to, let's say, creatives, you know, and people who are, you know, most of my own, it's in the gaming industry, people who are, you know, looking to do good creative work, what's the way that they should think about AI on a kind of practical level nowadays.
So I mean, right now with the level of AI is that whatever your best at, you're definitely better than AI, but it's probably better than you had a lot of things you're not that good at and game designers entrepreneurs have to wear many hats and a lot of our time is trying to find good people to help us out. The AI can be those good people plus the only tool we ever had that really is prove a duke seller creativity is coffee.
It turns out just in case you're wondering, pot does not it makes you think your ideas are better, but they don't actually get better on quality skills. And we're trying to get a prosthesis that actually improves in creativity and is engaging and we work with in this kind of way and I think there's a real power there.
So especially in the next step thinking about games, this is a really interesting space because game designers tend to be pressed for time, whether doing video games or, you know, board games, they tend to not have all the skills they need to get something done. They need everything from play testing to creative support and AI is really at the sweet spot of ability right now to help with all those things. So it's an exciting time, I think.
Yeah, and I've found in it takes this is a something that you've written about, but I found this to be very practically true. It takes like several hours of working with AI to really try to find that sweet spot of utility versus kind of, you know, it's a toy or wasting your time with it, right. It's just like it doesn't behave like other pieces of software where I can just sort of engage with it. And once I get past the initial UI UX, I can find the value right away.
Whereas here it's like the, I think what you would refer to as the jagged frontier feels like it keeps changing. And knowing when AI is going to be useful for me and when it's not, it's not it's it's surprisingly not intuitive. Very not intuitive because like an AI system turns it to be really bad at counting a 25 word sentence, but can write a pretty solid sonnet. And that's hard for us to deal with, right. They're not human abilities or missing things.
Although I find the best analogy is actually to think of the AI like a person. And what that gets you is the ability to think through, you know, it's flaws and it's BSing almost like a person whoo. So, you know, just you manage many people over your career and grow in organizations and you begin to have an end to a strength like this person is good at this. They're bad at this. I could throw this task and then they'll do a great job.
This one they're going to say they did a good job and didn't do it. And there is that kind of aspect to the AI piece. So I usually say about 10 hours of the model playing around, but it is changing, right. It's hard to get the maximum out of it. Like, you know, on the day we recorded this yesterday, Gen 2, or sorry, Gen 3 of Runway came out, which can now do incredibly good video like just on prompting.
Last week, Claude Sonnet 3.5 came out, which was a really powerful model that really changes the game on writing. And so it is hard to stay up with stuff. I think the thing to realize though is over the lifespan of game development especially, I kind of think you have to because the frontier is moving pretty quickly. And you know, the stuff that you do and the stuff that I've done with you does it is not fast.
It's not a, you know, no one makes a quicky game outside of it, you know, kind of jam situation that is like ready to go and player tested and go through all the steps you describe in your books, which are these excellent, excellent overview of the problem. Yeah, and I think it's like, if you tackle it at each stage of the process, right.
So the way I talk about with the core design loop, right, and this kind of mirrors off of design thinking with this sort of inspiration, framing, brainstorming, prototyping, testing and iterating. And I think you can approach every single piece of that, and AI can add value to that part of the puzzle, right. In terms of finding inspiration and just like pure raw idea generation, I don't think I've ever seen a tool as powerful as AI.
I mean, I get, I mean, I'm professionally creative now, like I've said my life study got to be creative. And I'm not saying that AI is going to come up with the best idea that I'm going to use. But in terms of just like the pure volume of like reasonable ideas and suggestions that can become a springboard for the stuff that you want to do, like that it's like amazing in seconds, what it would take, you know, a team of people hours at best to come up with.
Yeah, I think that you know, it's very funny. We started our code design career together building the breakthrough game, which is sitting behind me right here. And I'm super proud of that, right. That was used in organizations all over the world, the general idea is, and I think there's still value in it. But at the other side of the fence, like actually I can give the AI the directions to the breakthrough game.
And it does a pretty good job of just running it on its own and generating a list of two ideas for you. So I mean, I think as a creative tool, we've got something very new in the world. And I think that's, you know, both really exciting and but also a little nerve wracking because it beaters on supposed to be creative. Yeah, yeah. And this is, I guess, I guess let's let's address some of this stuff now. I don't want to linger too much on it because I prefer the sort of more optimistic use case.
But there's this is some of the dark side that feels here, right. There's a lot of people. And I know this is the most common thing I see in the game industry comes around illustrative work specifically. The people are worried that the, you know, the AI is going to destroy jobs is going to destroy the creative work.
You know, the classic idea of the of the AI robot is that it's just going to do all the grunt work. It's going to do all the busy work. It's going to do all the things that we don't want to do. And the reality that we're seeing from the current level of AI technology is actually really good at the creative work. I mean, it can create beautiful pieces of art in seconds with a few words of prompts. It can create.
Sonnet says you mentioned it can do, you know, I mean, it's not it's not great at making games yet. But I have no reason to believe it won't be like what for the people that are out there that are are afraid of this, you know, really just destroying the jobs that we actually really like.
Like how, how should we think about that or how should companies be thinking about where they want to be using these tools or not using these tools like where's the, where's the ethical line we should be drawing here or is there one.
So there's like three or four complicated questions you asked there. The first is the ethical line. That's a tough one, right, because these systems have ethical compromises throughout their whole process. They're trained on, you know, stuff that you and I wrote without our permission.
So what I really feel about that is that legal is that ethical, then they're fine tuned using labor that is very exploitative, you know, instead of using our own data, right, and then on top of that, it produces work that can be derivative in some way or reproductions of other copyright work or it just displaces our labor and stuff we care about a lot.
A lot of ethical constraints when I when I think the problem is is that the system is are so good and there's, you know, systems have systematically tried to address these so don't be firefly, for example, is only trained on data that they supposedly have legal and ethical claim to right where they can they can they've sourced that they pay they pay license fees. So that gets rid of the oh my god, it's just copying people's art, but at the same time we can still produce displacing art.
I think it's a really difficult challenge, especially in the creative world that we're in. Now the positive view of this is, you know, the same kind of thing that happened when the synthesizer came out, right, it was like in a destroyed professional music and turned out to create new forms living through that sucks, right, like it's it's not like living through industrial volitions good, and I think there is going to be displacement.
I don't think we could say that jobs are going to be unchanged. You know, I do think that it enables new kinds of creative work most, you know, they're I think freelance art is going to change what what are you doing? We can 10 times more productive. How do you offer guidance and you know, taste is something I doesn't have right now.
You know, so it might start to be more that you're supervising a small firm of AI rather than doing your own work or your own the luxury of work gets turned into other cases, but that's going to require some change and that's not necessarily comfortable.
For the individual creator, you know, I'm talking I've been in contact with one man games to do is build a lot of games before and now he's doing one piece of, you know, one game entirely on his own that would have been possible for how do we feel about that right is that great because there's more creative content coming out from more indie developers. What does it mean that people who couldn't code can code that people who couldn't draw can produce art of a sort.
I mean, I think we're just in for a very large change and I think that the anxiety is and and distaste is understandable. I don't know if it's going to make a difference. Yeah, and that's a lot of this is where I come down, you know, like we spend hundreds of thousands of dollars on art. We spend, you know, even at least that much and more on programming and engineers and all this stuff and that's a massive, massive overhead to have to make the kinds of games that we make.
The idea that someone can make those games at a fraction of that price is just a reality now and it certainly is going to only become more true over time. And so for the people, you know, for some people that means, you know, they're not going to get as much work or they'll, they'll work has to change. But for others, it means that they're able to create things that were just completely implausible or impractical otherwise. And there's that means there's new types of games that can be made.
There's entirely, you know, plenty of people out there with amazing ideas that don't have a million dollar budget to launch a massive TCG or a digital client. And that could theoretically be done. So to me, like I'm more excited about those possibilities, right, as someone who's never learned a program, but have had to manage a lot of engineers.
Boy, would it be nice if I could build my prototypes on my own and move quickly without having to necessarily hire engineers and spend their time and just be able to iterate cheaper and faster. That's the name of the creative game. And that seems like exactly the type of tool that we've been presented with, you know, if whether or not it's opening Pandora's box, it seems like one we can't close. Yeah, I mean, I think they're not closing as an important part, right?
Like there's going to room for artists for people with taste. I don't know whether the market looks the same though. Like I still hired, you know, I hire artists to do the work that we do and I intend to keep doing that. But I'm going to be competing against people who are not hiring artists as much, right? Right. And one by one, the barriers fell, right? Steam was holding out for three months. And then afterwards it was like, okay, air, it's fine now.
And the tools by companies like Leonardo that are doing stuff is doing that for digital art assets. I mean, they're there, right? So I think we have to decide what if we're doing a rearguard action, what that rearguard action looks like for the people doing that. But realistically, for anyone who's running a business in this place, right, it's it or wants to enter the space, something it's changed.
I expect you to see a burst of creative energy of new game types we never saw before, new approaches, you know, the event, you know, and there's secrets we discovered, right? There's, you know, how do we use this to do digital game testing, which is something you and I have talked about before, having personas play the game and can we accelerate the process of doing this, getting into markets incredibly expensive and difficult right now. What happens in this new world?
Part of what we get to do is think about how this is what an ethical use looks like and start to model the kind of behavior that we think is the right kind of model. Having a human art director is with AI help. It looks a lot different than I'm just going to cheap out and use cheap AI art everywhere. We have to do some thinking about this. Yeah, yeah, I think that's right.
And I think, you know, so let's let's dig into this because I think it's it's fun and interesting and immediately practical to me. And I'm sure too much of the audience, right? So on one level, there's the hey, look, you're an individual creator, you're somebody out there, you know, you should be playing with these tools and using the frontier models and just getting a sense of like where we're going to do this.
And then you're going to be able to do something and you're going to be able to do something and you're going to be able to do something and just get a sense of like where it can support you and the things you do. And it's worth some amount of investment to stay current on this because it's moving quickly. As a now as a CEO and entrepreneur, I have the the challenge of deciding how to integrate this into my company, right? I will I like you.
I'm committed to continuing to use human artists and you know, human and and and I would much rather expand the capacity of my current team to do more rather than, you know, reduce team size, right? But how I found it to be difficult to create that encouragement and to create that culture to allow people to use AI to encourage people to be able to be better at using it. And there's always these practical barriers, right? Where it's like you try to use it.
Does it quite do what you needed to do or it's just wrong often enough that it's not reliable. Like when you're advising organizations and you know, in my case, sort of this, you know, small to mid to organization, but whatever tier makes sense. Right. How how do you how should we be thinking about this in terms of being able to maximize for a team of creatives that we want to be able to be more empowered collectively?
Again, I think there's a lot happening there, right? So one thing we've seen is that there are lots of ways of accelerating this that are 100% clear sort of out of the box, right? So you helped teach one of my classes at WarTen, where I had just after chat, DVD came out when it was still GWD 3.5. And we ran a class where we taught people to be game designers and had them build a game.
And I had MBA students who, you know, many more creative industries. I know there's a few like MBAs or you know, but like there was, you know, hip hop promoters and all kinds of interesting people there. And they they were said, if you can't code it, I said you need working software. If you've never cooked a bit of the website, you need a working website, you need to have built a full game, you know, with with music 3D printing and and centering and all of his other tools.
And they did it. And some of them were actually quite interesting with some interesting design space and they used AI to sort of supplement their situations. And you know, I asked them right about some cases, the AI helped the OK said, hurt. So right now the title of the book is co-intelligence partially because right now it's a supplement.
So you spend your 10 hours learning what this does and you'll find that that the shape of the frontier and you'll be like, listen, it's really great for some stuff and bad other things. And that doesn't mean doing the work for you. I spoke to a Harvard quantum physicist who's like, all my best ideas come from AI and I'm like, does it know quantum physics as well? And he's like, no, no, but it asked me good questions.
Right. And so I think part of this is like, where does it unburden you? And you will find that there's things I can't do. I cannot tell it make a compelling board game because for example, all the AI systems hate conflict because they've been trained to avoid that to not because controversy. So everything ends very nicely in an AI game, but you've got to convince it to do other things if you want to.
It, you know, it falls on tropes quite often. There's a lot more tropes in the training sets. So you have to be one who kind of pushes it out of its trope weakness into something interesting. So it is a capitan. It's a person you're working with that adds resources. It doesn't solve the problem for you. Now there is a secondary issue here, which is like, how good do these systems get? And that's the big unknown question we have right now.
Yeah. Yeah. So it's like, I want to, I definitely want to get into the kind of how good is it get? What's the future like? But I want to, I'm really eager to linger on the like kind of short term. Like here's where we are. This, you know, the next six months, the next year, even though sometimes that's hard to predict. And if that's, if that horizon is too far to be practical, then that's worth flagging, right? But like to me, there's the future world, which is very, very hard to assess for.
There's the current world, which I think there are practical steps. I think a couple things I want to highlight about what you said, right? One is using the AI as a tool to ask good questions and like be a stand in for either your customer and your player or your reader for a book or your, you know, whatever, you know, your customer for a new product or even like, you know, for, you know, I want advice from, you know, Gandhi and the Buddha on what I should be doing with my relationship.
Even like those things like it's, it's shockingly good at giving you a kind of coach slash reflective mirror on what's happening and what's going on. I found that to be just like this incredible tool just for own for self reflection. So I think that's like a very powerful piece. And then I wanted to the second issue that is when it comes to sort of you use the phrase like sort of doing what would have been impossible before.
And pushing you to like just try to do things that are like way outside the scope of what you thought you could do. For people that are like practically, I'd love to just sort of dig in more like what practically speaking, what does that mean for someone to actually use it. So either the tools of, you know, kind of self reflection general like the quantum physicist or in terms of like, okay, great.
Now you told me I can, I can do things I couldn't otherwise do in my organization. What would be the next steps to say, okay, let's do that. Is it a game jam where people have to have something working program and 3d printed stuff at the end of the week. Or is it, you know, some other tasks that I make unreasonable assignments for or something.
Yeah, so let's break it down. I mean, you can go through each step of your process almost and think about what you would use it for. So I would just give you a couple highlights. So first of all, ideation, you should be using this for ideation, right. Come on with 45 games. No more like this more like that. It's very important to interact with the AI. You're not just typing in a command and getting something you're saying, no, no, you know, more like the fifth idea, you know, like making more diverse.
We have a paper showing how you prompt the AI matters and if you prompt it to do a chain of thought approach where at first, you know, list out some ideas, then come with even more diverse ideas, then modify those again, you know, with better ideas if you're just like, give me 30 ideas, right.
And all the techniques that you talk about in your book and that I discuss, you know, and with you on these things about, you know, ask you to come with portray ideas. Imagine your infinite budget. How would you do this? Imagine a digital computer system of the future. How would you do it? That all helps you come with really good ideas.
Then so ideation is great and that interaction is really good. Another thing that's very good at is prototyping, especially if you use something like cloud for cloud 3.5, sonnet right now, it has, if you haven't played with it, it does artifacts. It actually creates like images and other stuff.
You know, mock up with the website would look like now make this dial functional. It will actually do all of that for you, right. So you can actually play with some prototyping, you know, which is an important part of the game process very quickly. Then it actually works reasonably well as a game tester. So if you show it a screenshot of the game and say what confused you here's the instruction manual, what's weird.
Pretend to be a high school student playing this pretend to be an experienced board game designer. What would you say? You get pretty good testing data. Not as good as a human test, right, because you can't see everything that emerges. But for that first order catching the errors that would have taken you two to three rounds, it's pretty good at doing that set of stuff.
It's also very, very good advice. So this is a great paper showing that experienced entrepreneur, the Kenya who were doing well beforehand got an 18% improvement of their profitability when they got advice from the AI. Now if they were doing badly, they actually did worse because they couldn't even implement the ideas or know what questions to ask. But if you're experiencing design it right, like, you know, what should I be thinking about in terms of how to get production outsourced on this.
What are some things to worry about in this contract? You know, those kind of basic questions, you get a second opinion from an experienced if slightly, you know, slightly overconfident colleague. That's a very useful call to throw in throughout the entire process, right. And so you have this kind of multiplier mixed in that's actually quite valuable in every step of the way in a very practical way.
Then the two best ways to use this are just get GPT-4 on your phone pay the 20 bucks a month and use the voice mode and just talk to it. You'll be surprised at how far you get, but just like, I'm in the car, I'm on a walk. I'm just like, let me chat with you about my design problem. You know, at the very minimum, you get the rubber ducking experience that QA designers have of talking to a rubber duck and turns out you come with good ideas for just talking through with someone.
At best, you get really great stuff. It's also good motivation. You know, I spoke to the guy who invented positive psychology and he's retired now, but he was one of my most famous therapists of the planet. And he says he uses GPT-4, a version of having read his books as giving all of his psychological advice down because it beats him.
So I think that there is like, you know, there's an emotional intelligence piece. There's a QA piece, but the valuable, at least for right now, where AI is, is it's not that great. It tying together all these pieces. That's where we still need the human designer, the human entrepreneur to make all this happen. Whether that's long term true or not, I don't know. So I mean, I feel like artists is the most directly threatened.
Although if you talk to any artist, they will absolutely say, like, look, it's not consistent enough yet. You can't quite pull the pieces together. And true, if you really want high quality work, you're going to be humans in the loop right now. So, you know, I think that's kind of a liberating perspective in some ways. Yeah, yeah. In many ways. And thank you. That's like amazingly detailed, practical, implementable right away.
So I hopefully, you know, people are taking notes or are relisting to the section because it's exactly where I wanted to get at. Like things people you can all do right now. I've about half of those I've been doing and I'm happy about and half of those I have not I haven't played with the 3.5 son it. So that's basically what I'm doing right after we're done here. So thank you for that.
But now I think it is, it's worth moving into this next phase of like the future, the near future to to to medium future. Because you're right, it feels like we're in this golden age and it may just be a few months of golden age where you as a super talented as a talented creator, this thing can accelerate you enormously.
And as someone who has is very new or has no skills, it can accelerate you enormously. I think the people in the middle of the pack, if you were just okay at stuff, you're kind of in a bad spot right now, because now everybody's just okay at everything. But but as this evolves, there's no reason to believe right this was not going to keep getting better and there's a lot of reasons to believe it's going to keep getting better.
And that that threatens a lot of the foundation of what we do for work and how we value ourselves as humans. And so there's a lot of steps to this right there's there's baseline even with the technology we have today a lot of industries could completely get get eclipsed even just you know things like accounting and law and you know it's giving better medical advice and a lot of
doctors and this is just a matter of kind of regulations. I think it seems like it's just a matter of regulations clearing the path. And then there's this area where as you said right now the human making the disparate connections and the expert being better than the machine is true.
But who knows if that's true a year from now and so you know there's a lot of areas we could go here but I'm fascinated by this kind of like how we should think about the next you know not just the next six months to a year but the next you know year to five years range.
And I mean I talked to all the major AI companies are going to base us and I will tell you nobody actually knows how good it's going to get I will say that having had a lot of conversations over the last three months I think that people at Microsoft at open AI and a
lot of the topic seem to have gone from like who knows how far this goes to like we've got another year to exponential growth left whether right or not I don't know right and where that ends up I don't know. You know it was funny you're saying if you're just average you're not going to do great but you were going to do great in games anyway unless you got very lucky because the people in the space just like a Hollywood and other places to get where you are sitting right now just and the kind of interview here.
And I think it would be really good I mean luck helps you know perseverance helps all the personality characteristics of being a successful person help but you know not the only way to get ahead but there is some talent that has to be at the heart of this thing.
And if you are making in games you are you know top 1% calendar the very least probably top point 1% or point 0.01% in something it may not be at everything maybe you're not great at the art piece but you have a you know knowledge of systems maybe you have an sense of what's fun maybe you're just going to executing stuff and pull stuff together in a way other people can't and just willing to grind through the play testing maybe it's the you know the classic sense of taste but
we're ways off from threatening the point 0.01% that being said right that's the explicit goal of all the AI companies right AG eyes common definition is a machine better than human and every creative task better than the best human every creative task or you know they were in knowledge based task and you know if that happens it's not just a
game design is the least thing that gets gummed for and some ways you know the bankers get replaced the lawyers get replaced long before the game designers because there's still a matter of taste and difference and having a human guiding a process still matters right and so I don't know the full answer to that question right it's hard to know but I think we need to plan for scenario where that's possible if I was building a triple a games you know in video game world
with a four to five year lead time I'd be spending a lot of time thinking about this problem right because I've got hundreds of coders and first of all they're already all using AI to help them code we don't know you know at this point the code may not be good enough for a large scale you know code set we don't know but I would be using it but then the question is okay not just this but like I've been seeing the first really
interesting generative games come out you know a lot of people probably heard of AI dungeon which was using a very early version of GBT to sort of be a DM the current versions are very good dungeon masters and you can ask it to come up with a completely
novel powered by the apocalypse game and you get really good results even funnier I threw one of the most complicated manuals for any game I've ever ever ever played or I haven't actually played it because no one I know actually plays this thing but it's like 800 pages and manuals one of
the labors of love kind of things and it was able to roll up a character and doing it required understanding page 112 274 you know 380 and this is Google's Gemini pro 1.5 which has two million tokens of context winners can hold the entire manuals you know so like that's it's already doing some pretty amazing things the most fun of the of the actual AI power games I've seen so far is an infinite crafter game you know where you
take cards and stack them on top of each other and there's an AI version which can just keep generating new things to craft basically then make sense in context right so I drop the moon on the earth and what happens you know it could keep generating new ideas so we're at the early days this is a really interesting design space being enabled by this like you know what is it
do when you have an AI companion and just you and I slip through this a bit as we've got this very successful game and we're trying to adjust and the days of the I were like okay the AI will be a tool to help you and it's like oh no the AI plays the game on its own what is the role for the human the loop and I think these are things we all have to think about more as we move forward right right when this is exactly I let's use this as the illustrative example right because like as you said
the breakthrough game something we're both very proud of we've run it for you know 4500 companies we've had great success it definitely makes people more creative makes teams more creative I still use it with my team but and then as AI came in that we went through this process over the period of just a few months from chat GPT you know from from 3.0 or 3.5 I forget when we started having the conversation I'm like hey
okay let's have the AI be a seat at the table and then there otherwise the game is the same and that's perfect oh my God so cool and then now within a few months of us starting the work on that it was like oh actually no AI we just feed all the rules of the AI let the AI do almost all of the
work and then feed output a result there and it's basically just a you know kind of a GPT whatever the hell they call them terrible naming conventions for just pre-built rules right so it's not really even a business anymore so if you're training build a business
that builds on AI or that uses AI technology or that's a game that's going to launch in three years like how could you possibly plan for that like you can't you're taking a bet I mean what are you so entrepreneurs are used to taking bets designers are used to
taking bets right we're betting you know what it's creative work there's a lot of ways things to go wrong so you're making a bet I would want to bet at least on a little bit on the side of technology of improving right I think you need I think there's
needs to be added here bet equation right so maybe that means you move faster than you'd be for you use AI to move faster and we're going to iterate quickly right you know you look at you know like maybe there's a space I want to move into right now
maybe I take advantage of the fact that there's now so many more options for printing on demand or such in the board game space you know an AI companion piece that's useful in some interesting way but you know I think there's going to be categories of games
they get more effective than others there's going to be categories of entrepreneurship that are more effective than others it's always I think entrepreneurs are not being ambitious enough in what they're attempting there's a lot of like little
rapper kinds of things that the AI companies are going to just take away from us like if you've got a really good workflow based around GPT-4 I don't know what tell you but GPT-4 is not the end point of of open AI's products and they want their product to do everything you do and more like that is their explicit goal they believe they can do it where they can't or not so I think part of what you want to do is bet on something where there is taste and creativity and you're pushing something
further out past the you know the border yeah yeah it's it's man it's such a it's such an interesting space right so that my my general philosophy here is that there's there's two resources that that really matter in the modern society for us which is attention and trust that there's this ability to earn people's attention that they have a reason to be here and reason to pay attention to whatever you're doing that they're going to trust when you actually say you're going to do
a thing your product is going to deliver thing your games are going to be the case right in that those things you can leverage with you have AI backend helping you to do stuff and make things in terms of the you mentioned you know kind of taste making as a as a skill set that is you know something to lean on I I'm skeptical of that like you know the AI out of them first Spotify sending me you know or or YouTube
or whatever choosing what I'm going to like next is pretty darn good like is is is taste making really going to be something that's a sustainable advantage for humans maybe not I mean it's hard to put it's going to something right there is something inevitable of being really good at this stuff and anticipating where the market's going you know in kind of in in ways that other people can't do but you know it is a hard
problem to solve right I mean you know for right now we're in this great complimentary space I'm in a Dujas a dragon game that meets every week and it's awesome because we've got an AI listening to our entire
conversations and summarizing everything that happened you could query the game about what happened yeah so we don't have to know what you know what this character said under these circumstances we can you know actually ask it in character what happened or not everyone's generating songs and you
know and you know and and illustrations and none of us are could draw so like this is super fun like it's you know a huge enhancer I think part of what you need to do is think about where it's a handsick stuff I think the other thing you didn't mention is that most of the games especially the board
game space that you do most of your work in writer the you know with the higher hybrid kind of you know I don't know what we call this now but like there's a people element to it then I think thinking about how AI removes drudgery from that process is also really interesting like I think
a lot about the you know trend and sort of legacy board game kind of approaches which is all about kind of like adding in complexity to make a video game like like there's no doubt that you could more elegantly implement a big box game in a video game format is that but there you know
people want to have these kind of experiences are different so when I talk about taste part of it's a taste of what humans might want in that kind of interaction together to have really interesting human to human interactions in the room and you know the board game is a is a way of getting people into that human to human interaction there are ways to play with that as well.
Yeah yeah I think that's I think that's that's well said the you know it's frankly I'm shocked that I have a job in just forget AI I'm shocked that like board games and tabletop games and RPG games are so popular nowadays are more popular than they've ever been right when I saw
like the iPhone and the iPad come out and I saw you know VR technology and everything getting to where it is I was like oh well people are just going to want to do that now they're not going to actually want to play tabletop games and not going to get together and play RPGs and the exact opposite
was true and I'm very grateful for it and that speaks to the point that you're making which is like no people want to be with humans and they want to have you know physical tactile experiences and be able to enjoy that connection and what that provides and so how does you know it'll be a while before the robots take over that role I think but I think that how we enhance that is a big deal.
Yeah but I think you have to be opinionated like that is a perspective that we wouldn't have guessed right like who would have guessed that there'd be a board game and you know Guggen's records Renaissance that was powered by you know the pandemic and a whole and Kickstarter and stranger things all
sort of happening at the same time live plays like it was a bunch of elements that all came together to make this thing happen that is you know and once people start playing it they're into it right and I guess as the as millennials age out Gen Z will find this dumb again but also it also has to do with failure of like you know as RPGs got more expensive like computers it just didn't become the genre that people thought it would
compared to computers became FPS shooters iPads became all about you know pay to play free games and leaving this huge design space available so I think you could have an opinion about where things are heading and we could be wrong with this you know they're we you and I have been around long enough
to live through various crashes of various game system you know types where it's like this is going to last forever and then it's like oh actually know what ever wants to play this kind of game ever again right you know
MOBAs were supposed to be a big thing for a while and sort of like there's still a few players but it wasn't like it became the dominant space in the same kind of way you look up you know video games are weirdly stagnated even as tabletop has taken off and if you look at the top 10 games on steam
it's the same top 10 is a few years ago like there's this weird kind of loopiness happening I don't know what AI on leashes but it's not like this is an untroubled industry that has never had cycles before and difficulty you know there's a bet you're making in a world where you know this matters
right yeah and I think I think stamp sizing that you know it's a deeper point well beyond AI right you know you're a designer you're an entrepreneur you are taking a bet that you are right about the way the world is going to go and that your bet has to be you know both like non consensus and correct for you to be a success right you have to have an insight that you know other people don't all share and then be right about it and creative industries that's the roughest because most ideas suck
so most people you know ideas generally non consensus for the beginning because everyone's like I do not want to play you know a feat themed game in which I'm a cheese monger and you know medieval France and there are 7000 collectible cards you know like you're like yeah look I'm doing something not consensus yes also terrible right so the problem is you know having like so that's going back to the things that you've talked about in your books
and I'm sure in other podcasts which is testing and learning is really important too like one of the great things about these tools is they lower the friction for testing and learning and I think that that is just so vital because that's where we are right now how do we test and learn faster than anyone else becomes the most important relevant question and if you could do that without putting your job before you know like before you invest a lot of money in the new product
like that's exciting in of itself that that's where AI can help accelerate things too it's not just about accelerating good ideas it's about accelerating bad ideas one of the things that we notice you know when we study things like my combinator of these famous accelerator spaces
is they don't actually make you necessarily more likely to win you just win or lose much faster and if you lose you're still part of a community of people they give you another chance and I think there's some value in that kind of approach too
yeah this is I mean honestly if I were going to boil down everything that I teach into one thing it would be pretty much that like accelerate the rate at which you can iterate and learn like that's it like the faster you do that the better you are at what you're doing
and that's like you know there's a lot of nuances to unpack that but I really do think there's nothing more important to creative work entrepreneurship anything like how fast can I test my idea and how well can I take the information that I receive and actually implement it without you know
losing my ego and my my entire sense of self-worth the way it's bad yes and and that's and someone's really what you train people to do in your lessons and when I try and train people to do my entrepreneurship classes is how do you listen
for failure and interesting things and you know do that in a way that you actually make progress so it's part of the reason why when we do entrepreneurship we force people to have hypotheses that they're testing because if they're just going out and looking at how people like their product
they're only going to hear the good stuff and the bad stuff is going to be from idiot they're going to be from you know idiot to don't understand my genius and they don't really get that it's not going to look like this in real life it's going to be much better and they and you to lose yourself right so part of this is about how do we get people to listen and fail in interesting ways that they learn from pretty quickly and again I think that's where the AI you know boundary is actually helpful
yeah the AI is actually really great at this and I think I think you've mentioned this in one of your one of your some of your writings but correct me if I'm wrong but you know the people I just I'll speak firmly for myself anecdotally I'm a lot less impacted when the AI
tells me and critiques my stuff than I am when a human does it doesn't I don't mind sharing my you know I mean I've shared my entire upcoming book with AI and gotten feedback on chapters and different things and like not that I agree with everything
AI says but the ability the psychological impact of negative feedback is a real factor right it's one of the hardest parts of dealing with it how do you how do you manage it emotionally regulate and then be able to sift out data that's worth acting on from you know from the noise and and AI is like it's an incredibly powerful like there's almost no I feel almost no emotional resistance to it and I think as in my correct there was some kind of study or something about people being willing to
like put their stuff up and and and work with it more or is that yeah we're finding this kind of algorithmic joy of like also personally because it's so sniveling it's like oh you're so great but like maybe these ideas would be good ones you're like I am so great and thank you for recognizing my genius and giving me these suggestions right so you know and we find me by the way when people the kind of AI we had was algorithmic where it was just telling you yes or no
this is running the map like doctors hated using it because it would tell them 83% chance of cancer and you'd be like no I don't believe you I'm better at this and you're not understanding what the real situation is but when you get advice from these
talking large language models they feel more like companions that's only going to grow more right like it's like you know in your genius let me help you kind of you know like oh it's such a well-ridden piece and you're so smart but you know paragraph two
could use some help so part of it is you telling it to be more critical is actually kind of helpful too yeah yeah I definitely how you have to prompt it to to be more critical I think some of the pieces of advice I've again I've read from your book and elsewhere which is amazing where you
like give it a persona right you are a casual consumer of board games that doesn't really like to read rules tell me what you think of this or you are a hardcore player who loves these kinds of games and you only want
the most like competitive and most you know crunchy thing tell me what you think of this and it will take a very good job of like morphing itself into that persona and giving you at least a framework by which to be evaluating your work absolutely and I think that so I think that that idea of like
it fits into the bigger picture too which is one of we have to do R&D on this thing like that I could tell you the opening of people have never thought of its use for games when I tweeted when I tweeted out about the RPG book the CEO of Google retweeted me on it
they didn't even thought about using it for games right so if you can nail this you're in a place nobody else is and one of the things I would urge people to do is figure out what communities are part of that you could talk about this with right like we have to do R&D as a group
to figure out the right way forward otherwise we're going to be like people are going to say you're obsolete but it's really that we're not discovering fast enough use cases yeah yeah no creating a culture where this is you know we can share discoveries where we can be open about this where we're not afraid right like for companies that are not afraid that their employees are using AI because I guarantee you they're using AI whether they tell you are not
like it's just too too tempting of a tool being able to have groups where we can and we can talk about what we're doing without also being afraid publicly right I I definitely have a fear around like saying hey we're going to use a game with AI art because we're going to get lynched by people from the community even though we spend tons of money on art with most of ours all done by real artists but we are considering using it for some projects
that would otherwise never see the light of day I think being able to create a culture that we can be open about those discussions I think is really important to be able to move forward in a way that's you know fair of course everybody and understands the trade-offs but you know that helps us actually leverage the incredible powerful tool that we have available yeah and I think trying to think about how we think about that
to help people thrive in our industries is also important just like you were talking about there right like so you know you want to support artists because they're your lifeblood and because you care about art what these feel what we're doing in these fields but we also want to be in a situation where you can build games with you know the kind of budget and you know the capabilities you have and you need to keep
you need to keep everyone in your organization fed too right and you know like cost matter so it's it's this kind of logic that's hard on both sides and I think we have to work together to figure out what thriving looks like what do we how do we make sure artists continue to do their work
and don't get stuff stolen from them and how do we make sure that the high end has room for artists doing the kind of work that they're doing maybe the kinds of work they do shifts a bit and the kinds of commissions they do shift maybe instead of doing you know expective art they're doing
one set of projects where they're monitoring art for many other people maybe instead of having you know there's a lot of like what we do that came from constraints right like you know having a single portrait picture of each of each you know character and on a character sheet suddenly I can do 10,000 portraits what does that do for representiveness and what gains can I get from that having an artist oversee that process instead of overseeing drawing a single picture you know like
what do I there's there's questions we have to answer and we need to start answering them as a group or we're going to get divided up and it's going to be the lowest cost winner who wins and everyone else is going to lose Yeah and I and I love I love seeing what's possible at the boundaries of technology and design right like this is what we've done you know throughout our process as a company what we've done with you know
so for fusion as a hybrid deck game we're using what you know digital technology printing technology that wasn't available a decade ago and we're having it you know connect to a digital game we're having it connect to blockchain technology we're having it you know we played in worlds of VR like AI clearly allows for the types of games designed that didn't exist before I know a lot of companies I've spoken to that are using it for you know
NPC generation and for helping to create you know more free flowing content but you know what does this mean in the tabletop world what's this mean in other genres what's this mean for procedural you know generation
and different aspects that previously had to be pretty formulaic that now doesn't have to be like the the core of what we find fun doesn't change but the ways of which it can express and the ways and the tools that are available for designers is just something I'm un I'm incredibly fascinated by
and I can't wait to both explore myself and see well how what other people do with it right just more fun games is a good thing yeah I mean I think about this in the video game space which I'm pretty familiar with right and you think about what a team like a you know a remedy or something that did Alan Wake and control where every detail they can afford to make every detail matter right what's on a whiteboard matters and if I'm if I'm doing this in any game right I have to
have repeating textures and like I can't do what I could do before so we make a virtue out of the problems right it's like oh well it's very clever because I've only used pixel graphics not because that's my artistic choice but because that's a much cheaper way to do the art right and I think a lot of what we're thinking about as artistic choices are choices with constraints and when they can train shift that gives us new choices
for art that we can keep doing the same kind of labor pushed to a new bleeding edge of what that looks like right and I think changing the constraint structures is super interesting way to think about how creativity changes yeah no I think that's great I'm okay I want this one I know we're we're running a little short on time I want to talk about one other topic that is of interest which I don't know how much you've you know I don't know if you've covered this as much but
the you know not only is the I continually getting better at every kind of skill set that it has and all the different pieces that it can contribute to but it's also getting better just feeling like a person like feeling like someone that you talk to someone who you can be in relationship with and so this to you know in the context of obviously NPC characters in context of NPC social relationships of you know having different kinds of things this feels like
this is just the beginning of where we're headed do you have a sense of like as these things become more personified what that does to our culture what that does to our to this next chapter in society we have no idea right I mean early evidence is you know the people who use this and use like things like replica to so far have been fairly desperately lonely the early evidence which is qualitative studies that you have to take with the grain of salt have suggested
it makes people less lonely and actually more social because they get their anxieties out with the replica you know and then they they are better in real life but we don't know like we're kind of unleashing another experiment like social media that's kind of
unstoppable at this point of like what will we do my impression is that you people will have relationships with AI have various kinds both big capital R and small R relationships but that those will be balanced with other kinds of human relationships I don't think it becomes
so addictive that you only want to talk to the AI all the time but we don't know right and we don't know how who's going to use in what ways now from an NPC generation perspective the interesting problems becomes how you can strain it to be on you know
and not so it's not like you ever met a person I fell in love with the first bar you know with the first bar tender I met in the first village is sort of like you know and like we talk for seven hours and like I forgot about the dragons is so interesting problem right so I think we have to think about those kind of things yeah yeah I think it's like you know yeah you mentioned yeah social media is this sort of you know unleashed platform where it's like incredibly powerful you know
I'm able to stay connected with people all over the world but we have seen you know increasing rates of depression and challenges especially amongst youth and young girls in particular you know how this tool plays into that positively or negatively is something that yeah it's it's obviously nobody knows where it's headed but it is interesting and fascinating to speculate on I mean part of this is we could talk about all we want this is already going to happen I mean the open source tools
are already out there in the world and our legal will be evenly restrained and restrict use in the US which I don't think is going to happen in your future will still be available from other countries for you know like that's done already there's a lot of change baked into the system that we're just going to have to live through and come with social norms and laws to deal with the consequences of them but I mean I don't think there's a realistic you know I one thing I sometimes see
in the creative world is a this thing will be destroyed like something is going to stop it and I think that's kind of the most like there are negative effects we have to be honest about it but I think that the next step of like you know like I hear people say let's talk about model collapse the idea the AI is going to train its own data and then collapse that doesn't look like what's going to happen at least in the next few years training on synthetic data the AI
creates is perfectly fine I think people hope that the copyright cases against you know AI system are going to shut it down unlikely and even if that happens then they'll move to Japan where there's no copyright restrictions on AI training material I just like I think we have to be clear right about this the social pressure system campaigns don't use AI or we'll turn on you I don't think that's a long term feasible thing I don't think most people care that much and they're going you know
and even if they do it's still going to not matter as other people like consumers are not going to turn on AI systems the way we expect governments cannot regulate the way we expect we can talk about these things as if we want change to happen and I totally get it but I worry that there's a lot of people with a blind eye of like this will go away somehow and I think they sort of view you know the early stages of like some other technical moves like everything is going to you know
I think it's still room for sort of blockchain stuff in games but the idea that everybody is going to be a hundred percent blockchain based for all get it forget trading card games it's going to be you're going to have a portable gun between video games like that that stuff you know the fact that blew over I think people might take too much lesson from in some ways yeah that's right I think there's you know this I think there's the sort of common like hype curve right where people think
oh my god this is going to do everything and then there's this disillusionment and then there's like oh okay wait here's the real value right you know the idea I think I think well so I did ask GPD4 I gave it all the hype curves for every other technology and it found that there was no actual pattern of hype curve I think that's absolutely right I mean like there's you know I mean I think you can always find curves of like people are really excited
then they don't find it's as useful but then that's another case of coping right of like oh you know it look it's not that useful here in this case in this case but there's plenty people who it's hyper useful for this is the faster technology we've ever seen like it's not this genie is not going back in the bottle and even if it's like oh look it can't create entire our standards of what AI can't do is a disappointment of changing rapidly
from like oh it can produce a pro so like it can't write like Hemingway from like it can't do you know character you know original character design to it can't do original character design as good as the best people on the planet right right yeah we've never used to go yet yeah you know I remember everybody making fun of the AI because it couldn't do hands right it was kept making fun of the AI's handshake you know capabilities and then it was like okay well within three months now
that problem is solved it's sort of like the I feel like you know back when you know kind of the you know religious doctrine started getting pushed back as science sort of explain more and more things it was like well okay this is going to be over here now this is going to be explained over here and the this shrinking kind of rearguard action just as is doomed to failure the god of the gaps it's called right where you just are you know like whatever you can't explain that's what the AI like
and I think that's a dangerous thing for humans like we have to kind of like one of things I talk about in the book is you kind of need an essential crisis to work with AI it is weird it is upsetting like we're kind of talking like it's no big thing then it's creative it's up for some people are listening they're like no it isn't it is not and I totally get that and I would just urge you to spend five or six hours with with you know cloud 3.5 or with you know or with you know GPD 4 if you
use the free GPD six months ago you have no idea what these systems can do and you're just going to have to get through the crisis like we have to figure out a way forward because these systems are not going away yeah yeah I think that's I think that's right I think it's like you know we should be moving forward with open eyes understanding that any industry disruption we should be trying to make sure to protect you know the parties that are getting hurt
and try to be conscious of that and I'm all for people disclosing you know when they're using AI when they're not people can opt in to choosing products that they want or don't want but the idea that this is not you know I think it's crazy to to poke your head in the sand and pretend this isn't a thing that's transforming the world you know
even if the technology literally froze today and never got a day never got any better it will still transform the world over the next couple of years as adoption spreads and that's absolutely absolutely and we can we can feel however we want to feel about that and there's a lot of room for social critique and a lot of concern over what this means and you know
what this means for our social systems and what this means about capitalism and what we do next but I also think there's a practical aspect of like we just got to get like we were living in this world now and we can pretend we're not living in it but that's not going to be helpful yeah yeah and I mean when it comes to like you know just a speaking back just to creativity for a minute like
it's hard for me to really entertain any of the arguments that AI is not creative I mean when I talk about what I break down creativity too in general it's taking two different ideas and combining them together in a way they weren't combined before you're just like taking these different fragments and putting them together and that's all I have ever done in my career and then occasionally filter away the stuff that sucks and find something that's good
and it seems like exactly the way I works you know just from a pure practical standpoint like it's just it is it is I cannot I can't understand any arguments it says it is not a creative force because it just it's like it's it's incredibly powerful tool for that
and there may be some categories of creativity of true breakthroughs it may or may not be able to do but as you put out most creativity is combinatorial like work of binding stuff together you know there are occasionally run into these get you know these games or
it's like how would anyone ever think of that but in most cases like oh yeah this really innovative idea came I can see the origin of it in these other three ideas right and we look at how we describe games to people it's like chess meets magic the gathering with the smattering of you know of monopoly or whatever like that's a
recombination all right even I I always love our conversations I think it is amazing to be able to you know see your whatever you had a brilliant and deep insight into creativity and work and and the fact you know the sort of frontiers of entrepreneurship long before this new
frontier of AI has pushed you into the limelight I think it's a it's well deserved and exciting to see I encourage everybody to not just pick up your book intelligence which is the again I think by far the best most practical breakdown of how to think about AI today but also your your
sub stack where else could people go or what are the tips would you have for people that are you know have been convinced that this is something they need to be paying more attention to for both your work and AI in general.
So you know for better or worse Twitter or X or whatever you want to call it is where a lot of the interesting discussions happening in this space but also whatever your game design spaces I'm part of like three or four different communities I'm sure there are others near you like you need to be talking with colleagues like that's always been important and you know to success anyway and one of
you know just one thing you're amazing at is network he part of why this podcast is such a success is there's so many people know and respect your work and you treated well in the past so they want you know they are excited to be part of something you do for people starting off networking is the only thing that is just guaranteed is it help you succeed across any
industry and the more you bridge gaps between sets of people and the more network you never you do the better off you are so my advice is the same I give it everywhere which is figure out your community that you should join and then join that community and talk about this be someone who's helping lead the direction exterior experiments working public as much as you can and you'll get a lot of part of the way there.
Yeah that's fantastic and I will for anybody wants to come join our community we have a discord active discord for stone blade where we talk about some of these things we also will be at Jen Khan you can come meet us in person there I love to have these kinds of conversations both in real life as well as online Ethan thanks so much for taking the time man I know you're busy and this was such a fantastic conversation and I look forward to to many more as we see this future unfold.
Thank you so much and I'm thrilled to be part of the conversation as always and hopefully the recording work this time. Thank you so much for listening I hope you enjoyed today's podcast if you want to support the podcast please rate, comment and share on your favorite podcast
podcast platform such as iTunes, Stitcher or whatever device you're listening. Listen reviews and shares make huge difference and help us grow this community and it'll allow me to bring more amazing guests and insights to you. I've taken the insights from these interviews along with my 20 years of experience in the game industry and compress it all into a book with the same title as this podcast.
Think like a game design in it I give step by step instructions on how to apply the lessons from these great designers and bring your own games to life. If you think you might be interested you can check out the book at thinklookagamedesigner.com or ever time books or something.