Today, on episode number 787 of cxo, talk, we're discussing data and intuition. The role in marketing, we call that quantitative marketing. Our guests are two folks from the Columbia business school at Columbia University in New York City ODed. Netzer is the vice Dean for research and a professor and author of the new book decisions over decimals and Amy Jake is the chief marketing officer of the institution for me marketing.
Is that part of any organization that is on charge of facilitating and enabling successful transaction with the customer successful? Whether it could mean profit if it's for profit could be engagement for example if it's non profit but the ones who are in charge of making sure that such transactions relationships do exist and and now you want to layer on it. The quantitative part of it in order to create this. Successful interactions.
I really need to understand if under firm who's on the other side and it's not exactly the other side will have my partner's, my customers. And for that, I need a lot of data, a lot of information about my customers and who they are and good understanding of who I am, what am I capabilities and that again requires data and analytics. And specifically the analytics is often done in order to try and understand this match the met with the match between the
customer. And and they the firm and what the firm has to offer and even changing what the firm has to offer to make sure it matches. What what customers want?
Particularly in, in, in, in recent years where a lot of the effort went over Z, he's getting better and better understanding of the customers because the better and the more we understand the customers, the better, we can actually serve them, what it is that they want, and that requires a lot of data, a lot of analytics and a lot of quantitative marketing. Amy, what is the impact of this data driven approach on your
work right now? There was a period of time where it was somewhat difficult to explain to senior leadership the value of marketing. The increase in the data that we have, the ability to show quantitatively, and to connect an action that marketing is taken to a bottom line impact. That's what a lot of the data has done. It has allowed us to be better marketers, we understand a lot more. So dead was saying about the audience, we can engage in different tactics and strategies
personalization is so important. You know, we are inundated with material and it is very difficult for us as humans to have that, much flying at us and to be able to figure out what to pay attention to. You know, cognitively, if there's this influx of information, we have to pick, we have to sort, we have to think about what makes sense for us to pay attention to what limited time do we have.
And how We react to that and in Mass marketing sometimes you can get that benefit with your audience but more often what we're seeing is that the more personalized, that the marketing is, the more likely that person who is otherwise inundated will take a moment will stop and will engage with you. And the only way to do that is to be able to leverage the data that you have to say, this is the right course, for this
person, this is the right path. This is the right Next Step for person a But for person B, it's going to be a little bit different and every time that our audience or consumer or customer has that experience, the more likely, they are to come back because there's a benefit for them. They're getting what they need. They've seen that it works. They know it was worth their time and so I think data both helps us on the strategy side. Develop better strategies.
It helps us on showing results to senior leaders to, to shareholders to stakeholders and it helps Us to deliver a better experience to the customer. So they're all wrapped up in sort of one larger marketing effort. But I think those three components and thinking about it, in those ways is really helpful. I've been teaching the marketing core for quite a few years here at Columbia business. School is probably the course that has changed the most actually em out of any course we
teach. And the reason is that the world of marketing is changing tremendously. And if you think for example about advertising, The advertising. Again, many of us have watched madman, right? And that's that's really the old
days of marketing. But even up to the early 2000's, the center of advertising was here in New York was in Madison Avenue. If you think about what are the three biggest advertising companies in the world then not anymore in New York and there are not any more advertising company companies, there are Google Facebook and I'm an Amazon. These are these companies in Silicon Valley and they sit in, in, in Seattle, which means there are technology companies, and with that comes the
quantitative aspect of marketing. Amy made a very interesting point, she spoke about the enhanced ability to connect the marketing activities, back to impact on the organization. Can you talk about that? Because that's what Business Leaders ultimately care about, right? We're spending. A certain amount of money on our marketing and what is the ultimate impact? And how do we measure that? The one of the biggest impact of quantitative marketing on their
world of marketing? Was this notion of our eye and there is return on investment, right? I mean, will the famous quote and, you know, half of my marketing expenses are? Well, spent, I just don't know which half and then came the world of online marketing,
right? Then came the world of digital marketing and with this promise of we're going to see what people clicked on. We actually going to see whether someone clicked on the ad where the summit eventually even bought some finally, we can get our iron marketing, we know what is the impact on of a marketing on consumers? I think it's true. It we have moved a long long way in the ability to show return on investment on marketing. And we, it is much easier now to have that conversation.
And I think that marketeers that haven't taken that step have truly left behind. And if still struggle with are within their organizations, I will. So the mention that that promise may have been even too strong. It, we now started seeing the pendulum shifting on that statement in the sense that the fact that I see whether customer clicked on an ad doesn't yet fully tell me whether it's a full return on investment, right?
It could have been that this customer would have bought anyway, so we need some sometimes more sophisticated quantitative techniques in order to truly ask questions, like attributional in current through a mentality. And whether we truly have Roi on these. It seemed as if we solve the problem and now we realize the problem is, we have way way ahead, but there are still things to be done, which is where the world of quantitative
marketing works on today. A few years ago, a marketer had to say I see a correlation but I may not be able to show causation and now I think you're right there. Are there still are numerous questions that we have there's still challenges with attribution models but I think we are much closer to being able to say It's not just happening at the same time in, isn't that nice?
We're able to at least connect often some of those dots and say, well, I know that when my marketing spend goes up 13% 27 percent of customers, you know, purchase more or 13, you know my marketing spend goes up 13 percent and 50 percent more individuals enter this online store.
So we're able to do it in a way that you know, if our attribution models are right and we Have the right technology at some point in the path we can say there was a direct connection and we know that it's actually going to make a
difference. And I think directionality is important, but directionality with, you know, some very specific models that show that relationship is what ultimately gets you farther in terms of your budget, in terms of being able to enter into new markets, and really buying into sort of the confidence of the c-suite, I think, which is super important. I think one of the things that truly help with that, and particularly, in the world of digital marketing is the ability
to run these A/B tests, right? Yes. I mean it's very difficult to say, you know, how about my customer going to see one store and a half of my customer going to see a different version of my store. You know, my story. My story is very difficult to play with it in the online World it because it became very easy. And you know I split my time between Colombia and Amazon, I spend some of my time at Amazon advertising, the practice at
Amazon advertising is running. These A/B test these, let's try version a version version B, and when we do that, we truly can get to RI. We truly can get to. What is the difference between offering the customer version a and version B? It's still easier to doing digital environment where we can take, you know, we have 100,000 visitors, we can split them half or any any way. We split them randomly and see, throw to to different customers different different offerings and see how it works.
It's still a little bit more difficult in the offline mode. But even there, we are making progress toward thinking in the, in a language of A/B tests in the language of testing, in order to measure Roi. And in order to measure, truly what customers want? I'll give you just an example of that. Several years ago, I was working at a large company and my role and my team's role was focused around telling investors the story of the success of the
business. And when we had our hypothesis about which creative would work, The best to get them to engage with our financial results. I would have thought one thing and we a be tested over a period of about a year.
So that's four, four quarters worth of data and we actually found that in telling the story with our IP, so the show's themselves when we use that as a creative investors, now, this isn't an audience that you would naturally think would pay more attention to, you know, a character on a show than a financial times headline. But when we Talked about our
efforts. And when we use the creative, when we use our IP, we found that the interest in the engagement, in the information, the financial information about the company increased, you know, exponentially. And so I think to your point that's an a/b test that 10 years ago or 20 years ago, one, you wouldn't have been able to run but to you wouldn't have hypothesized that.
But when we just have the ability to continue to test and swap in and swap out, Out and we see, very clear results in the data, you know, it gives us confidence to both try new things, but then it gives us Security in the decisions that we make. Because we know now that we can see that that happening, please subscribe to our YouTube channel and hit the Subscribe button at the bottom of our website. So you can get our newsletter and we can keep you up-to-date
on upcoming live shows. What about the data? What kinds of. Of data, should marketers be collecting? What's the best way to get that data and ODed? Can you take us a little bit behind the scenes of the kinds of models that you're looking at? And again, my real interest here is the impact of that data. And those models on the real lives of marketers, hey, it sounds like a reality TV show, right? The real lives of ders. We have a fire hose of data, right?
We have more data than we ever had before. It's still the situation that we don't always have the data. We exactly need. We always sometimes, looking for the right data, we need there is this something we talked about in the book that you mentioned earlier, then we call the the certain teammate. There was this belief that finally when we have all of these data will get to the certain decisions. Certainties I me that we still make decisions under uncertainty, but we do have much
more data than we ever had. And Specifically, the data that marketeers value and cherish tend to be data that tells us about customer preferences, being able to measure and understand customer preferences and the reality of Eden. And the reason why it was difficult, it may be still is difficult in some ways is because of heterogeneity, because customers are so different than 0 to customers
that are the same. And, you know, I always tell my students to know they and always Find within my class. This type of students, those who treat the Apple Store, like a like a called. And those who would never be caught dead in an Apple store, right? And the marketeers needs to understand this and he's trying to stand this often with limited data, depending who you are, if your Google or Amazon, you have a lot of data.
If you just, if you are media Outlet, you may have less data about about the customer and who they are and we tend to see unlike. If you go to data for example, in finance, I can always Go one more year and have longer data about this about stocks, right? I can also just do an analysis of the entire Market when it comes to stocks, when it comes to to Consumers. A I cannot do data just on the entire Market because of the heterogeneity that I'm talking
about. I do need to understand each customer in their own preferences and I'm limited with respect to the history because if I'm talking about a travel, for example, if I'm Expedia one of these dry only see EU, you know, five ten times, that's all I have.
So I need to work with that length of history of what I observed about the customer, but and that's where economics helps so. Or statistics, there is a trade-off when it comes to data between how good your data is and how complicated your model needs to be the better data, you have the simpler, the model you can use. If we run the A/B test of a just talked about before. All I need to do is compare average.
Is a customer in the control but that much customer in treatment but that much, that's it. I mean, level of math, that that is, you know, a sixth grader. Could do if I need now to build a model where I am again a Expedia and I'm trying to understand customer preferences from this visit and the previous visits and so on, that's where I need a much more sophisticated type of modeling and type of analysis. The other, maybe distinction I want to make about data.
Is it extinct distinction between structure data and unstructured data until fairly recently, Circa 2010, most of our analysis was done on what is called structure data structure.
Data are numbers data that comes in the form that we've by the way, generally think about data in Excel, or in a table with numbers in it. But if you think about it, majority of the data we have as business people, as marketeers is actually unstructured data, it can Ones in text image, video, Audio customer calls us to the call center.
And we have data in terms of tens of thousands, if not hundreds of thousands of ads that we can, we can analyze ads are images and text, we have company reports, different estimates, depends who you ask, but 80 to 95 percent of data available. For business is likely in the form of unstructured data in text audio video and image. And it is only since 2010 that we actually know how to analyze this data at scale. I mean I'm structure that existed probably since the Ten Commandments right.
We're waiting on a stone but but really they build it or to analyze this data at scale, came with machine learning type machine, learning methods, type tools and and we seeing more and more companies are using it. And, of course, recently, the whole chechi PT and generative AI is an example of Jing, unstructured data, Amy your marketer. I'm assuming that you are not and expert statistician and data
scientist. And so given what ODed was just describing in terms of the quantity of data, the quality of data, what can marketers do to take advantage of these important points? But without Without having to be a deep level statistician. For example, don't be scared of data. I'm not a statistician as you very correctly, pointed out, I never have been.
But I think for marketers, when we talk about quantitative marketing or we talked about data-driven marketing, for some especially individuals early on in their career, it sounds like they're going to have to be gathering the data sets. It sounds like they're going to have to be building the model. It sounds like they're going to have to be interpreting.
And I think we're very lucky in that, we have a whole generation of individuals who understand data, who are data scientists, who are insights and research-driven colleagues. And the more that we can develop very close relationships with those individuals as marketers the better. Because most marketers, I know aren't as you said
statisticians. But what they do know is how to look at data and I know we're going to talk a little bit about this and and say, well, based on my understanding of the audience, this seems odd, let's explore that or they have a question that they know they want to answer. And they know that the data is an important part of that and so they can ask for help in building the model. So the first is, you know, you don't have to be the expert. Don't be scared, don't not use it.
Use it, find a partner, find a colleague, you know, find somebody who is the expert and the second is think about the data in Meaningful way. So you as a dad said, there's infinite data available so it really is about what are you looking for? And then when something and I know Dad's going to talk a little bit about this, when something doesn't feel right, ask more questions and a good example of using kind of numbers and qualitative elements
happened a couple of years ago. I worked in a publishing company and it's very common for people to unsubscribe from subscription-based businesses, including magazines. And so, you build a model and, you know, that churn will be X
based on historical data. And you know, that there might be some macro or micro conditions that will impact that Year's churn and you go out and you acquire customers, knowing this percent is going to leave and you need to have, you know, X percent of new people coming in and subscribe. But as a marketer it's not just about making sure that you can, you know, fix the revenue equation on both sides. It's also about understanding your customers.
So you have this model we have this model, we know that X percent, you know, are unsubscribing. And so then you have to dig deeper to think about why what is it? That what's the reason that you know, they've chosen to cancel this subscription and we could get a little bit of insight in asking structured questions and our Hypothesis was it was a value situation. Somebody decided that that product that the magazine was no longer worth what they were. Paying feels pretty simple,
right? But we actually had done some brand research and had some qualitative components and what the qualitative components told us was that actually had nothing to do with value, it had nothing to do with the product, it was about unfinished ability. So people felt that this particular magazine Worth the price worth, you know, worth had great value, but they couldn't finish it, and they would stack it up in a pile. And they would say, I'll come
back to it, and they didn't. And every time they looked at the pile, it crew and their guilt. So this is an emotional reaction. Their guilt over not finishing, the magazine was actually, what was contributing to canceling subscriptions, so that Insight is not in sight. You were ever going to get from structured data. It's not even in sight, you were to get from some of your survey questions.
It came exclusive these through some qualitative conversations and there were some solutions to that digital. Combated the problem a little bit because people can't finish the internet. So if you can't finish a digital issue of a magazine, you didn't feel as guilty. You know, you can introduce newsletters and quick quicker, B of content and summaries and things like that. But I think the other important thing is I and I want to really Reiterate. This is that there's the
structured data. There's everything, you know, there's quantitative and then there's this other qualitative element. That is equally as important for marketers to pair together and I think it is important that will think about data in the most General sense of the word, a Tamiya conversation with a person is data or just maybe want to touch on one thing with respect to the, you know, when I build the end model statistical model and I truly mean it.
I would I would actually Really go to a me, first to tell me this model is right, before I would go to a statistician to tell me whether it, whether they think my model is, right, because the True Value comes from, let's say it's a model that predicts click click through on an ad and the model showed, you know, twenty percent click-through rate. It's the decision wouldn't know to tell me that 20% clicks. All right, on an ad is unheard of, unless you're Ed truly
offered money to customers. No one. There's not, there's no one in five. Customers click and click on a man. I needed. Cat ears. I need a me to tell me. I don't know what you've done wrong, but I can tell you this is wrong because it's it's impossible. That customers. 20% 1811 in five customers would click on your ad. We have a couple of questions that have come up on Twitter, actually, to folks, Chris Peterson, and our Salon Khan, who are both regular listeners of cxo Takin.
So thank you guys are asking independently. Essentially the same question and that is, where do we draw the line for? Privacy in terms of data-driven marketing, that's from Chris Peterson. And arsalan, Khan says how, and who should strike a balance between collecting data that can affect privacy and who decides, what's private? What's not? And also, then you have the issue of biases.
So can we talk about that? And then we'll switch over to quantitative intuition privacy is an extremely important topic to think about to discuss. To realize that that companies have huge responsibility when we are collecting individual data. I think there are, there are there is the issue of data being shared and then there is the issue of data being used by, by the company. I mean despite what actually most people think most, at least the large companies do not sell
data. Now, I'm not sure it is, by the way, being done because and because of pure Ethical issues of because data is just way too expensive to sail, the use of data by by company. By again, the thing about the top companies like Facebook, Google Amazon is so important and useful for their own purposes that actually selling. It would be, would not be very wise. So generally, at least from these companies data generally,
do not do not flow around. It generally does tend to stay Within These these companies but even there they should treat it with the with the Ability that these data should be treated with and that's where recent regulation came about first party data where you're allowed to use your own data. But you're you're not allowed to actually use someone else's data, which by the way in, and of itself, there is an interesting tweak there. It's on the one hand, it sounds
right. It's indeed protects customers. In terms of their data, being being in floating around but it also increases, maybe the Gap between the top companies that have really good first-party data their own data and the smaller companies that don't have their own data. And that's what that where they could have closed the Gap by getting data, Maybe from, from someone else. I do think that we shouldn't let the cat Garden milk there. I think we do need regulation there, right?
And Europe is much better than the u.s. in that science. And we've made that mistake previously, with social media, where we let the the Guard themselves. We've made it with the privacy and I think we need to move there and I'm happy to see if there is generative an RA. I this conversation is starting already now. And not once this is fully fully out there and I mean, I don't know if you have other selves. Yeah, I mean it going to an end even kind of larger question is, who owns the data?
We're now in a world where our Behavior has been tracked for a very long period of time, and there are reams of data, people probably know that I prefer baked macaroni. And cheese, you know, over brussels sprouts, but who doesn't, you know, that, that data is gone. It's with somebody, I hope I get an ad for macaroni and cheese. I will let everybody know shortly, but I think there's a larger question as we think about a next-generation, right.
Let's talk about kids for instance, for whom maybe there's some data, there's not a lot who owns that data, who should own that data is my child, the holder of that data or is it the platform? Who's on which they're engaging, and there have been these really large high-level conversations around, you know, should I be able to own my data and I choose where when, and how to monetize that?
Or is there a trade-off? I get to go and watch cat videos or, you know, whatever it is videos that I like and in exchange, I'm giving somebody an insight and I know that because I understand that's, you know, the way that a lot of practices work because I'm getting a pop-up that says There are cookies and I'm willingly engaging in a behavior. So I think there's both the kind of what is happening now.
And then how do we think about data the value of data and ownership of data for a whole new generation, little bit off topic. But I think an important conversation for any of us who are, who are using data and whose data is being used in traded, in some way and we need to move too much more of an opt-in environment, right? We're at least, we make the decision of whether to Charity. Obviously, this is a crucially important topic. This issue of data privacy and data ownership.
And we're not going to solve that one now, but we can get a better understanding of is, oh, Dave. Aude of your concept of quantitative intuition. What is that, and why does it matter what even Division? I think this particular is important in the world of marketing. Where again, there is a, the more If you will, they the the unstructured data that Amy talked about or the the conversations that we have, we tend to use our intuition regularly in our private lives.
You know, we talked about surprising thing, an interesting thing. Sometimes we even call them gossip, but we are very worried of using our intuition, when it comes to the world of business, to our to our work life. And the idea here is not to just, you know, trust your gut.
The idea here is You are approaching data, bring in your your judgment into it. In fact, we have to bring in your judgment into it. So on on its surface quantitative in intuition sound like an oxymoron but in I, in fact, together with my co-authors for the book, Chris, Frank and permanently, we believe that not only that, they're not oxymorons, particularly the level of leadership. It's the only way, the only way to make decisions is to combine the data with a good sense of of
judgment. And when I say judgment, I mean, Intuition or judgment. This could come actually, in in at least three stages of the decision making process in the questions we asked in being a very clear about what problem we're trying to solve more often than not, we are just going on a straightening. Oh, we've been collecting all of this data, then got to be something interesting in. Its right, we need to guide the process.
To me, the one of the biggest Nono in making data-driven decision, making is we expect the data to provide both a questions and There's we should ask questions then. Hopefully data. Can provide answers. In fact, we've learned it better than ever since November of twenty twenty with with a 2022. Sorry with GPT. We need to provide good proms. If you want to get good answers,
we need to ask good questions. Second is something you already talked about how do we interrogate data and, and exactly what I mentioned that you do need the context as humans. We are very good in context and particularly, if you are an expert in your domain, as Amy is in, Amy is in in, in marketing, it's much easier for her to look at the model of very sophisticated analysis and interrogated. Not from the p-value of from you didn't use this three-letter acronyms or another, but from
this number doesn't make sense. I don't know what you've done wrong, but I can tell you that that it doesn't make sense. And finally, you're in a crucial place for judgment is in the synthesis of the information generally data and Analysis will tell you the what it would not tell you the server. We need to do it. What does it mean? And then now what are we going to do about it? And these are different components where we want to combine the quantitative together with intuition.
I use the so and learn a lot all the time. After you taught me that it's those two questions for marketers. Truly will change the way that you approach your, your data and your strategies. And in fact we said that one of the reasons we went on this journey of quantitative intuition was we were simply tired of meetings of horrible meetings or meetings that go on the water. And so we had a meeting and someone shows data and now we're going to spend an hour talking
about the water. In fact, there is a Persona, we talked about in the book, we call them the seymours, the seymours in the organization are the ones who in every meeting of only one comment can, I see more data? And then we can have meetings over meetings and postpone any decision Forever by asking for more data by by keep digesting. And, and, and slicing, and And slicing again the data moving from from the walk, right from what's in the data to? What does it mean? Right?
What are the implications? And again this is a place where you know people are even fearing a lot with chat GPT and so on with generative AI what's left for us as human humans. If all you do is the what, yeah, you will be replaced by a machine but if your focus is on the so, what on What does it mean on the now? But what are we going to do?
Not yet. I mean, I'm not saying that machines will Not get there at some point but not at least in the near future are so and calm comes back and he says, as AI continues to be used more and more, do we really need marketing people? If a I can understand the context around that data. Yes. Definitely. You do you do and I'm sure your
dad has a lot to say as well. I mean, where we are in terms of chat GPT right now is I would say it is a Full assistant in many ways, you know, having used it myself, my team uses it, it gets you, I'm making this number up but let's say a third of the way but it's not replacing what we're doing, it's not replacing
the knowledge that we have. And it's not doesn't always get the context and the nuances that are very specific to humans and then even more, you know, kind of granular or smaller there more, Subsets of nuances amongst different audiences. Now, you could argue all of that is data, and overtime, maybe those things could be put into a formula and maybe it could be
understood. But I think we're still a bit far away from saying this will resonate because or from understanding that a word, that means something in this context can be taken in an entirely different way in that context. And that context is, Medic. And so yes, is it helpful for instance, first summarizing some research that you might have. Absolutely. Does it give you something to gut? Check your approach? Can it provide more information? Yeah, it can.
But we're not at the point where it could run this sophisticated campaign for you and get equal or greater results. And I think Michael you you really hit it on the head with it within the day Ward context. I think this is the key. The key is truly context. Both in terms of, by the way, how these methods have improved. So the difference between, you know, GPT to which was the previous version and the GPT three and a half, which is Chad upto.
Now we already GPT for is context meaning and what I mean by that is in order to predict the next word on door in order to interpret a board understand the world with these tools. Do they take the previous that many words to understand in the context of the board and the reason why and it truly was. I mean Church uppity Was a huge leap in Innovation. Over the previous versions is, literally because of context because they had they use usable is called 8000 tokens.
Think about 8,000 tokens, something like 6,000 words because it includes periods and so on but six the previous six thousand words to understand a particular word, the previous versions, for example, GPT to use two thousand words to understand any particular word. Is humans. We are tremendous actually in doing that in in context, I'll
give you two examples. The first example is when I say the word model and in fact, it Michael, when you use the word model, both of us understood that in the in this context, we are not talking about the fashion type. We're talking about the nerdy type, right? Because we have enough context to understand that in this conversation, unless I'm going to give you a real clue with my previous words that we are talking about the fashion type model. We are talking about the nerdy
type. Type right? And and in fact, we don't even understand how good or how our brain actually in understand context. I'll give you an example for that. Think about a toddler at one and a half year old does not have the intelligence to speak yet. And we show them illustration in a book and the book has an illustration of a cat. And we tell them. This thing does meow. And then we show them illustration. The same book of a dog kind of book about animals as we often
read. You are a one and a half year olds and he tells them. Well this thing does wolf and then we show them maybe 45 times illustration, different illustrations from different books and so on the next day we're going to take them to the street. They cannot speak here they're going to see a cat or dog a
real. A real cat and a dog will first time ever they see a real one and they're actually going to be up meow and wolf without even again they can speak yet but they do it with five observations a few years back. The first time that machines went close to human level, in detecting cats from dogs from images and they've done it because the researcher made available to million observations take by humans million of cats and dogs.
How can a human do it with four five observations and a machine needs to million. We don't understand how we do it with only four, five observations. If we did, we would have trained machines to do it because it will be very useful to have less data. And I think that's gives you the the The understanding of why when it comes to The Sword. And the novel words, we don't have enough data of similar. So what an hour maybe if you are a physician and we have enough decisions of the kind given
symptoms, maybe it works. But for typical business decision, which is doesn't come that often. It's unlikely. It's unlikely that at least not with the current tools and where we are today or even in the near future that machines would have enough context is 8000 context. But repeatedly, Only with enough data to truly go to to that step. Again, may happen at some point,
not yet. If you want one place, where it did happen already and happened before generative, AI online advertising, the allocation of Heirs of which atoms. And seeing when I go to Google, or when I go to Amazon or Facebook is almost fully automated apart from the choice of the creative of The Advertiser. Yeah. That's has been already automated because the rules are fairly clear there isn't a Oh so
what does it mean? Well it means that this end is likely to lead to a higher click-through rate, then another ad, that's a predictive model machine. Learning can do that already, how can marketers avoid being data myopic, which is to say losing sight of the fact that business decisions involve people circumstances. We use that term context and not just numbers when marketing has a seat at the table and is involved.
And in the Strategic decisions things like growth or new audiences or understand where value is derived for a company that's when they can put together the pieces and they're not only seeing their small part. So I think being part of having a marketer, truly, truly involved at the senior level, with the seat at the table, who's, you know, just as involved as anyone else in the growth of the company is really important because it then becomes top-down. You ask your team, not to look
at this slice. You ask your team to think about how it connects to something larger. Oh, Dad, how can organizations strike, the right balance between relying on quantitative marketing techniques, versus trusting, their own intuition and experience bringing the two together. Ask yourself as you looking at data. Ask yourself what surprised you And it's amazing. How this fairly simple. Deceptively simple question often cut straight to the
straight to the chase. You're either finding a mistake or you're finding an Insight either way, you benefit. And so, in other words, don't trust your gut. Trust your doubts, look for these days surprises. In the data, Amy, you're going to get the last word here. What advice do you have for marketers given everything? We've just been talking about your best friend, is the
personal? One who can build the models, your you need to be embedded in business decisions to drive True Value. And I don't think that chat GPT, we'll take our jobs, but if you don't know how to use it, to make yourself more efficient, or more effective as a marketer, it will so it's not a substitute but it's an important complement that we all have to be using and working with them on a daily basis. Can I accurately Phrase that as saying, be part of the business, understand the tools.
Understand the data and bring all of that together to make the decisions that rely on both the data and your experience with the business and with the context even better. And with that a huge thank you to ODed netzer and Amy. Jake from the Columbia Business. School Columbia University. Thank you both for taking the time. Time to be with us and share your expertise with us today.
Well thank you, it's beneficial. Yeah thank you by going to thank you for the audience and thank you to the audience. You guys are awesome and your questions are great. Now before you go, please subscribe to our YouTube channel and hit the Subscribe button at the bottom of our website. So you can get our newsletter and we can keep you up-to-date on upcoming live shows. Just like this one. Thanks so much. Everybody, check out cxo talk.com and we'll see you next time. Have a great day.