The Internet Is Secretly Powered By Billions Of Tiny Auctions

for websites to actually run? Yeah. Basically, it's like if you think about virtually anything we do on the internet, if you think about at least some aspect of our careers having written on the internet, if you think of all the social networking, literally everything in some way, it was probably paid for the online advertising. Yeah, I guess you're right. I mean, none of these things are really

provided for for free. And I guess the cliche is always if you're not paying for the service, Uh, then you know the product they're selling is is probably you and your personal data, and they're selling that to advertisers, right exactly. And of course, as we know, a lot of the practices and online advertising these days are pretty controversial, and people are concerned about privacy and as you said, your personal data and the reader or the user being

the product. But have you ever wondered, really how they figure out what ad to serve you? In the split second you go to a website and there's already an AD that's perfectly tailored to your interest. Have you ever really explored how that happens? I'm not lying when I say I have actually wondered this specifically, I've wondered why the pair of shoes that I looked at for like five seconds two weeks ago follow me around the internet from months later begging to be bought. That really annoys me.

So if this is what we're going to talk about, then I'm on board, Joe. But what does it have to do with markets exactly? Well, it's uh, first of all, this is exactly what we're gonna be talking about, So you're in luck. And of course it's a market thing because someone has to buy and sell that ad inventory and essentially you go to a website or go to Facebook or whatever, and there has to be some sort of process for allocating that advertising space and how much

are people going to pay for it? And how much is that worth and how much are uh they is the website going to sell it for? And that is a marketplace. So the Internet is fundamentally run by market structure, is what you're telling me. I love this episode already exactly. Well, I'm very excited about it. So without further ado, I want to bring in our guest. His name is Option Bdelli. He is a systems engineer for online ad platforms, and we are going to talk about market structure as it

pertained to the world of online advertising. Thank you very much for coming on. Thank you, and hi Tracy Hija. So what happens when I go to a website? Wow, let's just let's just jump right in. Great, so today on average for the average website, let's go through what happens. So you load up your favorite web page, whatever it is, let's say bloom Bloomberg dot com of course, and chances are you visited Bloomberg dot com before. This is not the first time you visited this site, so that's important.

We'll get back to that in a bit. What happens the moment you load the page there is a race. People want to find out who you are and whether it's worth their time to show you an AD, And the hope is if they show you an AD, maybe you will in some way engage or convert or respond to that ad in different ways, and there are different

methods for measuring exactly how you convert. So the whole game at this point is to show you an AD as quickly as possible, that is, as revenue impacting as possible for both the person purchasing the ad and the purchase the person selling you the ad, the publisher and the advertiser. So I have a bunch of questions already about how the buyers and the sellers kind of come together.

But I guess before we get to that, can you describe what kind of personal data is available if Joe or I, you know, actually click on a website, be at the Bloomberg website or something else. Sure, so we can divide this generally into three buckets of data. Let's say there's the data that we know about the user as as soon as they visit the page, the data that is live that is part of the request. Then there's the data the Bloomberg in this example has from

your past visits. And then there's the data that the advertiser has purchased from a third party that has been collected from you over the course of months and years, from all sorts of different sources. The party that is in charge of providing this last piet of data, which I'm guessing is the most interesting to you, is called

a d MP, a data management platform. It's the job of these DMPs to find out as much data as they can possibly get about you, and aggregate from all sorts of different databases and data sources, and correlate it to the information that is returned to them by the publisher, in this case, Bloomberg. So I have a bunch of potentially different interests. I'm interested in financial market, I'm interested

in boxing, I'm interested in barbecuing. So in theory, there's a lot of different types of advertisers, very different from each other, that would might want to compete for my attention, that might want to put up some product that they're selling in front of me. So how what is the process by which that collection of data that the d MP has on me turns into someone putting an atom from ah? So that is the job of the DSP,

the demand side platform. Are you enjoying the outfits? I I am already I have a feel like there's more to come to there are Okay, So the demand side platform partners with several d MPs, and it's their job to look at your incoming data, your page visit data that happens when you visit bloomber dot com in this one moment and correlated to all the different data it gets from the d MPs. And how they go about

that is sort of proprietary. It's the special sauce. But imagine they have just about any imaginal data stores that they can possibly call on you based on your email address and how who you've given to you in the past, cookies that you've often websites in the past, even location begins from from brick and mortar real life stories that you visit in the past, and they're correlating it all and that single moment while you're viewing the page in order to figure out which ad is best to serve

to you. It's weird to think there's like a behind the scenes battle happening every time you open up a new web page. I'm wondering how how did this structure actually come into place? Like who made the decision that this was going to be the way um that the Internet essentially works and that advertising is sold online. So let's go through a brief history of how the market micro structure of online advertising evolved. Those are words I never thought I'd be saying before, but here we are.

So in the beginning that this is why they welcome to the outlaws. So in the beginning there were advertisers and publishers. So in this case, there's let's say Bloomberg and someone who wants to advertise on Bloomberg, and around let's say twenty years ago, was fairly easy for these two parties to find each other and match up. There were not that many people who were advertising, and they're

not many people publishing websites. But more and more advertisers and more and more publishers makes this market microstructure really inefficient. Imagine the equipment of an open outcry pit, but only about five people in each pit, scattered all around the world. How to buyers and sellers match up with each other after they've reached a certain volume. The solution to that first became advertising networks. This would be an aggregation of

advertisers under one umbrella being presented to publishers. That solves one problem, how do you get more supply to satisfy demand, but it exposes another problem. To give an example where that problem is, let's put some color on what ad we're serving on Bloomberg dot com. Let's say we're serving an ad for men's shoes. So if I present that add to Joe when he's being a boomer dot com, there's probably a higher percentage of a chance that he's

going to convert actually engage with that ad. Then if I were to show those same men shoes to Tracy, who I'm guessing is not in the market for Adidas right now, so we need to wait now. So we need a way to solve that problem. We need a way to actually show Joe the ads that he cares about and Tracy the ads that she cares about. So that led to add exchanges, and that in turn became

demand side platforms. The evolution how we went from ad networks to add exchanges to demand side platforms basically came from the recognition of this problem that although we have the ability to match up advertising publishers, buyers, and sellers, neither of them really know how to match up the

best inventory with each other. They know how to give each other, you know, a certain lot size, a certain amount of certain quantity in terms of number of ads are in terms of revenue spent, but they don't really know how to connect to Joe to give him the best add that he wants. Tracy, I'm listening to off

and describe this. I've been thinking back to some of our early episodes with Chris White talking about bond market structure and just the sheer multitude of potential players in the game and varieties of bonds that people could be selling, different coupons, different times, different structures, and the sort of

complicated challenges that that poses in terms of bringing everyone together. Yeah. Absolutely, I wonder in that case again, how they're able to do it so quickly that we don't even notice that it's actually happening in the background. So Offiction, what's the

technology that's actually allowing people to do this? So the technology is referred to as you ready, Joe, another acronym RTB Real time didn't and it's meant to happen as quickly as possible to connect advertisers and publishers through supplies lie platforms SSPs and demands like platforms DSPs. Why it's so fast. Well, there's an interesting wrinkle or two there.

First off, the publisher always has the option of showing you a house AD if they don't get a proper real time bid that they can match open in time. The reason you might want to show a house ad versus a targeted AD is because you never want to leave a user waiting for an ad load. I'm sure you've all had the experience of visiting a page and waiting for to load, and waiting for to load, and when it does, you get a pop up or something

more innoxious. Generally you want to avoid that. What URTV allows publishers to do is present the best possible targeted AD. But if they can't do that in time, they will just be happy to show you any ad. Maybe they have their own personal brands they'd like to show off to you. Maybe they have a preferred private partner who's as they have an inventory that they would front and run in front of their RTV ads. It really depends on the publisher. So explain to us the auction process

real time bidding. It's essentially describing an auction, So to walk us through what is that micro second or whatever that or that add is delivered. How the bidding works? Sure? Okay, so let's walk through our example again of Joe visiting Bloomberg dot Com. Joe visits Bloomberg, Bloomberg sends Joe's data to their SSP, their supply side platform. The SSP in turn transforms this data into a bid and passes it off to the ad exchanges. The ad exchanges in turn

pass this off to their d MPs. Are you following the microxtra platforms? Sorry? I say d MPs and a d MP DSPs Even I get confused. The d MPs are the data management platforms. They're in charge of gain the data. The DSPs are the demand side platforms, and the demand side platforms in turn look for the best inventory the bids that match up with the eyeballs that they're getting from Bloomberg and try to make the best match. It's the DSPs side too, or a job to service

sort of the match engine here. So this all goes through three counterparties and end if you're counting. So I'm curious what pricing actually looks like under this system, because if I think back to old style advertising, um, and you know, please let me engage in some media naval

gazing here. But if you were selling newspapers, um, for instance, you would know roughly what the audience for your newspaper circulation was, and you would have a decent idea of you know their income levels and where they live, and what they like to spend money on and things like that. But now these online ads are so targeted, does that drive down the pricing because you're not You're not really targeting like a big swathe of the population that you're

hoping might buy your product. You're actually targeting a specific person. It actually seems to drive up the pricing because you don't really want to flood the market with ad volume that most people won't engage, and you want to hit the lowest number of people because remember there's a ton of technical effort going behind all this. There are servers that need to run and engineers that need to be paid,

and all that. You want to put the least amount of effort into pushing your ads to the most the people who are most likely to engage with them. You never want to have any extra ad target hit that hits someone who's unlikely to click it. Ideally, not think about it like this. Let's say you have a thousand ads that you want to push to people, you have

the choice. And this is how it used to work about ten or fifteen years ago, of just agreeing to a rate upfront with a publisher and saying these thousand ads will sell for ten dollars and leaving the publisher in charge of running those ads as fast as they can sell. And some people will be engaging with those ads Monday around two pm, some it's Saturday four am in the morning. Some from the United States, some from Indonesia. They'll all get the same rate. That's really not efficient

from an advertiser's point of view. They're really not getting

the most bang for their buck. So what they would like to do instead is push to people who they know as much information about and for whatever reason, they've decided have the highest confidence and their ability to convert with the add Now I'm curious about sort of you know, if we really dive into the sort of financial market aspect of this, are there players in the market that attempt to for better lack of better warriors engage in arbitrage buy up cheap, buy up inventory that they think

they could get on the cheap, and then sort of repackage it in some way and resell up more expensive like essentially, you know, essentially become traders. Absolutely, And there are several different arbitraged strategies, just as you'll find in any other market microstructure. There are plenty of companies who are in the business of taking in raw ad supply and sort of sharry picking what they think is the

best possible converting ad. And on the other side, there are plenty of people who in the business of aggregating eyeballs and only presenting the best possible eyeballs to vertisers. Fortunately, I'm not sure I can give any of their names on air. How profitable is that business? Then, like can you make a ton of money out of it? Given that the pricing is quite low, it is extraordinarily profitable

in the tens of billions of dollars a year. So from time to time you mentioned secret sauce, I think at one point and you talked about this, and you hear about this world of companies called ad tech, and I never am totally clear what they do. And a lot of them seem to rise really fast and fall really fast. So what are some of the competing strategies? Some company comes along and says we have something new.

We have a new way that you can reach reach your clients more officially, or we're going to use AI or machine learning or big data or whatever. Like. What are these ad tech companies? Where are they in the process, and how do they attempt to make things more efficient? So there are really two basic strategies for these at tech companies. One is to make the process faster to present adds faster to eyeballs that are willing to view

them and gain a sort of latency arbitrage edge. The second strategy is to gather as much information as possible from all these different accurate sources and as quickly as possible match it up to the visitor that's trying to view the ad. Most add to companies focus on the second strategy. There in the business of gathering tremendous amounts of data, correling it all very very quickly, and presenting it to you quicker than the page will load. I'm

curious about the data gathering aspect. So let's say you and I or the three of us here, we're like, okay, we want to start a new ad tech company, and where are we how do we start gathering that data, like where are there wholesale brokers of it? Or that can we collected ourselves? Like what's the process there? So first we were probably partnered with several publishers like Bloomberg for example, to go back to the beginning, we would

also partner with third party data providers. We would want a combination of raw fresh data coming in from new visits so we can build our own database, and we want to have the ability to correlate it with as many pre existing data sources. You probably don't want to

reinvent the wheel plenty. There are plenty of public data basines and four paid databases out there where you can find out, for example, from your email address or from your IP address, what country you're in, what city you're in, what sites you've signed up for the past, what your LS you visited in the past, what time zone you're in,

what language you speak, all that sort of stuff. So what's the most lucrative area when it comes to this ecosystem of online ad selling is that the data collection? Is it? You know, the underlying technology of the bidding system? Is it if you're you know, arbitraging big blocks of

potential eyeballs to sell to people. What makes the most money. So, judging from the market cap of the at two companies out there today, I'm going to guess it's a combination of selling the ad and also taking a bit of the cut of every ad that's sold. So good example here might be Google, which most publishers use as a demand side partner. So Google gets their money two ways.

If you're using their product DFP double clip for publishers, if you're connecting to Google through DFP, they get a cut of every ad that is served through DFP, but also they have the option of serving ads from their own inventory through their exchange addicts. So for an analogy, think if, for example, on the COMEX, you both ran the exchange and also ran a gold mining company and offered a gold etf on the exchange, you not only get a cut of every trade, but you're sourcing inventory

that seems to be the most profitable. So Google not only runs the exchange, they also are participated on the exchange and are selling a raw commodity on that same exchange, and so they make money multip both ways from that trade from that environment exactly. In fact, they even it sounds like Google's pretty good business, yeah, which I thought of? That is Facebook similar in that respect, they have a similar business. It's slightly different for social media, but similar.

So I'm curious about the matching of the right add to my interest because maybe I'm interested in buying a yacht or a new car, some very lucrative thing, or if I clicked on the ad someone could make a ton of money. But I might also be interested in buying a book from Amazon that you know, it's probably the margins are small. So how does it balance those disparate potential outcomes to figure out which one is the best to serve? Well, the best way to describe it

would be the free market. The DSP is connected to several advertisers. Each one of them is willing to bid on your impression. So it really comes down to is the yacht seller advertiser in the bookseller advertiser? How much do they both value your eyeballs? Maybe the yacht seller thinks you're not really convert on an ad because he's looked up your date and says, oh, well, you know you live here in this time. I've never bought a

yacht before. Yeah, so it's say that you probably would in the future, but you've probably bought plenty of books, so maybe the yacht seller is not going to fight. It's harder for you using the price mechanism than the bookseller would. So if in one case, let's say the yacht seller bids fifty cents for your eyeballs and the bookseller bids a dollar the books, that would win and they'll end up paying fifty one cents, just enough to

beat the yacht sellers bid. Got it, So the bookseller might end up bidding fifty one cents and on a transaction that you know, maybe we'll make them fifty five cents or a dollar or something like that. Right, And the idea is you will probably lose money on most of these in aggregate, but the winners are real winners. And the odds seller occasionally is going to get that ad and it's probably going to serve thousands and thousands of ads that don't turn to anything, maybe millions, but

then when they do, those are huge jackpots. Yes, and I'll add one more wrinkle. We're talking about ads that ads come in many different forms. This isn't just a race for eyeballs, could be raising for video ad views. Or application installs. Even so I have a sort of related question, um sort of maybe even a philosophical question. But so much of the online advertising world seems to

be about targeted advertising. So does anyone just you know, throw stuff out there nowadays just on the off chance that someone might see a good product and want to buy it to Joe's y odd example, you know what if someone has a really really nice yacht and they think people will buy it if only they were aware

of it. Sure that does happen. It's also pretty rare because in order to actually get a high enough conversion ratio for this to be worth the advertiser's time, you really have to buy a tremendous amount of volume if you're not doing any targeting on who your customer is, and now she might be putting up a billboard versus sending a direct mail flyer. If you don't know much about the person who's engaging on the other side of the bye, who knows if they're going to bite? Is there? Though?

Like sometimes you see like a huge site takeover, like on the New York Times or something like that. You'll see some massive they'll buy every ad on the website or on the front page or something like that. It's clearly not particularly targeted. There is still some sort of like brand advertising on the internet. Right, Yes, those are private auctions, and those are great, but they're really only great for publishers who can demand that sort of leverage.

For bloomber dot com in this example, that would be a great example of the site that would have enough cashier to pull a private auction. I can imagine plenty of brands would want to have exclusive access to the eyeballs at the basing dot com. My personal blog probably not so much. I'm probably not gonna be able to land that big deal with Pepsi or Coca Cola to have them take over my site for the next thirty days.

And that's something that would be negotiated among humans, and there would be a deal maybe like there was just the big ad conference and canned or people like partied on yachts and stuff like that, and those are the kinds of deals that get negotiated there rather than through some sort of like algorithmic matching engine. Right, and think about it if you will. There's the difference between buying at the market price on an exchange versus setting up

an OTC Trade. We like to talk about changes to market structure quite a lot on the Pots podcast. Are there any big changes or big pressures or disruptions on the horizon for the current structure of the way these online ads are sold. I'm glad you asked. Let's talk about header bidding. Header bidding. Header bidding is what seems to be what's going to come next after real time bidding, and let's explain what it is and what the motivations are behind it. I mentioned Google's DFP double cliff for

publishers before as an interesting feature. First off, most people most of the time use Google for advertising, just in the same way that most people most of the time use Google for search, and most publishers are content to just trust Google to sort of drive the process and they don't really dig too much into it because Google wouldn't always serve an ad that seems to satisfy both

the advertisers and the publishers. The reason, in part that Google is able to do this, besides the fact that they're so great at aggregating data, is what's known as the waterfall. Here's how it works. You call out to Google, you asked them if they can give an ad to you so you can serve it to your visitors. Google goes through their inventory to see who the best ad

is and where it's coming from. They have the option of serving it from another exchange or their own exchange addicts, so they have the ability to front run basically any other advertising ex change if you allow them to. And because Google is where all of the publishers show up to, all the other exchanges sort of put up with it or forced you. Header bidding is meant to change that.

Header bidding is different from real time bidding, and that instead of where in real time bidding, where a publisher calls out to a DSP and waits for a response, header bidding allows the publisher to call out to multiple DSPs all at the same time and select the best bid incoming themselves. So you lose the convenience of just letting Google drive, but you gain the price competition between all of these different bidders. It sounds obviously better for

the publisher. So why is that not already the norm? Well, because Google doesn't like it that way. Most people most of the time are actually not as enamored with his market mixed structure as you and I are a publisher is probably the business of publishing content and doing whatever their core businesses. Actually managing and driving this process yourself

is really tricky. In order to for example, for head to be to be really profitable for you, you'd have to actually, you know, care about those bids and put some thought into whether you might for one partner over another. Would you need to build your own technology to determine If you're getting multiple bids from multiple exchanges, then it's I guess it's on you to actually determine which one is the best. Whereas if you're just on one exchange,

then that determines what's best. And so I imagine to take some more supply side infrastructure to build that up. Exactly. It's the difference between you know, setting up effectually your own prop shop for ads versus just trusting your broker

to disup why you with what you're asking for. Well, that has been absolutely fascinating discussion, and I just love the fact that the kind of market structure that topics that Tracy and I talked about frequently on this show are actually at the root of how the entire Internet works. And I didn't even realize that so I'll shooting Bedelli. Thank you very much for joining us. Thank you for

having me, Joe Yea. So, Tracy, you said that you've been thinking about online ads and you've been wondering about how they get served up to you. Do you feel that your questions have been answered? I mean, I definitely have a better sense of it now, but I will never look at a web page loading the same ever again. You know, every time I go someplace, I'm going to be thinking about the intense auction process that's currently happening in the background with a bunch of people trying to

bid on my specific profile. It is pretty amazing. I mean, sometimes websites or absolute load a little bit slower than you would like, but even with that, it is pretty amazing. How much has to happen in that short period of time. You don't even notice it, but all this information goes out, it's analyzed, it's put into some profile. There's a bitting habit, a bitty war happens in an ad is served to basically in an instant. I don't know if it's exactly

a miracle, because it's kind of creepy. It's it's kind of creepy, right, but it is kind of amazing, isn't it. Yeah, I guess it gets to a point that we've talked about before on the show, which is also about inequality and the ability of algorithms to sort of reinforce a

certain position that a person is already in. So, for instance, you know, if when you're twenty four years old, you need to pay day loan to survive until your next month's paycheck comes in, and you search online for payday loans, those ads might follow you around for years and years to come. Um, when other people who have never needed to pay day loan might see, you know, advertisements for

four one case or things like that. I've never really thought of it like that before, but I've been thinking. We had a whole episode on this, Joe. No, no, no, absolutely, I never thought about that before with regards to online ads specifically, But I have been thinking about how like I went through a phase where I needed to buy some clothes. I need to buy some shoes, and then I got served tons of ads for weeks and months

on and on similar stuff. And then I started worrying that, wait, am I buying more than I need to now on these things? Because I went through this period where I made these purchases, and so thinking about the sort of lasting impacts of a certain behavior on the type of ads were inclined to see for a long time is really fascinating and absolutely does speak to that discussion we

had earlier about the influence of algorithms on our lives. Yeah, you cannot escape your Internet history, no matter how hard you try. All right, Uh, well, this has been another episode of the Odd Thoughts podcast. I'm Tracy Alloway. You can follow me on Twitter at Tracy Alloway, and I'm Joe Wise though you could allow me on Twitter at the Stalwart, and you should follow our producer tofur Foreheads on Twitter at Foreheast, as well as the Bloomberg head

of podcasts, Francesca Levie at Francesca Today. Thanks for listening.