OK, good. So as I welcome everybody, it gives me great pleasure to welcome back Johnny. Johnny Books by Bartlett, who's going to talk to us about his time working in it in data science since leaving leaving Oxford and the doctoral training centre. Johnny started in 2012 at the CDC in the CIS Biosystems Biology Centre for Doctoral Training and is now in in the real world working on applied machine learning. One minor note where we're recording this this presentation.
So for GDP reasons, if you do not want to appear in the recording and want to preserve your privacy, please turn off your video and audio and you might want to change your screen name just to make sure that we don't capture your information. And if you can possibly remain muted during the presentation, that would be great, too. We'll have a little Q&A session at the end where we won't record. And so feel free to DeCock and share chat, ask your questions and get involved in the conversation.
So good. OK. So without further ado, I'm going to hand over to to Johnny Sumptuary, ok. Awesome, chef. This is. All right. Can everyone see that. I can do it. Okay, perfect. Great. So, yeah. So thanks for having me. And thank you guys for inviting me back to do this as it is to speak again. Yeah. So I wanted to go through what it's been like in data science. I've I left Oxford in December 2016. So it's almost four years now since I since I've started working in industry.
And I thought it'd be quite a nice thing to cover some of the things I have done, but also how did I get there as well. So actually, if I drive that, this is what I'll cover. So I'll give a brief introduction to myself then. So how and why I got into that science? Because that was one of the things I really wanted to find out when I was a graduate student thinking of leaving academia, you know, how do you get into industry? And in particular, data science was what I was thinking about then.
I want to go on to some of the data science projects that I've done and just aren't free to keep some people interested. I'll do a deep dive. Deep is probably too generous a word, but I'll talk a bit more about one of the machine learning problems I've done to give you a bit of context. And then I've got in to talk about some general reflections I have, which I hope will be useful in just thinking about what data science and machine learning is like an industry.
So, yeah, I, I'll come to that. So brief introduction at the moment. I work as a senior machine learning engineer at Spotify. And today is my one month anniversary there. So I've I've not long joined Spotify, so I'm still maybe still learning, find my way around things. But I've actually got a data scientist in industry for for almost four years.
And I'll speak a little bit about the difference between my experience of being a data scientist and what at least some expectations are of being a machine learning engineer as well. Because I think there are differences depending on on what what companies you go to and how they define it. But before doing that, I was a grad student and before that an undergrad students. I did a maths degree at University of Southhampton first.
And then, as Garrett said, I joined Oxford in 2012 on their systems biology. But then it was called a DTC, the doctoral training centre, and that's centre for doctoral training now. But you had a systems biology CBT and that's a picture of me in the lab in biochemistry when I was able to grow more hair and looking at protein crystals than a microscope. When I actually is doing stuff and not just in front of a computer, but yeah, that's that's a little bit about me.
But I want to go and spend a bit more time on the how and why I go into data science. So. How it happen? I've been in a graduation for about two 1/2 years and I started thinking to myself, I don't think academia is a thing that I want to continue doing. There are a few reasons for this. One was this like publish or perish sort of thing that at that time some of the practises.
So, you know, I felt there are times where it was we were doing enough just to get published and not necessarily to do the best science out there. And that sort of tainted at least my perspective of academia, because I you know, I got into it. I got into doing a defo because I said to myself, I'm going to be a professor one day. That was that was the plan. But, yeah, I that that's sort of tainted some of it for me.
Then I started talking to some of the postdocs in the lab and I think the postdoc life of complimented like family life very well. I'd been in a long term relationship and, you know, thinking about sort of family and, you know, with postdocs, you've typically got like, sure. Fixed term contracts, maybe like two to five years. And any one guy when you might be going hopping from place to place, whether it's in the U.K. or outside. And so feel like I wouldn't be able to to necessarily do that.
And then also, I didn't have the best work life balance unit HD. And maybe some of you also sympathise with this in the sense that you might work very late in evenings and gee. But also what, weekends? And you don't give yourself a break because you you often feel that if you're not doing work, then you're not being productive. And yeah, I sort of felt all of those things so that these are my sort of perceptions and my the way I saw things.
And some of you may feel the same, some of you may disagree with this, but, yeah, these were sort of things are gone from my mind as to why I didn't want to stay in academia. So what did I want? Well, I started to enjoy writing code, and this was something that was very different because dermo undergraduate degree in maths, when I was told that we were going to do a programming course, I absolutely hated that.
And I would do whatever modules I could to not programme because I liked pen and paper mass. But yeah, during the p h d i, I worked in a lab with a computer scientist and you know, it sort of got me writing code and got me liking someone like liking it. So I like doing that. I like complex analysis as well. I loved all the data analysis. Love's like right now the maths and looking at graphs and and trying to understand those.
That's what I really liked and I wanted to continue doing. I also wanted a job security as well. You know, as I said, I wanted to know, was it a long term relationship? I wanted to start a family. And and so it was some of that where I wanted some of that was was was secure and stable. I also wanted to be intellectually stimulated. So that was one of my worries about grains industry. I actually worried that if I left academia, I wouldn't be intellectually stimulated.
But I knew it was something I really wanted. So I. The other thing is, money's pretty sweet, too. Yeah. It's you know, you're going to get paid well in an industry as well. So I found out about data science. I'd read articles about it. And and it just seemed like a sweet deal. It seemed like everything I want it to code to do data analysis. Got the job secure a money and all of that. And it was it was great. I'm like, okay, I think I know what it is I want to do. But.
But. I don't know if you have the same sort of opinions or views, but when you when I've read a lot of papers or articles or watch videos, I often talk about the things that a state of the art. And, you know, I have a few examples. Something like couple of these examples happened after a joint left academia and in industry. But they always talk about like really these like state of the art neural networks and reinforcement learnings in these probabilistic graphical models that we're like,
really, really cool. And I was like, I don't know how to do any of that. Like, I don't. I didn't do what I thought was machine learning. And my my PTSD. You know, I could code, but I wasn't the greatest software engineer out there. And yeah. How do you how do you like help and write code for a self-driving car. He does that. So that's kind of where I was at with that. So I thought to myself, I need to develop some skills. So I decided to do a bootcamp.
And to do this, I figured that I'd need to write my thesis early. So I spent like six, seven weeks. Like, head down during the summer of twenty sixteen and just drop the thesis written up as soon as I could in the summer so I could take five weeks out to do this. Boot camp science today aside. So s2 D. S. So yeah, I don't know, boot camp, you know, you get put in different teams.
And when asked to complete a project for a company and we're not paid to do this and I had to pay the privilege of it, pay a hundred pounds for the privilege to to do work for a company. But it gave me some experience. Put the CV and a network as well. And actually it helped me get my first job, so I shouldn't complain about it. It was a hundred pounds worth spent, but I won't play the video or like I wrote an article about this.
The slides and all the links I've said you can have is available along with this recording, but you can watch it in your own time. It's in the core. It's linked on their mates on YouTube, if you want. But I hate watching myself back. So I'm going to skip this. But yeah, after the boot camp, I when I interviewed and found a job in the end and ended up as a data scientist and I'm happy to to answer questions on on that process.
If you get to the end. But I'm going to skip that and talk about applications of data science and machine learning and industry. So what I've covered so far, I've just had a brief introduction to myself, one slide, and then I've just talked about how and why I got into data science. So now I want to do is I just want to talk about some of the some of the things that, oh, the ways in which data science and machine learning applies in industry.
An important thing to note here is that it's it's far from an exhaustive list. So, of course, this sort of statement is used all the time. But I it's far from exhaustive because I don't know all of the applications. I've not been around it. So I've actually given a list of things that and projects that I've been involved with. So here are some of the machine learning specific projects that I've been involved with.
So like first off, in SCD s boot camp, I talked about I was doing price up to my zation for different products. So it was a company called the Parts Alliance. They like distribute and sell copouts. And we're also their project to work out how can we, like, price their products such that they can maximise revenue. So that was the sort of the first project I got everybody involved in.
And then when I finally got my first job as a company called News UK, I got more sort of like an LP, natural language processing style projects. So I work in the text. And so we're looking at automated topic tag extraction. So this is things like there's a news article, who's in it and what's it about? And, you know, should I extract Theresa May or is it about her? Is it a sports article and things like that. So trying to get a model to automatically extract those things from articles.
And largely this is done. This was a project there because they hadn't actually tagged their articles. And so for years. I can't remember how many articles, but we're talking hundreds of thousands, if not millions of articles that were untapped. And so they were, you know, unsearchable, you know, that then they they're not easy to find if they haven't got the metadata with them. It's a bit of anomaly detection, possibly.
The most ambitious project I've ever been involved with, and that includes all of my time now is this automated fact chequer for news articles. Safe to say that project was not completed. And I'll talk about some of the projects that didn't get completed later on. But this the idea here is that when a news article is is written by a journalist, it has to be fact checked by sub editors.
And so, you know, you've got people who go through the article and try and find sources online or verify those statements. And I mean, I remember one example where it gone wrong, which was an article that was published by The Times where I think NASA's sent a probe to Jupiter. And with the numbers that they'd given, they basically said that the probe would reach Jupiter in like 40 days or something like that. So ridiculously quick. So it was like Travian way too fast.
And that's the sort of stuff that are trying to find. So can you write a machine learning model that would scan text, find out what parts of the text are actual statements, and then go and do automated queries to find valid sources to then cheque that statement for its factual validity. That was that was a really tough project. We got some somewhere with it.
But the a lot of the natural language processing tools when the state of the art now was available, when we were doing this back in twenty seventeen. I know that twenty seventeen doesn't sound that long ago, but in at least natural language processing terms as an age away. So yeah. That we weren't able to do it with enough precision. And then I moved onto delivered and we started looking at things like compensation, abuse.
So when I'm assuming people familiar with delivery, I'll talk a bit more about it after this, because that's one of the deep dive projects I'll go into. But, yeah, it's it's basically an app that you can order on like food with. And so you can order food, but then you can claim compensation if the food code or items are missing. And so sometimes people that do that aren't actually being genuine. The food arrives, APSEY perfectly, and they just want a cash.
Surprise, surprise. So you're trying to detect that automated menu classification. Is this menu Italian? Is it got pizza things of that? Or is it. Is it Mediterranean? I'm we tried that in restaurant ranking recommendations. This is probably the project that I spent the longest amount of time doing. Probably a year and a half, I think. So that's the one I'm going to do a bit of a deep dive on. But yeah. And now. Now I've met the Spotify. I will be working on search relevant.
So, I mean the search relevance team at Spotify. So when you actually. So assuming for those that don't know what Spotify is, it's a music streaming platform. So you can go on and stream music so you can search when you use a search functionality, you get results. And I'll be working on the I'm on a team that works on traunch. Make those results more relevant. So, yeah, that's what I'll be dead.
So that's like a bit of an overview about some of the products that I've been involved with and that that specifically machine learning projects as a data scientist. Not everything you do is machine learning. In fact, most of the stuff that you do is not machine learning. And then some of the machine learning products, not machine learning products, have been involved in. So often it's about providing data. People just want to see the right graphs, you know.
So you've got people that want data because they want to make decisions and they want to make informed decisions. So dashboards are absolutely huge in an industry. Know there will be people that will be looking at dashboards and looking at figures and dashboards to make decisions about, you know, the work that they're going to do. Will actions take next? So one was this boom, this book.
So in the editorial team at the Times, they didn't want to go through dashboards to say they wanted to be able to ask the questions. So what I did was I built a little bot so slack for those that don't know, it's a messaging to imagine if you haven't used it. Imagine you have a load of group chats on WhatsApp and you have the ability to organise all those groups and different message people in the company. That's kind of what it's like.
But Slack allows you to build these bots that are basically like algorithms that can like post messages and extract things and shape things. So I wrote Slabaugh that basically responded. So whenever an editor would write. Can you give me the top 10 articles in the sports section today? It would actually take that question, turn it into a school query, query our database and then return those results. And they absolutely loved that because they wanted to.
They wanted the natural language interface. That's what they do. They they speak in words. And they wanted to use English to query things, not not code or or dashboards. The Chrome extension was just if you use the Google Chrome browser, they could just click on a button and it would bring up facts about an article. Eighty tests for those who don't know. And a B test is basically of experiments as well.
And industry often, but not like a lot of experiments. They basically you've created one feature. Let's say you added a button somewhere in your app and now you want to test if it's good. So in your age group, they don't have the button in your big B group. They do have the button and you test to see if you get increased click through rate or uptake or whatever it is you want to measure. So I've conducted several of those. And also lots and lots of ad hoc analysis.
So that's what I call. This is just what happens when someone asks for for some information. And this will happen a lot, particularly around Kovik, when when when it's out and in when the first lockdown down hit back in March was we can deliver. And, you know, the things change a lot. A lot of restaurants had had closed because they couldn't be open. But a lot people still want to get their essentials. So, you know, they're like Joanie, like what's what? What are consumers demanding now?
So, you know, I start analysing a lot of search results and I find out people what I'm not, what groceries all of a sudden and things like that. So then that tells people all we've got to get lots of grocery stores on on the app. So you've probably may have seen that delivery along with, I imagine, just the end of the eats. And the competitors have got in a different grocery stores and convenience stores like Carwarp and Marks and Spencers and and even maybe a local petrol station.
So, yeah. So there's lots of lots of things that say. That that's like a sort of overview about some of the things that I've done. One of the Timor sections now. So this one is going to be a deep dive. Talking a bit more about a particular machine learning problem that I did deliver. And then the last section, I want to talk about some general reflections that I've had. Sorry, I feel about 20 or so minutes and then I've got give us 15 minutes for four questions.
So I hope that will be heard. That's gonna be enough. Cool. So just say you're all aware. Like the slides that I'm going to show you. So I actually made this presentation delivery. They're available online. I've given this talk multiple times. And so the video of the full talk. So I'm what I'm going to do here is a section of it. But the full talk is online. So there is nothing. So it's all publicly available already. So I'm not sure I knew anything that you have to keep yourself.
And I've given links to the slide and two videos of the full took. If you are interested in learning more than what I'm about to tell you, then then you can go on and hear about that. So that talk is just me talking about how we do restaurant ranking at delivery. Cool. So first things first. Again, for those who like having these delivery before delivery, is Lochley a software platform? So it's it's an app essentially that connects different entities.
And we call is a free three site marketplace. So you've got restaurants and they're connected to consumers through riders that deliver food to those consumers, essentially. And on the on the restaurant side, there's been tens of thousands of restaurants and tens of thousands of riders. And what you may not know is delivery doesn't just operate in the U.K., so it operates in. Well, I was at the time I left LA in September. It was in 13 countries when I wrote this.
It was 14. And just they put a Germany summit 20, 19 sometime. Yeah. So they operate globally. And so, yeah, that's that's a sort of scan of of how delivery operates. And it's a platform that basically, if you haven't used it, this is what you see when you open the app. You essentially if you've got a list of restaurants and we want to be able to show those restaurants in an optimal order. Ashok, I get to that. So I just started a team.
Well, not not me. I didn't stop the team, but I started in the team that was created in October 2018 called Merchandising Algorithms, and so newly formed and we decided to first go. Our initial goal was gonna be, let's present the most relevant restaurants to the consumer at the top of the feed. Right. So you open the up and given who you are, what you voted before, what we have available, what should we show you? A top that you will most likely one.
And so bear in mind, this particular thing, this go is a business problem, is nothing machine learning about this. And this is one of the things that you'll end up doing if you decide to do the science is you need to take a business objective and then decide, can I solve this with a machine learning approach? Or does it need something simpler? Does it. Is it even something that is is something I can solve? Often the people that ask that question don't know and don't care.
At this point, they just they they think you're a dinosaur. You've got the data. You can solve any and every problem. You have to then work out whether you can solve the problem and give them the answer, whether they like it or not. But this was one in which we believe that machine learning was a good candidate for. So we created a model to rank those restaurants. Now, bear in mind, I've said given a list of restaurants like we want to present the most optimal restaurants.
What do you mean by optimal? Again, what? Well, what we've said here is we want to rank in order of relevance to the consumer. That was our definition of optimal. Bear in mind, this can change. So the business might say, I don't care about the relevance the customer. I care about the profit. So, you know, can we rank such that I'm going to get the most profit or can we rank so that it's the most fair and we get the highest distribution, like spread of orders across different risk. So something.
So you have to define what we mean by optimal. And again, when I love, like, say, you know, someone who presents the question to you from the business and says, I'll just just present these restaurants. Ultimately, they often don't define what Tomalis. And so it's up to you to make sure you define the problem space. Again, it's just one of the things that as a data scientist and industry, you learn how to do. This is the machine learning and it's a data science is the technical stuff.
But then there's also the soft skills and the business skills that you have to develop as well. So we've set that optimal. We want to rank in order of relevance. But how do we quantify this? This is all quantitative. It's wishy washy. How can I quantify this so I can build a machine learning solution to this? So the first thing I want to do is I want to say what does relevance mean and how can we measure that something is relevant to the consumer?
So one of these I call these online metrics and they are often called online metrics. What I mean by this, by online is that users see these in reality. So we're going to show a ranking of restaurants and users are going to see that ranking and then they're going to order them into place in order. This is in contrast to an offline metric when, say, you might be running a machine learning model locally on your laptop, on your machine.
And so you have to measure something different. And I'll talk about online metrics later. But online metrics, we talk about order volume. So this is just a proxy for something being relevant. Right. So we're assuming here that if all the volume goes up with my ranking, then those restaurants are ranked. We're probably more relevant. That's that's the proxy. And again, we have session level converging. We've stopped using it. I won't go into the details of that.
But it's it's to do with the fact that it's it's a question and it's hard to interpret changes in that number. But we did initially start with looking at that. So when we're framing the problem, if you've done some machine learning before in the past, you'll notice you need what in machine learning? Speed is called a target variable. This year we've got our target, which is given a list of restaurants. What restaurant that they use it purchased from here. I've done a purchase.
I've called it converted. We often say convert. If I say order or purchase or convert, I'm using them interchangeably. So on the left hand side, we have one session and a session is just assume it's somewhere. I open it up. I have a look at some stuff. Don't like it. I closed the app. That's one session. That's a session that did not convert because I didn't purchase anything. But then in the evening, I open the up again. Have a look. This time I did buy.
That's a session that converted. And that's a separate session to the first one. So I owe a session. Can be different uses. So when the session on the left, someone convert it on on to whereas on the session. On the right someone convert it on the bagel factory in position one. And yet that list can be. In central London, that list is like thousands in saving Paris. It's it's huge because there's so many restroom options. So you decide to frame this as a classification problem.
I won't go into the technical details here. As I said, you can you can watch the video and talk a little bit more about it. But the idea as a classification problem is what we'll do is we say take a restaurant. How likely is the user to purchase from that restaurant? And then we'll give it a school. I'm going to use score and probability interchangeably again here. But if you're technical and you know that often with different classification models, you don't get a well calibrated probability.
So it's probably better that I use the word school, but I have used probability here just because people find it easier to say what's the probability that someone's going to invent again? Sometimes in the business and industry, you end up using words because it's easier for people to understand. But yes, essentially it's a school. How? Let me go out to convert between zero and one. And we can use the logarithmic lost function, which is on the right to train the model.
So the idea here now is we've got our target variable. Are they going to convert on a restaurant or not? Now, I need to find out what the dependent variables are all in machine learning because those features. So on the right, you can see you reduce features like how long will it take for the for the order to arrive? What is the popularity of the restaurant? Did they get a lot of orders in the last 30 days? What's the restaurant rating? Does the restaurant have an image on the up?
Sometimes they don't have images in all of these things are factored into whether someone will purchase from the restaurant or not. Yeah. And then there's some function, some machine learning model that will take those features and outputs some school. The important thing, first stop simple and iterate. You know, we didn't start off with a machine learning model. It was just a weighted average of the restaurant's popularity and the estimated time of the order arriving.
And this started simple allows you to build infrastructure to to like what we say we serve that rankings are actually present the ranking to users. And I'll talk a bit more a bit later about what I mean by like the serving infrastructure around it, because that was actually very important. But once we were able to do that, we built the infrastructure. We did then look at using different models them to logistic regression.
And then we started using more complex models a bit later, like neuro networks. But then when we went to evaluate these models. So before we actually present them to users, we want to evaluate are all the ranking algorithms we're creating. Are they actually any good? So these are what we call off line metrics, metrics that we measure on on a laptop. And Berrima, I should say, before I started this project, I'd never done any ranking at all in my life, like in a machine learning sense.
So if you're looking at some of this stuff and you like what is going on, I, too, was in the same position as you as aggression like. So don't feel like all of this has to make sense because it only made sense to me two years ago now. Then I started looking at these things. But essentially these like all of this stuff I'm talking about, like, you know, you guys are already a world class university.
It won't take you long to learn and do. And that's largely what I do, is I don't know how to do it. Start off with. And then I just learn on a job. But yeah, we have a bunch. These are fly metrics. The one that we used was called mean reciprocal rank, the way that's calculated. If you just I've got five columns. Just look at the column on the far left. So in that column on the far left, there's five different rectangles.
We call them restaurant cards. And I've written convert it on the card that's in the fourth position down. So that says it's rank is full. So it's reciprocal. Rank is one over four. And I do that for every single one of these columns, every single section. So now I've got a bunch of reciprocal ranks. I can then take the mean of that. And that is called the main reciprocal rank. That is a an evaluation metric. And like information retrieval, they call it, or ranking.
And the idea, it's a number between zero and one. If you place the converted restaurant at the very top of the list in every session, then you mean a single rent will be equal to one. So you want to get it as close to that as possible because that that suggests that you'll reach your permitting the list. You're changing the list such that the most desirable thing is highest up. That's the idea for this. So that was all the centrepieces in terms of like the workflow.
How did we do this? Well, I said we needed to get the data. We need to plot the sections. We need to plot all the. Just so we have a data warehouse, I mean, right, a bunch of sequel to to extract that data from database and rewrite all of the rest of the code in Python. So we validate that the model we train, several models say, and models for each one each. A single model is one permutation of that of that list. So I can calculate the mean reciprocal rank for all of the sessions.
For one model. So that would be our one. And then I can do that for all the different models. And the best one is, is the one with the highest Niemans reciprocal rank. That's the model. We then choose to go into production. If you listen to the rest of my talk, I will talk about why that was not a good idea and why I and supercool rank in our case wasn't a best evaluation metric. But I won't go into that here. But once we've chosen that model, we then run the AP test.
So what I'm saying is half of the people in group, they get the original ranking. Half of them, a group B, get the new ranking. And then we test to see did the order volume go up or did the session level conversion go up? If it did, great, then we wrote up the new model. If not, then we need to think again and iterate. And this is a complete iterative process.
So even if we do well and we we get a successful experiment, there's probably features we missed out or there's a new better model that we can use. So this is completely iterative. So you can keep going through this cycle, picking up different projects elsewhere in a team. But but this is a particular thing that will keep going. Yes.
So then I say this is current work. Back then it was current work, but we started looking at more complex models and on the right is a wide and deep neural network which we use to implement, which is currently in production at deliver at the moment. So that's that those. That is like a sort of deep dive into one of the projects that I've done. And I said more information from from the video or feel free to ask me questions and I can talk a bit more about it in detail.
If you're interested. But I want to talk a bit about general reflections that I've had from work and into science. I think, like for me, this is like the most fun bit about writing. It's presentation was thinking about like, what do you actually think about my time? First off. Is machine learning models need to be in production to provide value. Right. So if I just train a model and it's on my laptop. Great.
That's cool. I've learnt something, but it's not gonna do anything for your company unless users are using it. Or people are making decisions of it. It's it's not providing any value. And actually getting machine learning goes into production is not an easy thing to do. So I'm at a time of. I wrote this presentation a couple of weeks ago. I was in the middle of reading the article. And in the first paragraph, it states this.
This is a company. VentureBeat reports that eighty seven percent of the site's products don't make it to production. And another one is that it says that it's 90 percent, although the second one, she talks specifically about machine learning products rather than data science products linked the source at the bottom stack. Overflow Block isn't a link to the article was reading. But I've also linked to the two separate articles.
In case you're interested. But this is a large, like really, really big problem. I think a lot of what people read about and doing what I had thought about the science was I'm going to build machine learning models, I'm going to do basic stuff. I'll be doing these new networks and what is this cool stuff? But most of the time it's not. It doesn't make it a production. And it's so it's it becomes useless. And so what I'll say before is learning.
If someone for the business asked you a question, you have to decide whether you're able to answer that question. That is a huge skill, because if I just say yes all the time, it's going to end up with most of my products failing. And I've mentioned it about seven infrastructure. Briefly, I said I talk about it. So this is a figure from a paper published by Google. I think it's 2015.
And what you see in the middle is it is black and okelo code. That is the amount of machine learning code required to get something into production. But there's a whole bunch of stuff around this that needs to be built in order to to actually make your machine learning model or any data science put it. You do useful and use them into use in production. And often I. Like scientists themselves don't usually have the skills to do that.
And I don't. I didn't. And this is one of the reasons why I wanted to switch. This is the reason I switch to genetic engineering, is because I want to learn the skills to get a bit more into all of the other parts of the infrastructure so that I can be less reliant on other people to get motors into production. So I went back to the seven, the machine learning models I mentioned I was involved with before and the ones with the green circles sorry if and when it's kind of blind.
That's why I made the shapes different as well for the circles. They're the only two machine learning projects I've done that have actually made its production. All of the other ones haven't made it to production and so haven't provided the value other than perhaps learning that we can't do this or this was not the right time. So, yeah, I can talk about all this cool stuff, but whenever you go to, like conferences and stuff, you get that evil person, the audience, he's just like at the end.
Ask a question, did it go? Is it in production? That's when you hear the speaker start to scream because they talk about all this cool stuff, but it doesn't make it there. So that's one thing. Secondly. Second reflection, Engo is not in the ALP. It's the decision that informs all the action that's taken. So take, for example, the product I was doing on compensation abuse. We are meant to get machine learning model to decide or to work out if a compensation claim was abusive or not.
That wasn't going to. That model wasn't going to make that decision. It was going to give a school would go to a customer service representative who would ultimately ultimately make the decision based on other things. She he she you he saw I say she. Because we were working with with a lady when we're doing this. But yeah, it was them who he was going to make that call. So that was that's the that's the thing. So with a machine learning model, it's not the put specifically of that model.
It is the action that it's taken or the decision that it informs. That's the important bit. So here is the the model itself doesn't have intrinsic value. So that's that's another thing. So when you're doing a project, you have to figure out what is it going to be used used for? How is it going to be used? Who's going to use it? Baselines and constellation, they can be hard to beat.
Right. You know, a lot of times you're seeing these articles and all these things about a deep and models being the the best solution and coming out and being step the. Often that that's in certain cases, particularly cases with unstructured data like images and text and things that for a lot of structured data and tabular form data, a linear model will, if it doesn't be a deep, deep neural net like it will do basically as good a job.
And we found that so far up until this year, the dispatch algorithm at delivery that decides when to dispatch right is to go and pick up an order was a linear model, is just a linear regression model. And that does the trick. It does the job. It does very well. So, yeah. And sometimes it's the case that even you don't need a model to do. You need a human. So when I worked at News UK for The Sun, you're trying to work out what what articles to promote on social media.
Now, if anyone's read The Sun, I know how many of you do. I don't. I still don't. But like yeah. Tilghman's and want to editors and she was like nudity and celebrity deaths. So as soon as that's in an article that's gonna blow up. I don't need a model to tell me that. And she's she's right. So that that's that's kind of the things that we're talking about. Like think about when a model is actually needed.
Plan how you're gonna show that your plan, how you're going to share that you've added value so you can release it. A machine learning model. We can release some sort that assigns product. But ultimately, someone's got to be convinced that it's actually doing the right thing because it's as much as we might be quantitative and, you know. Influenced by a data. There are a hell of a lot of people in industry who aren't. And sometimes they just need a story or the right cell.
And that was the case in particular, news UK site delivery and Spotify. They use a lot of a B tests. And so we do just test if it actually is good. But D UK, we didn't do any of that. We just needed the right person to be convinced. And so it was about how we gonna sell this story. And the last thing I'm gonna cover, I'm going to be brief here because I want to give time for questions. So is data science is just an incredibly broad time.
There are so many different skill sets that come with being a data scientist. I forgot to put on here, so I put on afterwards. There's a link to the article that I actually got this. He's, um, definition's from. But yeah, as a data scientist, sometimes you think you'll know absolutely everything. I use it. I work to deliver as a data scientist and algorithms.
So I you know, I'm not an expert in inference and causal relationships, nor am I great or necessarily expert in analytics and built into right dashboards and knowing what new metrics to create. They are different skill sets. And I think that knowing your type of skill set and knowing that you don't have to be all that broad is an important thing. So, yeah, I'm just going to summarise there. I talked about how and why I went in state of science covered.
Some products have done in the last four years, but I ended up talking about a bit more about the ranking problem and some some reflections. So I hope that has been somewhat useful. I know not technical, but that was my intention, was not to give a incorrect technical talk, but to give you at least some reflection on that. Yeah, and that's that's the end.