Computer-Generated Communication

I'm sure many of you have seen social media posts that say things like type I am the on your phone and then generate a result using the middle option of predictive text. So you know, just for example, I did that. If I did that on my phone, then I get I am the only one who can help me with this. Oh two, real predictive text. I mean, I'm the only one who researches and writes these episodes.

That's it's way too real. But the whole meme of using predictive text to generate seemingly meaningful or you know, sometimes wildly absurd phrases is just part of what I want to talk about today. Now. The reason this topic jumped at me is because of a recent news article that I read over on the Verge The article that was written by Kim Lyons has the title a college student used GPT three to write fake blog posts and

ended up at the top of Hacker News. Now, as the headline indicates, a computer science student used a predictive text engine called GPT three, a beta build of it in fact, that stands for Generative pre Trained Transformer, and then generated a blog that was featured on a site called hacker News as if it were a piece written

by a flesh and blood human being. What's more, a threat on Reddit showed that only a few people were picking up on the feeling that something hinky was going on, and that perhaps the blog post had not been written but generated. And Lions goes on to point out that the fact that there's a lot of, you know, not very good writing on the Internet makes it a little harder to sus out a decent generated post as opposed

to a written one. It's not so much that the AI has become super awesome at writing, but rather that we've kind of lowered the bar more than a little. This kind of plays into the whole concept of a touring test. So, just to go off on a tangent here, this isn't in my notes, I'm just going to speak off the cuff. The Touring test is named after Alan Touring,

famous computer scientist, and the idea. Nowadays, it's kind of evolved into this idea of you have a series of interviews that a person does over a computer, and some of the interviewees are people and some of them are chat bots essentially, and the goal of this whole exercise is to see if the person who's doing the interview can consistently tell if the other entity on the other side of the interview is a person or if it's

a chat bought. And if you pass with a certain percentage, you would say that the chat bought has passed the Touring test, that people are unable to tell the difference between the chat bought and a real human being, and that this is kind of one of the markers for artificial intelligence. We're gonna be dipping into that sort of thing with this discussion as well. So today I'm really wanted to dive into the whole concept of predictive text and how it's done and how it could absolutely destroy

platforms like Facebook in the future. That's all I'm going to end this episode, So stick around, but we have to build on this gradually, So let's start at the very beginning, which, according to this woman who's singing outside my window, is a very good place to start. And we are going to start with a particularly tricky concept for a former English Lit Major to try and explain,

and this is called a Markov model. It's named after a mathematician named Andre Andreyevitch Markov, and he was born in Russia in eighteen fifty six, and he did a lot of work on an area of mathematics called stochastic processes. But that just raises another question, right, what does stochastic mean? Well, a stochastic variable is one that is randomly determined. A stocastic system has a random probability pattern that you can study,

but you can't dickt it precisely. There's always uncertainty. So you can assign probabilities as to how the pattern will form, but those are just indications of how likely a particular pattern will form, not a guarantee. So let's take a very simple example, and let's pick something really random. Let's talk about my two year old niece. So let's say my niece is standing in the middle of a room and I walk in. Now, based on my past interactions with this random creature, I know my niece is likely

to do one of three things. She is going to run at me and grab my hand, and then boss me around and put me someplace and tell me I have to stay there. She's going to run away from me and then hide and then demand very loudly that I come find her. She is not, i should add, quite grasped the concept of hiding. Or she is going to ignore me and say and or dance. Those are the things that she typically does. There are other things she might do as well, but they happen much less frequently.

So let's say I want to sketch out this scenario on paper. I might start with the scenario is my nieces in a room and I come into the room. Then I would draw a little bubbles on my paper to represent the potential actions or states as we would call them, in a Markov chain that could follow this input of me walking into the room. Now, based on the number of times I've seen her respond before, I

could wait each of those states with a certain probability. If, for example, she runs at me and grabs my hand then bosses me around more than half the time I can wait that outcome, as you know, And does that mean the next time I walk into a room that she's going to do that? No, each incident is random. I'm just illustrating how likely a particular outcome is going

to be. I would then assign probabilities for the other two outcomes I outlined, and and maybe just ignore all the outliers and say that one of them is you know, likely, which means the third one is only five percent likely to happen because it has to add up to now. The example I just gave is ridiculously simple, despite the fact that my niece is already incredibly complicated, And it just gives us the odds of one starting state that I'm me walking into a room that then transitions into

one of three outcome states. Markov models can have lots of variables, with some variables dependent upon the value of other variables. So you might see a chain as something like if outcome A happens and there's a sixty chance that it will, then there's a thirty percent chance that a subsequent outcome A three will happen, And it can become a really complex branching path of possibilities, but we can stick with simple. Let's take the coin flip, the

classic example of a random variable. We know that the odds of a fair coin landing heads up are and landing tails up. Our fifty percent. Flipping a coin many thousands of times should show that collectively you're gravitating towards those probabilities, that about half of your coin flips will be heads and the other half will be tails. But that does not mean you won't get on streaks where you flip heads over and over. Allah, Rosencrantz and Guildenstern

are dead. And if you don't know that reference, I highly recommend that you read that play or you watch the excellent film version that has Tim Roth and Gary Oldman in it, because it is fantastic and it kind of dives into a fun discussion of probabilities and what does that actually mean Anyway, The odds of flipping a coin heads are for a single coin flip, but what

about a second coin flip. Well, if we look at just that flip in isolation, that second coin flip, it's still a fifty pc chance that's going to land on heads. But if we frame it a different way, if we ask the question, what what are the odds of flipping heads twice in a row? This is a different question because you're not thinking about individual flips. You're saying, what

are the odds of this happening twice sequentially? Well, now we have to take the odds of it happening once, which is, and then we have to multiply it against itself. It's a fifty chance again that it would happen twice. So oft is let me do the math. It is or one four. So if you were to do a pair of coin flips, and you were to repeat this experiment over and over and over again over the long run, you would find that of those sequences would end up

with heads followed by heads. But what if we wanted to say, how what are the odds of flipping three heads in a row? Well, then we have to have it again. So instead of one out of every four trials, we would see one out of every eight, or twelve point five percent. And we can keep extending this out. We can figure out the odds of some ridiculously long stretch of flipping heads in a row. Now in Rosen, Cranston,

Gillenstern are dead. We are told that it happens and astonishing ninety two times in a row, that streak has a probability of one in five octillion. That would be a five followed by twenty seven zeros. This does not mean that it would be impossible, but it is unfathomably unlikely. Clemson University has a useful lecture available online in the form of a presentation, and it's titled Introduction to Markov Models, and it uses weather forecasting as an example. And their

example takes three initial states, sunny, rainy, and cloudy. Consequently, those are also the three potential output states, so each state can transition into three states, including transitioning into itself, so you could go sunny to cloudy, sunny too rainy, or sunny to sunny. That's a valid result as well. And in their example, the ideas that we have based on past observations figured out the probability for specific forecasts

based on whatever the current weather happens to be. So, for example, we've figured out that rain tomorrow is likely if it's raining today, but it's only likely if it's just cloudy or sunny today. So if it's cloudy, if it's sunny, if it's raining today, that we'll see rain tomorrow. But our model would need to have probabilities assigned to each pair of starting and ending states. So I'm gonna follow through with that just for the purposes of this conversation.

And we've covered the probabilities of tomorrow being rainy based on whatever today's weather is. But the example from Clemson also gives the other two outcomes states. So if we're looking at the probability of tomorrow being cloudy, we see that based on our past observations, that if today is sunny, it's a chance of cloudy tomorrow. If today is rainy, it's a thirty percent chance, and if today is cloudy,

there's a fifty percent chance. And finally, if we want to know if it's going to be sunny tomorrow, again this is all just based on the example. We see that if today is sunny, there's an eight percent chance that tomorrow will be too. If today is rainy, it's just a five percent chance. If today is cloudy there's a fifteen percent chance. Now, the reason we need to know all of these probabilities will become clear in a second. And again these are just examples, they don't reflect real data.

Markov got very clever and began to use math to describe probabilities for predictions that are further out than one state. So, for example, you might say, what is the probability that, if today is cloudy, that tomorrow will be sunny and

that the following day will be rainy. This is kind of similar to us asking the question of what are the odds of flipping heads two or three times in a row, except we're looking at the probabilities of weather that are based on what our current conditions happen to be.

So using the example probabilities that were used in that lecture, we would find that sunny days follow cloudy days just fifteen percent of the time, So there's a fifteen percent chance that tomorrow will be cloudy if today is sunny, and rainy days follow sunny days twenty per scent of the time. So if tomorrow is sunny, there's a twenty

chance the day after tomorrow will be rainy. So then that means that if today's cloudy, we've got that fift chance tomorrow will be sunny, and if it is sunny, there's a chance that the day after tomorrow will be rainy. So we have to multiply those probabilities together. We have to multiply that by twenty or point one five times point two. That gives us point zero three, which we

convert to a percentage. That means there's just a three percent chance that if today is cloudy, tomorrow will be sunny, and the day after tomorrow will be rainy. That's just a three percent chance of that happening. And the further out we try to predict a particular sequence of whether, the lower the probability will be, meaning you know it could happen. It's not like it's impossible, but it gets less likely the further out we go from our initial state.

So a Markov model is a stochastic model that describes putten chill sequences. It is temporal in nature. That means we are really concerned with the state of things and how those states will change over time, and it gives us a way to explain how current states will depend upon previous states. It's not just about predicting the future, but also understanding the present. Why are things the way they are right now? And it gives us the chance to weigh the predictions of the future based upon past

observational data. This is why we see weather forecasts that give us percentages for rainy days, Like a chance for rain tells us that it's probably a good idea to bring an umbrella if we're going outside, because based on past observations, there's a decent chance it's going to rain today. Now, let's get more complicated. What if we don't actually know

the current state of the weather. Let's say that you are stuck inside and you can't see out a window, you have no windows in the room you're in, and someone else comes into your room and says, what's the weather like outside? Well, the only hint that we have in this experience is if the person that comes in is carrying an umbrella or not. We don't actually know the current state. We can only make an educated guess based on the presence or absence of an umbrella. The

reality of the current state is hidden from us. This leads us to a type of sequential analysis that's used in computer science, the hidden Markov model. So with these models, we're trying to learn more about the initial states by analyzing the outcomes that we can observe. And another way of putting it is we're trying to answer the question Why are things how they are right now? Why did

this happen? Let's look back and figure out the probability that a particular initial state led to what is going on right now now. The whole reason I spent time talking about Markov models and probability is that it ties heavily into predictive text. It's also used in tons of other computational processes and analysis, from natural language analysis to genome sequencing. It's really powerful stuff. If we think about language,

we know that there are certain rules to things. You can't just string random letters in a sequence and expect that to make a word that other people can understand. We have developed languages that have their own vocabularies and syntax and grammars. We know that in English, for example, the letter Q is nearly always followed by the letter you. We know that it would be very odd to see the letter H follow right behind the letter J in English.

And so we can start building out a dictionary and a matrix, and the dictionary would include lots of common words, and the matrix would include basic rules to help us identify when someone is making a typo or misspelling something. And with these tools we could build out a method for predicting a letter based on the letters that were

already typed. So if I typed T and then H, my predictive text might helpfully offer out the letter E because I frequently type the word the If I ignore that and I hit the letter A, I might get the prompt of using van or thank or maybe even thanks or maybe something else. And we're starting down that journey toward generative text. When we come back, I'll explain more about this and some really cool experiments with using machine learning and what that all means. But first let's

take a quick break. Okay, So we're building out a tool that quote unquote understands basic probabilities of words appearing in a given language in a given order, and it understands that, for example, a Q will be followed by you nearly of the time in English. We build into this model all sorts of probabilities, so that words that are more common are going to pop up as autocomplete options more frequently than uncommon words. But we can do better than this. We can pair this with a learning model.

Learning models evolve over time, They adjust based on the input fed to them, and we're talking about lots and lots of input, they refine themselves, so, in other words, they learn. So with learning models are predictive text begins to adjust to the specific individual who uses the predictive

text over time. Like a phone. So let's say you and I each have the same particular model of smartphone, and we're both running the same operating system version and everything, like our phones are are essentially identical, at least at casual glance. And we've both been using these phones for a few weeks. And in that time, you and I have each used our phones to send various messages to our friends, our family, our colleagues, you know, your arch nemesis,

Ben Bolan, you know the usual. As we do that, our predictive text keyboards start to pick up on how we use words, and it can build up a frequency matrix, which isn't just looking at words that are common in general, but words that are common to us as individuals, and the way that we use words, and sometimes the way

we generate words. Maybe you happen to use the word balder dash a lot, and so you start typing the word and the autocomplete for balder dash will jump up much faster than it would if I were typing it on my phone, because my phone has never heard me use that, so it doesn't automatically assume that's what I'm typing. Maybe I use the word folder roll a lot, and the same happens with my phone compared to yours. The models learned the words we use, not and not just

the words that the words we create as well. So let's say that I was, for some reason a big fan of How I Met Your Mother, which I'm not. But let's say that I am a big fan of Neil Patrick Harris, which is true, and his character often says that is wait for it, legendary. Uh, And it might extend the word legendary. So to do that, I might throw in a whole bunch of extra ease at

the beginning of legendary. Well, my phone might pick up that I tend to do this, and so it includes that as a legitimate word, even though any sort of spelling check would say this ain't a word, stop it, But my phone's predictive text is going to include it as saying this is something that is meaningful and thus a valid option. Also, the phones can learn to adapt to our own sense of syntax and grammar. Perhaps for

purposes of a particular effect. One of us tends to tweak the syntax of the language that we're communicating in for some reason. Maybe it's for comedic effect and it's not following the established rules of grammar for English. But our phone starts to understand that's how we communicate, based on how we order our words and how we generate our phrases, you know, how we communicate that. While our choices aren't necessarily in alignment with an established formal system,

they represent a particular approach to communicating. Predictive text can start to get a handle on that if it's built properly, and even someone who communicates in an idiosyncratic way might find that their phone is offering up particularly relevant suggestions. So how does all this work? How do machines actually learn stuff? Well, there's not one single method, but there are a collection of related processes that computer scientists develop

to train machines. And you can look at two major types of categories of machine learning, and there are a lot of subtypes under each of these, and those would be supervised learning and unsupervised learning. Supervised learning involves training a computer model using known input and output information, so Let's take an example that I like to use a lot, and it's about image recognition. So let's say you're teaching a computer to recognize images of coffee mugs, and you

have an enormous supply of images, millions of them. Some of them contain coffee mugs and various shapes and sizes and colors and orientations, and the lighting can be different. You might have the handle pointing to the left, and some or pointing to the right or the other. Some cases it might be on its side. But you've got tons of these, and you also have millions of images of other stuff. Some of it might not even resemble a mug remotely. Maybe it's an airplane or Christopher walkin.

Others might look kind of like a mug, you know, it might be a glass or a bowl or something similar. Now, as a human being, you can tell straight away if the image you've got in front of you represents a coffee mug or not, But machines don't inherently possess this ability. You could feed one photo of a generic off white coffee mug, the handle happens to be pointed to the left, and you tag that photo as a coffee mug, you give meta data to the computer to classify that as

a coffee mug. And if you create a database of images, maybe you do a search for coffee mug, that one would come up as a result because of all the work you've done with tagging this thing and effectively telling

the computer this is what I mean by coffee mug. However, if you fed a new image and this one is of a red coffee mug that's of a different size, maybe the photo has different lighting conditions, maybe the mug is a little closer to the camera, the handles point to the right and on the left, would the computer automatically know that that's a coffee mug. No, it hasn't learned that. So you would have to build a predictive model for a computer to follow based on the known

input and outputs. Your output is you want the computer to classify photos as either having a coffee mug in them or not, And you might use an artificial neural network. In this case, you're creating nodes that accept input, then they apply some sort of decision making process to that input and then pass it along further along the network. You can almost think of nodes as essentially making a yes or no judgment on a piece of data. Does

the input qualify or does it not? Does it have this particular aspect of whatever it is you're looking at, in our case, coffee mugs or does it lack that? With our mug example, it could be a simple question like is this mug shaped? But the nodes are asking lots of questions and making lots of judgments and passing them throughout the neural network until you get to the final output, the final judgment of is this a coffee mug or is it not? And computer scientists influence how

the computer processes information. They adjust the waiting of answers waiting as in like weight, as in heavy W E I G H T waiting. So you create your model, you use nodes that are making a series of judgments on images. You wait those decisions so that you're hopefully going toward a more accurate result, and you feed your photos through and you look at the output. Now you know whether the photos have a coffee mug in them or not. You're looking to see if the computer can

recognize that. So you're looking to see if your model succeeded or failed. And then you go back and you make adjustments to your neural network. You adjust the waitings of those decisions so that the nodes process information in a slightly different way, and you always have the goal of improving the accuracy of the overall system. You feed the images through it again, and you do this over

and over. You train the computer model so that it gets more accurate as you make these adjustments, and ultimately you get to a system that can accept brand new images, ones that haven't been deliberately chosen, and then sort those into images that either are of a coffee mug or are not. And this is in an area called classification.

So in our simple example, images just fall into two broad classifications, photos with mugs or photos without, though we're gonna get a little more complicated a little bit later, so you can have all sorts of classifications. Medical imaging systems make use of this sort of machine learning process to indicate whether or not an image of a of

a tumor is benign or not. Handwriting recognition program ms do this to speech recognition can do this as well, so supervised learning systems can also use a different approach called regression as a means of training a system regression is all about predicting a continuous response, like how much electricity a community is going to need over time. It's

about predicting things to which you can assign real numbers. So, for example, predicting a change in temperature, temperature happens to have a value that is a real number, so that falls into this category that's supervised learning, where we have the known inputs and known outputs. We know definitively if the information the computer generates is accurate or not because

we can actually check its work. It's kind of like a teacher grading student tests and then working with a student who has a low score to get a better understanding of subject matter, and then on the next test hopefully they score better, and you keep working with that student over and over until they have reached a high of level of consistency of being correct. Unsupervised learning is more about finding patterns or meaning in data where no

such patterns or meaning is initially obvious. When we talk about sifting through big data to find patterns, this is the kind of thing we're talking about. Those patterns might be subtle, or they might only be obvious when you're dealing with truly enormous amounts of information. We humans are really good at spotting patterns up to a point. It's part of our survival mechanism. Recognizing patterns helped ancient humans recognize prey or predators, so it's a key element to

the survival of our species. But when you get to really, really big quantities of data, it's hard for us to see patterns. It would be kind of like if you jumped off a boat in the middle of the ocean and then you were told to look for patterns that are the size of New Zealand you'd be lost right away. The scale is something we can't deal with. But computer systems can handle data far more efficiently than we can, and that means they can potentially spot patterns where we

would not. Unsupervised learning techniques are best for this, and they have a few different approaches. One is clustering, which is pretty much what sounds like. The system looks for groupings and data indications of clusters, pattern clusters. And now I need to get back to my image recognition coffee mug analogy. If we were just feeding images that are either a coffee mug on a neutral background or something else,

then we could go supervised learning all the way. But if we wanted to create a system that could recognize if a coffee mug were in a larger scene, like a crowded kitchen table, lots of other stuff is on it, we could probably rely a bit on unsupervised learning, in which we would use clustering to teach the system to look for data that collectively appears to represent a coffee mug. We're trying to create a system that can pick out the shape of a coffee mug in an image that

has a lot of other shapes in it. The system needs to understand which shapes, which lines and curves represent the borders of objects. So what is a coffee mug as opposed to say, a tablecloth or a shadow or a bowl with a spoon next to it. Unsupervised pattern recognition can lead to that outcome. Again, it requires a lot of training. You feed millions of images to a system numerous times to refine this approach. The method often relies upon hidden Markov models. Oh and this also ties

into something else that's you know, tangentially related. But I thought I would bring it up in case you guys have been experiencing it as much as I have. If you've noticed a lot more instances of websites demanding that you prove you're not a robot with a capture. By the way, this is a good reminder that if you go to the tech stuff store at t public dot com slash stores slash tech Stuff, you can get a shirt or you know, dare I say, a coffee mug

with this capture robot idea on it. A lot of those captures involve a series of photos, and it's your job to click all the photos that have something specific in them, you know, like bicycles or crosswalks, or traffic lights or fire hydrants. If you've wondered why that is, well, it all comes down to good traffic versus bad traffic. There's a lot of traffic out there that is uh powered by butts for various reasons, and that can clog things up, and so systems and companies like Google want

to prioritize traffic that's good traffic. It represents actual people trying to do stuff, and give them preferential access to other methods that might be malevolent or just might end up making things run slower if they get unfettered access.

And the reason these captions are getting so difficult is because machine learning and image recognition software has gotten really good, and so to protect against bad traffic, companies like Google are using difficult capture systems that present fuzzy, dimly lit, or otherwise you know, bad photographs to you, and your job is to stare at them, possibly on a tiny smartphone screen, and figure out which ones are legit. The whole goal is to present photos that are so lousy

that machines can't really deal with them. The problem is, over the long run, machines get better than doing this sort of stuff, whereas we kind of, you know, we have a cap on our performance. There will come a point where an image will be get you know, too fuzzy or too dim for us to make out if there's a fire hydrant in there or not. The machines will always get better at stuff at this than than

we are over the long run. Heck, older capture systems are completely obsolete now because computer systems can complete them at a success rate that's actually higher than humans. We've got a lot of science fiction stories about machines becoming sentient and ruining humanity, but the truth of the matter is they don't need sentients to be disruptive. If they are directed by someone for a specific malevolent purpose, that's bad enough, even if the machines aren't really you know,

thinking for themselves. Okay, but let's get back to predictive text. After all of this. You could create a machine learning model that has a huge database of words, you know, a dictionary, and you could program the system to classify the words. You can sus out which words are nouns and verbs and adjectives, and then apply rules to how

those words can go together to make sentences. Or you could just you know, analyze a ton of literature and have the computer kind of figure that out for itself, just through statistical analysis, understand how words fit together based upon the history of the written word, at least in modern English. For example, if you went further back to like old English, first of all, your vocabulary would be totally different, but your grammar would be too, and suddenly

things would not make much sense. It would everything would sound like yoda. So the system could go through millions of pages of materials building a statistical model that shows how frequently certain words pair together and in which order. Effectively, you're analyzing how humans put letters together to make words, and words together to make sentences. You could move up from there. You could try and analyze how sentences come

together to make up paragraphs, but it starts to get tricky. However, you can work on a system that can present a series of sentences that are related enough to be a coherent presentation of ideas, at least in the short run. It might not be super compelling or as effective as what a human could do, but it could be a lot more impressive than just, you know, a string of

totally unrelated words. When we come back, I'll talk a bit more about how computer systems can put words together for us and what that could mean in the future. But first let's take another quick break. Okay, So, AI systems, if sophisticated enough, can use stuff like hidden Markov models and machine learning to put together strings of words that, from a probability standpoint, a statistical standpoint, at least are

likely to make some sense. There's no guarantee it will actually make sense, but if things are going well, the phrases will be grammatically correct, and if they're going really well, the word choice will be reasonable enough to pass muster. But this is still pretty hard. Computer systems typically lack the ability to build on context and meaning because they're effectively looking for what is most likely to come next, rather than looking back at what has already come before.

Does that make sense, Well, let me put it in another way. In our weather example, I talked about how the predictions for future weather depended on current weather. So what is it doing today? If it is sunny today, there's an eight percent chance it will be sunny tomorrow according to our example. But the predictions don't depend upon the weather that came earlier, like what happened yesterday. The

system doesn't care about yesterday's weather. We might care because we're using long trends of weather to act as our data source to train the computer model, you know, to create those probabilities. But yesterday's weather, as far as the computer system is concerned, has no impact on tomorrow's weather. So if yesterday we're rainy in today is sunny, the computer doesn't really care. It just cares that today is sunny. The same thing can hold true with systems that are

creating predictive text. The goal with standard predictive text is to save users time and effort by suggesting likely words as you, you know, start typing, So if you start typing the word technology, at some point, the system recognizes the letter pattern and offers that up as an option, And for words that are frequently used in pairs, you'll get those suggestions right away after you type the first word. Since this is typically presented as an option, you know,

something you can choose to use or not. It's pretty simple to avoid going wrong unless you, as a user, fumble things and accidentally picked the wrong word, which can get kind of embarrassing, or if it autocompletes after the fact, thinking that you made a spelling error and then you have accidentally spelled Tim mentions name as Tim Munchkin and

I am deeply sorry for that. Auto replies with email get a little more complicated as the system is analyzing the message that is coming into you before formulating a possible response. So I have email systems that do this for me. And one common example for me is that our sales team here at our company will send me an email asking if I'm okay running a particular sponsors ads on my show. Now, normally I like to do research on my sponsors, so I'll take time to look

into things and then respond myself. But sometimes the request is for a sponsor I'm familiar with and I definitely want or you know, occasionally definitely do not want on my show, and I'll see on my phone that I have the option to pick a quick reply of something like sure or yes, that's fine, or something similar. In this case, the email program is using natural language systems and predictive text to suss out that there is a request and that the common responses I might make to

that request should be options. Now, it's not that the computer system actually understands the nature of this request, but more like the structure of a request. In other words, it's saying, this looks like it's a yes or no question. Let's present him with responses that are in a yes or no format. The fact that the system doesn't really have a deeper understanding can become evident in other use cases.

So for example, Janelle Shane, who is a research scientist and who has a delightful blog called AI Weirdness, took time to try and train a machine learning system to tell jokes. It became clear that the system could construct something resembling a classic question slash punchline style of joke. But it was also clear that the punchline rarely had any connection to the question. It actually reminded me a lot of how little kids like my two year old

niece tell jokes. These jokes are some of my favorite in the world, not because the jokes are inherently funny, but because they are absurd and they show how little children can recognize the structure, but not how to build an actual joke. My favorite of the AI generated jokes almost got it right, and it went like this, what do you get when you cross a dinosaur? They get a lawyer's I mean, that's that's almost a real joke.

I actually love that one. Shane pointed out the bit that I mentioned earlier that these systems have next to no short term memory, and so building any lengthy response is pretty much impossible because the computer system is so focused on choosing the word that comes next without an understanding of the connection or context of what came earlier.

And you may have come across stuff like a social media post that says something along the lines of I fed a computer ten thousand movie scripts and asked it to write the next you know, Highlander movie or whatever, and then you get a little screenplay, and inevitably they end up being silly and absurd, with crazy stage directions and dialogue and descriptions. They also tend to be written entirely by human beings. Most AI systems are incapable of

keeping things consistent, like character names. A computer system might create a character name and give that character align, but that name is not likely to return later on in the screenplay. It's not necessarily going to show up in any stage directions or descriptions. It ends up being more dreamlike and free form. It's still absurd, but it's not as internally consistent. So if you come across a long piece of absurd ast humor that was quote unquote written

by a computer, chances are it wasn't. It was written by a person who was emulating the dreamlike absurdism of computer generated text. They're still really funny, they're just not necessarily actually generated by a computer. So about that blog post that ran on Hacker News. How did that get past so many people? It started with Liam Poor, a college student, a computer scientist, who made contact with a PhD student who in turn had access to a private

beta build of the GPT three autocomplete tool. Poor created a blog post title and an introduction to serve as the launch point for the system to build upon. And together they ran a few trials with this machine learning system and auto generated text system and uh with those prompts, and then Poor picked one of the results to submit as a legit blog post. Now, I'm going to read a little section of it. Now, the blog post title was feeling unproductive, maybe you should stop overthinking. And here's

a segment that comes from the middle of the blog post. Quote. When you engage in creative thinking, your brain starts working more efficiently. It becomes more active and more open to new ideas. It also helps you think outside the box and look at things from a different perspective. So how does this all tie into productivity. Well, if you're a creator, then you should be engaging in creative thinking on a regular basis. The more you do it, the better your

brain becomes at thinking up ideas. This makes it easier for you to work on your projects because you won't get stuck as often. End quote. Now the phrasing makes sense. It's in a very casual style, and other parts of the blog post get, you know, even more casual, sometimes straying into grammatical error territory. It's not terribly precise, nor

is it saying anything really. The example I gave to a friend of mine is that this blog post is just like if I said, you know, if I'm caught outside when it starts pouring down rain, I get wet. I mean, yeah, that statement is true, but it's also, you know, not saying anything, or at least not anything that isn't already evident. All that being said, the blog post impresses the heck out of me. And that's because

the paragraphs follow in a logical pattern. It's not well written, but there's so much bad writing out there that it also doesn't stand out. If I had read this without knowing a computer generated it, I'm not certain I would pick up on it again. Not because it's great writing, but because I've read a lot of really bad writing out there. Heck, I've probably written some of it. Think of some of the content farms out there that post

thousands of blog posts a day. There's not as many as there were maybe you know, five years ago, but there's still quite a few. Well, a lot of that content is written in a very quick, slap dash style, and and no, no shade being thrown at the writers. They're trying to make a living, but it's not exactly well crafted work. This piece could have passed for one of those, and the piece does actually seem to build

on itself. New paragraphs reference a point made in an earlier paragraph, something that you didn't see so much of in other systems. New paragraphs build on those earlier ones, not in substantial ways, but there is a coherent link from one paragraph to the next. It's not as free form and absurd as other generative texts that I've seen. As for the autocorrect on our phones, those get more

individualized as we use them. Like I said, if I type a proper name like my dog tim Bolt, my phone starts to pick up on this that it's a word that has a particular meaning to me, that it's also a proper noun because I always capitalize it, and that it's not a typo, it's not a misspelling. So while the name wasn't in my phone's dictionary when I first got it, it has been added to that now that I've been using it so much, and it can even auto complete the name as I start to type.

Now we have some really impressive examples of generated text or generated language applications in AI. A couple of years ago, Google demonstrated how the Google Assistant could make a phone call to a real human being operated business and make an appointment for you. In a demonstration, the assistant called a hair salon and had a brief conversation with the salon employee to okay, haircut appointment, and it all sounded, you know, fairly natural. This approach to natural language recognition

and generative language is really powerful stuff. In this case, the assistant was relying upon certain parameters. Right The assistant knew which salon the user wanted to call. They knew

the time frame that the user had outlined as being appropriate. Uh. In this particular demonstration, it was an appointment slot anytime between ten am and twelve pm, and knew what day the user wanted an appointment and had all the basics, and then the assistant could respond to questions and statements from the salon employee on the phone and book the appointment, all without obviously revealing that it was an AI program.

The appearance is that the assistant is able to have persistent knowledge, but that's more of an illusion than anything else, it does show that computer scientists are making a lot of progress towards building systems that can generate language at if it's not deeply meaningful, can at least be useful. I'll close out was something that I covered at the

IBM Think Conference back in twenty nineteen. To demonstrate the power of the Watson platform, which is a foundation for various applications that all tap into deep AI processes, IBM organized a debate between a debate champion and a system called Project debater or, and the debate was on the topic of subsidizing preschools. IBM had drawn the pro side of the argument, and I got to watch this debate

live in person, and it was impressive. Not that I felt that Watson was able to outmaneuver the skilled, logical, eloquent human champion, but it was able to construct a pretty sound and consistent argument. It wasn't as strong and rhetoric, but it appeared to parse the flow of the debate properly for the most part, constructing arguments and supporting them with information wherever possible. It didn't come across as quite human,

but it was still really impressive. I think it will be quite some time before machines can generate text or speech at a level that compares with skilled humans, you know, humans who incorporate so many things from creativity to insight to intelligence in order to build communication. But progress is being made all the time, and thanks to a surplus of you know, not so great communication out there, we're more likely to not notice the computer generated stuff as

it improves. This opens up a lot of thorny problems. We've already got a problem with fake news. In a world where computer systems could generate endless blog posts and articles supporting narratives that don't reflect the truth, we're really going to be in trouble. And I think that's why this news about the blog post passing for a real

article should scare platforms like Facebook. If we reach a point where computers can lad Facebook with fake news and other computers are running bots that interact with that fake news, fewer people are going to stick around on that platform. They're going to it's just gonna get a turned to a cess pit of of total nonsense. You know, some people stick around, but a lot of people are just

gonna bail. People have been bailing already. We're gonna see a lot more leave, and once the advertisers get win that the majority of activity on Facebook isn't even human and therefore doesn't represent actual potential customers, advertising money will start to dry up, and then even a behemoth like Facebook could crumble. Now I'm not saying this is going to happen quickly, but I think it definitely could and probably will happen at least in some respect over the

course of the next few years. So hey, Facebook, maybe think about your oncoming existential crisis and you know, get ahead of it. It would be good for everybody, including your shareholders, and I know you really care about those alright. That wraps up this episode of tech Stuff and how artificial intelligence and machine learning and predictive text are all evolving rapidly in ways that are both cool and you know, concerning, if we're being totally honest, But I want to know

what you guys think. I also want to know if you have any suggestions for future episodes of tech Stuff. Reach out to me on Twitter. The handle is text stuff h s W and I'll talk to you again really soon. Text Stuff is an I Heart Radio production. For more podcasts from My Heart Radio, visit the I heart radio, app, Apple podcasts, or wherever you listen to your favorite shows.