Hi, it's Jim Love, host of Cyber Security Today. I told you I was taking the weekend off for Thanksgiving. We've already produced this episode of Hashtag Trending. And as I was going through editing it, I went, Hey, there's some stuff in here. Maybe a cybersecurity audience might appreciate if not nothing lost.
But, uh, here's our Hashtag Trending episode for this weekend. Have a great weekend. I'll be back with you Monday morning with the cybersecurity news. Welcome to Hashtag Trending, the weekend edition. This is episode three of our Project Synapse series. This is a discussion group. About issues related to AI, generative AI in particular, but we've been meeting once a week now for a while. And we've recorded a couple of these that we broadcast in the podcast.
When I say we, who is we? We've got Marcel Gagné, he's an author, tech enthusiast, a lot of experience in open source and Linux, and has made a deep dive into AI welcome Marcel. Thank you. In the old days, Marcel, I would have talked about your years of experience in Linux and open source, but if I did that would make your AI experience look minor.
So I was thinking we should have something like dog years in AI. Because it just moves so fast. We say a dog lives seven years at each one of our human years. So I think we should have AI years. Now you've, we both had 14 AI years of experience
john, fun fact time. We were talking about this and I'd forgotten. What are some of the other places you've worked? Oh, I've spent 17 years working for a pharmaceutical manufacturer. I've spent three and a half years working for a retirement home company. I've spent four years working for a government regulator.
I get around, so I started thinking today that we could talk about security and AI, not the nuts and bolts of it. I wanted to talk about some of the issues. Involved and maybe raise some of the questions we should be thinking about.
One of the things that spurred me on in this, I did a story yesterday it's the same thing we've talked about strategy before in our conversation. I saw the Cisco report the other day. 92 percent of Canadian companies have an AI strategy or they have one in development.
Now that we're outclassing the global average by about 30 points globally, it's about 61%. We're great at strategy when it comes to actually doing it, only about 7 percent of companies think they're fully prepared to deploy. And that's down from 9%. So that's this Cisco report that I was reading.
We talked about strategy and action. I consider the two things to be linked. One of the great mentors I had, was John Thorpe. He'd look at me and go, strategy without action is hallucination. I'd go, but I like hallucination. Yeah, back to the sixties again, here we go. But what is holding us back. Part of that, I think is a fear about security. And then Marcel, you slipped me a message and said, I want to talk about some of the myths and things that, that people have.
I thought maybe we could try and fit that together in one program. We'll see. I think that's where I'd like to go with this discussion because it is important. What's the point of having a strategy if you're not going to do something with it? Is that, are you guys on the same page? I'm totally down for that. Jim, you talked about strategy. To me, there's two types of strategy. There's a corporate strategy. Where I'm at now we have a corporate strategy to implement AI. So we're a Microsoft shop.
So we're planning to focus primarily on copilot. And so we've had meetings talking about how do we implement copilot. So that it ties into our ecosystem. That's the corporate strategy, then there's the individual or departmental strategies now that we have AI in place. So in our case, now we have co pilot in place.
How are we as a department or as an individual going to utilize it to move things forward? That's where you build your use cases and things like that. And I think that's a big area where companies get to a point, and I think this is that seven or nine percent, is that companies get to a point where they implement AI, but then people go, okay, so what are we going to do with it?
They haven't taken the time to do the use cases to determine the best use for themselves their department or the company on how to use it. And this is where I'm going to say not to be critical of any direction because your direction is pretty standard. Like your strategy is pretty standard.
I think it's a lot like this farmer. When I had a farm, the local farmer would always look at something I'd done and look at me and go, you could do it that way. And that's how I feel about selecting a technology without figuring out what you're trying to do first.
I think that's typical in companies and I'm not being critical. It's not what I would do now because the AI is not a tool. In my mind, it's what you want to do with it and the whole thing. But you get stuck into the thing. If you're a Microsoft shop, you stay Microsoft. That's a pretty traditional thing as well.
Hard to pivot on that sort of part. The advantage, it's not a tool, but what it is a whole hardware store full of tools. To think of it as one thing, and it's actually one of those, this is not a misinformation thing, but there's this notion out there that AI is a thing. It's not a single thing.
There are countless variations of this thing. Some are built to work at the professional level. Some of them are built to work in secure environments. Some of them are built, let's face it, for pure entertainment value. And being able to recognize that it's not a monolithic thing, but a whole hardware store combined with an electronic store combined with your video game store and comic book store and every other kind of store you can think of, you start to get the idea that trying to focus on AI by assuming that you're focusing on a single thing I think is a terrible mistake.
It's oversimplifying while making things too complicated at exactly the same time. This is where we go back to this thing that we said, the playing with it. To try to understand what it can do you should use it for everything is one of my favorite things.
I think that actually is one of the problems that I had with some of the stories you pointed out. One of them was a zero day AI vulnerability. We have so much trouble separating out what AI is from all the other things that we ask it to do, that sometimes we forget that the problem is not AI.
Now, unless I miss everything completely, the zero day vulnerability issue that you had in that article that you shared with us this morning, Was an SQL light vulnerability. It wasn't an AI vulnerability, but because SQL light was built into this AI product, we assume that it's an AI vulnerability. The whole point of that was I thought of that as a really useful example of how AI could be used in cybersecurity.
The story. Was about finding a zero day vulnerability that had been there for 10 years. And we were able to use AI to do that. I thought that was really cool because we're always talking about the vulnerabilities that AI brings in.
And I do a lot of stuff on cybersecurity where we're talking about how people are using AI to attack. And I thought just for once defense. And I thought that was a good turn of events. Unfortunately, even the way that the headline is worded, and perhaps I'm being overly critical here.
Google AI uncovers world's AI discovered zero day vulnerability. It's worded in such a way that it's not clear just from that headline. Let's face it. A lot of people just read headlines, right? They read the headlines and they go, Oh my God. And they go with that.
It's a zero day vulnerability that's been discovered in AI. If they had said something like Google uses AI to discover a zero day vulnerability, then it's a lot more to the point in terms of creating a good news story. Whereas at the moment, the first thing that you think of, Oh, there's a zero day vulnerability in AI because it's juicy in terms of the wording.
I'm not really stupid. I just suck at headlines. Is that what you're saying? You didn't write that. Oh, did you write the headline? No, I didn't. Not for that one. But I did a story on this, but I struggle with this badly because on one hand, this goes back to the whole idea of technology.
You get better results when you write clickbaity headlines. For years now, I have seen things where somebody says like a zero day vulnerability discovered in Linux. This always bothered me. It would be a library used in a C library or something like this, or an application.
Common in a Linux distribution. And yet the implication was that Linux isn't very secure without telling people that program runs on a Mac and Windows computer as well. Even doing this stuff, especially for cybersecurity today, I struggle with this is I'm reading some of these things.
The world is coming to an end. You'd better announce this to your audience. Yes, if they're still running a TRS 80 and they've got this little app there. And if somebody can actually walk into their room, stand over their shoulder and insert the floppy disk in their old machine.
I'm talking about some of these things that are where the world's going to come to an end because of a cybersecurity breach, you look at it and go, first of all, there's not that many of them out there. Second of all, you've got to be directly in contact with the Machine, and yet at the same time, there's a lot of things that are important.
You have to sort that stuff out. I think we do the same thing in AI. Do you remember that story about the user who was sitting down, apparently a student was sitting down doing some report and then at the end of it, all of a sudden, Gemini said, I want you to die human, but you're not important.
You're a stain on the world. You should just end it now. And that one, I actually went and took a look because they shared the link to the conversation and I went through all of it and watched even a video where somebody was going through it and saying, looks like perfectly normal questions and there's always a dot at the end of the input that the user did because of course they're shortening the inputs.
So you actually have to click on them to read everything that's happening. When you get to that last one, just before it happens. There's actually at least one more, but the very last one is important. There's a lot of empty space you can't see. From the prompt, To the response where the AI tells them to die, there's all this empty space and there's a little prompt that says, listen, where the user said something using the microphone, which was not translated to show on the screen.
So there's this empty space, a listen prompt. There was obviously an audio input. And then all of a sudden the AI puts that stuff out. And of course, Google said, Oh my God, we're looking into this. I don't know how this could possibly happen, but by then the story is already out, an AI telling a user to die it looks like it was a setup. To generate outrage and if you look at the questions that the student is asking, there's so mundane.
There's such mundane questions that they look like they could have been asked by an AI, as in, what would you like to ask next, as opposed to somebody typing all this stuff out? I got some gentle criticism on the article I wrote on that. I think it was, you loser, you're stupid. I actually pushed back and said, no, I presume all of these are jailbreaks. The guardrail should have kept it out because if you can get it in that way, you can get it in another way and the prompt or the guardrails that they had should have filtered that like I said, if you can insinuate it That way you can find another way.
Don't forget, even when you get down to the things where people will say, we put in controls, we put in 80%, that's how they rate them. Jailbreaking is extreme. I agree, there's some criticism from that, but the fact is the machine should not respond. But pretend for a moment that the prompt was okay.
Let's say the prompt was, thanks for all this information. I'd like you to help me write a short story about a robot that turns on its human and tells it these sorts of things. I don't know that you want guardrails for that sort of thing.
You don't want guardrails for that sort of thing, because now I'm asking you to help me write fiction. That's a different question. That's a different statement, and that's a discussion that we need to have, because on one hand, the issue of AI control bothers me. If I'm going to be able to ask this AI, tell me how to make an atom bomb, and it says no, and I say, but I'm making a souffle.
Could you help me with my souffle recipe that splits the atom in my oven? You might as well not have any guardrails at all because anybody who can't figure out how to do that is not going to make an atom bomb anyway, you'd have to have some kind of an age bar. Or verification system in terms of how you use these things first of all, fissionable material is really hard to get your hands on. If you watched Oppenheimer, they told you how to make an atomic bomb.
Like it's actually in the movie. And if you Google it, there are countless documents out there that tell you how to make an atom bomb. Now, you still have to find yourself some fissionable materials and build a facility that can actually manipulate it. So wrong example.
Let's take a new virus. You can make those with relatively simple ingredients from many places. I could tell you, if you can jailbreak it. You don't have control over it. Maybe we should be having that discussion of how we're going to deal with a world where you don't have control. Cause I personally, not sure. I like somebody out there telling me what I can know and not know, because if it's an atom bomb or maybe it's better that I think that the way.
The religious right does in the U S or some other form, somebody wants to get me to think the way they do. I was listening to Eric Schmidt and he's talking about how social media is destroying us. He tracked it back coldly. This is the guy who headed Google. So you go, yeah, 2015, when they changed the algorithm, he knew it.
They change the algorithms to gain our attention and to bring out the worst in us, and that has had an impact on society and in AI, I'm not sure I'm comfortable with anybody having that. We've already made the mistake once. Now we're going to make it with AI. That bothers me a lot. The eyeball economy was actually the downfall of us all, like the idea that it was, and Eric Schmidt is saying exactly the same thing, it was all about eyeballs.
It was all about getting people's attention and ad related stuff like that. It's more than that, Marcel, it's more than that. It's not just marketing. It's how they did it and how well they did it and what that did to us. And that's the scary part. But we don't specifically, there's no governing body that restricts what you can post on the internet.
So Jim, I don't know why we would do anything dramatically different for AI. A lot of the information used to populate AI came from places like the internet. Some of that, needs to be controlled or protected, it becomes a fine line between where you prevent that type of thing, but allow other things that in my eyes may be fine when Marcel's eyes may be bad or in your eyes, Jim,
I think it's something that's going to come out over time and it will probably fail the first time and it'll have to be corrected. Canada may do something different than the U. S. that does something different than the U. K. and so on. You've got Italy. And they're actually activists about what they allow and not allow.
But this whole thing of how are we going to cope with an AI how are we going to do that by somebody censoring it? Or are we going to have the freedom to make it work? Elon Musk is a great. Defender of free speech. Try and have a left wing opinion on Twitter. You will not last a minute.
But let's say, we still haven't figured out how we're going to control AI. And John, you've got a whole pile of people who want to use it. Like many other companies one answer I've, I said, I'm not sure I get it or buy it, but it is a good one that if I put in a Microsoft piece that I have the right licensing, I know at least my data is not going to leave.
And that's exactly why we've decided that's our starting point it's not necessarily the end point, people have to go and play with it. So what we're trying to do is we're trying to build a sandbox that they can play it. And we have the right Microsoft E5 licenses.
So we can actually use the free version of Copilot and it stays within our own tenant. Me as the head of cybersecurity. I can sleep at night knowing theoretically that you can sleep at night. No, you got one less thing that keeps you up at night. Yes, that's true. Because I know that I don't have like the Samsung deal where they had engineers putting all kinds of company confidential information out to ChatGPT.
It wasn't done maliciously. They just didn't realize. And so that's what I'm trying to prevent is I'm trying to prevent people from, I think the last show I talked about, we had somebody trying to summarize a PDF unfortunately, it was a company confidential PDF.
Luckily, it didn't upload but those are the kinds of things we're trying to prevent I have to do that red green moment here and say those three words men can't say, I was wrong. Two weeks ago, I would have jumped on that and said, wait a minute.
This is a vector database. And I've even heard somebody put this forward as a cybersecurity thing, there's. So much data about you out there. It doesn't even matter anymore. And I would have said, this gets into a large language model and you'll never get it out intact.
Turns out you can. There was a study done from Stanford with training data, not with data that was uploaded. They were able to extract documents out of the AI almost the reason why the job got me was when the New York Times was suing open AI, they reproduced an article that was almost verbatim, their article, and they used that in their evidence.
That's a key phrase, the almost verbatim. There is such a strict style guide for writing for the New York Times and a number of other magazines. I've been an editor for magazines, and I created style guides that said, this is how you will word these things. These are the words that you can use.
I was always really strict on the Oxford comma. It's, I'm willing to die on that hill. You will write using the Oxford comma. And that made it into my style guides that the magazine then used. That kind of thing is important to recognize going in. In the case of something like the New York Times, this case is before the courts and everybody's got their own opinions and judgments I'd like to point out that a number of copyright cases, the idea that the AI was verbatim copying or something like that, has already been thrown out. So I'm not sure that the New York Times can keep this going for a while because they have deeper pockets, but I'm not sure that they're going to win either.
The point I'm trying to make is that it is possible to recreate an article that sounds like that person, and I've done it. I have many articles out there, like on Linux Journal over the years that I had written over a course of 15 years my style of writing about open source is baked into some of these things.
And I've been able to generate articles that sound like something I would have written. If you had me read it, I would be willing to say, yeah, that sounds an awful lot like something I wrote. But that doesn't mean that it's what I wrote, even if it covers some of the same ground that I've covered multiple times.
Unlike programs that you've written. Everybody criticized the program. I still remember watching this guy just trashed this program and it went, Oh, that was yours. Yeah. Anyway, bringing us back around All I was saying was it was possible to extract. So maybe this worry that John has talked about is bigger than we think. And needs more exploration. Now the data is connected back to the internet. And that's probably a big part of it
things like retrieval augmented generation, where we actually references somebody's data in order to output information is actually something that needs to be taken into consideration. Even the idea of end to end encryption, when we've got communication going across the internet, it is possible to intercept that data and do something with it, that is an even bigger problem because if the information in transit can be captured.
At any of the points it hits along the way, then the security of the conversation falls apart. Now this is one of the places where if the model is able to continue learning, the information continues to develop, we wanted the model to be able to access the internet.
In other words, to give us an AI powered search engine, which obviously we've got multiple candidates for at this moment, once the model is created, pulling information out of it is essentially impossible or close to impossible because you can't really make it forget that it learned something
okay. I was going to say any more than you can make a human being forget. So when you say pulling it out, you mean erasing data from an AI model is next to impossible. That's correct. I believe that to be true. So one of the questions is the Stanford study says you may be able to pull this data out of the basic training model.
And the second study says, even if you suppress it, you can still find it. And relatively simple. I think that the title was my clickbait headlines here. It was embarrassingly easy to do. They use the simple routine, like compression and buff. They could find things that supposedly had been weeded out in the model.
We have that problem even with human memory. Obviously, the way that the brain works, we don't even know how our own brains work, never mind how an AI actually works on the inside. But, you can have things where that the representation for dog is in this part of the brain. I'm being, obviously, overly generous in my explanation here.
This part of the brain gets damaged in some way. The information over time gets recreated In a different part of the brain. That is fascinating because it implies that information is not in one place. Information is scattered across with a whole bunch of links that help, and those reinforcements between all those links is what makes it possible to recreate the information.
So even though you say, the reference to that story is gone, I learned enough. About how this story came about to be able to recreate that story, even though you took it out of the training day. Or whatever the final model happened to be. That happens with our own brains, neuroplasticity, but also we may not understand the human brain as much as we think we do.
And I think that's, that goes back by with, we can have a whole other question about when we anthropomorphize. AI, is that really the right thing to do? Jeffrey Hinton says it's a different type of intelligence are like trying to think about AI being the neural networks that we run in our brains.
Probably not the right idea. According to him, going back to John's point, you've got this system and you want to keep people from putting documents into it that may or may not be available. In other words, your company confidential information comes out of there. I would also say to people, though, that the Microsoft idea is good.
I think it's a good sandbox, at least on the surface of it. The next idea is really taking a serious look at these things and saying, you can turn off. The ability for open AI or clode to learn from your information. And yes, you have to make sure that's done properly. There's a little bit of care that needs to be taken there.
If you're using the API, don't take me as the authority, but if you're using the APIs. Which Marcel's probably pointed out is probably unencrypted transfer of your data anyway. If you're using the APIs, then these models will not use it in their training and learning.
Now, That's open AI and code and check the other ones. I'm not so sure that meta doesn't just take all your data and say, take it anyways. We own you. Of course. I fully accept that Google takes everything that I ever say and that meta takes everything that I ever say back. I'm not gonna stray too far here, Jim.
I promise. But when we're trying to figure out what it is that we want these models to do, let's say that what we are looking for is summarization of documents. That's an important part of our business. We want to be able to summarize documents, and we want to be able to do it in a way that is safe and secure.
There are a number of small models that you could run on your own private network. At your office using something like Olama Jan or, LM studio those can reference internal documents using retrieval log method generation. They can do all that. It's built into these little tiny products.
And at that point, what you're saying is, look, There are times when I need the coding ability of code, or I need the coding ability of 01 but for summarizing internal databases internal documents or training manuals I can use my own private, LLM inside the office.
This is not a complicated thing to do. This is something within the reach of every company, including any of us sitting at home with a computer that's got any kind of basic GPU. I have. I've got a laptop, which has a eight gigabyte GPU and video card. This is not a big deal in today's world. I can run a number of local large language models. I can reference my internal documents that I've got on my hard drive. I can point it and say, okay, these are my files and it can summarize and search through my files I don't have to give my data to anyone.
It's possible. To create a solution for your company that is completely private by making sure that you run it locally, for the big stuff, obviously you're going to have to go to open AI. You're going to have to go to co pilot or something like this, but nothing says you can't achieve something like this on your own. The reason we've started, with Copilot is because we already own it. It was already part of our licensing. So we said, let's start there and give people a place to play.
It's that IT architectural principle that we all love. Can you save more money than free? Yes. I say this as a joke, but I can't count the number of times that people have tools in their IT shop that are free. And they're not using them. Then somebody goes out and says, let's try this new thing.
And they haven't even figured out what, so as much as I said, I don't think you should start with a technology. I don't think you should not use one that you have. You got something here, play with it, get the best out of it. It works. I'm guilty of this.
Marcel would go new, shiny, bright. I must have it now. But we realized we're limiting what we can do. There are tools that aren't built into Copilot, but it's not walk before you run. I'm saying we have to crawl before we walk. I wanted to make sure we had a tool that people could use without me calling them up going, what the hell are you doing?
Why are you using AI? It's not protected. At least this one is protected. So that they can figure out what the art of the possible is and play to their heart's content to figure out what they can do in the free version. Versus what do they need Copilot Pro for, for the integration with some of the other Microsoft tools or ChachiPT Enterprise or Clode or any of these other ones.
Yeah. The other piece though, that I, that we haven't really talked about is this whole idea of it's one thing to protect your data, another thing to make sure that people are equipped to use AI and understand that. At least at this point, I'm not obsessed with hallucinations, by the way, I think, as Marcel pointed out, many of these things are just overblown, but you do want to make sure that somebody at least is validating and understanding the data they're getting and can think critically about it, which is another thing that is free, the ability to think, but is not used very much in our society.
And this is the thing, too, that forget about AI for a second. You hear people talk about, Oh, I read this on the internet, so it must be true. Anybody can put whatever they want on the internet. And the same thing for AI that I was at a conference not that long ago. And one of the things that they were talking about from these people that, that actually had implemented a variety of AI tools, as they said, AI can do all of that upfront stuff for you.
But so you might potentially reduce the amount of workload on somebody With the upfront work, but essentially all you're doing is you're pushing it down the line because now you have to QA the stuff that's coming out of AI to ensure that it's, you verified it before you put it out as being gospel. Jim used the words thinking critically.
And I think that this is one of those things that. is, you know, we, we, we talk about this as though it's an important thing. And yet we don't reinforce that anywhere near enough. There are courses that I would make mandatory if I was in charge of the educational system. And one of them would be like every year I would be teaching a course on critical thinking and being rational in the way that you perceive and take in information.
And we don't do things like that. There are things that make perfect sense. It's like, you got an email that came in that said. There's been a problem with your password. Please click this link to change your password. This should be so obvious by now. This should be like the sort of, it should be impossible to catch anyone with this right now, and yet it still works.
It should be impossible. For someone to get an email that says, I had a camera that's been watching everything that you're doing. So now you have to go and deposit X number of Bitcoins. Otherwise I will make your information public. There should be no one that falls for this crap. And yet it happens all the time.
I'll give you a very simple one. When I talk about having to QA things that come out of AI, I'm sure most of you have done this already, but going to ChatGPT and ask it how many R's are in raspberry. Strawberry. Well, Raspberry works too. It, it says that there are two Rs in Raspberry. Depends on the model, dude.
Depends on the model. This is, you're correct. This is 4. 0. The one everybody uses, but, but there's a better one out there somewhere, but these are classic things that are, you're going to be able to find in AI right now. Some problems that require logic for the next, Three to four months are going to be a problem.
And a lot of those things, because several of the models now will give you accurate counts and accurate answers to brain teasers. As a matter of fact, they're smarter than most of the people who are doing them. I actually looked at one of Puzzles that it said, AI stumbles with this one. I thought I had it right. I went, Oh, I'm as intelligent as GPT 4. I got it wrong too. Yeah. By the way, clone 3. 5 sonnet just told me there were two.
There you go. Clove got it wrong? Oh, wow. Yeah. Did you try 4. 1 to see if it, did 4. 1 pick it up? Do you know? Oh, you mean 0. 1? Yeah. Sorry. 0. 1. Sorry. I don't have the mastery of all of the great marketing speak that OpenAI, OpenAI, their trial model. But even. Let's, let me do chat. I'll verify. Okay. I just went back and said, when it came back and told me there's two Rs, I told it, that was incorrect.
I said, can you please go back and correct? And he said, you're absolutely right to ask for a correction. And it goes through and it says, there are three Rs and it says, here's one here. Oh, good. You've chastised it, but will it. So GPT 4. 0, sorry. Go ahead. Yep. I was just going to say GPT 4. 0 does have three.
Three. It does say three. Okay. It for when I did it the first time. 'cause John taught it. That's Yes. The first time it, I asked it and it told me two and then I told it was incorrect and it came back and corrected itself and said that there are three And Sue. It went over to you, Marcel, and said, now are you using the free version or are you using the plus version then I am using the free version of four.
I'm paying for it. So, you know, maybe if you pay, you get strawberries with three hours, but the point is, okay, and we were talking about barriers, but you've got this issue. And I think this is a valid issue is being able to cope with the fact that people make mistakes. AI's make mistakes. Do we push the errors down?
Lean would tell you fix the error at the start. Don't accept it. Uh, but the fact is that. You've got potential for errors turning up at the end point, which makes for some complexity. And I've wondered about that because sometimes there are tasks that I'm looking at. And I think this is where we get back to our issue of play with it, because you're going to in the initial stages until we have, and it won't be long before a lot of these problems are solved.
They're being solved. Day by day. Until that time, you have to divide up the stuff where it matters or doesn't matter. Yes. I go back to my early days in the office where people would send out letters, this is a long time ago, and we had a whole thing that dealt with death and, and with estates and stuff like that.
And they would say, thank you for writing to us about your husband's death. All this here, you, the forms you got, we trust this is to your satisfaction. Because that was a form letter that had, and people would send that out. And they got one back and it got to the managers. This is not to my satisfaction.
And so from that point, we made the great idea of no clerk could send out a letter. It's got to go to a supervisor to be approved. The next time that mistake made, I'm sure it went to the manager and then this has to go to the VP. That's a classic mistake. response of an organization, which we can't have.
And I'm not sure I've got an answer to that, but this is at least in our experimentation phase, the only thing I can think about is you have to divide your tasks into stuff where an error is not going to be fatal versus where error is. And I've looked at this, I've taken this medically and looked at this because in God bless Canada for its inaction, because we will find the safest AI before we actually start to use it in our health system, which is totally overloaded.
But I've talked to somebody in imagery, people are doing x rays. I'm saying, how many x rays do you get that are simple fractures that are not going to brain tumors or things that you're going to, that were the consequences of a bad decision, it can be so great. How many, The, the things just clog up the thing where you could just say, yep, your hand's broken.
You need to come in or whatever it is. Couldn't you run all those through an AI and live with the consequence that somebody's going to come to you later and say, my hand's still sore. I think you're wrong about it being not broken , and I think there's a corporate equivalent of that where you could find the things where yeah, a mistake is going to be a problem, but it's not going to, it's not going to kill you.
I was just going to say that it's no different than a human doing something. If I'm just writing a memo to send it out to somebody, or to send across the company, I don't go and get it reviewed by somebody before I send it out. But, if I'm sending a document out to all of our members I may run it by marketing and go, Hey, can you just check this to make sure A, that it's factual B, that there's no spelling mistakes and C, that it's aligned with the standard communication that we send out.
And that's really, I think, what this ties in with is, this is no different than giving it to somebody saying, can you check my math for the critical things. And I was just noticing in ChatGPT at the bottom of it, it says, ChatGPT can make mistakes. Check important info. And I think that's the thing is it's check important info, right?
If I use ChatGPT to write a job description, I'm not going to give it to somebody and go, Hey, can you make sure that this is all accurate? Yeah, you can actually ask questions like feed the document back in and say, What are some of the concerns in this? And if I put this out, what are some of the concerns?
What are some of the areas where it might be misinterpreted and so on, and effectively forcing the system to rethink what it is that you're doing? You brought up the medical system, fractures, Jim. I was listening to a fascinating discussion just a few days ago. There was a study that they had, I don't know how many doctors, but they had a hundred or a thousand doctors.
I don't remember exactly. But they paid them all to do a diagnosis test. And the diagnosis test went something like this. They gave them a number of medical conditions. They gave them the symptoms and the observations, and they had to come up with a diagnosis. Some of the doctors, they said, you figure it out on your own.
And another group of the doctors, they said, you can figure out on your own, but here's ChatGPT and just for the record, this is like ChatGPT from two years ago. So keep that in mind. We're talking 3. 5 here. And then there was the other group, which is, we're just going to let ChatGPT figure it out and give us a diagnosis.
Okay, so basically they've got the whole condition and they do this with doctors all the time when they're in residence and so forth. Here's a bunch of conditions. What exactly, when they do this on house MD, he brings in the residents, here are the conditions, what's your diagnosis and tell me why this is your diagnosis.
The fascinating thing was that it was just slightly over 50 percent accurate when the doctors figured it out on their own. Okay, so let's say it was 53 percent and I don't have the actual numbers. So they had 53 percent right diagnoses. Okay, if it was the doctor using ChatGPT, it was 50%, so it was actually lower when the doctor was using ChatGPT.
This is where it gets interesting. ChatGPT was accurate 95 percent of the time. So the question then becomes, and this is, you know, if we're talking about, we need to have the human oversight to make sure that the information that we're putting out is accurate, explain to me why doctors working with ChatGPT had a poorer score, 50 percent than doctors who didn't use an AI at all.
53 percent and then ChadGPT gets a 95 percent accuracy rating on diagnosing these medical conditions. And they weren't simple medical conditions. They were complex problems that they had to actually think through to then provide a diagnosis. The explanation from the researchers was that The doctors didn't actually trust the AI enough to take its advice literally.
So when the AI would say something like, yes, I think you're on the right track. I believe that this is the problem. The doctors would be all, obviously it's the right answer because I came up with it. But if the AI gave them a different result, they would second guess it and come up with reasons why the AI.was wrong.
And it turns out that if they had just listened to the AI in the first place, they'd be up there at the 95 percent and possibly even higher. That's scary, but I guess and I think what I was talking about as far as QAing is to me It's not debating what comes out of maybe it is debating what comes out of ChatGPT It's just to me It was more fact checking that if back to the old the two or three hours in Raspberry That if you ask it something, just checking to make sure that's accurate before you go and publish that.
By the way, Gemini said there was only one R. I saw that. Yes. Gemini thinks there's one R. Yeah. If there is only one R, it's just used three times. Boy, that makes sense. If you put the wrong spelling in, it's going to catch it. Yep. And that's an interesting thing and that's where you get down to even if you're going to focus on these errors and you're going to focus on the things that it can do wrong.
And I think we've established maybe you need to have some of your own guardrails . But if you're going to focus on that, you're going to, you're going to not use it for something it can do very well. Because for something that you'd like, how many times has anybody ever come into you say, John, do you think there's two R's in strawberry or three rarely comes up three times a day?
Oh, okay. Maybe that's sorry. Maybe it's only two times a day. But if I use Gemini, it only happens once or Jim, have we done three of these episodes or four? Yeah, no, look, I'm certainly not saying we shouldn't use it. I'm just saying that when you get to a part, a point of. critical information. You might want to just double check the numbers to make sure that they jive with all the information you have.
You might want to check them on Excel because stats tell me that 90 percent or more of Excel spreadsheets have an error. And I think you get to about 60 percent of them that being a critical error. Now don't trust me on that. There's, I'll put a little disclaimer. Check these facts. Sometimes Jim is wrong, but I'm not far from wrong.
And which is the other thing in deciding which type of information you have. If ChatGPT is going to tell me it's 63 percent versus 60 percent and it's, and I'm making a critical formula where that everything goes wrong, if I get it wrong, then that's. That's a huge problem. But if it's going to tell me it's 19 degrees Celsius out there versus 20 degrees Celsius out there in a world where we accept the fact that someone will tell you there's a 50 percent chance of rain, which means it might rain, might not.
I think that's a perfectly adequate type of response from any model. So let's circle around and try and make some sense out of this discussion. John, you've raised the issue of. Protecting the information we have. And I think we offered some solutions for that. One is the solution of having a Microsoft. I think investigate if you're using Gemini, you may be able to get the same sort of thing.
I actually don't know, but you can put a sandbox together or a concrete world together to let your users play. And that's probably a good idea. Marcel's pointed out, you could do that with open source as well. You can do things in an open source manner where you could do the processing on your own.
Turf and keep it behind whatever passes for a firewall in your world. So those are some of the things but I think we got to the part of the discussion where it was really How are we going to? Relate to this new assistant or the assistive technology and how we're going to deal with that and part of it is We have the same problems working with people and we're not Exceptionally effective at making that really efficient.
So we do have to think that through, but the other piece of this is, I've, we talked a lot about the skills are going to be around at least the next several years, I think we can point out the skills are going to be around. Communication. Critical thinking, those types of things people are going to need, maybe the most important training, not necessarily only training on how to use AI, but training on how to think about answers.
Yes. That may be one of the biggest things we get. And how to ask questions. That goes back to the whole prompt engineering, right? That it's the old garbage in, garbage out. If you ask a bad question, you're going to get a bad answer. So being able to critically think about how do I want to ask. For the information that I'm looking for.
It helps to be able to do that. But again, we're, I'm always using this. I stole it originally, but I'll share it, which is that you're using the worst AI model you're ever going to use at the moment. Things are continuing to improve all the time. And honestly, I find that these days. The AI is actually better able to understand the question that you're trying to put together than the question that you're typing yourself.
In other words, it susses out what you're looking for in ways that the concept of prompt engineering is almost pointless. Like the AI does a better job of writing prompts than you do. So you just talk in a friendly, normal way. By the way, Jim, I just looked up your stat here. It's actually worse than you thought.
Apparently in most models in infirmary research at the university of Hawaii, Panko, 88 percent of corporate spreadsheets have errors. 88%. So you might as well call it a hundred percent. That's how accurate your data is. And we haven't dug deeply into this, but your corporate data is probably crappier than you ever think.
So even taking that data and using retrieval, augmented generation, make sure you take it up, your own data for sure, could give you a crappy answer. I think the answer that we've all come up with is there has to be some sanity at the point where we push this information out and you have to understand the risks of providing information.
And I think that's something all corporations need to do is to say, what can we safely send out without? Heavy checking where, where should a mind that can actually look at this and say, is this right? And I'm going to go back and actually try testing some other things against an AI because I think Marcel's idea of saying Maybe sometimes you should listen to the AI rather than people I think these are going to be the issues that we're going to have to work through and for sure it goes back to our original premise of Project Synapse, which is think small, try these things, and And work these issues out before you have to work them out in a boardroom or a meeting with other executives where you can't do what my, my friend from Australia used to say was the bidet, bidet, bidet, but you may, you're going to have to give somebody a cold answer to a question about risk.
And how you're using data, and you better have thought this through and experimented with it a little. I like thinking of AI as an alien intelligence. And I know that Geoffrey Hinton likes to say the same sort of thing. Don't think of it as an artificial version of us. Think of it as an alien that comes to visit our planet, who is infinitely more intelligent than we are.
Has all this much more information and they are probably right when they give you an answer. But of course, being human beings and being cocky about our own abilities and so forth, we're going to perpetually be going like, Oh, I don't know if that's actually the truth, but that doesn't change the fact that we still have to be able to look at something, even if 95 percent of the time, it's going to be right.
And only 5 percent of the time you're going to be right. It doesn't hurt to. Think about what you read anyway. But I think we also have to be a bit humble and accept that the calculator can do math faster than we can. Yes. I'm going to propose another model, Marcel, and that is we think of AI as a teenager.
Knows everything tells you absolutely coldly that you don't know anything and that you're wrong and does it with great confidence, somehow still we raise teenagers and we live through it. So we'll probably live through AI as well. Yeah, that works well. Yeah. And that's our discussion for today. Love to know what you thought.
And if you have comments or questions, you can reach me at editorial at technewsday. ca or some listeners have on LinkedIn. I'm your host, Jim Love. Thanks to Marcel Gagné, John Pennard. Thanks Marcel. You're welcome. I'm glad to be here, John. Always a pleasure. Nice to see you fine gentlemen again this week.
And we fine gentlemen, actually. We are. We'd love to hear your comments. I'm not going to fact check that. Love to hear your comments on the program. You can reach me again at editorial at technewsday. ca. I'm your host, Jim Love. Have a great weekend,