Welcome to CyberFocus from the McCrary Institute, where we explore the people and ideas shaping and defending our digital world. I'm your host, Frank Cilluffo, and have the privilege today to sit down with two luminaries from the intelligence community to talk about artificial intelligence. Hopefully find some of the signal through the noise and explain what it is, why
it matters, how it's being applied, and where we go from here. First, we have Teresa Shea, who ran signals intelligence at the National Security Agency, worked at In Q Tel, has more presidential and intelligence awards than we can count, and is just a phenomenal public servant, along with Glenn Gaffney, who ran science and technology at the Central Intelligence Agency. He, too, has more presidential rank and intelligence awards than anyone I know.
And they both were doing AI to one extent or another before it was cool. Teresa. Glenn, thank you for joining us today. And I thought we'd start with you. Ask seven people. You're gonna get seven different definitions for artificial intelligence, how it delineates from machine learning. Generative AI, general AI. Glenn, why don't we start with you and then, Teresa, jump in. Let's do some level setting to. To define the issues. Sure.
So I think I'll start with general AI. I'll start with the more complicated one and work backwards. Sounds good. To get to some building blocks that I think will be part of our conversation. But when we talk about general AI, we're really talking about a cyber brain, a bit of silicon, or a machine that creatively thinks and acts the way a brain acts and things. It's the thing that Hollywood dreams about and writes about, and people are most afraid of when they think about AI and
AI making decisions for which there's no control over those pieces. We're not there yet. There's a lot of work going on in the general AI space, and there's great progress being made. I say this not to scare people. I see real value in it. But when we're talking about general AI, we're talking about a thinking computer, an independently thinking computer. When we talk about generative AI, which is really the rage right
now, we're talking about models which are built up of algorithms and data. You have to have both. It's not just software. But once the machine learns on a corpus of data, it can then take what it has learned and then generate new content, generate new content around things like music and literature and art and any one of a number of different things. And so it has enough learning and enough fundamentals that it has learned from working on that Data that it then can create something original
in that space. Then we have AI, which is where most of the AI today to date has been focused until we saw. Until ChatGPT. Right. Changed everything. And with AI, we're looking at specific application. You train a model and that model then has a specific task and oftentimes is augmented to some other set of tasks or things that we do. They have different levels of attendant value and attendant risk. So a spam filter is a good example of an AI model that has learned about what
spam is and what spam isn't. A lot of times by you clicking reject that, that's spam, I don't want that. And the risk is relatively low. So we accept the value of it because the risk is once in a while we get some spam in our mailbox as an example of a simple function that we accept the risks almost every day and use it, not even realizing sometimes that we've used it and we've helped train the model. And then machine learning is really a fundamental building
block underneath all of that AI. It's the way a machine learns. And so you adopt a particular set of methods that are out there to help the machine learn from the corpus of data and then take what it's learned from that training set and then apply it to a larger corpus of data. Really well stated, Theresa. Anything
to add to that? I think no, I thought that was, of course, excellent. I've
had the pleasure of working with Glenn through the years and he is such a big thinker and I just love that. So it's great to be here with him. But a couple of practical points I would make, you know, in the machine learning space, for example, we've seen that applied in identifying faces and a whole corpus of data, photographs, for example. So you see that, you know, on your mobile phone every
day or in the. If you're using ChatGPT, which I hope everybody is using and practicing on, because that's a great tool for getting used to how AI works and asking questions and it creating great, sometimes content to answer those questions in whatever language you ask them in. And the only thing I would add to that wonderful description is warning, warning, red lights are going off here because not everything you get back
from these AI generating machines is accurate. So be sure and check the facts. So
the human in the loop will always be relevant. What do you think? Well, I
think it's critical. I'm a big believer in you need to have the human in the loop when the human was the reason for the loop in the first place, I didn't coin that phrase. I picked it up from somebody else. But it stuck with me for a long time. It's a good one, which means we actually have to spend the time. And like Teresa was saying, people should get familiar. People should be, you know, experimenting with ChatGPT. But experimenting is the right word. Try it out
on some things. Remember that it's a probability machine, not a truth machine, which means you need to spend some time looking at what it's brought back and evaluating that, and that's getting familiar with it and getting comfortable with it. On the experimentation side, I think, because the risks do change when we talk about implementation. I gave a
simple one on spam filter. But when we start talking about active cybersecurity and applying an AI process that says this is ongoing and I need to shut these things down, otherwise we're going to lose a system, there may be consequences to shutting things down that we're not ready to accept. And so we need to actually experiment and work through those things at a practical application level. And Teresa, your thoughts in terms
of the. The interaction between. At the end of the day, critical thinking is still, for now, largely a human skill. But I'd be curious. And not all of us do it, including myself. But what are your thoughts there? Yeah, no, I absolutely. Getting
back to this human in the loop, I'm 100% with Glenn on this, that this is going to be needed for the foreseeable future. And having that human, the critical thinking aspect that they bring, asking questions about the answers, that is exactly where you need that human brain, that it can infer different informations, different sets of information, and synthesize that in real time, unlike a machine today. So we have a lot of fear, I think, built up about AI and, you know, it taking over the world,
et cetera. I think we need to overcome that and start to learn how it works and find out where those risks are, where it's vulnerable, because I know we're going to talk about this, but, you know, it is vulnerable. There's certainly a lot of adversarial security. For AI as opposed to just AI. And how do the machines learn? They learn from data. We all know data can be poisoned, it can be mislabeled. It has its own vulnerabilities that can cause results. In your model, in your
AI model that you use in the end, dirty. Data in, you're going to get
dirty data out. That's the reality for our students. Don't just go to ChatGPT to write your term papers is the bottom line. But let's get to sort of that. Red blue security. The cybersecurity implications around AI and we've had previous guests pontificate and make some very strong and bold statements. Phil Venables just sticks out because he was very strong on AI will help. The defender will help. What do you guys think on that? I think it has to help the defender. Cybersecurity in
particular is one of the areas that I feel like we need to apply AI first and foremost because the sheer level of complexity and speed that is at hand in all of the networks that we have, all of the machine to machine communication, let alone the things that are nefarious. Right, that are there, it's easier and easier to gain access to a machine. It happens quicker and quicker every day. The amount of time it takes to set up shop inside someone's network continues to be there
and exists. But the opportunities are huge in that space. There's so much coming at the cybersecurity professional today that the augmented cybersecurity professional let machines do what they do best, which is talk to and understand signal to noise from other machines, and then let them bring to the forefront the things that the cybersecurity specialist needs to pay attention to right now, present some consequence or prioritization, begin to tune that professional to
what's happening and where their attention needs to be and the speed at which it needs to be applied. And I think what we're going to see, and I believe we need to see it here first, is this evolution of people, machine teams. So we can talk about the augmented analyst, we can talk about the augmented cybersecurity professional.
But this teaming arrangement is really important. At the first level. We need to experiment with it and think about the outcomes, look at the outcomes and test some outcomes in that space, and then realize that the human machine team is a system in and of itself, which means it will have its own set of emergent behaviors that
come from that interaction. And it's one of those things that I think is important for us to realize is that you can create a system where the human being does everything the way that they were trained to do, the machine does everything the way it was trained to do, and you still get a bad outcome because we haven't actually learned and developed these things together as a system. It's a thoughtful set
of issues. Theresa, any thoughts on that in terms of AI for the Defender vis a vis? There are a lot of concerns also that AI improves the capacity of the attacker. And for a long time the initiative has remained with the attacker. But do you think we could flip that equation a little bit? Well, you know, you're
right, Frank. AI is certainly a dual use technology as most technologies are today. So it can definitely be used by the offense as well as the. And assume it
is. And it. Absolutely. We have examples of where it's being used on both sides.
Right. And on the defense though, I think Phil's right in that this is really going to empower the defender to act with speed. It's going to be able to. Machines don't get tired, they work 247 and they are constantly can be trained to look for example, anomalies in user behavior, anomalies in devices, things happening on your network, your complex networks that Glenn was referring to that alert you to be able to
take action faster. So you're being able to isolate attacks. The whole idea is raising that bar on the attacker. And I do think that's going to benefit the defender like it hasn't in the past. And I think that's the way we started Glenn
unpacking the definitional we've been doing for fraud and for spam filled. AI has been around for years and machine learning and other modalities, but it's actually now getting to the point where I think you can realize if you can get to that interaction in a constructive kind of way, yield some great rewards. Right? Absolutely. And I think
there's plenty of published examples of machines beating humans at games, but chess in particular, but when you have the human and the machine playing together as a team, it beats the machine almost every time. And that's, that's the power of that teamwork and that interaction, you know. But don't lose sight. I mean, let's, let's pause here for
a moment. This isn't going to be easy. Not at all. Right. It's going. AI
is never is. You know, costs money, it's going to take resources, it's going to
be operational costs, maintenance costs. But also people. Yeah, you know, training people. That's what
I was going to ask. What do you think the workforce of tomorrow looks like? Because it is going to require both. I mean one side of you says to do what humans do best and bring that critical thinking, but they also need to understand what's behind the machine to understand how to maximize that. Right. What would you be. And we're jumping way ahead. I wanted to get to this at the close, but what do you think the student of tomorrow should be focused. On today, I
think one of the things we're seeing even now is that the nature of the business in terms of the utilization of these AI engines is that we're seeing the shift begin from the data scientists to really, I think some of the early data is showing that it's quality analysis or people with an analytic background that seem to be getting the most out of these systems the fastest or the earliest, because. They'Re
looking at it systematically. They're looking at it systematically. They also been trained and have
some experience in taking a problem and breaking it down to its constituent pieces and then asking the right questions. All right, and so the, the art of the, the art of asking the right question and framing the question with a knowledge base around how was the model trained, what was it trained? Iterating and then iterating on that, right, is a, is an application of critical thinking. Right. And bringing the machine into
that process. And so I feel, I feel that we should be training, right, educating this current generation as well as the upcoming generation, earlier and earlier ages in not just the critical thinking process. But how do you break that down into its fundamental questions and then how do you begin to leverage the power that is the machine that's there to assist you in this space? Teresa, when I think public information, public,
the President's daily brief, majority of what's in there is going to be gleaned through signals intelligence. Just because it's so much data at the end of the day. You've been doing this to some extent or another for a while, right? Trying to make sense of it, of all the data that comes in. So what do you think that career path looks like for a siginternational? Yeah, I am very excited for the
signatures of today and tomorrow. First of all, this is Teresa Shea speaking. I'm not, of course, a government employee any longer. And so, you know, this is a personal, personal opinion, but we're, we're seeing the commercial world and the intelligence world increasingly overlap and AI is no different. You're seeing, for example, you know, we used to really
struggle with characterizing signals in the RF environment. Now they're using AI in RF and they're able to characterize signals and identify them immediately, which is just going to enable you to be able to filter on the ones that you want to get to those important ones, which helps get you to processing. The processing state is the next
state where you're now applying machines to conduct your processing. They understand and can identify different protocols and get to underlying text much faster than in the past now we haven't talked yet, but the translation and the large language models, that's a big issue. That's being able to do translation at speed. And just using the human analyst for the very, very tip top of that pyramid is just raising the level of pristine
intelligence that they're able to produce. And I'm excited for it, which is. Exciting, there's
no question. But I also think that for someone going to the intelligence community, they probably need to be thinking some of these issues through beforehand. Right. I mean, there needs to be some training. So, yeah, when we were talking, I couldn't agree more
about getting the education of AI and the use of AI across the spectrum, from my grandchildren to my children, etc. But remember, these generations are growing up with machines, with technology in their hands, so they're used to it now. I don't know when the last time you went into an intelligence agency, but you still can't take your phones in. You have to leave your devices. Unless it's an OURA ring. Right. That's
the one that I think is permissible. But you know, they are stepping back in
time. It's still true today that they're stepping back in time. And we have got to figure out how we bring these worlds more and more together. And I hope we're going to talk a little bit about open source and you know what that's going to bring to the forefront, what it's already bringing to the forefront. Absolutely. And
I do want to jump into open source, but the ethical questions are legitimate here because this may be trite, but there's certain decisions back to the human in the loop. I only want someone who's sworn to the Constitution to make on behalf of our country. Right, right. You don't ever see that fully changing, do you? I mean, science fiction would maybe think. Otherwise, but I don't see it changing. But I think
this. Got to be careful it doesn't. And I think there's a, there's another set of steps that are required in terms of this ethics and guardrails discussion. Because one of the things that I'm concerned about is that there'll be mistakes made in the application of it and then we're going to get knocked back. Right. And typical. And
go in correct. Because what happens typically is somebody will make a mistake and then we'll draw the policy line or the guideline three steps before that could ever happen again and we start slowing things down. Meanwhile, our near competitors are going to keep pushing forward and pushing forward. And so we have to be aggressive in developing and
testing these models. And I think that one of the things that I call for regularly is I believe we need to establish proving grounds that are focused on key aspects of application. So AI in cybersecurity for critical infrastructure and then begin to test a range of outcomes and include the social sciences and the public. Because the user is not just the person or the consumer is not just the person affected by the answer to the question they ask the the consumer is everyone who's affected by
the decision that gets made. And so at what point is it acceptable to shut down some aspect of the power grid, as an example, and what elements of that power grid? And as a society, we will mature and embrace different levels of risk associated with the value and the associated risk with those decision space. But that has to be tested. And in doing that, not only do we continue to move this forward, but we build a whole new level of trust with the public sector in
the way that we're approaching using these things. And that's true whether you're a government, you're the government or you are one of the companies that is developing AI solutions for application, right? Broadly, absolutely. Theresa, anything to add on that? Yeah, no, I completely
agree. You know, I do think that you're seeing some of these AI safety committees being formed within the government, within the IC, the intelligence community, DoD, but as well as in the private sector. And there's an awareness out there that you have to worry about bias. For example, you have to worry about protecting privacy and civil liberties because you're using a corpus of data that could potentially have personal information in there.
And we just had an AI officer for Department of Treasury on. So they actually have a council at OMB of AI officers. So it's sort of like a cto, a CEO, a ciso, AI is now starting to make that. And I believe the
administration called for every government agency to have a lead person in that space to develop these. Right. Responsible for it, which I think is all great and it's all critical, but I think again, to maintain that positive and competitive advantage in terms of operationalizing its use, we need to actually develop these kind of what I'll call open proving grounds where government, industry and the citizens, our citizens can engage. And I think
that's a. It's not just that opportunity, it's an opportunity for trust building. All right. But it's also going to help move us forward as a society in critical application that will really stand. It'll separate us from the competition. Let's Put it that way, yeah. But trust is the coin of the realm. And I'm sure you speak from
experience where a chilling effect could be in the that you make a mistake and no one's allowed to make a mistake anymore. Here's the reality. We're never going to be the intelligence community. They're estimators, they're not clairvoyance. Hugely important. Well said. Hugely important
point. And it goes. And it's why I think the intelligence community should have an advantage. And it's not new for the intelligence community. As Teresawas saying, and you said earlier, we've been using machine learning and AI and it grew out of our advanced modeling efforts for years. But these machines aren't truth machines, they're probability machines. And the
intelligence business is a probability business at the end of the day. And that's guided by expert opinion and critical thinking and quality analysis and questioning your sources all the time. Absolutely. Let's get to the open source discussion. I mean, the reality is, as
I'd say, it has its day in the sun and some would say it should have had it a long time ago. But the reality is I don't think you can go into compartmented discussion now and not have open source information drive a lot of that discussion. Am I wrong on that? I don't know if you're wrong on
that. I'm hoping you're not. I haven't been inside for a while in these discussions, but I believe that that's absolutely the way it needs to develop. I see the development of using again, models, advanced models, in gathering what is open, just like we're talking about focusing the cybersecurity professional's attention. I almost think a friend of mine, Bob
Glykoff, talks about each of us having an AI recommender on our shoulder. Right. For the tasks that we have to do and the things that we have to approach. And if we think about what that means for the analyst, whether they're an energy analyst or a Treasury analyst, the ability to see what's out there and make connections, discover patterns, alert those patterns to the critical eye and the critical thinker that can then evaluate those. That's gold, right? What I would have done for. I wish I
had AI engines and advanced modeling techniques like we have today. Back 100 years ago when I was doing analysis because we were using punch cards and writing models and it was a challenge, but the advanced modeling helped. The power of it today would be fantastic. Absolutely. Theresa, any thoughts? Yeah, no, I agree. I think this has been
true for a while. Right. I think that, you know, when I got out in 2015, I couldn't believe what threat intelligence companies were doing in the real world. I was like, what? That'd be classified on the inside. You know, so we do have to get over this classification concerns what should be classified, what shouldn't be classified. You know, we've got a culture that we've built up over decades and decades for all
the right reasons. And that really turning that ship is going to be, I think, key to being able to really bring in and embrace the open source and use it. And I love the way you said, you know, on your shoulder, kind of that recommender, the analysts all having access to imagine how they could get their jobs done if they could have that feeding their intelligence every day. And it really helped
with attribution and bringing the American people to have a better appreciation and understanding of what they're dealing with here. And I think to underscore something that Teresa was just
talking about is I think there's a real role for the intelligence community to play in the open world and in these public private partnerships, because so many of these things used to be the purview of nation states and nation states alone. They're not anymore. These tools are out there, the data is out there. But what the intelligence community and the national security community have is years of experience in working through these
knotty, difficult problems and applying some of these tools and techniques. And so the insight that's been gained through hard knocks and experiences behind the fence line can be shared outside the fence line without getting into the details of the problem that they were trying to solve or the cell that they were trying to uncover. And the community needs, I think, to both embrace what's out there, to help them shed some of
their guilty knowledge and then lean into sharing their expertise. That'll help everybody move forward.
And they would have to be able to build on that to be able to do the exquisite piece better. Absolutely. So in a weird way, you can't. Why compete but actually get even better? It should be mutually enforcing, and the public sector shouldn't
have to guess what the next set of tough questions would be. Yeah, and I
think we're getting there, aren't we? When I was in government, this is a thousand years ago. If it didn't have the stamp on it, people didn't read it. I don't think that's the mindset right now. I hope it's not. I don't think it
is. I think that's. I think that is changing. I Think there's still. It's a very human condition to be focused on what's immediately around you. And this gets to, you know, Teresa's point about all the things you can't bring into the building and all the things that you can't do while you're. Sometimes for good reason. And for good reason, there are good reasons for it. Absolutely. But the community needs to embrace what's out there in the open and look at the best way to help propel
it forward and to bring what's valuable inside. And so the example that I often use is Google Earth, right. Which has got nothing to do with AI. Well, actually it does now. I stopped myself. It does now. It didn't in the beginning, but in the beginning, the idea of a commercially available searchable system that was unclassified and just because the quality wasn't as good as some of the more sophisticated systems that were there, a lot of folks in the community thought, why would we ever use
that? How would we ever use it? And it was a very short period of time to where you went from people asking that question to you rarely went to a meeting that you didn't have a commercial image on the table. And then based on the understanding, you would then go apply more sophisticated sources and methods to try to get after some problem. And so I see the same thing happening now with, in this next evolution of open source. I hope we are. I feel like we're
moving in that direction and I hope we continue to, because I do think government's role is essential, but I also think that it can always use some calibrating and titrating based on what you can get cheaper, quicker, easier. The trap to be avoided,
I think, is the trap of the community feeling like they're competing with folks on the outside doing it right. And so if, if the community adopts a position of, well, their open source analysis isn't as good because they don't have access to the good stuff. We've taken the wrong posture. You bring up such an important point. And
both of you sat at an entity that I'm a huge fan of in Q Tel, where you have a full appreciation for the government's customers needs and requirements and everything else, which a lot of industry does not have, but also can build on the innovation and prowess of our entrepreneurs in this country. What do you think that, what are your thoughts in terms of how you blend both those worlds together? So
you're right. And I think that the advantage that our nation has always had as its greatest advantage, our Ability to. To innovate. We have the smartest people in the world and they're constantly thinking of these new and better and better ideas. When we were at in Qitel, learning what the hardest of the hard problems were that couldn't be solved because both the government and the commercial sectors are innovating very effectively to
solve problems today and to advance our nation in the economy. And so bringing those together hugely beneficial was a, in my opinion, you know, it was a very good thing. The hard part about that, Frank, is getting it into the operational baselines of the intelligence community. And is that because not invented here still or. Well, that's certainly part of it. Because there's a culture part of it. Right. The acquisition process, certainly
part of it. You know, the far is the far and they're very, you know, it's very regimented and you've certainly had enough conversations about that to know how hard that is to get over. So having the mindset, I think, in the intelligence community to take that risk and try it and it may not work. I think you had said earlier, you know, you can't fail anymore. Well, we need to be able to fail. And you fail fast and you recover fast, right? Absolutely. No company would
exist if they didn't. That's right. The other piece that I think is important here
when you're talking about whether it's startups or research, internal or external research, is we actually have to create margin for adoption. Because what happens is they will, we will, we have, we will build something, we'll bring something in from a startup, we'll test it, and then we'll, we'll make comments on it, we'll adjust it, it'll work. But
there's no room left in the program to bring anything in. Whether you're talking about program of record in an acquisition cycle or you're talking about the pace of operations and the cost of maintaining those operations. One of the things that I know we had to do and I believe needs to be done increasingly across the government, is that leaders need to actually purposefully create margin for adoption and be looking at the
things that are. Looking at the things that are being developed and tested and say this one needs to move into the operational analytic field or the operational field. And here's the margin that we've created to have that introduction and build that space in, which is hard to do unless you're starting a new program of record these days.
And so it sustained to go beyond the traditional acquisition cycles. If you're not going
to create margin you're going to lose your innovation edge because you'll always keep it to be a sub element of your budget over here because it's important to have, but it won't transition into the flow. Both Glenn and I of our previous agency
have seen where this has had huge impact when they are able to get it into operations. Hugely important. And it does come to the trust equation, which is the
coin of the realm. Right. Because unless you're seeing it day in day, it takes one time to lose trust and it takes a long time to build trust. That's
right. Especially in your fields and where people's livelihoods are at risk. Risk. So it's
understandable. But the only way you're going to see that combined is if you're in the same foxhole and you can actually learn from it and then build and iterate and improve. Right? That's right. How do we fix that? This is totally off where we were going, but that is a big issue that the government, I think, needs to at least provide some Runway. Yeah, I think there's been some interesting developments over
the last year or so where the government has put information out early. Right. To lay what I'll call the, you know, a factual baseline ahead of other things that have been coming out. Right. You're talking like RFIs or what are we. Yeah, no, I'm. Because we're off, we're off our line here a little bit. I'm grappling for the example, but there was, there was in anticipation of some things coming out from one of, from one of our competitor countries. I think the intelligence community put some
factual data out early. Right. And it had, I think, a really positive impact in this space. I think that kind of trust building, sharing that information, being more public in that space I think is important. I'm not sure if that's where you're going, but that's one of those areas in building trust. Because I mean, all things said
and done, we have to build in the ability to A, make mistakes, B, learn from it and see, continue to innovate. Because if, if this may be again going off piste in our discussion here, but I think one of the thing that things that differentiates our great country from others is, is the ability to innovate. It is the ability to fail. Fail and still be able to succeed. In previous countries, you fail once, you're never getting the credit from a bank to succeed again. Or I
don't want anyone waking up saying, I wish I were born in Beijing. I think that's our differential or anywhere else or Tehran or Moscow or whatever it may be. Point is, I think that is our. Secret sauce that in coupled I think with
something that you mentioned before about the tools being in the hands of people who have sworn an oath to protect and defend the continent. Take that seriously. And. And the protection in the defense of the Constitution is about the people more than it is about the paper. Bingo. Which manifests itself in the things that Teresawas talking about in terms of you take the privacy of American citizens and our allies seriously. Right.
And so you build systems. And this is where I get back to that proving ground concept. Because when the American people understand the level to which the rule of law drives as much as the innovation drives and the mission drives and they see the rules sets that are put in place, the precautions that are taken and the steps that are in place, if those things are violated, it changes the trust equation. It's not just some big opaque machine to be afraid of. You're right. Right. And
technology is moving faster than laws. So we got to figure out how to get to that trust. It always will. Right. Technology always changes the law at some level
or another. Right. And but again it's through that social impact piece and that exploration that we. Define then how the law changes application and technology. And it's always, it's
always the application. Gunpowder was gum. It was. It's always the application used. Right. It's
always the application. So let's talk then. You sort of opened up the discussion on
what some of our peer competitor, adversarial countries, and I'll be blunt, autocratic regimes, by and large, we can't afford to lose this. Right. The consequences are severe. You know,
I think we started this whole conversation around AI and that's absolutely a game, technological game changer. There's certainly other technologies I would put in that bucket as well. Microelectronics, biotechnologies, for example, supersonic weapons. What we've seen all of those technologies, the imperative of, and this is a race who's going to use and operate those technologies the fastest because we're all going to have some access to them and who's ahead
in that. And some say the US is currently ahead in AI. To stay ahead is a different question. And we have to continue to invest not just in the technologies and the development and the innovation, but also in the people. And we talked
about people in education. Let's not underestimate our adversaries either. I mean the amounts of
money and resources, for example, that China's Pouring into AI the way they're working together. China, Russia, Iran, North Korea. Let's not underestimate their partnerships and what they can do when they combine forces. Especially when they don't have to worry about protecting their citizens.
That's right, yeah. And in some regimes, they test things out on their citizens first. Right. And so like misinformation and disinformation. Absolutely. And so to underscore, build on Teresa's point, the US and our allies have, in my opinion, the best research in the world, and we do lead. What China is happy to do for now is to concede a certain amount of that lead as long as they can operationalize our thought leadership faster than we do. And that's where the autocracy has an advantage,
because democracies are inefficient by design. And so we have to be purposeful and deliberate in investing in what I'll call democratic principles of digital design. For how do we, if we're going to compete and we have a more inefficient system, but we have the best research and the best financial instruments in the world, how do we bring
them together in new and innovative ways in order to try, fail? Let a number of different approaches develop, but get it into an operational space at a speed that challenges the autocracies of the world. That's a great point because, I mean, you look
at unrestricted warfare, which was basically everything all of us in think tanks wrote years ago in their initial planning and strategy and doctrine around cyber. They just acted on it. They weren't all that innovative in terms of the ideas, but they acted on it. And that. That is a good point. It gets to the application discussion in a different kind of way. Think slow, act fast. Love it. Love it. That is
a good point. In terms of sort of public private partnership. We touched on this discussion in many different ways, but in terms of critical infrastructure in particular, where most of these companies didn't go into business thinking they had to defend themselves against foreign intelligence services or foreign militaries, and many of them are finding themselves in that position, and some are so critical to our economy, into our public safety and public health
that we have a responsibility. Where do you see AI fitting into all that? Into critical infrastructure and application? There. And I put Grid at the very top of the list. And the reality is operational technology. Some of this is decades old, but I can't differentiate it from OT anymore. The world's blending. We can try to put cute bows around it, but the truth is it's all coming together and the bad guy has a vote in how this all comes together. So we've got to start. And
I think that's a really important piece because in the end, it's the thing that actually keeps me up at night more than anything else. Is that the IT side? No, it's that through normal economic competition we're going to give up our infrastructure. Right.
Because. Because it's easier to pay somebody else to do it. Yeah. Right. And when you're, and when you're buying infrastructure as a service or you're buying services that are actually, I'll say it a different way, you're buying services that actually underlay your infrastructure. Right. You don't realize you're making infrastructure choices when you're just working with a, a
best bid on automating aspects of your city. As an example, if we're not actually thinking about the AI in what it brings in helping in that management of city functions, in developing a city, and we just approach it as consumers, we run the risk of buying a solution that changes our infrastructure without a paying attention to it.
And I think we're seeing that play out in sectors right now. Theresa, anything to add on that? So we talked a lot about how AI is enabling the defender,
and I think this is true in the critical infrastructure space. But it's very complex. And the critical infrastructure you talked about, the grid, it's a complex system of systems with many interactions and interfaces, and that's very hard to defend. So being able to use AI and train your machines to again, some of those basic functions, like looking
for those anomalies and understanding what user behavior looks like in a normal space. So when you see something different, you can predict in a lot of cases, if you're seeing something different on one of your many, many log files or machine measurement indicators, anything like that, if you can actually use AI to do that, I think you're not going to miss as many opportunities to get ahead of where you're going. It's
a really important point because you can actually do red versus blue in a modeling environment. They tell you where those adversarial. Adversarial, adversarial and tell you where the evolve, what, what your evolving threat face looks like. Because the highest probability of a threat vector coming at me might have been here last week, but it might be over here next week because of two or three other services or things that got added
into the space. Right. And so the ability to put that kind of compute power, modeling power to work, I think is really, really important. The Other thing on public private partnership here is we have to do this in public private partnership for places like the energy sector, because it got privatized. They own and operate. Yeah. And so we have no choice but to do that in a public private kind of partnership.
And I think the government has a role to play in helping buy down risk in this space, just like the government bought down risk relative to vaccine development right in the face of the pandemic. And I think if we look at these critical sectors that Teresa highlighted earlier, that the critical sectors of competition, I think there are key aspects of how to buy down risk in each of those that will help propel us forward. But that, again, is a public private partnership discussion, just like we
did in the vaccine development. If you think about the attack surface itself, it just
grows exponentially. So there I. Even if you wanted to, you can't do it without a genuine. And I'd go beyond partnership. I'd say collaborative. It has to be. That's
right. It has to be collaborative. And to your point, it's beyond our borders too.
Right. We're talking about. We have to bring in the international space here. Let me
ask both of you, because I've been told a pessimist is an optimist with experience. I'm still the ever optimist. Are you both optimistic as long as we're wary and cognizant of the risk? Optimistic, absolutely. On the AI sets of issues? Absolutely. And excited
and passionate about it. And if Sherman Kent were around, father of analysis, what would
he be thinking about AI today? So I think. I think. I don't know, but
I hope not. Then you're gonna get every conspiracy theorist. Yeah, yeah, exactly. I don't know. But I'll tell you, the analytic training, right. That Sherman Kent, really, you know, there's a reason why the school's named after him, right. I mean, he drove so much of the trade craft of analysis. And I think this tradecraft of analysis is exactly what needs to be continue to be developed, but then taught and extended throughout the department of education and school systems and training all over.
I think there's an interesting piece because with generative AI, we will have machines that generate insight. And how do you source that? Right. We know how to source signals and we know how to source humans, and we know how to source images. When the machine creates content that we go. That needs to be paid attention to. Think
about academic citations as well. I know the consequences may not be. As severe, but
as you think about it, you think it may only take months to work out a model that produces something brilliant in this space. It may take a lot longer to figure out how you're going to source it and report on it and what probabilities you're going to assess on that data on that content. And so it's an
important piece. And so I think the evolution of AI is not just a technical and application and recommend her on your shoulder, but it's a how we think about, handle and act on the information that comes out in applications like we've been talking about in cybersecurity and energy security, grid, power grid, but also in the way we report it. Yeah, that's a fascinating set of issues. And the cryptologists and cryptographers, many
of whom are nameless, who literally won World War II, what would they be thinking today? That's a, that's, that's a thought provoking question. Because they were brilliant. I mean
they, you know, they were. They won the war. The women and men, many of
whom we will never know their names. That's right, yeah. There's a great book called
the Code Girls. It talks just about all the women that, you know, came to, came to break the codes and did a. Yeah, kick butt. I like that. Yep. You know, I think that that's a foundation upon which we've built and if they could use tools like AI to speed up again, accelerate and get to answers faster. Now Glenn makes an excellent point about this ability to be able to have explainable AI. We have to be able to answer questions about the sourcing and how he
came to that answer. And there's a lot of research going on today to try and improve upon that and get us to a better space. And they would be part of that research. Right. They would help break that down into a dissolvable problem, which is awesome. What questions didn't I ask that I should have? I took us
on a journey I wasn't expecting, but it was awesome, so I enjoyed it. But what questions didn't I ask that I should, should have? I don't know that it's
a question, but it's a, it's a, it's all. About the question, isn't it? May, it's, maybe it's, maybe this is a, it's a last point on my part, but the, one of the reasons why I'm optimistic is because we are going to be
creating new infrastructure. And so as we think about new energy solutions and what's going to be required for distribution in that space and what does it look like to Buy down risk in that space companies so that incumbent energy companies and startup companies can all come and actually develop and test and get into a place where we
can invest and accelerate. Well, thanks. I was trying to avoid saying it again, but my point here is that we have the opportunity to build cybersecurity, data security model, security, AI into the design. And it's true there. It's true in the biotech sector. We have a tremendous infrastructure that is yet to be built in the biotech sector because it's going to be bio in everything. And I don't see how you separate.
It's converging. That's right. And so I look at that and I say this is
an opportunity for us to do this by design and not try to bolt it on after the fact like we've been doing for the last 50 years. Well said,
Theresa. A great lesson we learned in cybersecurity, right? Yep. So we have a presidential
election coming up, Frank. There's multiple elections going on around the world this year. And I think a better understanding of deep fakes and what's real and what's not so that, you know, normal individuals can understand if they're being manipulated, taken for a ride or not. I think that would be an interesting conversation. I just saw a new not for profit called True Media that allows you to put a video in there and it'll identify, at least give you some indications of where to flag and tag.
Yeah, that's awesome. Yeah. I mean, truth is, is that's not just one country. Multiple countries are engaging in that activity. Thank you to both of you, not only for spending so much time with us today, but for fighting the good fight for so many years. Our country is better off because of you. And I just wanted to say thank you. So thank you both. Thank you. Thank you.