Bloomberg Audio Studios, podcasts, radio news.
The Iran War showed little signs of de escalating this week, with Israeli forces targeting nuclear development sites in Iran, the US attacking Iranian mind laying vessels in the Strait of Hormuz, and Iranian drones striking Dubai International Airport. President Trump and Iran's new Supreme leader each vowed Thursday to keep up their respective military campaigns amid mounting oil prices.
Every day we lose about twenty percent of global oil supplies.
We are just shy of one hundred dollars a barrel on Brent now.
In a post on truth Social Trump wrote that preventing Iran from threatening the Middle East and having nuclear weapons was far more important to him than the cost of oil. Iran's new Supreme Leader, Iotolamus Stabahamane, said Iran would continue its blockade of the Strait of Hormuz, adding that he'd consider opening other fronts if the war continues. But the war has already expanded it to an important new front.
How it's being waged with AI enabled weaponry. In just the first twenty four hours of the Iran War, the US hit more than one thousand targets. According to the commander of US Forces, that is double the scale of the assault that opened the Iraq War in two thousand and three.
The decision to wage a campaign like this is intimately linked to how remotely the US can pursue this war.
That's Bloomberg National security and tech reporter Katrina Manson.
It's clearly really important for Donald Trump not to send US troops in on the ground, so conducting an air war with the help of AI suddenly becomes critical to his own pursuit of a war.
The US's first strikes on Iran came hours after the Pentagon severed its contract with Anthropic inn AI company that had been working directly with the US military. The Pentagon has formally notified Anthropic that it's deemed the artificial intelligence company and its products a supply chain risk to the United States. Pentagoon has said it needs the ability to
use Claude for quote all lawful purposes. Anthropic, which is best known for its clawed AI tool, wanted assurances from the US government that its technology would not make lethal decisions on its own or help to conduct mass surveillance on Americans. Anthropics ouster and a move by its rival Open Ai to start working with the Department of Defense have struck a nerve. Some Americans are a plauding Anthropics decision Claude went down last week because of unprecedented demand.
It's the latest turn in a long simmering debate over how the US military works with Silicon Valley companies and how those tech titans contribute to the development of AI weapons systems. Katrina took a deep dive into that story, and it's a big part of her forthcoming book on the US military and AI.
AI may be good for scale, it may be good for speed, but is it as good for accuracy and saving lives as these claims have been made in the past. We don't know yet.
I'm David Gera and this is the big take from Bloomberg News today on the show, as Anthropic and the Pentagon feud. How is the US integrating AI into its war fighting machine, How is AI enabled weaponry being regulated? And is the tech as good as it needs to be with so many lives at risk? About five years ago, Dario Amide and his sister left their jobs at OpenAI to found Anthropic, an AI company meant to enforce strong
safety guardrails. Since then, Anthropic has achieved mainstream success with Claude. It's even managed to partner with the US government, but that all changed at the start of the year. Bloomberg Senior editor Mike Shepherd has been following the saga closely.
The Pentagon began reviewing how it wanted to roll out artificial intelligence across the Armed Force, and in January they put out an order from Defense Secretary Pete Hegseth saying that they needed to accelerate adoption across the Defense Department, which he refers to as the Department of War. He even put up posters around the building saying the government
wants you to use AI as part of it. They included language that said they did not want to be bound by any usage restrictions that might come from an AI provider of any stripe. They didn't single out Anthropic, but Anthropic is not only one of the leading AI developers and providers in the world, they are one that has stood out for what many see as its safety first stance and its adherence to principles of trying to develop AI with a mind toward avoiding some of the
worst case scenarios. Now. For its part, Aanthropic has expressed continued interest in working with the military so long as the Pentagon A military officials abide by those usage restrictions.
And the two that have really been a redline for the company concern the use of its AI technology for mass surveillance of Americans domestically and then also fully autonomous weapons the climate they as the developers of the technology, insist that they don't have full confidence that it's ready for those uses safely.
Katrina, you have a book coming out later this month on the Pentagon's efforts to integrate AI into warfare. I'm curious when that initiative started. When did the Pentagon begin looking into this an interest.
The pentagonice to say it's been developing AI for sixty years. But the real project that I think we can set the timerby is Project Maven, which started in twenty seventeen, and it was an effort that really spoke to America's concern about falling behind China. The US began to realize it was using old tech and that this new age of warfare was coming that would require robots, autonomy, and AI, and Project Maven was this effort to experiment with AI.
At the time, the cutting edge was computer vision, things that could identify objects on a video feed taken by drones and process that quicker. But the people I spoke to also explained to me that all along they imagined that this would help with targeting.
You write about the software that comes out of Project man Maven smart System, which is made by Palenteer and incorporated technology from Amazon and Microsoft and others. Is this something the Defense Department thought they could do in house or was it always something that they needed the cooperation of private industry.
It's a real core point that obviously the Pentagon is still struggling with. For this very advanced tech, This recognition that has been happening over the past ten years that warfare is going to be what some call software defined, they need commercial companies, and so when Project Maven happened for the first time, really they were going outside the traditional primes as they're known, the traditional defense contractors and looking at companies like Google, Microsoft, and Amazon, but other
tiny startups as well who were just beginning to experiment with fascinating algorithms that could detect images on wedding cakes and then reapplying that for the tools of war, and that transition was incredibly uncomfortable, clunky, difficult, and they're backfired at one point rather spectacularly when in twenty eighteen, Google workers discovered that their company was working on Project Maven, and they feared that their tech and their traditions for
them could be subverted into what they ended up calling in a protest letter the business of war, and the parallel with Anthropic today isn't quite the same. The CEO of Anthropic all Along has said he believes in national security work more than any other AI lab. Anthropic has lent into classified work, and the classified cloud is where Penticon does its fighting, so they've been involved with lethal
operations in some way for more than a year. What I think is different is he had these red lines, and as he is trying to uphold them, tech workers in other companies are really paying note now.
Katrina ethics are so central to this program, and I'm curious how defense officials addressed those ethics of seeding life or death decision making to machines to artificial intelligence.
They would argue, I think very current now to the operations that are using AI tools against Iran, that humans are still making the decisions. But in a very interesting presentation by the Commander of US Forces just this week, they've made clear that they've hit five thousand, five hundred targets. That speed is exactly why they want AI and that scope, and the Pentagon has had to work very hard thinking through precision. The claim, of course is that US weapons
are the most precise in the world. But where you decide to put them, that is where America has had problems and the scale at which it's shooting at areas where we know civilians are present, because Scentcom is warning civilians to stay home, telling them to stay away from ports. There is a margin of error. There's a decision to accept or not accept a certain amount of collateral damage.
They haven't made public what margin they set on that collateral damage, but these are key ethical decisions in The targeting process is a multi stage effort where people feed in and eventually a commander makes a decision to sign off. But as you involve AI, that process speeds up. The decision making time is reduced, and fears for things like automation bias or the algorithms themselves going wrong. Hallucinating or drifting algorithms tend to naturally get worse over time. All
of that has really yet to be worked out. Now the US has got a policy for that. It's a directive. Some people to me frame it more as a process than a rule, but it does say that there needs to be human judgment over the appropriate use of force. Well,
that's supervision, that's not necessarily making a decision. And so I think from all the people I've spoken to, that role, that human role is reducing, and Sentcom has been particularly proud this week the commander to say that AI is helping them reduce operations decisions from what used to be days and hours two seconds.
After the break. How good is this technology really? We hear about one test where things went awry, and we dig into how lawmakers are responding. Proponents of AI enabled weapons say the technology can improve decision making on the back and put fewer troops at risk, which could save lives. Its promise of speed makes it imperative to national security experts who see it as essential in the widening AI arms race with China, but Bloomberg's Katrina Manson points out it also comes with risks.
AI is this fundamentally unpredictable black box technology. So it's brilliant at bringing a lot of data to bear. It just might be the wrong data, and it might be organized in the wrong way. And if you're not giving enough time to checking, or if you don't understand where it can make mistakes and you put it into critical a place in your own system, you've made yourself incredibly fallible. Now, from the US perspective, that brings a couple of problems.
One is you may end up killing civilians, which is against US policy. Certainly deliberate targeting inadvertent targeting under the rule of law has to be proportionate. But the second thing is you can also hurt your own troops and allied troops. And the US has a history of grappling with what are called friendly fire incidents, and AI was meant to help clear up this kind of claus witsy
and idea of the fog of war. One of the ways they've integrated AI into the battlefield picture is through the system that emerges from Project Man called Maven Smart System. For most of US, it's a really basic idea. It's essentially just a digital map. It's Google Maps for war. It's really hard for the military to create that because they have data in all sorts of places, they haven't
labeled their data. They have spent the last ten years really getting their equipment, their tech, their connectivity up to scale where something like this could work, but then integrating AI into it is really down to what checks they
make for it the algorithms themselves. I found cases where the quality of the algorithm, the ability of the algorithm to detect what it's meant to be identifying, can change if the weather changes, if you move an algorithm from a hot area to a green area, picking up something as simple as a truck or a tank, andinguishing between a man or a woman or a child, which is really critical to those life and death decisions and the rules under which the US military operates, really isn't at
the stage that you can give that over to algorithms. So the risk is if they are not sufficiently tire kicking their own algorithms when it comes to computer vision, or when it comes to using lms and chatbots to speed up their processes and analyze data, you could be going in the wrong direction for quite some time before you notice it.
Katran I wonder if you coul tell us a story of what happens when this technology doesn't work as it's intended to. And you write about a test that took place back in June of twenty twenty five, maybe you could just walk us through what happened then and what lessons we can draw from how that unintended result could pretend difficulties down the road.
The teams behind Maven had made computer vision and then this was the next iteration. This is where a program called replicator, so it would be to identify a target and then the drone could go after the target and then identify it and explode. And developing automatic target recognition is one thing that relies on creating algorithms that can find things. The other is this swarming technology coordinating between drones. Of course they were testing this. It's expected to fail,
that's why they do it in test conditions. And the aim was to deliver multiple thousands of drones by a certain date. And two months before that date, there was a test in California of some drone boats and in this one experiment, the drone was towed out to see
before it was to be switched into autonomy mode. That in itself was meant to be a safety measure What happened was, inadvertently a command was sent from the dock to the drone boat, and when the drone boat was activated into autonomy mode without anyone realizing it, it started trying to get away from the captain who was towing it. The boat started accelerating, decelerating multiple times between zero and six knots at pace semi circling action. The rope goes
taught and the captain is capsized. At that point, he's in the water, and then the drone boat turns and comes toward him. That's a very dangerous moment because the rope could strangle him underwater, he could be submerged. There's a runaway boat coming for him. A third boat was able to intervene and save him, and he was okay.
Then what actually happened in the investigation that ensued was that someone had mistakenly sent was called a zero command just by pressing enter on a command line, deployed the boat into autonomy mode, and then all of this ensued, so they fixed it. This is a very early stage of testing. They're meant to be putting explosives on these boats.
They're meant for these boats to be all cooperating together and able to defend an island such as Taiwan in the event of an invasion, And I think it showed that the tech is really hard to deliver and just simply not ready.
In a statement for Katrina's book, a Navy spokesperson said that safety is always their top priority, that they have backup systems to prevent danger. The lessons they learned during these events drive improvements in their systems. Mike, these conversations about ethics are happening among the Defense Department and these private companies. Where are lawmakers? Is Congress showcasing any interest in engaging with the subject.
Showcasing is a good word, because they do like to showcase their interest, but actually advancing a proposal that would codify some regulation or law even on how AI is deployed in warfare, we are a long way from that.
There's discussion of attaching some sort of amendment to the Annual Defense Authorization Measure, which every year usually gets caught up in the fight of the moment, and that could be where this ultimately ends up, But it's unclear whether there would be enough consensus between both parties to really come up with language. They all could agree on and that the Trump administration would not try to torpedo somehow itself.
Remember the tech industry, a lot of the big tech companies have really moved to align themselves with President Donald Trump in a lot of different ways, and it's difficult for them and a lot of other business fronts to challenge the president, including on this.
Katrina, I would ask you lastly, just about how much the horse is out of the barn here. I think there'll be a lot of people listening to this who will be impressed by how far this field has advanced, that is a integration into warfare, and may wonder if there's any recourse or anything that they can do they as citizens to slow down this process.
People talk about a costless war, We've already seen that from the US side alone. It's not a costless war. AI is meant to give you a riskless war. And probably where civilians and citizens come involved, are trying to understand the contours of is there such a thing as
a riskless war? And who is harmed by that? And if in any way AI isn't saving civilians or even there are misfires that involve AI, then really you have to re examine if it makes war more likely that in itself is a change for the way that the US had maybe been thinking about conducting wars under President Trump himself, who of course is the main political leader who has said I don't want any more wars.
Katrina Manson's book Project Mavin, A Marine Colonel, His Team and the Dawn of AI Warfare comes out later this month. This is the Big Take from Bloomberg News. I'm David Gura. To get more from The Big Take and unlimited access to all of Bloomberg dot com, subscribe today at Bloomberg dot com slash podcast offer. If you like this episode, make sure to follow and review The Big Take wherever you listen to podcasts. It helps people find the show. Thanks for listening. We'll be back tomorrow.