Zone Media. Hello and welcome to this week's Better Offline Monologue. I'm your host ed zetron. I heard something really worrying
the other day about a major hyperscaler. According to the source, said hyperscaler was allowing and even encouraging non technical workers to deploy code to consumer facing products, specifically those who cannot read or write code vibe coding their own projects using generative AI, with their code at some point theoretically reviewed by an actual software engineer before it gets pushed
to production. The cold comfort of that review is that it assumes that software engineers are at least the ones reviewing that code are actually adept at code review, or even if they are, that they have sufficient time to
look over the overlea verbose code that lllms spew. In some cases, I've heard management is actively encouraging and even mandating these non technical workers to use lms to make these features, creating a mutation of tech debt where somebody who cannot code uses a machine that doesn't think to create code with no intention that nobody really understands, and does so at such a velocity that it burdens the actual technical workers with constantly having to monitor and fix it.
LLMS do not understand anything, nor do they think, which means any solutions they build or theoretical bug reports that they may make are immediately questionable. Their hallucinations are such that even features you believe a part of your code, after all you can't read it might not be there, or might be poorly designed, or might have some sort of unforeseen problem that neither you nor the LM are aware of, because its training data is based on code
that already exists, versus any ability to solve novel problems. Also, the idea of having any number of non technical peopleship code is fucking insane, an indicative of an overwhelming ignorance
on the part of management. Of even a few years of having overwhelming amounts of code written by LMS, even by engineers who know what it says and have some intention in the prompts they do, is going to create a situation where most of the code is written without any intention, making it much harder to the debug because nobody really knows why it was written that way, because
LMS doesn't think. I know. I'm repeating myself already, but this situation is chilling me, even outside of the vibe coding. There's a larger problem of developers writing code with llms that they barely review in some cases because they don't feel they need to and just skim read it. In many many more because their bosses are demanding their ship features faster than is responsible or safe. Remember many many tech companies are mandating llmuse harassing their workers checking how
much they use lllms. I've heard this from multiple companies, and really it's not just for them, but it's especially hard on software engineers. Adding a layer of code written by people who quite literally do not understand what it says or does guarantees future situations where major services simply break. And the more of this nonsensical code that's allowed to be stood up on these services, the harder it will be to fix. Code isn't just something you write once
and leave forever. It needs to be maintained by other people. Sometimes he is in the future, especially when people keep being laid off, which becomes much harder when there's lots of code to go through written by an LLM piloted by somebody who doesn't know what they're actually doing and relies on the LM, which doesn't know anything to tell them what's going on. Again, I realize I'm repeating myself, But llms do not have thoughts or feelings of knowledge.
They are generating based on the parameters of their training data, which means that all of their code is at best an abstraction of somebody else's back and forth with a chatbot. The code is not written efficiently or with any consideration of who might have to work with it in the future, or indeed what other things might be involved. Lms only know what they are fed or what they're connected to.
They don't know the nuances of code. They don't know the nuances of software engineering or architecture, which means that they only know so much based on their training data and the environment around them. Kind of. They don't know the nuances of how service let's say Meta or Microsoft's has been built over decades, and indeed, the more of this code that's used to build those services, the less that these companies know about how their actual fucking software works.
This is setting up the software industry for disaster after disaster, and it's already started to happen. To quote the information from this week, according to internal Meta communications and an incident report seen by the Information A major security alert occurred last week after a meta software engineer used an in house agent tool similar to openclaw to analyze the technical question that another employee had posted on an internal
discussion forum. After doing the analysis, the AI agent posted a response in the discussion forum to the original question, offering advice on the technical issue. According to internal communications, the agent did so without approval from the employee. It's so cool that this is happening. It's so cool, it's great, it's actually brilliant. How fucking insan what? And I'm I'm kindly going to assume that the person using this knew
what they were doing. But the idea that we have and I think this is what's happening with open source too. We have people with llms who are like, yeah, well the LM tells me I'm good, so I must be. Let me just run this LM pasture problems. It's why we're getting all these junk pull requests on GitHub on open source projects. People that think they're competent because an LM told them to are fucking up the entire software world.
And according to the Information, Meta systems storing large amounts of company and user related data were accessible to engineers who didn't have permission to see them, and this was marked as second one incident, the second highest level of severity on an internal scale that Meta uses to rank security incidents. And again that's quoting the information. The incident follows multiple problems caused to Amazon by its Kero and qllms.
I quote Business Inside is Eugene Kim. On March second, customers across Amazon marketplaces saw incorrect delivery times when adding items to their carts. The incident led to nearly one hundred and twenty thousand lost orders and roughly one point six million website errors. Amazon's AI tool Q was one of the primary contributors that trigger the event, according to
an internal review. On March fifth, another outage caused a ninety nine percent drop in orders across Amazon's North American market places, resulting in six point three million lost orders. One of the internal documents stated one key factor was a production change that was deployed without using a formal documentation on an approval process called Model Change Management. Very cool. I also want to be clear that it appears that
these incidents were created by use of these tools. By actual software engineers, people that ostensibly know how code and software architecture works. Reliance on large language models, especially at a time when executives are putting more pressure on engineers to deliver more features and ship more code, means that software engineers are being incentivized to be sloppy and to
ship slop itself. There is nothing inherently good about automating code, nor is there any inherent value in shipping a lot of it fast. Lms convince you that what you're writing is good and stable and does the thing you want it to. And if your skim reading the outputs, or of course unable to read them at all, it's easy for you to assume that because you asked the model that does not have thoughts whether it thinks you got something right, that you actually did so, and that it
got it right. To be explicit, allowing an LM to write all of your code means that you are no longer developing code, nor are you learning how to develop code, nor are you going to become a better software engineer as a result, nor are you solving actual problems. You are just handing work over to something and taking dog shit out. I'm not saying that all code is using
lms are inherently bankrupt or anything. But I hear these stories about writing all the code, and I'm they give me the willies, And I know what I'm saying sounds like an insult or hyperbole. I don't mean it in that way. If you are just a person looking at code, you're only as good as the code the model makes.
And as Mobitar recently this discussed, these models are built to galvanize you, glaze you, and tell you that you are remarkable as you barely glance at globs of overwritten code that, even if it functions, eventually grows to a whole built with no intention or purpose other than what the model generated from your prompts. I'm sure there are software engineers using these models ethically, who read all the code, who have complete industry over it, and use it like
a glorified auto complain. I can see the value. I'm also sure that there are some that are just asking it to do stuff, glancing at the code and shipping it. It's impossible to measure how many of each camp there are, but hearing Spotify's CEO so that its top developers are basically not writing code anymore makes me deeply worried, because this shit isn't replacing software engineering at all. It's mindlessly removing friction and putting the burden of good or right
on a user that it's intentionally gassing up. And ultimately, this entire era is a test of a person's ability to understand and appreciate friction. Friction can be a very good thing. When I don't understand something, I make an effort to do so, and the moment it clicks is magical.
In the last three years, I've had to teach myself a great deal about finance accountancy in the greater technology industry, and there have been so many moments where I've walked away from the page, frustrated student self doubt that I'd
never understand something. I eventually did. It took time, It really took time, and really, that luxury of time is important, and sadly, many software engineers face increasingly deranged deadlines set by bosses that don't understand a single fucking thing about their job or the software industry itself, let alone what lms are capable of, or what responsible software engineering might be.
The push from above to use these models because they can and I quote write code faster than a human is a disastrous conflation of fast and good, all because of flimsy myths peddled by venture capitalists in the media about LM's being able to replace software engineers. It's fucking stupid. It's a disgrace, and there are real problems that are going to happen as a result. The problem is that lms can write all code. Theoretically, they can just put
the code out that you might have rent yourself. Doesn't mean the code is good, or that somebody can read it and understand its intention, or that it works, that it will work in the future, or that you can build any kind of sustainable or I don't know, like stable in any way organization on top of it, or even that having a lot of code is a good thing, both in the present and in the future of any
company built using this generative code. Adding the variable of code written by people who quite literally do not understand it guarantee something severe and calamitous in the future, though why i'd argue that was the case without their influence. Increasing the volume of code contributed to a company naturally increases the amount of time needed to read it. And the amount of effort needed to maintain it, which naturally
encourages people to use llms to summarize it. And then well, you have to rely on the lms to tell you what good looks like, and they don't know a sincle fucking thing. And it also creates a new burden on the technical workers to have to clean up the slop in their day to day lives. Generative code is a digital ecological disaster, one that will take years to repair thanks to company remits to write as much code as fast as possible and use llms as much as possible too.
Every single person responsible must be held accountable, especially for the calamities to come as lazily managed software companies see the consequences of building their software on sand I'll see you all next week.