Welcome to the Sentient Code, where intelligence is engineered, autonomy is emerging, and a line between human and machine grows thinner. Each episode, we decode the algorithms, explore the robotics, and examine the ideas shaping the future of artificial minds.
So I want you to just take a second and look around the room you're in right now, or you know wherever you happen to be listening to this today. Yeah, just do a quick scan of your environment, right, and I want you to try and find a standard, maybe somewhat dim, incandescent.
Light bulb, like the kind in an old desk clamp maybe.
Exactly, or even just the little bulb inside your refrigerator. That specific bulb, it takes about twenty watts of power just to maintain its glow.
Which is I mean, it's barely enough light to read.
A book by, right, it's practically nothing. So I want you to hold on to that image, that tiny twenty watt energy budget.
Because we're going to use that as the bits line for something pretty.
Wild, completely mind bending. Actually yeah, because that exact same amount of energy, those twenty watts, is what your biological brain is using right this very second. To do well literally everything.
It really forces a complete recalibration of how we view our own biology.
Doesn't it. It totally does we just walk.
Around completely oblivious to the sheer, I mean, the thermodynamic miracle happening inside our skulls every day.
Yeah, think about what you, as the listener, are doing on just twenty wants Right now, you are processing the auditory signals of my voice.
You're converting acoustic waves into electrical impulses.
Exactly, and then you're parsing individual words out of that continuous stream of sound.
And if you happen to be, say, walking down the street while listening, it's even crazier.
Oh yeah, you're instantly recognizing familiar faces. You're navigating this complex three D environment, keeping your balance against.
Gravity, regulating your core body temperature.
Right, and you can learn a complete, letely new abstract concept from a single example and instantly integrate it with decades of stored memories.
All of that high level computational work is happening simultaneously, And.
While you're doing all of that, that identical twenty watt budget is sustaining the profound, completely unresolved mystery of human consciousness. Itself. It's just staggering all for the energy it takes to dimly light a closet.
And you know the contrast becomes almost absurd when we look at our current technological landscape.
Oh absolutely, let's talk about that.
Like, consider the state of the art artificial intelligence models today. The systems that are performing complex pattern recognition or generating photorealistic images.
Or simulating human likesuage right.
To achieve even a tiny fraction of human cognitive capabilities, and usually only within incredibly narrow domains. Current AI requires physical infrastructure on a planetary scale.
We are definitely not talking about twenty watts anymore.
No, not at all. We're talking about sprawling, warehouse sized data centers, just packed floor to ceiling with specialized hardware.
I mean, these facilities need dedicated industrial cooling towers just to prevent the silicon from literally melting.
Exactly. They consume millions of watts.
Millions of wants for a single model. It's basically the energy equivalent of a small dedicated power plan.
Just to run a specialized software program that can write a passable email or identify a stop sign and a photograph.
So the ratio of computational capability to energy consumption in the human brain isn't just like a little bit better than our.
Ai, No, it is superior by multiple orders of magnitude.
And the most critical realization here is that this isn't some temporary engineering hurdle, is it not?
At all? This isn't a problem where we can simply wait for the next generation of silicon microchips and just assume the efficiency gap will organically close.
Because the foundational architecture of how we build computers is fundamentally at odds with how biological brains operate physically at odds. Yeah, so we have this massive multi order of magnitude disparity and efficiency, and the traditional roadmap of computer science is essentially a dead end for solving it.
Which brings us to the terrain we're exploring today.
Yes, we are looking at a radical, incredibly ambitious field of engineering called neuromorphic computing.
The name itself derives from the Greek roots for nerve and form.
I love that. And just to be clear, this is not about writing clever software code to run on the exact same hardware we've been using since the nineteen eighties.
No, this is the engineering quest to build actual physical hardware, new types of silicon chips that fundamentally think, physically operate, and are structurally organized exactly like a biological brain.
It requires a complete paradigm shift, It really does, because the vast majority of artificial intelligence today operates by simulating a neural.
Network in software, but then executing that simulation on conventional, non neural hardware.
Right, It's just a similar running on a normal.
Computer exactly, but neuromorphic engineering abandons the simulation. The goal is to instantiate the physical operating principles of biological neural computation directly into the atomic structure of the machine itself.
Okay, So to understand why we need to completely reinvent the physical computer from the ground up, we first need to talk about the fatal flaw baked into the computers you.
And I are already using, right, the von Neuman architecture.
Yeah, if you're listening to this on the smartphone, that incredibly advanced device is constrained by a design choice made back in the nineteen forties.
We have to go back to the mathematician John von Neuman set.
The stage for us. What was happening in the forties.
Well, during the era of those massive room sized vacuum tube computers like the Nis, von Neumann formalized a blueprint for computer architecture.
And that blueprint became the standard for virtually every single digital device we manufacture today.
Right exactly, and the defining characteristic of this von Noyman architecture is a strict physical set of duties.
Okay, break that down for me.
You have the processor which performs the actual mathematical computation and logic, and then physically separated from it, you have the memory unit which stores the data and the instructions.
So they basically live in two completely different neighborhoods on the.
Motherboard they do, and they're connected by a communication pathway known as a bus.
A bus got it.
Because they're physically separated. Every single time the processor needs to execute a task, whether that's adding two numbers or changing the color of a pixel on your screen, it can't do it alone.
It has to ask for the data, right.
It must send an electrical request across the bus to the memory. It has to wait for the data to be retrieved. The data is pushed back across the bus to the processor, the operation is performed, and then the result usually has to be sent back across the bus to be stored in memory. Again.
Wow, if you really visualize that, it's just endless shuttling back.
And forth, constant shuttling.
Let me train an analogy here. Imagine a chef working in a high end restaurant kitchen. But there's a massive design flaw. The pantry isn't in the kitchen.
Okay, where is it.
The pantry is three blocks down the street.
Oh, that sounds awful, right.
So the chef is standing at the stove, water is boiling, and they realize they need a pinch of salt, so they have to leave the kitchen. They have to stop what they're doing, run out the door, sprint three blocks down the street to the pantry, grab the pitch of salt, run all the way back to the kitchen and sprinkle it in the pot.
And then they probably need something else.
Exactly Then they realize they need a chopped on you Boom out the door again, three blocks down the street, grab the onion, sprint back over and over millions of times a day, so by the end of the night. By the end of the night, the chef isn't tired from the actual act of cooking. The chef is completely exhausted from commuting.
That is a perfect way to picture it, and the physical consequence of that commute is the crux of the problem.
The von Neumann bottleneck exactly.
Historically, in computer science it was viewed as a speed limit. The processor is incredibly fast, but it spends most of its time sitting idle waiting for data to travel back and forth over the bus.
But nowadays it's not just about speed, is it No.
In the context of modern artificial intelligence, the bottleneck is a massive energy crisis.
Because pushing electrical signals back and forth across a physical copper wire billions of times a second that generates heat.
Massive amounts of heat. Due to the physical capacitance of the.
Wire, it actually takes physical energy to push electrons down a wire.
It takes enormous energy. When we run complex AI neural networks on conventional hardware, we're shuffling massive matrices of data millions of parameters back and forth continuously.
So the commute is basically bankrupting our energy budget.
Precisely, the sheer thermodynamic cost of moving the data is astronomically higher than the energy cost of the actual mathematical operations being performed.
We are burning our energy budget on the commute, not the computation.
Which naturally leads us to ask a pretty important question.
Right, If that's how our current devices arrange their kitchen, and it's clearly a disaster for energy efficiency, how does the human brain around its kitchen?
How is your brain managing to do all of this high level processing on just twenty watts without completely melting down?
Yeah? What's the biological solution?
The biological solution entirely eliminates the commute?
Wait, entirely.
Yes. In the human brain, computation and memory are colocated. They occupy the exact same physical space.
Okay, I need to understand how that actually works physically.
Well, the brain contains roughly eighty six billion neurons, and they communicate with each other through microscopic junctions called synapses.
And we have what trillions of those.
Trillions of synactic connections. Yeah. In the biological paradigm, the synapse operate simultaneously as the processor and the hard drive.
So the wiring itself is the memory.
The connection itself is the physical medium through which the computation occurs, and it is also the physical substrate where learned information is stored. That is wild when you learn something new, say you're practicing a new language or memorizing the layout of a new neighborhood. The physical structure of your brain actually changes like.
It physically alters. Its shape.
The synaptic weight, which determines the strength of the electrical connection between specific neurons, physically alters.
See if you open up a traditional computer, the memory is just a file saved in a discrete sector of a silicon ship. But in the brain, the memory is the physical shape, density, and chemical strength of the literal wiring.
Computation and memory changed together. They are intrinsically linked in the same physical space.
So when I see something what happens.
When an electrical signal passes through a network of neurons in your visual cortex, It doesn't have to pause and fetch the memory of how to process a straight line from a different lobe of the brain.
The instructions are just there.
The processing instructions are built directly into the physical pathway the signal is currently traveling through. There is no shuttling of data.
There's no bus, no bus at all.
To use your analogy, the kitchen and the pantry are perfectly integrated at a microscopic level.
And replicating this exact physical colocation in silicon hardware is the foundational premise of neuromorphic engineering exactly.
That's the core of it.
But you know, the physical layout of the von Normann bottleneck is really only half the problem, isn't it.
That's true.
It's not just that traditional computers separate the kitchen in the pantry, it's the very language they use to communicate internally, the fundamental way we treat the materials we build computers out of.
Right, we need to look back at the origins of neuromorphic hardware to really understand this, which.
Shakes us to Carver Mead right at Caltech.
Yes, Carver Mead at the California Institute of Technology, in a landmark nineteen ninety paper, Mead, who was already a highly respected pioneer in integrated circuit design, actually coined the term neural morph.
Okay, what was his big breakthrough?
His most profound realization wasn't just about moving memory closer to the processor. It was about the fundamental physics of how we utilize silicon.
Because right now the global tech industry uses silicon to mass produce transistors, and we treat those transistors in a highly rigid binary way.
Yes, we treat them like tiny light switches.
They're forced to be either entirely on, representing a one, or entirely off, representing a zero.
Conventional computing is entirely digital and binary. It operates using discrete, strictly controlled, clocked operations.
Everything is forced into that binary state of one or zero.
Furthermore, millions or billions of times a second. A rigid internal clock forces the entire system to step forward synchronously.
Just marching to a beat exactly.
But med looking closely at biology, recognized that biological brains do not operate like binary calculators.
They don't use ones and zeros.
No, the brain operates in a fundament only analog, continuous time mode.
So carver Meede looked at these silicon transistors which we're using to build these rigid digital calculators, and said, what if we stop forcing them to be strict on and off switches.
He proposed operating the transistors in what physicists call the sub threshold regime.
The sub threshold regime, what does that mean? In plain English?
In standard digital logic, a transistor is hit with a relatively high voltage to snap it cleanly on, allowing current to flow freely, or the voltage is dropped to snap it cleanly off.
It's a brute force approach.
It is. But if you apply a very low voltage below that clean switching threshold, the transistor doesn't just go dead. Its behavior changes entirely.
It does what happens.
It responds in a graded, continuous analog way. The tiny trickle of leakage current that digital engineers usually try to eliminate the stuff they see as a bug, exactly the bug becomes the core feature. Mead realized that the analog physics of this sub threshold silicon, how electrical charge slowly accumulates, how capacitance builds, how ions.
Diffuse, It acts like biology.
It could naturally mimic the continuous physical dynamics of biological cell membranes and ion channels.
So he wanted to use the raw physics of the material to just do the math organically, rather than forcing the material to act like a rigid mathematical abocus.
That's a great way to put it. By utilizing the natural analog physics of the substrate, you bypass the massive thermo dynamic overhead of encoding every single piece of sensory information into long, complex strings of binary digits.
And shoving them through that serial bottleneck we.
Talked about, right, But this analog shift fundamentally changes how information is transmitted across the network.
How so well?
Modern artificial neural networks in software exchange precise, continuous numerical values floating point numbers like point seven four to three fro sheats of numbers basically yeah, But biological neurons do not exchange spreadsheets of numbers.
They speak in spikes.
They communicate through action potentials, commonly referred to as spikes.
What exactly is a spike? Physically?
These are brief, discrete, essentially identical electrical pulses that travel down the axon of a neuron and trigger a chemical or electrical response at the synaps.
Okay, I want to make sure we truly grasp this because it's a massive departure from how we normally think about data.
It is a huge mental leap.
If the biological brain isn't sending specific numerical values to convey information, how does a literal zapp of electricity actually carry any complex meaning? I mean, a spike is just a spike, it's a great question.
In biological neural systems, the information is not encoded in the size, the amplitude, or the shape of the signal.
Because every spike is basically identical, right.
Instead, the information is entirely encoded in the timing and the rate of those spikes.
The timing and the rate.
Okay, if a sensory neuron in your eye fires rapidly, producing a dense, high frequency cluster of spikes, it might be conveying a stronger stimulu, say a very bright light, compared to a neuron that fires rarely.
That makes sense, more spikes brighter light.
That is called rate coding. But more importantly, there is temporal code.
Temporal coding like the exact microsecond.
Yes, the precise microsecond timing of a single spike relative to the spikes of other surrounding neurons, and the network can carry incredibly complex, high dimensional information.
Let's use an analogy to really visualize why this timing based spiking communication is such a monumental advantage for saving power. I'm all ears think about a traditional computer processor like a massive symphony orchestra, but the conductor is an absolute tyrant holding a metronome.
A very rigid conductor, very rigid.
The metronome is the computer's internal clock, ticking away billions of times a second. Every single musician in the orchestra is forced to play strictly to that beat. Right even if a musician, let's say the triangle player in the back, has absolutely no notes to play for a full twenty minutes of this infanty, they cannot relax.
They have to stay engaged.
The conductor forces them to stand at attention, physically tapping their foot rigidly keeping time on every single beat, exhausting themselves just to remain perfectly synchronized with the global metronome.
They are burning massive amounts of energy just to actively do nothing.
Exactly, and the physical reality of conventional digital chips mirrors that orchestra exactly doesn't it.
It absolutely does. The global clock cycle forces electrical activity and power consumption across the entire chip, even.
In sectors of the processor that are momentarily idle, even.
When they have no new data to process. The wires are constantly charging and discharging just to maintain the rhythm.
But a neuromorphic system, a spiking neural network, is fundamentally different.
How would you describe it.
It's like a cool late night jazz ensemble playing completely without a set tempo. There is no metronome.
I like that.
A musician in this ensemble will sit in total relaxed silence. They expend absolutely zero energy. They aren't tapping their foot, they are actively keeping time. They just wait. They just wait at rest until exactly the moment they were organically inspired to play a single note, a spike. They play their note, and then they immediately return to resting in total silence.
And the energy savings of that jazz ensemble approach are staggering a bit, because when those artificial neurons are silent, the power draw of the chick drops precipitously. This is the essence of an event driven architecture.
Event driven, so it only reacts when something happens.
Spiking neurons are purely event driven. A neuromorphic chip consumes practically no power when there is no new data changing in its environment.
It assists there.
It sits in a quiescent state. Energy is expended only when and where information is actually flowing through the network.
This means the overall energy cost to the system scales entirely with the activity of the network, right, not the physical.
Size exactly, not the sheer physical size of the network. You could build a massive chip containing millions of artificial neurons, but if only one percent of them are actively spiking at any given microsecond, your power draw remains astonishingly low.
Which perfectly explains how you get a human brain running on twenty wants.
It makes perfect sense, now, doesn't it.
It's a massive, dense network of eighty six billion neurons, but at any given moment, the vast majority of it is resting.
Just whispering to itself in perfectly timed, sparse spikes.
So the theory is beautiful and biology proves it works. But I want to shift gears from theory to cold hard silicon. Are we actually forging these jazz playing analog brains in the real world?
We are. The engineering community has taken this challenge head on, and there are several major projects taking distinctly different architectural approaches to forging spiking neural networks in hardware.
Okay, let's hear about them. Who's leading the charge.
Let's start with one of the most prominent research platforms developed by one of the largest chip manuf facturers on the planet, Intel's lowy heat architecture.
Intel. Really they are the undisputed giant of traditional von Neumann computing. How did they approach building a brain?
Well? Intel released the first LOWI heat chip back in twenty seventeen and followed it up with a more advanced architecture LOWI Heat two and twenty twenty one.
Okay, and the sheer.
Scale of what they engineered is breathtaking. A single LOWI heat two chip contains over a million programmable artificial spiking neurons and over one hundred million synapses.
A million neurons on one chip.
But the true innovation is how it is structured. It is organized into a dense mesh of neuromorphic cores, and crucially, there is no global metronome, no clock, no clock. These cores communicate entirely through an asynchronous on chip spike routing network.
How does that work without a clock?
When a neuron spikes, it essentially packages that spike like a piece of mail and routes it through the mesh to its destination, independent of any global clock.
A million neurons on a single chip, communicating like a postal system what does the power consumption actually look like when you run it?
This is where the event driven philosophy really proves its worth. When the LOWI heat chip is actively computing processing complex inputs, it consumes power in the milliwat.
Range milliwats and for context, a normal.
Chip to put that in perspective, a standard GPU might consume hundreds of watts. Wow, But it gets better. When lowihi is at rest waiting for a sensor input, its power draw drops down to the microwat.
Rank microwats, literally a millionth of a watt. You could run that off a watch battery for years.
It represents a reduction in energy consumption of several orders of magnitude compared to running the exact same computational workload on a conventional CPU or GPU.
That's incredible. So what are people actually doing with it?
Intel provides this platform to a global community of researchers, and they're using it to demonstrate incredible edge applications like what things like real time gesture recognition, complex robotic control systems, and even artificial olfactory sensing, essentially giving machines the ability to process.
Chemical smells the smelling computer.
Yeah, because the architecture relies on spiking networks, it naturally excels at adaptive learning tasks where the system needs to learn and adjust to new patterns on the fly. In the real world.
That's Intel's approach, but IBM has been a major player in this space as well. Right.
Yes, IBM introduced their true North chip back in twenty fourteen, and it represents a slightly different philosophy within the neuromorphic design space. True North also featured a million artificial neurons, but paired them with two hundred and fifty six million synapses. However, True North was engineered with a highly regular, incredibly rigid, but extremely low power architecture.
Rigid in what way.
It was specifically optimized for deep spiking neural network inference inference.
Okay, when you say inferns, you mean it was designed to take a neural network that has already in fully train somewhere else, loaded onto the chip and just run it as efficiently as physically possible.
Exactly. The focus was on execution rather than on chip learning, got it. True North demonstrated that you could perform real time complex pattern recognition tasks like classifying multiple moving objects in a live video feed or processing spoken audio streams at power levels measured in just tens of milliwatts.
Tens of milliwats for video processing.
Yes, if you attempted to perform that exact same real time, high frame rate video processing on conventional hardware, you would easily be burning through hundreds of watts or even kilowatts for a dense enough network.
That's a massive difference.
True, North proved definitively over a decade ago that neuromorphic hardware could handle practically useful, commercial grade tasks at a dramatically lower energy cost.
See I hear that, But I am struggling to see how this is financially practical. What do you mean You are telling me we need to build entirely new analog or highly specialized asyncre chifts from scratch. Yes, but the manufacturing pipelines for standard digital silicon transistors are worth trillions of dollars globally. Yes, we have perfected the art of making digital processors. You can't just throw that entire global infrastructure away.
It's a very valid point.
There has to be a middle ground, right, Yeah? Is there way to use the conventional digital chips we already mass produce? But somehow forced them to act like a brain.
That is the exact pragmatic dilemma that birthed a completely different approach to neuromorphic engineering, the Spinnaker project. Spinnaker, Yes, it was developed at the University of Manchester, led by a researcher named Steve Ferber.
Steve Ferber, why is that named? Legendary and computer science.
Ferber is one of the original principal designers of the ARM processor architecture.
The chips in our phones.
Exactly, if you are listening to this on a mobile phone, tablet, or even many modern laptops, you are almost certainly using an ARM based processor.
Wow. Okay, so what did he do?
Ferber and his team took a fascinating hybrid route. Instead of designing highly experimental, custom analog neuron circuits from scratch, they took an array of small, standard conventional digital.
ARM processors just off the shell processors, yes.
But they fundamentally altered how they interact. They interconnected these standard processors using a completely custom, highly specialized asynchronous spike routing network.
So the individual brain cells are just traditional digital processors, but the wiring connecting them forces them to speak the brain's language. Of spikes.
You've got it. The individual processors run software models simulating the biological behavior of neurons, but the network architecture physically forces them to communicate via discrete spikes.
That's a brilliant compromise, it is.
This hybrid approach provides immense flexible programmability. Because they are standard processors, you can easily rewrite the software models of the neurons while still capturing the system level efficiency of an event driven spiking network.
How big did they scale this?
The scale is astounding. The second generation Spinnaker two, which is currently deployed at the Technical University of Dresden, scales this concept up to over a million interconnected arm processor cores.
A supercomputer made of a million cores just to rut spikes.
A network capable of routing billions of individual spike events per second in real time without traffic jams.
That's incredible.
Spinnaker serves as a vital bridge between computer engineering and computational neuroscience. It possesses the sheer computational power to run practical neuromorphic applications, but its primary design goal is to model massive biological neural.
Systems, so scientists use it to study the brain exactly.
It allows neuroscientists to simulate biologically realistic neural circuits at a scale that actually approaches the complexity of small regions of an actual mammalian brain.
That is wild is using millions of digital brains to simulate an analog brain by enforcing a spiking language.
That's one way to do it.
But what if we want to abandon digital completely. What if a research team wants to go full analog physics, no digital processors, no software models at all.
For that pure analog vision, we look to the Massive Human Brain project in Europe, specifically a system called brain scale S developed at Heidelberg University.
Brain Scale brainscale S.
Is a pure analog mixed signal neuromorphic platform. It does not use digital code to simulate biological neuron models.
Go how does it work?
Instead, it physically implements the differential equations of biological cell membranes directly into the electrical physics of its custom silicon circuits.
So the physical electrical currents flowing through the chip are literally acting out the biology. The silicon is physically behaving like a cell wall.
The capacitance and resistance of the physical circuits are tuned to perfectly mirror the ion channels of a neuron. And because it relies on pure analog physics happening at the speed of electronics rather than waiting for software to calculate mathematical equations, it operates at an astonishing speed fast. It does not run at biological real time. It runs neural dynamics set up to ten thousand times the speed of biological.
Real time, ten thousand times faster than an actual living brain. Yes, if you're a listener, just pause and think about the implications of that for a second. If you're a scientist studying how a specific brain network learns, adapts, and evolves over say a full year of biological life, you don't have to sit in a lab and wait a year.
No, you don't.
You can run that entire year of complex physical brain evolution on the Brain scale S platform in about an hour. You're essentially putting physical brain dynamics on fast forward.
The value proposition for basic science is immense.
It has to be.
It allows researchers to study the long term evolution of neural network behavior, to test complex plasticity rules over massive biological time scales, and to observe phenomena that would be impossible to track in a living organism, all in mere seconds or minutes of real time.
But you know, all of these chips, whether it's Intel's digital mesh or a fervers million arm cores, or Heidelberg's accelerated analog physics, they face a massive wall.
They do.
You can build a billion artificial neurons, but a brain isn't a brain unless it can learn. It has to adapt to new information. How do you actually program or teach a billion silent spiking nodes when you don't have standard code to write.
To fully grasp how a neuromorphic chip actually learns, we first have to understand why the dominant learning mechanism used by almost all conventional artificial intelligence today is completely fundamentally incompatible with biological brains.
Okay, let's unpack that.
Conventional deep learning, the technology behind large language models and image generators, trains its neural networks using a mathematical algorithm called backpropagation.
Backpropagation. It is the undisputed engine of the modern AI boom. It is, how does it actually work? And why couldn't a biological brain just use it?
Backpropagation is mathematic elegant, but biologically impossible. Let's walk through it. In a conventional AI network, data is fed forward through multiple layers of artificial neurons to produce an output. Let's say it's looking at an image and trying to guess if it's a handwritten number seven. It makes its guess. If the guess is wrong, the system calculates the exact
mathematical error of that guess right backpropagation. Then takes that error value and mathematically propagates it backwards through every single layer of the entire network. It uses calculus to calculate an exact gradient, a specific adjustment for every single weight in the network, so that the network will be slightly more accurate the next time it sees that image.
I want to visualize this. It sounds like an incredibly overbearing micromanager and a massive corporation.
That's a very apt comparison.
Like the company makes a mistake on a product, the micromanager immediately hits the pause button on the entire factory. They look at every single one of the million employees.
Simultaneously, and they calculate exactly how much of the blame below belongs to each individual person.
Right, they synchronously update every employee's instructions, and only then do they unpause the factory to try again.
And the biological brain simply cannot function that way. The physical requirements of backpropagation are immense.
Why can't the brain do it?
First, it requires a global error signal, your micromanager that has an omniscient view of the entire network at once, which we don't have right. Second, it requires the system to temporarily store all the electrical states from the forward pass in a massive memory bank, so the backward pass can use them to calculate the.
Blame, and we don't have that separated memory bank exactly.
Third, it requires perfectly synchronous updates across the entire system. A biological brain simply does not possess the anatomical hardware for this.
There's no biological manager neuron that can pause your brain, calculate a global calculus error, and physically adjust eighty six billion synapses simultaneously.
No, that would be impossible.
So if the brain can't do global backpropagation, how do we get smarter? How do we wire new memories?
The brain relies entirely on local learning rules. It learns from the ground up, not from the top down local learning rules and the most heavily studied biological mechanism, which is now being directly implemented in neuromorphic hardware, is called spike timing dependent plasticity or STDP.
Spike timing dependent plasticity. That's a mouthful. Let's break down the mechanics of it.
It is entirely dependent on highly localized physical information, specifically the precise timing of spikes between two directly connected neurons.
Okay, let's isolate just two neurons.
We'll call the first one the presynaptic neuron, the one sending the signal. The second is the post synaptic neuron, the one receiving the signal.
Got it pre impost.
STDP dictates that the physical strength of the synapse connecting them changes based entirely on the exact relative timing of when they both spike.
Just the timing, just the timing.
If the presynaptic neuron fires spike just before the post synaptic neuron fires, the physical can between them strengthens. The biological logic is that the first neuron likely played a role in causing the second one to fire.
It's recognizing cause and effect exactly.
However, if the timing is reversed, if the post synaptic neuron fires first and then the pre synaptic neuron fire slightly later, the connection physically weakens.
Why we caedday, the.
Logic being that the first neuron clearly didn't contribute to the second one firing, so their association is meaningless and should be diminished to save energy.
I want to frame this as a strict rule of cause and effect. For you listening, think about the relationship between lightning and thunder. That's a great real world example.
If you're standing outside and see a brilliant flash of lightning and it is consistently followed right after by a massive crash of thunder, your brain notes that specific timing lightning first, then thunder right.
Because the timing is consistent and strictly ordered, your brain physically wires those two concepts together. The synapse strengthens. You learn the rule causes thunder perfect. But imagine you hear a random crash of thunder, and then ten seconds later someone flashes a flashlight in your eyes. Your brain completely ignores the association.
The timing is wrong, the order is reversed.
STDP means the neuromorphic hardware is teaching itself based solely on the timing of highly localized events. It doesn't need a massive global manager telling the whole network it made a mistake.
It learns autonomously synaps by sunaps based purely on what it experiences.
That is just so elegant.
This is the physical realization of heavy in learning, a concept proposed back in nineteen forty nine, often summarized by the famous neuroscience phrase neurons that fire together, wire together.
I've heard that phrase.
Modern neuromorphic chips like Intel's lowiha actually have highly configurable learning rules built directly into the silicon that replicate these forms.
Of STDP, so they can learn locally.
Yes, this enables the chip to learn continuously on the fly, directly from streaming sensory data in the real world, adapting to new patterns without ever needing to connect to a massive cloud server to run a backpropagation algorithm.
But wait, if we follow this logic to its physical conclusion, we hit a roadblock. What's the roadblock if STDP requires the physical synapps to remember its own history to know if it fired before or after its neighbor how do you actually do that? In silicon? A standard digital transistor forgets everything the exact second the power drops.
That is the core issue.
Yes, you can't build a brain out of amnesiacs. Did Carver meat have an answer for that? Or did engineers have to invent a completely new material?
That is the exact problem that pushes neuromorphic engineering into the realm of exotic materials science. To truly replicate a biological synapse, we need a physical component that naturally remembers its own electrical history.
And what is that?
And this brings us to what is widely considered the holy grail of neuromorphic computing, the memorister.
Memorister, it sounds like a piece of alien technology. What exactly is it?
The word itself is a portmanteau of memory and resistor. In standard electronics, a resistor is a basic component that simply resists the flow of electrical current by a fixed unchanging amount.
Okay, like a bottleneck in a pipe, right.
But a memristor is a two terminal electronic device whose physical resistance fundamentally changes based on the history of the current that has flowed through it in the past.
The physical properties of the material permanently change based on what it has experienced.
Yes, as current flows through it in one direction, its resistance might drop. If current flows in the opposite direction, its resistance might increase.
Wow.
And crucially, when you turn the power completely off, the memorister remembers its last state of resistance indefinitely.
That is deeply profound.
It is the literal physical embodiment of a biological synapse in a living brain. The weight of a synapse, how strongly it connects two neurons changes based on the history of ions passing through it.
Memristor does the exact same.
Thing electrically in a neuromorphic chip utilizing memoristers, The electrical resistance of the device directly encodes the strength of the connection. That resistance physically alters when current flows, effectively implementing a local learning rule directly in the atomic physics of the device itself.
So we don't need a separate memory chip down the street to remember the synaptic weight, and we don't need a processor to calculate how it should change.
No, you don't.
The memorists are just is the memory in the processor simultaneously.
And the engineering implications of that collocation are staggering, particularly when it comes to the core mathematics of artificial intelligence.
The core mathematics, we're talking about matrix multiplication, right.
Yes. To appreciate this, we need to understand the massive mathematical burden of AI.
Okay, I need to understand this matrix math thing. Why is multiplying grids of numbers such a massive burden for normal computers?
Think about how an artificial intelligence actually sees an emas image. Let's say we have a tiny image just one hundred pixels by one hundred pixels.
That's ten thousand numbers representing brightness exactly.
Now, feed that into a neural network layer with ten thousand artificial neurons. Every single neuron needs to connect to every single pixel. Oh wow, that creates a matrix of one hundred million distinct weights. To calculate the output, the computer must multiply the ten thousand pixel values by that one hundred million weights.
So the processors basically doing hundreds of millions of tiny math homework problems one by one.
And because of the von Neumann bottleneck, it is fetching the textbook from the library down the street for every single one of those problems.
Just endlessly fetching and returning.
It is fetching weights for memory, doing a digital multiplication, adding the result to a running total, and storing it back. Trillions of operations for a single frame of video.
It's exhausting just thinking about it.
Now, replace that conventional architecture with a massive grid of memoristers arranged in what we call a crossbar picture, a microscopic to tech toeboard.
Okay, a dense grid of intersecting wires.
At every single intersection where a vertical wire crosses a horizontal wire, there is a memorister connecting them. In this physical setup, we don't do digital math at all.
Wait, no math.
We map the input data the pixels of our image as electrical voltages sent down the vertical wires. The stored weights of our neural network are physically represented by the conductance of the memorisers at the.
Intersections, conductance being the opposite of resistance. How easily they let the electricity through exactly.
Now, basic high school physics takes over Alms law states that current equals voltage multiplied by conductance.
Okay, remember that, so as the.
Voltage flows into the memorister, the resulting electrical current coming out the other side is the exact literal mathematical product of the input and.
The weight It multiplies it physically.
Yes, and then Kershoff's current law states that currents flowing into a single node naturally add together. So the currents from all the memoristers in a column naturally sum up as they flow into the horizontal wire.
Wait, the physics engine of the universe calculates the multiplication and the addition for us instantly, just by letting electricity flow through the grid.
The entire matrix vector multiplication, the grueling operation that requires billions of digital steps and massive energy on a standard GPU, happens in a single instantaneous analog step.
That is unbelievable.
There is zero fetching from memory. The math happens naturally as the electricity flows through the material. The promise here is an improvement in energy efficiency for AI inference tasks, not by a factor of ten or twenty, but by several orders of magnitude.
I have to play the skeptic here, though, Okay, go for it, because what you are describing sounds like pure magic. If we have a physical component that acts exactly like a biological sinn apse, eliminates the von Neumann bottleneck, entirely uses zero digital computation and saves massive amounts of power.
Why isn't it in my smartphone? The fair question why isn't every tech giant completely abandoning traditional silicon and churning out memorisarch chips, Because the engineering reality of working with these novel materials is brutal. Bridging the gap from a beautiful theoretical physics concept to a mass produced, highly reliable commercial chip is an incredibly difficult materials science challenge.
It's just hard to build them.
Memoristive devices suffer from several severe practical physical hurdles that have slowed their widespread commercial deployment.
What kind of hurdles are we talking about it?
The first major obstacle is device variability variability. Yes, when a foundry manufactures a standard digital silicon transistor, the process is so perfected that the transistors are incredibly uniform. But when you attempt to create millions or billions of nanoscale memoristers in a dense array, individual devices tend to have slightly different.
Physical characteristics, so they aren't identical.
Their atomic structures aren't perfectly identical. They don't all react to electrical current in exactly the same way. In a digital system, minor variations don't matter because a strong signal is still read as a one, But in an analog system, the exact value matters.
It's like having a world class orchestra where every single violin is tuned just slightly differently. It might still loosely sound like a sympathy, but the high level precision required for complex tasks is totally.
Lost exactly the analog noise overwhelms the signal. The second major physical challenge is drift.
Drift.
What's that even when a memory star is not being deliberately programmed or changed, when it's just sitting there holding a memory its internal resistance the stored synaptic weight can slowly shift or degrade on its own overtime on its own. This is due to thermal fluctuations or the natural movement of atoms within the crystal lattice of the material. Imagine your computer's hard drive slowly randomly changing the text inside
its own five while the power is turned off. It progressively degrades the accuracy of the neural network over time.
That sounds like an absolute nightmare for reliability. If you train a self driving car in a memorister chip, you don't want it slowly forgetting what a pedestrian looks like over the course of a year.
No, you definitely do not. And the third major hurdle is endurance. Endurance biological synapses are remarkably resilient. They can dynamically strengthen and weaken continuously for an entire human lifespan a century or more. Memorisers, however, rely on physically moving atoms or altering crystalline structures at the nanoscale every time they are reprogrammed, so they wear out. They eventually suffer
from physical wear and tear. The number of times a memorister can be reliably programmed before its properties degrade irreversibly is limited. This endurance limit is a severe bottleneck for systems that are meant to learn continuously on the fly.
So it's brilliant in theory and it works in the lab, but the physical materials keep breaking down, drifting, or forgetting things when we try to scale it up.
It is an active, intensely competitive battleground in material science. Right now. Researchers worldwide are investigating numerous exotic candidate technologies fighting for supremacy to solve these.
Exact issues, like what kind of technologies?
We are looking at phase change memory, which uses heat to change the material from amorphos to crystalline. We are looking at resistive RAM, conductive bridging RAM, which physically builds and breaks tiny metallic wires. At the atomic level, we are looking at ferroelectric devices.
That sounds like a lot of options.
Each of these exotic materials offers a different trade off. One might have incredible endurance but terrible variability. Another might be highly stable but require too much energy to program. Progress is steady, but we haven't found the perfect Goldilocks material yet that can be seamlessly integrated into standard commercial silicon foundries.
Okay, so the memborister revolution is still fighting its way out of the materials science labs. But despite these massive physical herds, neuromorphic chips, even the ones relying on standard silicon like Intel's LOWI heat, are already leaving the lab and entering the wild. They definitely are, and they are thriving in very specific environments, environments where traditional power hungry AI simply cannot survive.
If you want to find the perfect natural habitat for neuromorphic technology. Today you look to a field called edge computing the edge.
Edge computing refers to environments that are physically far removed from the cloud and those massive warehouse sized data centers we discussed earlier.
So out in the real world.
We are talking about devices operating out in the remote, chaotic physical world. These devices are constrained by strict physical limits. They're usually running on limited batteries, they cannot be plugged into a wall. Furthermore, they require instant, zero latency reactions to their environment. They simply cannot afford the time or the energy to beam sensory data back to a server farm, wait for a massive GPU to process it, and wait for the instructions to be beamed back.
And a major part of operating effectively at the edge is how the machine actually perceives the physical world. We talked earlier about how traditional processors are forced to march to a rigid metronome clock cycle. Yes, well, it turns out our traditional sensors, the eyes of our machines, do the exact same thing, which brings us to the concept of event based cameras. Right. Traditional cameras operate on a
rigid frame based paradigm. A standard digital camera takes a full complete picture, let's say, sixty complete frames every single.
Second, regardless of what's happened exactly.
It captures the light value of every single pixel, millions of pixels, over and over, completely regardless of what is actually happening in the scene in front of it.
I love the analogy for this. A traditional frame based camera is like a terribly inefficient security guard sitting at a desk watching an empty hallway.
A very bored security guard.
The guard is forced by protocol to pick up the radio and call headquarters sixty two times a second to give a full report. The hallway is still empty. The hallway is still empty. The hallway is still empty.
That sounds exhausting.
The camera is generating massive amounts of entirely redundant data, and the processor downstream has to expand mass amounts of energy to process all those identical pictures just to mathematically confirm that nothing changed.
And the consequence of that setup is a massive waste of power and computational bandwidth.
So what's the alternative.
An event based sensor, often called a dynamic vision sensor, is intrinsically tied to the neuromorphic architecture, and it fundamentally changes this paradigm. Instead of capturing full frames on a rigid clock, an event based camera only outputs a signal, a spike, when a specific individual pixel detects a meaningful change in brightness.
Every pixel acts independently.
Every single pixel acts independently.
So our security guard completely goes to sleep. They use absolutely zero power. They only pick up the radio and call headquarters when someone actually opens the door at the end of the hallway.
Exactly the camera produces a sparse, asynchronous stream of spikes corresponding only to movement or change in the visual field. If the scene is static, the camera outputs absolutely nothing and consumes practically no power.
But if something moves, But.
If an object moves rapidly, the individual pixels capture that movement with microsecond precision, far faster than a standard sixty frame per second camera could ever hope to catch. Furthermore, because each pixel manages its own exposure based only on local change, event cameras don't get blinded by looking directly at the sun the way normal cameras do. They have immense dynamic range.
And because this sensory data is already in the form of discrete spikes. It forms a perfectly matched, seamless pipeline directly into a neual morphic chip.
It speaks the same language.
The chip doesn't have to translate a massive JPEG image into a neural network. It just receives the spikes and reacts instantly. Where's this hardware actually being deployed today?
The most compelling and rap rapidly advancing application domain by far is robotics.
Robotics that makes sense.
Autonomous robots face the exact same fundamental survival problems that biological animals face in the wild. They have to process incredibly rich, noisy sensory data from a chaotic real world.
They must adapt their physical behavior to unpredictable changing environments.
They must make rapid life or death decisions without the latency of calling a cloud server. And they have to do it all while carrying their own limited energy source a heavy battery.
Right picture an autonomous drone trying to navigate through a dense forest at forty miles per hour.
That's a perfect example.
It can't pause mid air for two seconds to upload a video frame to a server, wait for a neural network to calculate the depth of the forest and wait for instructions on how to dodge a branch. It will crash into a tree before the data even reaches the cloud.
When researchers equip these robots with event based sensors directly wired into neuromorphic processors, they achieve extraordinary results.
Oh like, what kind of results?
These robotic systems can perform adaptive locomotion like a multi legged robot learning to walk over uneven shifting terrain on the fly, and high speed obstacle avoidance at power budgets that would absolutely crush a conventional computer.
Because conventional computers are just too heavy and power hungry.
If you tried to run a traditional state of the art computer vision model on a small flying drone, the battery would be entirely consumed just running the processor, leaving zero energy for the actual rotors to keep it in the air. Neuromorphic chips solve the fundamental power and weight constraint problems inherent in mobile robotics.
It's essentially giving machines the survival instincts of an insect, fast, incredibly energy efficient, and fully self contained.
But the applications are not limited merely to building better, faster, robots. There is a profound bidirectional relationship here. Building these physical neuromorphic chips is proving to be an invaluable tangible tool for the field of computational neuroscience itself.
Wait, how so does building a silicon brain actually help us understand the wet biological brain better immensely.
In theoretical neuroscience, researchers can sometimes get away with vague mathematical models or high level assumptions about how large populations of neurons.
Interact because it's just theory.
But when you are forced to build a physical piece of hardware in silicon you cannot be vague. Hardware requires explicit, rigorous, quantitative commitment.
Right. You have to actually build the circuit.
You must specify exactly how the voltage operates, exactly, how the timing windows work, exactly how the local learning rules are applied. The physical implementation of these theories forces neuroscientists to be absolutely rigorous.
It acts as a strict physical reality check for their theories.
Exactly when they run their biological models on physical platforms like the Heidelberg Brain Scale s system, the behavior of the hardware provides a tangible test of their theories about working memory, temporal coding, and sensory at adas adaptation.
And what if the hardware doesn't act like the brain.
If the silicon network does not behave the way the living animal does, it indicates that the biological theory is fundamentally flawed or incomplete. It creates a beautiful reciprocal feedback loop where engineering constraints inform neuroscience and biological discoveries inspire better engineering.
We spend a lot of time celebrating the hardware. The analog architecture makes complete sense. The power savings are literally orders of magnitude better, the event based sensors are brilliant, and the edge robotics applications are undeniably the future. Yes, so I have to address the elephant in the room.
The elephant in the room. Let's hear it.
If this hardware is so perfectly designed, so biologically inspired, and so vastly efficient, why isn't the entire tech industry pivoting? Why is the artificial intelligence landscape still almost entirely dominated by massive power hungry GPUs churning through conventional deep.
Learning Because of the software. The software we have hit what the industry calls the software wall. The harsh, pragmatic truth of neuromorphic computing today is that programming a spiking neuromorphic chip to do complex tasks is incredibly difficult.
Why is it so hard? We have millions of brilliant software developers in the world. We build massive operating systems in complex video games.
We do. But the entire global ecosystem of software development, the massive, highly polished libraries, the intuitive training frameworks like PyTorch or TensorFlow, the decades of accumulated mathematical knowledge is entirely built around conventional continuous digital math.
Oh right, Because we're used to the von Neumann system.
The software ecosystem for neuromorphic computing is by comparison, very immature. But the problem isn't just a lack of polished tools or developer familiarity. It's a fundamental mathematical mismatch at the core of the.
Technology because of the spikes.
Because of the spikes, we talked earlier about backpropagation, the calculus based algorithm used to train on almost all modern AI to high levels of.
Accuracy, the micromanager.
The micromanager backpropagation relies on calculating gradients, smooth, continuous mathematical slopes that tell the network exactly which direction to adjust its weights to reduce errors calculus. You can only use this calculus if the mathematical function you are analyzing is continuous and smooth. It must be what mathematicians call differentiable.
Meaning you can calculate a precise slope at any given point on the curve.
But spikes are not smooth or continuous. A spike is a discrete, binary event in time, it either happens or it doesn't. Mathematically, a spike is not a gentle hill. It is a sudden vertical sheer cliff face. You cannot calculate a smooth slope on a sheer cliff face. Therefore, spiking neural networks are fundamentally non differentiable.
I think we need a grounded analogy for this. Imagine trying to train a dog okay to dog with traditional AI, where you have continuous math in calculus gradients, it's like training a dog using a leash. You have continuous control. You can gently pull the dog to the left or smoothly guide it to the right. It's a smooth, continuous correction. I like that, But a spiking network, it's like a light switch. It's either on or off. You can't gently
flip a switch. You can't give a nuanced correction, you can't use the leash of calculus. How do you train the network You can't directly use the loosh. That is the central algorithmic challenge of the field. How do you train a network that speaks exclusively in discrete, non differential events to perform highly complex, nuanced tasks at the exact same level of accuracy as a conventional network trained with
advanced calculus. So what is the compromise? How are researchers currently trying to bridge this mathematical gap.
The current dominant approach relies heavily on techniques called surrogate gradient methods. Surrogate gradients essentially scientists mathematically cheat during the training phase in their computer simulations. They take the sheer, undifferentiable cliff face of a spike and replace it with a smooth, continuous curve a surrogate just for the purposes of doing the calculus.
To use my analogy, they pretend the light switch is actually a smooth volume dial, just long enough to figure out which direction they need to turn it.
That's a great way to think of it. They calculate the approximate gradients using backpropagation, update the weights, and then map those newly trained weights back onto the discrete spiking hardware.
It's an approximation. You are forcing the analog hardware to learn using a digital math translation.
It is an approximation. Then, while surrogate gradients have driven significant progress and allowed spiking networks to tackle much harder problems, it remains a mathematical.
Compromise, so it's not perfect. No.
Currently, spiking neural networks trained using these surrogate methods still generally lag behind conventional deep neural networks when tested on highly complex industry standard benchmarks like massive natural language processing or hyper detailed image generation. The raw accuracy isn't quite there yet.
So we have built a physical hardware architecture that is theoretically perfect for energy efficiency, but we haven't cracked the code on how to teach it complex tasks as effectively as our flawed brute force power hungry systems.
Closing the software performance gap, figuring out how to achieve state of the art accuracy natively on spiking hardware while maintaining that insane energy efficiency is arguably the defining engineering challenge of the next decade in this field.
Let's step back and look at the massive journey we've just taken. We started by looking at the crippling heat and fundamental inefficiency of the von Neumann bottleneck, the processor and memory forever commuting back and forth burning power. We cover a lot of ground, we really did. We explored Carver Meade's radical vision of using analog physics to mirror biology, ditching the rigid digital metronome for this silent, event driven jazz ensemble of spiking.
Networks, and we looked at the incredible silicon blains actually being forged.
Today, from Intel's lowy heat to the massive semi relations of Spinnaker and the accelerated physics of brain scale less. We dove into the holy grail of membristers, physical devices that literally remember their own electrical history like a living synapse, allowing the laws of physics to do the math for us.
And we saw how this technology is finding its true calling at the edge of the network, powering the survival instincts of next generation robotics.
It is a vast, incredibly complex landscape.
It is, but the overarching trajectory is clear, and I believe inevitable. Biological brains have already provided an existence proof.
They've proven its possible.
They have proven definitively that general, highly flexible, adaptable intelligence can run on just twenty watts of power. Replicating that precise physical operating system in silicon isn't just a quirky alternative to mainstream computing.
It's essential.
It is perhaps the most ambitious, fundamental and consequential project in the history of human engineering. The biological brain has solved the thermodynamic problem of intelligent computation in a way that our current machine simply have not.
I want to leave you, the listener, with a completely new thought to mull over, something that builds on everything
we've discussed about physically mirroring the brain and hardware. What's the thought if we truly succeed in making this hardware, if we overcome the massive materials science turdles with membisters, if we finally crack the software wall, If our machines start learning locally, adapting, dynamically, and physically rewriting their own electrical pathways exactly like our biological minds do, will these neuromorphic computers eventually inherit our biological flaws.
Wow, that's a fascinating question.
Think about it. We know human brains are highly susceptible to optical illusions, specifically because of how our neural pathways take rapid shortcuts to save energy.
We suffer from cognitive fatigue when our synaptic neurotransmitters are depleted.
Exactly, we are prone to irrationality and emotional biases because our physical wiring connects logic centers directly to keep survival instincts.
That makes perfect sense.
If we build an artificial brain that physically thinks exactly like us just to save power, what else does it start doing like us? Does a neuromorphic AI start seeing phantom patterns?
Does an autonomous robot experience the silicon equivalent of fatigue or a loss of attention after processing too much sensory data without a reset.
If we build the machine in our exact physical image, do we inevitably build our biological vulnerabilities into it as well? Something to ponder