Reverse Engineering the 6502

So it's a marvelous, marvelous thing to do. And it's so rewarding. And as you add more bits and pieces, you get more functionality. Eventually you get bitmap graphics, and then, then you can get the joy that you mentioned, which is playing games. And that's fun. That's super fun. But what you've kind of start doing is realizing that the, the manuals that you've got, or the memory of how computers work you've got, is incomplete and wrong. And some of the video games of yesteryear, maybe less so today with more, uh, random hardware available. But you know, back in the day, video game, programmers would discover

weird things that the graphics chips did or the CPU did, or the interrupt system did. And they would incorporate that into their game, or they would incorporate it into the copy protection for their game. So that, It was really, really hard for you to either copy or hack.

Or they got extra performance because they were relying on some extra weird thing, but of course everyone had the same physical hardware, so they could genuinely rely on these kinds of things. But you didn't know that as an emulator writer. So you just emulate it, like it says in the, you know, the 6502 manual says, yeah, well, this thing, this is, this sequence of bytes is meaningless. It's not used. And you, so you don't do anything with it. And then you discover games and you've, you've hit your debug trap that says, no, no, some, some game used that opcode. And you're like, whaaaa? The first time I hit this, it was,

I think it was the first time I hit it was in a game called Repton, which is like a Balderdash kind of game. And as part of the code actually jumps to an ASCII string, right. It starts interpreting an ASCII string as bytes now, as it happens, it's that doesn't have any meaningful side effect, but they're all these undefined opcodes in there. It doesn't do anything. And eventually it falls through, into the real code and carries on whether that was intentional, whether it was a mistake, I don't know, but it was like a first wake-up call. So like, hey, well, it doesn't like stop these computers, didn't have the facility to like throw an illegal instruction, exception at the hardware level, they just plowed on. And of course it's all logic gates at the end of the day. And so those bit patterns still meant something to the hardware and it did something and so the game designers would discover that, Hey, if I use this opcode, it's, it's the store instruction, but with one bit different now, normally there's, there's the store a, the A register, store x, store the X register, store y, the Y register.

And then in the fourth, if you were to literally just looking down the list of opcodes, there was like, No, don't use that one. And then it went on to the next instruction. Yeah. And of course, you know, you're thinking to yourself, wait a minute, if I, any normal human being

Who was designing this would see that essentially the only thing that was changing from one instruction to the next are the bottom two bits, you know, in the store A, It's 00. Store X is 01, 10, and then 11 you're like, well, what the heck is this doing? And so it turns out that through whatever means people realize that what's happening at that point, is it storing a, and'ed with the X register. And now I have literally no idea why that would be useful in a game context, maybe for masking actually. Now I think about it. I can't believe I've never taught this before, but if you had like a sprite mask in the X register, you've saved yourself an opcode, right? You don't have to and with the, a register and then store the a register, use this, this SAS sax command, which isn't, again, it's not an official thing, but it just comes out because that's the way the hardware physically work.

And so you start finding out more and more of these things, you start realizing that the systems are not as well understood as the manuals would have you believe they're not as well as understood, understood even as the designers of the original hardware necessarily understood them. And so to sort of give some context to this when I was about 14, 15, a good friend of mine, and I were trying to take a game that was stored on cassette tape back in the day, when that was a thing, you know, you plug in a cassette tape into the port on the back of your computer. And the beeps and boops were like really low, um, modem speed sounds. And it would eventually load, it would take, you know, four or five minutes and it would load, but we had disk drives, right? We were lucky that we had disk drives.

We wanted to take the game that we had on tape and put it onto disk, but that meant hacking it. Now whether we actually legally owned the cassette tape that we were hacking it from is one for the pub. Although in fairness, I have now met and spoken to the, the author and I've kind of made my peace with him over this particular, um, thing. But anyway, this was a particularly, um, interesting hardware protection that was put on the cassette tape version. So effectively the code was, was loading the beeps and boops from the, the, the cassette tape. And then it would jump to the, the beginning of the, uh, the, the code. Now I could write a piece of code that listened to the beeps and boops and gets this big block of, of, uh, of code. And then I could probably just save it to the disk and then just run it, but that it didn't work.

And I forget for the reasons and my friend rich, who would maybe listen to this, will perhaps remember why, but that wasn't possible. What you needed to be able to do is essentially decrypt. And just pull out the bit of the code that you needed for the game. Now, the author didn't want you to copy it. The author didn't want you to put it on this. The author didn't need to look at his code, right? His, his admittedly compiled or assembled code. So he would write an encryption routine right at the beginning of where the program started executing. And that decryption routine would decrypt the game that was immediately after the decryption routine. So the code literally follows off of the last bite of the loop of the, for all the bites in the remainder, E OR with FF worst possible protection system. And then it would fall into the middle of the game.

And there was no point in between those two where you could put a break point or similar. And the reason for that there were no hardware break points. And you know, the first thing you might want to do is you might want to modify the first bite after the end of the decryption routine to, to say, no, no return here. But of course you don't know what to write there because it's going to be decrypted. So you have to write the encrypted return, and you could probably guess there's only 255 combinations, so you could try all of them. But then if you actually looked at the decrypt routine, you see that things that go into the key, you know, I just said exclusive or with FF, but the things that go into the key that decrypts the rest of the code are the bytes of the decrypted code, the bytes of the decryption routine itself, the decryption routine also modifies itself while it's running.

So the key is mutating as it's going, and there's no way for you to poke at it, um, to stop you from doing like time-based attacks where you kind of stop it, jump out of that, and then patch up the routine instrumented in some way. And then jump back to the beginning of that part of the routine that you patched out. Um, it would use hardware timers. So they were timers that were VAT changing variably as the thing was going on, it also would configure interrupts, and it wrote its own interrupt handler that deliberately did not like leave the state of the machine alone. And it came back from the interrupt routine, having modified another part of the key, so where this interrupt was triggered and all that stuff, all factored into it, which made it basically impossible for us to.

Richard and I, back in the mid nineties, we wrote, uh, an emulator for the very computer that we were writing an emulator on in the basic routine, basic sort of language of the computer that we had in an attempt to emulate the protection system perfectly or emulate the BBC perfectly, such that we could get to a point and then stop it and then save. And we spent a couple of weeks on this and we gave up because it was just so complicated. All these things I just described to you, we're in this code, we'd have to be perfect and they would have to be cycle perfect. Like literally every tick of the clock, all these numbers are changing. Hardware states changing, interrupts are being scheduled or descheduled or canceled or not canceled. And the whole thing was in there and it was brilliant and we'd like, well, we can't crack it. So we gave up on it, but fast forward, 30 years, Oh gosh, really is. Fast forward several decades. And I'm writing an emulator for that same machine, but for recreational purposes, not for nefarious purposes. And I came across this game again, and I'm like, no, this time, this time game, I'm getting you, I'm having you

So, one of the evil. I mean, in addition to all the things we just talked about, one of the things that this particular, uh, protection system does is it used a rotate instruction. So rotate is like a shift where the top bit goes back into the bottom part. So a bit seven goes off and comes back into bit zero. And if you think, um, uh, and this particular rotate instruction was a rotate, a memory address, which was unusual on the 6502. Normally you'd have to load something into a register, do something with it and then store it back. But in this particular case, you could rotate directly a memory address. So if you think about what's happening inside that instruction it's going to take several CPU cycles, it's going to have to read from that memory location, it's going to have to do something, and it's going to write it back out again.

And this is all within a single instruction. Now what Kevin's protection system did is he actually rotated the address of a hardware timer. So the hardware, this is a piece of hardware that pretended to be memory, but actually was, was not memory was a counting timer. And it was, it was configured on the system such that reading would give you the current value of the timer. Writing would reset the timer and start it again from that value or, you know, something equivalent to that. So that's interesting now, suddenly in order to modify, uh, to, to emulate this instruction perfectly, I have to emulate within the instruction because as the instruction is executing, that timer is ticking down. So does it read from the timer on the third cycle or the fourth cycle does it write to the time on the fifth cycle or the seventh cycle or what what's going on?

And so you're there now you're assigned to answer, this was never put in the, in the, in any of the, the data sheets to my knowledge, right? This is just like, no, a rotate takes seven cycles. So how do you answer these questions? Right? I didn't, unfortunately I didn't have a 6502, well I did, but it wasn't in the BBC micro. Uh, it was in a, on a breadboard in front of me right here, but, um, how do you answer these questions? And so this is when I went digging and I discovered a whole world of folks who are taking chips, decapping them that is taking the protective plastic or ceramic layer off of the top of the chip using extremely distressing chemicals to etch way at the, the plastic and the surface, really horrible, horrible stuff. I mean, there's terror stories out there, uh, for, for what happens if you don't, if you don't look after yourself.

So please, if anyone's listening to this, do not attempt any of these things,

But these are like folks who are just more, um, uh, what's the word, more relaxed around scary chemicals than most people who are interested in this kind of stuff. And, and in particular, the 6502, because it was such an important processor, a they, they, they targeted that first. So there's a website. You can go to visual six, five Oh two.org, which shows you the high resolution photographs and the polygonal data that they've extracted from those high definition photographs showing the different layers of the chip. And like these, these chips are old enough that the manufacturing process, A is visible under a high quality,

But affordable, um, light microscope rather than anything kind of scanning tunneling, electron nonsense, but also that there are, so there are a few enough layers that it's practical for you to extract the layers one by one, by like washing it and another, and counting to 10, and then quickly washing off the solution and then looking at it again and like taking off these layers, you know, there's a metal layer at the top, and then there's various level layers of polysilicon and there's various doped regions, which sometimes you can see cause they look a bit different. Other times you just have to infer that they're there, but the, the whole, uh, premises that they have now, a gate level, like I was going to say transistor level, really. It is what it is. It's a transistor level simulation of a 6502 that you can go and run in a web browser.

And they've more over, you know, as well as just emulating it and all the pads that is the big connections that come up around the outside of the chip. There's been enough work that they can tag areas of the die and say, well, this is the, A register, right? These, these flip flops hold the, A register. This is the X register. Here's all the other internal state. Here's how the PLL works. Here's how the, um, the, the the instruction decode works. So with this information, they were able to explain comprehensively things like how the store AX, uh, instruction works. You know, that SAX that we found, they could trace the two low bits that we, I was talking about before the O the O one and say, well, what happens when the one and the one, well, this just activates the, A register and the X register at the same time now, because of the way the physics works.

And this is in the particular case of the, the 6502, it's an NMOS, um, type of part, the, the sort of pulling low of a zero wins out. So if there is a one being pushed onto the, onto, like the output bus and a zero being pushed onto the output bus, because both the, A register and the X registers kind of like enable, has been turned on at the same time, then the zero wins. Cause it pulls the one down to zero. So that's the, and that you're getting, that's why it's store A and X. If you put them both on, at the same time, now, different fabrication would have come up with different other, uh, sort of side effects. But that's like one example of like, that's what's happening in that particular instruction. So I was able to go to their site and do essentially a gate level, transistor level simulation of the rotate instruction and see exactly what happens and what cycles the read happens and what cycle the write happens.

And I was able to discover, and this was actually well-known before this is, I'm not discovering anything new here, but the, in between the read and write, there is a one cycle delay while this chip is like thinking it's like reads the value into a temporary part of the chip, does the rotate, the next clock tick, and then writes it out. But there aren't enough pins on the outside of the 6502 chip itself to sort of say, I don't need to read or write to memory this clock cycle. So it's always reading and writing from whatever the pins are.

In between step the chip, isn't doing anything, but these pins are still on the outside. And the RAM doesn't know that you're not talking to it or you don't care. And so normally it wouldn't matter, right? If it's no harm, no foul, it's like I'm reading or writing to a piece of memory that, okay. It doesn't, it doesn't matter. Or hopefully as long as you're not writing zero to something, but it turns out that what's actually happening is it's. It is already updated. The address pins

To, sorry, the address pins remain the address of where it just read from. Okay. And the databus still has the old unshifted value on it. And it's, but it's already told to, write. So it's done a read.

Read the address, got a value 12 out. And then while it's doing its rotate the microcode inside. It's not even microcode, but it's not going into that. But like the, the, the, the design of the chip inside, it said, well, we might as well toggle to a write. Ready to go, because in a minute, we're about to write out whatever we've just, we're calculating. But of course what's really happening is a write of the old value has happened to the memory location it just read from.

Okay. So, so far, no harm, no foul. It doesn't really matter, right? If you've just read from a memory location and you mainly wrote back to it, and then now you're going to update it with the shifted value. That's not a problem. Except of course, if that hardware was hardware, it wasn't memory because writing to that address changes the state of the hardware you're writing to. And so this was a really significant find because the timer that is actually on the end of this rotate gets two writes. And each time it resets itself. Now you are about

to write a new value over the top of it. But what happens if in that one clock cycle, the value you wrote to triggered the timer to go off, now an interrupt scheduled. And so this side effecting right, actually was important to model. And so that was the first surprise. That was like, my gosh, okay, there's this redundant write that happens. That's normally no problem. But now is, is, uh, is important to model. And it was, it was an important thing and getting this correct. Then there were some other things to do with the way site, the, the bus specifically to the BBC micro worked in terms of slowing down. Um, did you ever go through the apple? I know I'm frothing away here, but what did you have an Apple, um, Apple two or anything?

And then finally I wrote the actual value that I wanted to write. And I got that through, and that was also split out. So like finding this out was amazing and a godsend and, and piqued my interest entirely because like staring at a bunch of seemingly hugely cryptical, cryptical seemingly hugely cryptic, rectangular regions that don't appear to be connected in any way was somehow something the wizards could read and say, Oh, that's an and gate. And I'm like, what? No, it isn't. It's like you highlighted that one green and that one red, why did you do that? Oh, well, this one's clearly doped this way, because it's an even number of things from the outside. And I know that, and I'm like, whaaaa?

How are you doing this? What is this witchcraft?

And as it happens, you know, the 6502 is, and I'm probably definitely belittling the work that went into it. Cause it was a huge amount of work. But as I've learned a little bit more, the NMOS fabrication technology that was available to them in like the early seventies was, is easier to understand coming in from the outside. And you can sort of see, Oh, that's a really big squiggly bit. That's probably a resistor. Um, this bit over here is like a power transport, a transistor this thing here with like, you can sort of see it if you squint. But again, it's one of those things where I, I don't know how often you have this experience when something you're being explained makes perfect sense while you're listening to the engaging, present presenter.

And so I've really loved that it's been a really fun experience. Um, and so, you know, obviously it's touched me in terms of my emulation work. And then more recently, some friends of mine have started looking at a disk drive controller, which was something that came alongside the BBC micro to control the disk drive. I mean, I know on the Commodore, for example, they actually just throw another 6502 inside the disk drive and say, well, that can just drive the disk and do the, all of the, the, the magic to understand how to, you know, shift, uh, the stepper motors in and out and how to turn on the sense detection and all those kinds of bits and pieces, how to do the MFM and FM decoding, which made it really a, an amazing target for like, well, here's a co-processor if it's not running the disk drive, surely we can do something else, which is exactly the kind of cool thing that was done back in the day.

Because again, you could guarantee it on the end of a wire, I just loaded it off desk. Um, I got a spare 6502 brilliant. Now the, the disk controller that, um, that, uh, my friend took apart is an Intel disk controller. And it's long had this again, a big list of here's the commands you can run. And you're like, well, there's a gap somewhere in this list of couple, I wonder what it does. Right. But yeah, through, through decapping it and getting a friend to photograph it and then go over it. They were able to isolate an area which looked like ROM. And so it became clear that it was basically a CPU, an embedded CPU, probably with a very strange, um, uh, opcode list. But they were able to isolate where the ROM was and then using some real tricks of like known bytes that were written to the disk drive when you like format it, for example, they were able to like find regions of the ROM that corresponded to that, and then theorize and later on, essentially reverse engineer, the opcodes for this CPU.

That was the drive controller that lived inside the, uh, the, the, the, the BBC micros, uh, disk drive system. And I mean, that's just amazing. And I love that kind of stuff. It's just so interesting that there's always a layer lower than where you are. Right? You can be you can be writing in, in a high-level language and be vaguely aware you're interacting with an operating system. You can be writing an operating system and you could be using C++, or you can be using C and you'd be thinking like, yeah, I'm laying memory out. I know exactly what I'm doing and then you realize well, there's a DMA controller that's could also read and write to memory, or the network card can also read and write to memory, and there are interrupts and, you know, you kind of think, well, okay, I know where I with with interrupts, okay.

I've written an interrupt handler. I've written games before, and I've kind of had to feed the graphics unit, by every time it ran out of work, I would, you know, all these cool things. And you think, I I've, I'm talking about myself here. Like, I think I've reached the bottom of the stack. Now I've written an interrupt handler. I'm that's it I've, I've, we've done now. It was like, you know, when the Lord Kelvin said, there's no more physics we're done here, but then he goes, no, no, no, no, there's, there's more, you know, you can dig down and on Intel processes, you can go where there's microcode. Um, you know, what you put into the processor is not even remotely what it actually executes you like, Oh, really? Well, that's fascinating to me. How do we know? How can we observe it? You know, there are all these clever tricks and you kind of start learning down further and further and optimizations. And you're like, wow, that's amazing. And then somebody else comes along and blows your mind by saying, you know, you can take the lid off that chip. Right. See how it works inside. And like, Oh my gosh. Yes. It's so cool. It's so cool. And I've, yeah, I've really enjoyed the journey. And I'm still learning a lot, a lot about this, but it's just a lot of fun.

Okay. So there was still no way you couldn't go in and say, let's take out the jump to the reset and kind of get in there because again, the CRC code itself that the bytes that implemented the CRC were part of the key. So by modifying it to say, Hey, don't do that. You would destroy the key and it wouldn't decrypt correctly and all that stuff. Right. So essentially any minor perturbation to the code anywhere would either crash it, or it would get to the end of the decode routine, the CRC check would fail and it would wipe the memory of the computer. And it would have to start again, loading off tape for another four or five of your life. Right.

And so Richard theorized that Kevin had desoldered the ROM on his motherboard and replaced it with a ROM that did not wipe memory when you jumped to the, it failed the CRC routine. So far. So, okay. How do you use that to, to help, right? Because you run it and if you've got, if the CRC was wrong, well, it would just reset the computer, but the memory would be okay. Still. The second thing is that Richard realized is that the way that the cipher worked essentially is cyclical. It's like a random number generator style, like co uh, encryption key generator, right. Dynamic, but, but cyclical. And so it would repeat after a certain amount of time, certain number of iterations. So what the process was is Kevin loaded up this decrypt and CRC put the known CRC of the un-encrypted game into the CRC code, because that's what, you know, that's how it should, that's how it knew it got the decode right.

And then he put the un-encrypted game after it. And then he put in his desoldered ROM with the don't wipe memory. And he executed it once. So what of course happened is that the decrypt code ran on the un-encrypted game and caused it to change in some totally unpredictable way, but critically the same way every time. And in a way that if you were to continue doing it, it would eventually cycle back to where it started. Ah, okay. So then it would get to the CRC code, the CRC code would fail and it would reset the machine, but keep everything intact. Kevin would save that image to disk and then he would run it again and then he would run it again and keep running and saving and running and saving. And every time it reset, he knew that he was another iteration along.

that a hacker would use timers and counters, and, you know, self-modifying code and all this kind of nonsense to stop them from hacking it, it was, um, understanding enough about the way that this particular type of encryption works, that it would cycle. And it was at the end of the day, desoldering something off the motherboard of your machine and replacing it, which like nobody thought of. Wow, it's so cool. It's so cool. And so interesting. And, and yeah, I'm, I'm glad to say that, um, jsbeeb will run the alien eight protection system. And in fact, uh, Kevin gave me permission to make it a test so we can get back to testing. Uh, one of the tests is to actually run all of that, that I just described and make sure that it gets to the end and executes the first bite after the CRC code, which incidentally I don't, um, I don't even have right.

The rest of that code is like, it stops it there's, there's, there's not enough for it to be the game. So it's just the code, but it's so fun. It's so cool. And there's so much to learn from it. And this is again, 40, 50, 45 year old computer now. Quite. I mean, not 45. Gosh, now I'm 45, 35 year old computer. That's slightly better. Yeah. Wow. So do you have any experiences like that? I mean, we, I mean, we we've taught that like at, uh, you know, late teens, early twenties, we, we, our paths diverged.

You know, you can put a ribbon cable on the back of it, and then you could like plug it into turtles that drove draw lines on the, on the ground or hardware, things. And magazines would even have like printable stuff. I remember having like a, a, an eight, uh, element led, um, wand that you could build. And there was inside of, it was a, uh, like a motion detector or actually it was a mercury switch. And so it could tell how quickly you were waving it backwards and forwards. And for those listening on the podcast, they're not looking at me in the video, which is only Ben I'm wiggling my hand back and forth, like I'm batting a bat. And so, sorry. And then allowed you to like, write a message in the air by serializing out and turning on the LEDs. And that was so cool because I was building with a regular computer.

So anyway, that exposure to hardware and the low level, and the fact that just hackability was what inspired the raspberry PI. And so I feel like there's a nice warm glow. I get from like, realizing that now on my desk here, as you can vauges see just out of shot, I have a whole bunch of raspberry PI related gear. Cause I'm hacking on this kind of stuff again, and it's brilliant. And so people can get raspberry pis, they can get Arduinos. The Arduino IDE in particular is just really, really user-friendly. So you can go write an emulator. You can hack around with this hardware. And there's just a lot of cool things that people can still do this nowadays, even though we, it feels as we're staring at these, these huge supercomputers in front of us. Well, even the ones that we have in our pockets are super computers by anyone's, um, yardstick. These are all there. There's still an accessible route to playing around with interrupt handlers and assembly and, and, and just understanding at a deeper level how computers actually really work.

However many we've got found this interesting too, if we haven't just alienated everyone, then I'd be surprised. But I mean, I'm just hoping that the, the, the enthusiasm, which I feel for this subject comes across and some, you know, at least one person is inspired to go and have a look or hack on some hardware or dig down through the chip layers or read up how their CPU isn't really doing, what they're telling it to do is a deep, deep subject. But yeah, I guess we can talk about other things another time, right?