TechStuff Classic: How Image Sensors Work

twenty nine, two thousand twelve. It is called how Image Sensors Work, and we're talking about the various sensors that you would find in digital cameras. So I hope that you enjoy this classic episode as Chris Palette and I tackle this topic. So today we thought we would look

at something. Actually it was Chris's suggestion that we look into this particular topic, which was the the topic of image sensors and what they do and what the two main types of image sensors, how they are different from one another, and uh and I thought it was a great idea. It's also a fairly complex topic. We do have an article on how stuff works dot com that says, what is the difference between c c D and CMO s image sensors in a digital camera? And that's really

what we're gonna be talking about here. Um. So that there is an article on the site, and that's a nice short article if you want a quick overview, but we're gonna go into some detail a little bit in this podcast. And really the first thing you need to know is that an image sensor is it's taking the place of film, right, Yes, that's correct, Yeah, a long and a long time ago in a galaxy that happens to be right here where we're sitting. We did a

podcast on the megapixel myth um. I think a lot of people equate uh, numbers with a way we have with quality and they say, oh, well, I've got a twelve megapixel camera that's obviously better than that six megapixel camera I used to own. Well, it depends on what you're doing with the photo. It also depends on again the other qualities of that camera, right, and image sensors have a lot more to do with the quality of

the photo. But in a way it really depends because again this uh there, there's this this idea that there are two different kinds, which kind is better? It depends on what you're doing with that what are you taking photos of? Um? And Uh, As it turns out they're

they're not really better than one another. Inherently, they're they're better than one another for specific applications of the photographic technology, and the quality of the two sensors is constantly getting closer and closer, so that the things that one sensor does better than the other start to become less distinct

over time because the technology is improving on both sides simultaneously. Uh. Now, if we were to go back a little bit to the early days of digital cameras, the distinction was was clear. You know, you would say that, well, a professional photographer would more likely have a cc D image sensor in his or her camera. CCD meaning charge coupled device, charge

coupled device, that's that's one of the two types. And someone who has say a relatively inexpensive of course, back in the early day of digital cameras, that was definitely relative eight billion dollars, only a thousand dollars, yes, princely going back to that a thousand dollars as opposed to say eight thousand dollars. But a person holding one of those cameras might have a CMOS or a complementary metal ox side semiconductor image sensor. Yes, they come up and say,

that's a wonderful shirt you're wearing today. That's that's such a great picture you've taken. Have you lost weight? No, it's not that kind of complimentary. I have a whole joke about that, but I'm going to spare everybody because we've already said the punch line. Anyway, these are the two different sensors, and they do go about capturing data a different way. Let's let's go into the basic way

a camera captures an image. I'm going to talk about still camera here, so we're talking about cameras in general, not necessarily film, more digital. Right, So, in general, what happens is you've got a camera and you're pointing it at something that you want to take a photo of. Light is coming towards you. It's reflecting off of the

the subject of your photo. If light we're not reflecting off the subject of your photo, it would either mean you were in total darkness, in which case taking a photo is not very helpful, or you're taking a picture of a black hole because not even light can escape it. That being said, they're actually looking at making a physical picture of a black hole. Using radio telescopes, which is so cool. That's the tangent. Anyway, so light is coming from the subject a little bit is awesome. We should

do a full podcast us on that. But anyway, lights coming from from the subject toward the camera and uh, and the light passes through the lens. The purpose of the lens is to focus that light toward a specific point within the camera. It moves through the aperture, which is the opening behind the lens that allows light to pass through. There's a shutter that's there behind the aperture which actually directs the light up towards the view finder.

For the old style cameras, you know, the ones that don't have the you know, you're not looking at a screen on the back, you're looking actually through a view finder. Well, that light gets directed up by a mirror and that's essentially attached to the shutter that makes the light go up inside the camera. Then it hits a prism which inverts the light. Because you may not know this, but the light the image that's coming in. That's saying the

sensor is actually upside down from our perspective. Gasp. So if you didn't have that prism there, the subject you're looking at would be upside down. It would be like everything you're making photos of was in Australia. That unless you're Australian, in which case it's all in Detroit. So that's the that's the way. If you're wondering why there's

all these giant car factories in Australia, it's not. It's just because you didn't have that prism in there, right, Um, Okay, that's a terrible joke, but no, the prism does invert

the light, so otherwise again upside down. So when you press the button to capture an image, the shutter, the shutter, the shutter release exactly, the shutter, the shutter moves out of the way and instead of the light hitting that mirror and going up to the prism and inverting, the light hits either film in a film camera or an image sensor in a digital camera. So really the shutter just moves out of the way and then the light

hits the sensor and then you're good to go. It's a little different with the digital cameras that are out right now, but that's in general how the process works basics. Yeah, now, and with cameras now, light maybe hitting the sensor constantly, and the shutter itself is not a physical shutter. It's just the way that the sensory captures data. And we'll talk about that when we get to that point. There are two different major types of shutters that we can

talk about. So that's the general process. Now, with film, it's a chemical process. Light hits the film and then some chemical reactions take place, and that's what allows you to capture an image. Right. With image sensors, it's not chemical, it's electrical. Right, you're converting light energy into an electronic signal. Yes, and then you're gonna want to store to some medium. Yes, uh, you know, typically some kind of flash memory device, depending

on on what kind of camera you have. You know, there were some I think that that stored on CD, so you know, your mileage may vary, but in general, some sort of flash device on onto day's cameras. Yeah. The old digital camcorders could record on on different kinds of media and so, and digital camcorders are working under the same general principles as digital still cameras, with some

you know, other differences, but we'll talk about that. Like I said, so, now we get into the differences between the two major types of sensors, the charge coupled device and the complementary metal oxide semiconductor. So we're just gonna go do cc D and CMOS from here on out, I think, otherwise I'm just going to have tongue twisters for the rest of the podcast. Yes, well, I just wanted to make sure that people knew what it what

it stood for, obviously very important. So in in a c c D sensor, every single pixel now, pixel, remember, is a point of light. An image is made up of pixels, millions of pixels. That's where the megapixel comes from. Right, So a twelve megapixel camera is going to take twelve mega pixels worth of pixels and within the dimensions of that image, whereas an eight megapixel camera will use fewer pixels for that same size. Right. But and that's where

our idea about resolution comes in. Sometimes you hear people talk about a low resolution image, it may be that it's got fewer pixels in that image so that you can actually start seeing if if the pixels are large enough and few enough, you can start seeing the borders from one pixel to the next. It's not very smooth, it's almost jagged. Well, yeah, I mean that this is

the benefit of having a high megapixel camera. If you shoot it high quality, then you are capturing more more pixels for a specific region of the image, and you can you can render that photo in a larger format. Um because when you shrink, when you when you compress the size of photo and reduce it in size um, the compute it or is able to you know, throw out unnecessary information and that the image still is pretty

good looking. When you try to increase the size, the computer has to sort of guess on you know, pixel by pixel basis. Well, I mean, this color is sort of a brown color. It looks like I could throw something else in here similar. And that's why when you increase the size of a photo, a digital photo, that it starts to look kind of jaggedy and rough because the computer is having to guess at what that information is. So if you take a ten megapixel photo and shrink

it down, it's it's gonna look pretty good. But if you try to take a two megapixel photo and blow it up, it's not gonna be so pretty. Yeah, if you think about it like a puzzle, Let's say that you have a puzzle that has four pieces to it, well, then you're gonna be able to see the division of those those four pieces very clearly. If it has four million pieces, then it's each of those pieces are individually

much tinier than those four giant ones. That's the air issue is that the larger you blow something up, if it's if it doesn't have enough mega pixels in it, not megapixels, but enough pixels, then you're gonna start to notice. But that being said, the general digital cameras that are out there for the consumer market and the general way the consumers use digital cameras, megapixels really don't matter because most of us are not blowing images up to poster size.

Most of us are using them for online photo albums. We might print a few out, but usually eight by ten tends to be about the largest because most people don't have printers capable of printing at a larger size, and when you take it to somebody to have it printed, it's kind of expensive, so a poster. Most of us don't do that, so most of us don't need to worry about megapixels at this point. Professional photographers, it's a different story. So cc D sensor, each of those pixels

has a charge. The photons that are coming in and hitting that image sensor are being transferred from from a light energy from photons into electrons. Now UH, they have UH there's an output node with a c c D sensor where that is converted into voltage. It's buffered and then sent to a different part of the camera so

that it will become an analog signal. So a CCD sensor it's a very it's a very UH specific device that doesn't it doesn't have a lot of other functionality to it apart from the fact that it's taking in light and converting it into voltage. UH. Now the pixel is completely devoted to capturing light and it has a very uniform output. So the that's that's sort of where the the idea of CCD being high quality came from. UH. It was very good at capturing the true essence of

whatever it is you're pointing your camera at. You don't have to have a you don't have to worry about low lighting effects that kind of stuff, or having uh an image turn out too grainy if the light is too low, which can happen with CMOS images, particularly from a few years ago. It's a depending on where you know where the manufacturer for your camera got the c m O S sensor. Uh, you might not have as

big an issue taking low lighting uh images. But if you've ever used a digital camera in a know, either a dark or just a dem environment, and you look at me like, this just doesn't look good. Now, when I take a photo outside in the middle of the daytime, it looks gorgeous, beautiful colors, very very distinct. Um. That's part of the problem is that the CMOS sensor captures it in a different way. In that case, every single

pixel has its own charge to voltage conversion. The c c D, it's doing all of the pixels at once. In CMOS, it's doing each pixel individually. And then the sensor itself has other elements added to it that the c c D sensor does not have. Remember we said c c D kind of offloads the information once it's been converted into electrical impulses to other chips, right, Well, those elements are actually on a CMO S sensor. So it's got amplifiers, it's got digitization circuits, so it's actually

converting the electricity into bits. On the sensor itself, it's got noise reduction capabilities, and so that means that it actually speeds up the process and it decreases the amount of space you need within a camera because all of those elements are found on a single chip as opposed to having dedicated chips for these these specific functions. Unfortunately, also reduces the amount of space it has for image

capture because all that stuff is on the same chip. Yes, so that that you know, that's a downside, yes, so you that was one of the arguments again early on, was that c c D cameras could take sharper photos than CMOS cameras, and that you know, it's almost there was also an expense issue, right, c c D image sensors tend to be more expensive than CMOS ones CMOS.

The process of manufacturer got so efficient that the price started to come down, and that that's why those are the sort of image sensors that you find in things like smartphones. You know, smartphones that have cameras tend to have CMOS sensors in them. They take up less space, they put out less heat, they take less energy to run um and they're very fast. So those are all the qualities that people who are having a who wants something in a nice slim form factor or if that's

what's important to them. So yeah, C C D S image sensor might take a sharper quality photo in certain situations, but it's also going to require a larger form factor, and it does take more energy to run, and that that energy is going to also mean more heat. Yes, as we know, as electricity runs through a circuit, one of the by products is heat. We haven't figured out a way to get around that yet. It's just one

of those one of those realities that it's uh um. Basically, it's inefficient enough where some of the energy is being converted to heat energy instead of you know what it is intended for. Right, Chris and I have a little bit more to say about image sensors, but before we get to that, let's take a quick break to thank

our sponsor. So, so now we've got down to the the idea of these two different image sensors capturing uh information in different ways um, and the fact that over time both both types of sensors have developed to the point where the differences between the two, apart from the fundamental difference about how they collect information, have started to

to diminish. Right, So that you can find some professional cameras out there now that you CMOS image sensors, whereas you know, a few years ago that was really unheard of. And you can also find some consumer cameras, especially in the cam Quorter realm, that are using cc D image sensors, which again for a while you just didn't hear about

because c c D cameras were so expensive. It was pretty much reserved for professionals, you know, just consumers just didn't necessarily have the money to drop on something like that unless they were you know, one per centers. So yeah, it's it's it's it's still a developing thing and we're still seeing that kind of level out. But that and

the two technologies do still exist. They coexist, so it's not like one has been abandoned on top of in favor of the other, although that tends to there there's usually someone predicting that every few years. Well sure, sure, um yeah. A lot of the research that I did for the podcast was from Teleedne Dolsa, which makes which makes both types of sensors, and they had some really interesting,

uh comparative white papers and other information. If you're interested in getting into the depths of it, it got some of it got fairly complicated. UM. But basically they they had one paper they said that they're, uh that image sensors can be measured on basically eight different characteristics UM. And these were responsivity, you know, basically how responsive that the sensor is. Uh, it's dynamic range, uniformity, shuttering, UM, speed, windowing,

and anti blooming UM. And you know, again this is kind of you know complex, but the the uh, it's kind of funny because the way that the image sensor captures information. UM. You know, depending on the type that you're talking about, they're not really uh, it's really application specific. UM some of them. Some of them really don't have that much difference over the others. Like, for example, UM, CMOS chips are known to be a little bit more responsive. UM. But c c D s are have an advantage in

dynamic range. But basically they didn't say, you know, the this one chip is better than the others. They said, it has more to do with the manufacturing capability and whether the chip has done right and is used in their correct setting than it does UM, you know, for a particular type of technology. Right, and you were mentioned mentioning the fact that there are different shutters. In general,

a CMOS image sensor uses a rolling shutter. Uh, there's nothing saying that it couldn't use the same sort of shutter that SEC the image sensor does, which is a global shutter. There's nothing saying that it couldn't. It's just that all the camcorders I looked at specifically, because this really plays more into video than than uh, still photography. Although there's some crossover between the two. Um it said that you could have a CMOS with a global shutter,

it's just that you don't find those. So what's the between the global shutter and a rolling shutter? Well, a

rolling shutter to me. And when I the first I read about this, the first thing I thought about was a copier or a scanner where the image sensor you put the document on the on the screen, you close the UM the top of the lid, and you tell it to go ahead and make a copy or make a scan of it, and the image sensor travels down the length of the document from the top to the bottom or what exactly, and and it is going you know from UM it's starting at a specific point and

capturing the entire document as it travels the length of it and uh, you know, because it's going essentially line by line. If you think about that in pixel terms, is taking a row of pixels and then another row of pixels and then a nighte you know, as it goes down. Right, Yeah, I was thinking of it sort of the way television works. Yes, where it'll it'll you have a line by line from the top to the bottom. Um will ignore the interpalation part, otherwise we have to

get really complicated. But anyway, the image is painted essentially on your screen from the top to the bottom at a rate that's so fast that your eye does not detect that. It looks like it's all simultaneously projected to you, but it's actually done line by line from the top of the screen to the bottom of the screen. Same thing with a rolling shutter. So when you take a photo or you're using a camcorder, let's stick with cam quorters.

So if you're using a camcorder that has a rolling shutter type of image sensor talking cmos, uh, the the images being recorded from the top to the bottom over and over and over again. Okay, so uh with a c c D camera, it's a global shutter which means that it's capturing all that data all at once, Yes, sort of like film would. Yeah, so it's not it's not um you know, it's not something that's gonna be scrolling.

It's all one image. So this means that the two different types of image sensors are also prone to two different kinds of flaws that can happen when you're using them. Well, of course, I mean that's that's like any other types of technology. Not everything is suited for every use, right, So let's say that let's i'll talk about the different flaws that you can find. C c D essentially has one type of flaw that you can encounter, which is

called the smear effect. So smearing is let's say that you've got a a you're taking an image of something that has a bright light in it. Um Smearing is this effect where you sort of see the light. You'll see like a projection of light above and below it or you know it's that's that's why it's called smear. It's it's been extended beyond just a source of light itself. It's kind of like a halo effect, though usually it's more of at least in the samples. I've looked at.

It's more of a vertical thing where it looks like it's almost like a ray of light that goes straight up and down the the the screen from the source. So that's one of the things that c c D

image sensors can fall victim to, but not CMOS. And it's all because that global shutter exposes that image, the whole image simultaneously, and it's all gathering that light, and once the predetermined shutter speed for that global shutter has elapsed, it stops gathering light, turns that that entire exposure into an electronic image, and then starts again. And the rolling shutter just doesn't have that same effects, so the smear does not happen with that, and it's you know, it's

very noticeable. If you see the the effects of this, you'd think, oh, well, that's unfortunate that there's this weird shaft of light right there in the middle of the frame. Well, that's that's it for the c c D. Okay, that's the that's the one flaw that's c c D image sensors can can fall victim to. But there's the one known thing that people complain, the one thing that people complain about. There are three three different ones for cmos.

The first is called skew. Okay, So you've got this rolling shutter and it's going from top to bottom as it's recording images. Now, this utter is going off u multiple times per second. But let's say that you are panning the camera very very quickly from one side to another, so you're changing the view. Well, you're having a rolling shutter and you're panning the camera. This can cause the

idea of skew. We have just a bit more information to cover in this classic episode of tech stuff, but before we get to that, let's take another quick break to thank our sponsor. So let's say that you have something that's uh, that's significant, a big thing that's in the frame of the photo, maybe maybe like a tower. All right, So you've got a tower in the frame

of your image, and you quickly pan from left to right. Well, as you're panning, that shutter is rolling, and if your pan is fast enough, then the shutter is actually going to start building an image where the pixels at the top of the image are further on one side than the pixels that are at the bottom of that image. Because it's not capturing all that data simultaneously. The outcome of that is that you get a skewed image when

the output image itself is skewed. So that tower, which might be perfectly straight when you look at it, when you start looking back at the video and you're playing it back really slowly, it suddenly looks like it's leaning or it's diagonal. It's like that, you know, it's suddenly not it's not true anymore. Now I understand what's wrong with all those vacation pictures I took. Yeah, exactly, that's the that's it. You know, it's no, it's not at all.

But anyway, that's that effect is because of that rolling shutter, you know. And again a global shutter would not have that problem because it's taking all that image, you know, it's taking all the information all at once. The rolling shutter is taking it bit by you know, line by line. And again it's only if you're panning very quickly, because it's this is going so many times per second that

if you're doing a nice slowe hand, it's it's not noticeable. Also, you're more likely to prevent the kind of nausea that's associated with the quick panning of Yeah, we'll get to the human advantage to that too. Next is the wobble. Yes, so you don't have this problem. No. This is wobble is when you get sort of a weird, stretchy or rubbery look to stuff that's going on in the video.

And it tends to happen with handheld footage, right because you're when you're holding the camera, you don't have that steady base that you would if you're using a tripod. So let's say like a found footage film sure becoming more and more popular these days, So something like all onlines of Blair Witch or clover Field or or one of those movies or or or vhs made by friends of mine. Check it out. It's us. It just premiered over at Sundance. Um, that's a shout out to my

buddies anyway, So same sort of thing. It's it's because of that rolling shutter. The information is being captured line by line. If your camera is not steady then and if it's moving around quite a bit and at a fairly fast pace, then it's the The images are not going to be uh, they're not gonna be clear. They're

gonna end up having this wobbly, stretchy look. So let's say you're panning uh down, so you've got you you maybe you've got your looking at the top of that tower and you start panning down very very quickly to say, simulate a fall. So uh, we're panting down very very quickly. That rolling shutter is going up from the top to the bottom very quickly. As you are going down, the shutter is going to uh. If you're matching the shutter speed or getting close to the shutter speed, it's going

to make that building stretch out, it's gonna look very odd. Um. And so that's another one of those issues. And again the global shutter doesn't have that problem because it's not it's not capturing information the same way. Uh. And then finally there's partial exposure. Partial exposure happens when light is hitting the shutter or the the image sensor at a very particular moment and and the light is hitting it just fast enough so that when the rolling shutter starts,

the light's not there. But before the rolling shutter has finished, it's it's a journey across the image sensor. The light has coming gone, which means that part of your image is going to be much brighter than the rest of

your image. So if you think about your image as h let's say you're taking a picture of, say a poster, all right, you gotta you're looking at a poster and there's a flash that goes off as you are taking your image, and the flash is moving at a speed is a very quick, flat moving it's moving on speed that's faster than the rolling shutter is when you actually look at that picture, when you're looking at the poster in the back, it's gonna look like there's this one

band of the poster that's much more brightly lit than the rest of the poster, and that's going to be the moment when that flash hit the image sensor as the rolling shutter was going down the sensor. So this is another issue you have to work with your lighting in order to avoid it. And uh you know it can if you're using a flash that's a longer based flash,

you don't have to worry as much. This is why partially why anyway part of it because it's most of the the smartphone flashes or l e D s, But it's also part of why if you ever take a photo with a smartphone that uses an LED flash. It tends to last a while. It's because if it didn't, then your all your images would come out with this weird banding issue. And you don't want bands in your in your pictures unless you're at a concert. I'll be taking some tonight. Awesome, I'm gonna go see. Day might

be giants and that's a shoutout. Today might be giants. Everyone's getting shout outs today. It's free plug day on tech stuff. Well, you know a lot of stuff on tech stuff requires a plug. Yes, it's true. Not everything is better reoperated. So, yeah, the c c D is only prone to the smear issue, whereas CMOS has those other three. If you have a decent camera, you shouldn't have you know, in our are taking precautions. You just

have to worry about it, exactly. Yeah, if you if you know what you're doing, you can get around these problems. It's just that these are the ones that are the cameras are prone to based upon the technology they use. So it's not that every single image you're gonna take, or even even like a significant percentage of the images you'll take, will have problems associated with these issues that I've talked about, but some of them might and the reason why they they have those is because of the

technology itself. And again, you know, you just little basic tricks that you can do, you know, just for example, using a tripod whenever you can helps a lot, it'll it'll really remove a lot of this. Also, you know, most of most people aren't running around and jerking the camera left and right so fast that these are really coming into play. Uh And if you're using Instagram, really you've made your image look so crappy already you don't need to worry about these effects. That's that's just a

joke mostly. Alright, My wife uses Instagram a lot. What a lovely old tiny photo of the Space Shuttle. I'm so glad anyway, Uh so, yeah, I mean, so, which which is better? Really kind of it really does depend on what kind of photography you're going to be doing. Um, you know, probably the biggest difference is whether you're doing

still photography or or video. And most of the time when you're shopping for cameras, the type of sensor that's and it is not necessarily the easiest information for you to find out, although it does pay to to look into that if you can and and actually do some research on the sensor itself, because, like we said, the sensor and the lens of the camera is going to have a lot more to do with the quality of the images that you get using that camera than how

many megapixels it has. So even if you go out there and you buy a twelve megapixel camera and your buddy has an eight megapixel camera, your buddy's images maybe may look sharper and more vibrant than yours. Again, not to do with the megapixels. It's more about the lens and the sensory And of course, if you're you're planning on dropping a lot of coin on a new camera, probably would be a good idea if you read some reviews from professionals to give you an idea of what

you expect to see. If if other people are using it the same way, you will be um to get to really get an idea of how you know whether it's going to suit your needs, and that's the most important thing. Very good, Yes, good advice from Mr Bled And that wraps up another classic episode. Hope you guys

enjoyed it. If you have any suggestions for future episodes of tech Stuff, why not, right in The email address for the show is tech Stuff at how stuff works dot com, or you can drop us a line by going on over to tech Stuff podcast dot com and following the links there to our various social media accounts. You can also click on the little store icon in the menu there and go on over there and purchase

stuff in the merchandise store. We have lots of fun designs, and every purchase he make goes to help with the show, so we greatly appreciate it, and I will talk to you again really soon for more on this and thousands of other topics. Because at how stuff works dot com, wal won