Wednesday, February 17, 2010
Is a "ghost in the machine" creating an intermittent failure of Toyota's complex electronics? How difficult is it to find and repair such a glitch? How will the next generation of electronics improve on the systems in use today?
MAUREEN CAVANAUGH (Host): Toyota has recalled nearly six million vehicles in the U.S. for acceleration problems the company claims arise from misplaced floor mats and sticky gas pedals. There are those in the auto industry who are not satisfied with those reasons and point to a possible problem in the increasingly complex electronic systems of the vehicles. So, just how much are late model cars relying on electronic controls and how much do we know about them? And should we all begin preparing for electronic glitches in our next new car? I’d like to welcome my guest, Dr. Ingolf Krueger, UCSD professor of Computer Science. He directs the university's Service-Oriented Software and Systems Engineering Laboratory. Dr. Krueger, welcome to These Days.
DR. INGOLF KRUEGER (Professor of Computer Science, University of California San Diego): Thank you very much for having me.
CAVANAUGH: And we’d like to invite our listeners to join the conversation. Do you have a question about your car’s electronic system? Give us a call. We can’t fix it but we might be able to give you some sort of explanation for what’s going on. 1-888-895-5727, that’s 1-888-895-KPBS. So, Dr. Krueger, how long have we been using computer software in automobiles?
DR. KRUEGER: The first software modules in cars actually appeared in the mid-seventies and they were related to, you know, the injection system in the vehicle. So these were very isolated software systems, very small pieces of software, really, and, you know, they were not as interconnected as software in the vehicle is today.
CAVANAUGH: So it’s just grown and grown and grown from that point on.
DR. KRUEGER: It has, yes.
CAVANAUGH: And are these all models of cars, foreign and domestic?
DR. KRUEGER: Yes.
CAVANAUGH: Tell us about the current generation of car electronics. How much computing is in, let’s say, a high end car today?
DR. KRUEGER: Well, there are actually several measures we can look at to understand how much, you know, electronics and software is in a car. One is simply the number of computers you have under your hood, and so we have anywhere between 30 and 80, you know, separate computers that are all interconnected with one another via, you know, electronic networks just like the internet that you have, you know, connecting your PCs at home. And so we have a lot of electronic control units, each having its own computer system and we have on the order of 100 million lines of software code in a car today.
CAVANAUGH: That’s amazing. And tell us, so – I guess my question was going to be what do they do but I guess it’s more like what don’t they do? Do they do everything?
DR. KRUEGER: No, you know, I mean, there’s a lot of, of course, mechanics and, you know, mechanical engineering that still makes the car go so the engine is not a virtual engine, it’s a real engine. But as you see from, you know, the hybrid vehicles that are so much in the news these days, there’s a strong intercoupling between the mechanical parts and the electronics and software parts on the vehicle. And so, you know, most of the functions in a car, about 90% of the innovations of these functions is actually driven by software today.
CAVANAUGH: Now, as an example, describe for our audience the systems that are integrated with just locking and unlocking your car.
DR. KRUEGER: So this is really an interesting example you’re bringing up because in a high end car, you can find up to 19 electronic control units, 19 different computers being interconnected to supply the functionality for locking and unlocking.
CAVANAUGH: And what does – so when you put the key in, what starts? I mean, what do you activate?
DR. KRUEGER: Yeah, oftentimes you don’t even really put the key in anymore, you actually press a key fob…
CAVANAUGH: Of course, yes.
DR. KRUEGER: …and that’s where it starts. Already there, you have electronic communication, wireless communication, typically, between the key fob and the car. And then there will be communication, messages really, that are sent inside the vehicle to a variety of different systems. Let me give you an example. So when you start, you know, to unlock your vehicle, it will certainly send signals to the physical locks that are in the doors but it will also send signals to other systems, for instance in high end cars to your radio so that it can configure itself to the station you listened to last when you left the vehicle. It will also send signals to your, you know, car seat. There’s a separate electronic control unit that, you know, models the car seat, and to make sure that it moves into the right position for, you know, the person owning the key, and then there are other safety and security systems that are controlled by this as well. And, you know, of course when you drive away, in most cars today you have a feature called ‘lock on drive away’ which means that your very safety relevant motor control and the speeds it measures are actually connected to your locking system.
CAVANAUGH: And all of that with that single press of the fob, as you say.
DR. KRUEGER: Yeah, it gets started with a single press of the fob because, you know, it’s almost like starting up your home computer. That’s what you do with unlocking your vehicle.
CAVANAUGH: I’m speaking with Dr. Ingolf Krueger, a UCSD professor of Computer Science. He directs the university's Service-Oriented Software and Systems Engineering Laboratory. And we’re talking about electronic controls and how many of them are in our cars these days. And we’re taking your calls at 1-888-895-5727 if you have a question or a comment. Now, as I understand it, Dr. Krueger, most of these automotive electronics…
DR. KRUEGER: Umm-hmm.
CAVANAUGH: …have been getting high marks.
DR. KRUEGER: Yes.
CAVANAUGH: In fact, there’s an electronic stability control system…
DR. KRUEGER: Umm-hmm.
CAVANAUGH: …that’s actually going to be a requirement in…
DR. KRUEGER: Yes.
CAVANAUGH: …in a couple of years. Would you agree that these systems usually work quite well?
DR. KRUEGER: Yes. You know, I mean, if you look at the statistics—I mean, there are very few numbers really available that you can point to because analyzing accidents is not an easy feat—but there are numbers that indicate that for some car manufacturers the introduction of electronic stability control, for instance, has led to a reduction of about 15% of accidents with, you know, injuries and fatalities. And so, you know, we’re looking kindly at a crisis, quote, unquote, but, in fact, the electronic systems in the vehicle have contributed a lot to the safety and security of the drivers and traffic in general.
CAVANAUGH: What is electronic stability control?
DR. KRUEGER: It’s basically a system that interconnects your braking system with various sensors that measure what the driver wants to do, you know, in terms of steering and connects these systems with your anti-lock braking system so that by controlled braking application of braking to the vehicles, you can avoid a turnover of the vehicle in critical situations.
CAVANAUGH: I see, so it basically, if you’re overcorrecting or…
DR. KRUEGER: That’s right.
CAVANAUGH: …doing something silly, it stops you from doing it.
DR. KRUEGER: Exactly.
CAVANAUGH: That’s interesting. So what is the downside of all this – these complex electronics in our cars?
DR. KRUEGER: Well, I wouldn’t call it a downside, I would call it a challenge because all the manufacturers today face a complexity challenge. If you look, you know, we talked about the introduction of software in the seventies, and what I said back then is that those were largely isolated systems. Over time, more and more systems such as stability control, you know, anti-lock braking systems, throttle control, and so on and so forth have been introduced, step by step, into the car. And many of the features—I mentioned the number of 90% are software driven—are now interconnected which means that there’s a lot of communication between these, you know, 30 to 80 electronic control systems and you can look at the communication that’s going on on the internet and you have a small segment of it actually going on in your car. And so this interconnection between these systems, the many messages they have to exchange to make the car drive, literally, adds to the complexity. And the challenge for the car companies really is to get this complexity under control. And many car companies do active research on this topic. They, you know, form industry consortia for the next generation vehicles. But this is really a highly interesting research challenge that, you know, needs to be addressed, how to maintain control over the complexity.
CAVANAUGH: Now I was reading about this and it seems that one of the consequences of these highly complex systems within the car is that the systems have a tendency to interact with each other in ways that weren’t anticipated…
DR. KRUEGER: Umm-hmm.
CAVANAUGH: …when they were built.
DR. KRUEGER: Right, right.
CAVANAUGH: How does that happen?
DR. KRUEGER: Well, I mean, in – You have to look at how automotive systems are built today. The car company has a wide network of suppliers who build independent modules and the car company, in the end, has the challenge of integrating all of them into a working vehicle. And so if you think about, you know, 80 control units that are delivered to you as a car company, and you have to make sure that they all, and at the right time, exchange the right information, that’s a challenge. And they face this challenge and they, in general, do a very good job at it. But the challenge remains and it becomes, you know, worse and worse, you know, as we’re introducing more and more software into the vehicle because each of them will come in a new module or, you know, a subset of modules and this complexity has to be managed.
CAVANAUGH: We’re – I’m speaking with Dr. Ingolf Krueger and we’re talking about computer software in automobiles, the complex electronic systems in many of our cars these days. We’re taking your calls at 1-888-895-5727. And Steven is on the line and he’s on I-56. Hi, Steven, welcome to These Days.
STEVEN (Caller, Mobile I-56): Hi there. I’m actually on I-5 now.
CAVANAUGH: Okay, great.
STEVEN: But anyways, this may be slightly off topic but I have a Toshiba laptop and one of the features I like a lot about it is that I can just swipe my finger over a biometric identity…
DR. KRUEGER: Umm-hmm.
STEVEN: …pad and log on. I’m wondering if that’s on the way in for automobiles? Say where I could swipe my finger and get into the car by just touching something on the door instead of having to put in a key.
DR. KRUEGER: Actually, this is a very good point and this is one of the features that car companies are, of course, working on, you know, entry systems. And some have resorted to, you know, keyed entry systems and others already have electronic, you know, key pads that you can use and what you explained, you know, in terms of swiping your finger can be a possibility. In car systems, you always have to provision also for the fact that these are pretty rugged systems and so the – you know, I don’t foresee this very, very soon but it’s certainly a possibility.
CAVANAUGH: Now what do you mean by rugged systems?
DR. KRUEGER: Oh, in, you know, in a car…
DR. KRUEGER: …has to work in all weather conditions…
CAVANAUGH: Sure, yes.
DR. KRUEGER: …rain and shine.
DR. KRUEGER: And so if you want to bring a device either to the door or, you know, other equipment…
CAVANAUGH: I see.
DR. KRUEGER: …you have to face a challenge that it has to operate under all weather conditions, you know, washing devices and so on and so forth.
CAVANAUGH: It would need extreme sensitivity to…
DR. KRUEGER: Yeah.
CAVANAUGH: …pick up that fingerprint, at the same time it would have to last through rain, sleet and hail and all that.
DR. KRUEGER: Right, exactly.
CAVANAUGH: I see. Are these computer cars, are these systems – how have they changed car repair?
DR. KRUEGER: Oh, dramatically. You know, with the introduction of these many electronic control units, car dealerships and shops had to start becoming, you know, savvy in working with car electronics. And so many problems that, you know, make customers drive into the dealership or to the shop are diagnosed now by onboard diagnostics, and because of the diversity of the various systems it’s sometimes very difficult to pinpoint the problem and so they will actually start exchanging module after module until the behavior that causes the problem goes away. And so oftentimes perfectly fine electronic control units are exchanged just because it’s so difficult to pinpoint the problem.
CAVANAUGH: I see. It’s, in essence, easier to just change out the whole unit…
DR. KRUEGER: Umm-hmm.
CAVANAUGH: …than it is to find the specific problem.
DR. KRUEGER: Right.
CAVANAUGH: Let’s take another phone call. And Andrew is calling us from Talmadge. Good morning, Andrew. Welcome to These Days.
ANDREW (Caller, Talmadge): Good morning. Thanks for my call. I had a question. You’re talking about, you know, these rugged systems on one hand but at the same time I’m so amazed by how complex and how many computer systems are in our cars nowadays.
DR. KRUEGER: Umm-hmm.
ANDREW: I’m trying to understand, is there some kind of – it’s kind of a two-part question. Part one is, is there some kind of operating system that cars – that carmakers use kind of like what’s in our personal computers and is it standard between different carmakers or does each carmaker have their own kind of operating system? And the second part was, you know, whatever they – kind of system they have, it must be pretty stable to withstand all these elements and not kind of crash like a home computer. You know, it seems like the Microsoft could learn something from the operating system in cars or something.
DR. KRUEGER: Umm-hmm.
CAVANAUGH: Thank you for that, Andrew.
DR. KRUEGER: That’s a wonderful question, actually. The first part about the operating systems is that there’s a wide spectrum that you find. You cannot yet speak of a unified operating system for the entire car. You will find very focused and tailored operating systems on many of these electronic control units. Sometimes they are, you know, specifically tailored for one electronic control unit, sometimes they use standardized operating systems for real time jobs and these real time operating systems are then sold as standard operating systems. But actually there are industry initiatives to build an operating system platform that is standardized across a set of cars. It’s not quite like the operating systems that you find on your PCs because it’s tailored to specific functions of the vehicle but it is in the same spirit. And so this is actually where the industry is moving towards and there are industry consortia that are working on this. Your second question was about the stability and, in fact, you will find that most of these electronic control units have two things. One is, they have so-called fail safe modes so even if something in the environment, in the electronic environment, starts to fail, they go into fallback modes where they offer reduced functionality but keeping the safety of the vehicle intact. Sometimes there’s even a mechanical safety mode where, you know, even if, you know, anti-lock braking fails, for instance, you still have the mechanical braking available in the vehicle.
CAVANAUGH: That’s good to know. Let’s take another call. Nancy is calling us from San Diego. Good morning, Nancy, and welcome to…
NANCY (Caller, San Diego): Hi.
CAVANAUGH: …These Days.
NANCY: Good morning.
NANCY: Thank you for taking the call. This is a great topic because my mechanical engineering husband is wondering, and he’s a pilot so he’s gone and he can’t ask the question. I’m going to try and put it into his words. He’s saying that he’s wondering about electronic microbursts coming from a lot of the cell towers or other satellite type things that might be interacting with the computer systems causing some of these speed ups.
DR. KRUEGER: Umm-hmm.
CAVANAUGH: Thank you for that, Nancy.
DR. KRUEGER: That’s a very good question again. These, you know, I mean, there are effects that are challenging to describe but in general the car companies and their suppliers do a very good job at shielding the electronic equipment from external radiation sources. And what you describe as microbursts is only one phenomenon. We are constantly actually exposed to radiation coming from the sun that actually goes into or can go into these electronic control units and even alter the memory contents. And so car companies and the suppliers producing the electronic control units are shielding these against external and internal influences because even in the car, inside of the car, you have radio frequencies that could impact unshielded electronic control units.
CAVANAUGH: Now if I understand correctly, your group, Service-Oriented Software and Systems Engineering Laboratory’s done some consulting work with Toyota.
DR. KRUEGER: It was not even consulting work. We had a research project over three years, actually, together with Toyota Info Technology Center in Palo Alto.
CAVANAUGH: I see. Now, I’m wondering, Nancy referenced the speeding up problems and, of course, what we’re – what she’s referencing is the problem that Toyota has had with this accelerating problem, this – and they say it’s misplaced floor mats and sticky gas pedals.
DR. KRUEGER: Umm-hmm.
CAVANAUGH: From what you know, just from what you know about the electronic systems in cars, do you think Toyota could actually be facing a problem with its electronics?
DR. KRUEGER: I would like to generalize the question…
DR. KRUEGER: …slightly because this is actually, you know, not Toyota specific.
DR. KRUEGER: It’s a question that every car company has to deal with and one that they are actively working towards to solve. And I know Toyota is, I know that, you know, other car companies that we’ve worked with in the past are also. They invest a lot of money also into research, which is, you know, very good and important. And what they need to focus on, in my mind, not just Toyota, all car companies, is to maintain control of the complexity so that next generation cars will not have these problems or, to put it differently, can be designed in a way that problems like this don’t occur. And this is specifically what we do research on, so our research is about bringing different modules that, you know, have to work together into the same system and controlling their interactions so – and then proving, literally proving mathematically, that they will work together correctly.
CAVANAUGH: Some people say that a lot of this electronic software in cars has been rushed to market.
DR. KRUEGER: Umm.
CAVANAUGH: Would you agree with that?
DR. KRUEGER: It’s – That’s a hard claim. It’s very difficult to dissect. You know, I mentioned the number that 90% of all innovations are driven by software in the vehicle today. That’s a number we know. So I do know that some car companies actually held off on introducing new features because they first wanted to get the complexity under control. So, you know, they didn’t certainly rush to market. On the other hand side, these features, the new features like, you know, assisted driving, assisted parking and things like that, these are sales arguments.
DR. KRUEGER: And so people will buy vehicles because they have these fantastic features. And so, you know, the market asks for it and the companies are responding and on average, I mean, the – as I said earlier, the electronics that have been introduced into the car have reduced the number of accidents with fatalities and injuries.
CAVANAUGH: I’m interested, when you’re trying to find a mechanical problem, you know, you can do a crash test or you can do a stress test, how do you find a problem in the software?
DR. KRUEGER: That’s a very deep question. So, you know, there are many things we actively do in our software, whether it goes into the car, and I have colleagues at the computer science department who are working specifically on finding bugs in software. What do you do? You actually – you can test it but you can also work with tools that check every possible state of the system it can be in. Whether it is in a valid state or whether it’s in a state that will lead to a failure. And, you know, there are computer science researchers who are developing these tools to actually do these checks very efficiently. It’s very complex because you have 80 electronic control units who all can interact with one another, so the number of possible states the system can be in is – it explodes and so you have to apply, you know, strong mathematical reasoning to find these errors.
CAVANAUGH: I think we have time for one more call. Let’s speak with Aaron in San Diego. Good morning, Aaron, and welcome to These Days.
AARON (Caller, San Diego): Well, thank you for taking my call. I do have all the respect for all the technology and computer and this and that but (unintelligible) aviation and going to (unintelligible). I have a ’71 Volkswagen Bug. I can go across country back and forth and if accidentally a cable should go out, it’s only two dollars and I can change it on the side of the road. To me, cars are, A, to be transportation. They need to be dependable and basically cheap and good so I can get to my destination. Thank you.
CAVANAUGH: Aaron, thank you for that. So why do we need all these electronics anyway, Dr. Krueger?
DR. KRUEGER: Well, you know, I mean, if you’re happy with a car that does what, you know, you described then, by all means, you know…
DR. KRUEGER: But at the same time, the electronics on the car has reduced the, you know, the environmental impact of cars dramatically, have increased the utilization of the fuel so that, you know, it’s much more efficient to actually, you know, drive a car today with the limited resources we find on this earth. And so if everybody was driving cars made in 1971, we would, you know, have much higher gas consumption.
CAVANAUGH: I wonder, what’s the next thing we should look for in cars? Is there something that’s right on the horizon?
DR. KRUEGER: Well, you know, I mean, there are certain things that have been on the horizon for a while and have been held back by the complexity…
DR. KRUEGER: …and regulations, so one thing is, you know, so-called ‘x by wire’ technologies. It’s a drive-by wire so that you’re no longer – I mean, you’re still interacting with the steering wheel but rather than performing a mechanical task, you’re actually doing an electronic task so that it’s translated to the wheels electronically via the electronic systems in the car. And brake by wire, steer by wire, all of these technologies will be here within a few, you know, within a few years to a decade.
CAVANAUGH: It’s terribly exciting. Thank you so much for speaking with us today. I really appreciate it.
DR. KRUEGER: A great pleasure. Thank you very much for having me.
CAVANAUGH: I’ve been speaking with Dr. Ingolf Krueger, UCSD professor of Computer Science. There are so many people who wanted to join us but we couldn’t take all your phone calls. Please go online, post your comments at KPBS.org/thesedays. Coming up, a chat with the author of the 2010 One Book, One San Diego selection, “Outcasts United.” That’s ahead as These Days continues here on KPBS.