Debugging Complex Embedded Systems

Figure 1: A typical embedded system has many devices
utilizing a wide range of signal types and bandwidths.

In the next few weeks, we’ll present a series of posts describing the debugging of various components of embedded systems, everything from working out the timing of power on sequences to extracting analog sensor data from serial data streams. All these examples of “real world” debugging can be performed using only standard tools available on most mid-range Teledyne LeCroy oscilloscopes.

Let's start by defining what we mean by embedded system and deeply embedded system. For our purposes, an embedded system is a fixed function, self-contained control system on one printed circuit board. Typically, printed circuit boards (PCBs)  will have multiple passive devices, a few active electronic devices, analog devices, digital devices and a few serial data devices. There tends to be a microcontroller device to process data and control other components. There will also be some type of system memory, often embedded inside the microcontroller. And, there will be some power conversion devices and power distribution elements that power all the other devices in your embedded system. 

An embedded computing system is when the primary processor/controller is a higher speed, higher cost microprocessor, instead of a low-speed microcontroller. Whereas microcontrollers tend to have memory embedded inside them, microprocessors will typically have memory interfaced outside. 

A deeply embedded system, or deeply embedded computing system, is essentially either of the systems defined above, but one that additionally detects something from the environment, processes it and does something with the result. Because of the need to respond to environmental cues, it will contain some type of sensor devices, which are typically analog sensors. PWM controllers of subsystems are commonplace in deeply embedded systems. 

The complexity of all such systems can vary from low to high, as can the size, which directly links to the cost of the embedded system. A fairly simple embedded system would be a traditional washing machine control system. It's a fixed function, it needs to do only one thing, based on the input you give it. On the other hand, an in-vehicle ADAS system with multiple sensors feeding data to a high-speed microprocessor can be considered a complex deeply embedded computing system: it has to make critical decisions based on all the input that it gets, and actually drive the car with it. (Modern washing machines would probably be closer to this description!)

Figure 1 shows a block diagram of a typical deeply embedded computing system (click on the image to expand it). There are digital devices utilizing various signals, such as DDR memory, EEPROM, flash memories, BIOSs. There are interrupts that feed into the microprocessor, carrying general purpose I/O signals. There are low-voltage 12 V, 5 V, 3.3 V, etc. power rails that power the various components. And controlling everything is the microprocessor, which is connected to the other devices using off-board, low-speed serial data such as UART, USB 2.0, I2C and SPI.

Throughout this series, all the tools that we’ll talk about apply equally to both microcontrollers and microprocessors, embedded systems and deeply embedded systems. For purposes of debugging, there is no difference between the two. Typically, any embedded system requires some sort of system validation in the design phase. And during system validation, you're going to debug cause-effect problems that arise from the interactions of the many different signals that pass between various devices. 

Because of the mixture of signal types found in complex embedded systems, the instrument you use to debug them needs a range of capabilities that make it possible to acquire and view signals of very different types and bandwidths simultaneously, because time correlation is the key to debugging. If you cannot confidently relate very different events to each other, you cannot find the root cause of problems. Obviously, mixed-signal capability is a must, but so is high resolution, low noise, long capture times, and high sample rates. As we address different debugging tasks in the next posts, it will become apparent why any instrument used to debug complex embedded systems must have all these.

Watch William Kaunds explain many facets of debugging embedded systems in the on-demand webinar, Debugging Complex Embedded Computing System Issues (Part 2).