Debugging Complex Embedded Systems

Figure 1: A typical embedded system has many devices
utilizing a wide range of signal types and bandwidths.

In the next few weeks, we’ll present a series of posts describing the debugging of various components of embedded systems, everything from working out the timing of power on sequences to extracting analog sensor data from serial data streams. All these examples of “real world” debugging can be performed using only standard tools available on most mid-range Teledyne LeCroy oscilloscopes.

Let's start by defining what we mean by embedded system and deeply embedded system. For our purposes, an embedded system is a fixed function, self-contained control system on one printed circuit board. Typically, printed circuit boards (PCBs)  will have multiple passive devices, a few active electronic devices, analog devices, digital devices and a few serial data devices. There tends to be a microcontroller device to process data and control other components. There will also be some type of system memory, often embedded inside the microcontroller. And, there will be some power conversion devices and power distribution elements that power all the other devices in your embedded system. 

Measuring Dead Time in 48 V Power Conversion Systems, Part 1: Static Measurements

48 volt power conversion systems input a 48 VDC power bus and output other voltage levels. The key section in the conversion process is the inverter stage. The inverter subsection topology may be half bridge, full or H-bridge, or cascaded H-bridge for 3-phase systems. There is typically a filtering circuit after the inverter before power is passed on to the Load, and a control system that takes care of controlling the entire conversion system. 

Figure 1:Three common topologies for switching inverter stages. The load is connected between upper and lower power switching devices, which are selectively switched on and off to supply power to the load from the DC bus.

The three inverter topologies shown in Figure 1 are all similar in that they use stacked or “totem pole” power devices for switching. 

In operation, each of these topologies provides a path from the DC bus through the load and then to ground (0 V) by turning on selected devices. At no time should there be a direct path from the DC bus to ground, an event called “shoot through.” Designers intentionally add dead time to eliminate the risk of shoot through.

Debugging Dynamic Link Behaviors with CrossSync PHY for PCIe

Figure 1: These upstream lanes show unexpected equalization
behavior. If you only had the electrical signals to work with,
how would you begin debugging them?

Recent generations of PCI Express® rely heavily on the link equalization process to support signal integrity at bit rates of 8, 16 or 32 Gbps. In Phase 1, both sides of the link exchange TS1 ordered sets to establish an operational link. In Phase 2, the upstream port requests the downstream port to configure its transmitter equalization to compensate for the link channel. In Phase 3, the opposite happens, and the downstream port requests the upstream port to configure its transmitter equalization. 

At PCI Express compliance events, or when doing pre-compliance testing in the lab, a transmitter link equalization test is performed to determine whether a device is capable of correct link equalization in isolation. A piece of test equipment, usually a protocol-aware BERT, acts as the link partner for the device under test. The BERT requests specific preset changes from the device, in response to which the device (in theory) changes its preset to provide the correct channel compensation. The changes are captured by an oscilloscope, which is capable of visualizing the transmitter equalization changes in the electrical layer and measuring first of all, if they happen quickly enough, and secondly, if the device actually changed to the preset levels requested, which occur in a known sequence. 

But what happens when you suspect "weird" equalization behavior in a live link between two devices—for example, something off with the upstream equalization in Phase 3?  How would you capture that to begin debugging the problem? Where would you look for a clue as to what is happening?

Debugging L1 Substates Timing Errors with CrossSync PHY for PCIe

Figure 1: CrossSync PHY for PCIe lets you easily map the
electrical to the protocol layer of L1 substate events.

Debugging the L1 substates used for PCIe® power management has been a real pain point for engineers, especially those using the M.2 form factor. The L1.1 and L1.2 substates allow you to take a PCIe link to a deeper power savings state than L1 alone. L1 is entered by either of two mechanisms, an ASPM message or a Power Management message, but in both cases the net effect is that the link goes into electrical idle. Unlike conventional L1, a clock request (CLKREQ#) signal is used to enter and exit the L1 substates. Upon entry, the device or host deasserts the clock request line. By saying, “I don’t require a clock anymore,” the clock can be turned off for additional power savings. Upon exiting the L1 substates, the CLKREQ# is asserted and the reference clock resumes.