Get Ready for PCIe 6.0 Base Tx Testing--Compliance, Jitter and Eye Diagrams

Figure 1. The four levels and three eyes of the
PCIe 6.0 PAM4 signal.
Click on any image to enlarge.

The PCI Express® 6.0 Base specification was officially released in January 2022 at version 1.0, meaning it is considered final. The CEM and PHY test specifications are currently at version 0.3, and 0.5 versions of both will probably be released sometime in the third or fourth quarter of 2022. As both the CEM and PHY test specifications are still at an early stage, it's hard to predict exactly when the PCIe® 6.0 compliance test program will start, but it's safe to say it is at least a few years out. However, PCIe 6.0 Base testing is upon us, especially those of us working on chip design.

So, what can implementers expect as they retool for PCIe 6.0 Base Tx and Base Rx physical layer testing?

PCI Express 4.0 Error Detection Using a BERT and Oscilloscope

Figure 1: BERT and Oscilloscope connection
for PCI Express 4.0 error detection.

The PCI Express® 4.0 standard specification requires an oscilloscope with at least 25 GHz analog bandwidth and a BERT which can test bit rates of at least 16 Gbps. The BERT provides a known input pattern to the PCIe® device under test (DUT), and the DUT is instructed to regenerate the identical bit pattern while placed in loopback mode. Since a BERT can output a signal when a bit error is detected, this signal can be input to the oscilloscope to trigger a synchronized capture when an error occurs. By combining the capabilities of these two instruments, a powerful combination of real-time error detection and characterization can emerge.

Connecting the Instruments

In Figure 1, a known pattern is connected from the BERT PPG D1 output to the PCIe DUT input via a 2.92 mm K-type cable. The DUT attempts to regenerate the same data pattern, while the DUT output is routed to both the BERT error detector input and the oscilloscope channel 1 via 2.92 mm K-type tables and a power splitter. An error-free reference signal is connected between the BERT D2 output to the oscilloscope channel 2, along with the error trigger signal from the error detector output to the oscilloscope channel 3. An oscilloscope Edge trigger is set on the rising edge of the error detector output signal on C3.

Correlating Low to High-Speed Events in Complex Embedded Systems

Figure 1: A challenge when testing embedded systems
is to correlate events in a low-speed interface like SPI
to events in a high-speed interface like PCIe.

A common requirement when testing embedded systems is to measure the timing between signals with low data rates and those with high data rates. Looking at the functional block diagram of our typical deeply embedded system in Figure 1, we see low-speed serial interfaces like SPI and I2C along with high-speed serial links, like PCIe (often serving as the high-speed serial PHY in our diagram).

Take for example testing the initialization of the system. When power is first turned on, the ROM bios and flash memory initialize program elements that are required by the embedded system’s microprocessor. Once the initialization is complete, the microprocessor has to notify the motherboard via PCIe that it is active and ready to receive data via the high-speed serial bus. This all has to happen within 200 milliseconds. 

Debugging Dynamic Link Behaviors with CrossSync PHY for PCIe

Figure 1: These upstream lanes show unexpected equalization
behavior. If you only had the electrical signals to work with,
how would you begin debugging them?

Recent generations of PCI Express® rely heavily on the link equalization process to support signal integrity at bit rates of 8, 16 or 32 Gbps. In Phase 1, both sides of the link exchange TS1 ordered sets to establish an operational link. In Phase 2, the upstream port requests the downstream port to configure its transmitter equalization to compensate for the link channel. In Phase 3, the opposite happens, and the downstream port requests the upstream port to configure its transmitter equalization. 

At PCI Express compliance events, or when doing pre-compliance testing in the lab, a transmitter link equalization test is performed to determine whether a device is capable of correct link equalization in isolation. A piece of test equipment, usually a protocol-aware BERT, acts as the link partner for the device under test. The BERT requests specific preset changes from the device, in response to which the device (in theory) changes its preset to provide the correct channel compensation. The changes are captured by an oscilloscope, which is capable of visualizing the transmitter equalization changes in the electrical layer and measuring first of all, if they happen quickly enough, and secondly, if the device actually changed to the preset levels requested, which occur in a known sequence. 

But what happens when you suspect "weird" equalization behavior in a live link between two devices—for example, something off with the upstream equalization in Phase 3?  How would you capture that to begin debugging the problem? Where would you look for a clue as to what is happening?

Debugging L1 Substates Timing Errors with CrossSync PHY for PCIe

Figure 1: CrossSync PHY for PCIe lets you easily map the
electrical to the protocol layer of L1 substate events.

Debugging the L1 substates used for PCIe® power management has been a real pain point for engineers, especially those using the M.2 form factor. The L1.1 and L1.2 substates allow you to take a PCIe link to a deeper power savings state than L1 alone. L1 is entered by either of two mechanisms, an ASPM message or a Power Management message, but in both cases the net effect is that the link goes into electrical idle. Unlike conventional L1, a clock request (CLKREQ#) signal is used to enter and exit the L1 substates. Upon entry, the device or host deasserts the clock request line. By saying, “I don’t require a clock anymore,” the clock can be turned off for additional power savings. Upon exiting the L1 substates, the CLKREQ# is asserted and the reference clock resumes.

Anatomy of a PCIe Link

Figure 1: A PCIe link between root complex and end point.
Each device has its own transmitter and receiver.

Peripheral Component Interconnect Express (PCIe®) is a high performance, general-purpose input/output (I/O) interconnect designed for a wide variety of computing and communication platforms. Recent iterations of the standard take advantage of high-speed serial technology, point-to-point interconnects, switch-based technology, and packetized protocol. They rely heavily on link negotiation and link training, so much so that being able to capture and view dynamic link behaviors is essential to debugging PCIe devices. Nevertheless, it remains a pain point for PCIe engineers. Why is it so difficult? To start, let’s break down the PCIe architecture to understand what is going on.

The Important Difference Between ProtoSync™ and CrossSync™ PHY

Figure 1: CrossSync PHY captures everything from
physical through protocol layers at once.

With the recent release of our new CrossSync™ PHY for PCI Express® product, some of you may be wondering how it’s any different than ProtoSync™ for PCIe®, which has been around for quite a few years.

ProtoSync is an option for Teledyne LeCroy oscilloscopes with bandwidths that support high-speed serial data analysis. We’ve released ProtoSync options for PCIe, USB, SAS/SATA and Fibre Channel. ProtoSync links the same Protocol Analysis Suite software that is used with our protocol analyzers to the oscilloscope application, so that you can see physical layer decodings in the familiar PETracer and BITracer views right next to the decoded analog waveform. 

CrossSync PHY differs from ProtoSync in the three, significant ways:

PCIe Electrical Testing: Where Are We?

Figure 1. PCIe specifications through Rev 5.0.

You may be wondering where we are in the roll out of PCI Express test specifications and active testing. Figure 1 shows the status as of March, 2020.

Introduction to Debugging High-Speed Serial Links

Figure 1: These images depict the degradation of serial data
traffic as it makes its way from transmitter to receiver

In recent years, the data rates in serial links have increased exponentially across any number of standard protocols, including PCI Express, USB, and even SATA and SAS. With higher data rates comes more challenges for system designers, validation engineers, and test engineers with respect to signal integrity (SI). Some SI effects are much more prominent at higher data rates than they were for lower-speed versions of the same protocols. In this series of posts, we'll examine these SI effects, look at some methods of improving system performance, and discuss some SI analysis solutions as well as measurement considerations.

Debugging Ethernet, SATA, and PCIe for IoT Devices

Figure 1: A generic IoT block diagram
shows serial-data links in blue

In our ongoing review of debugging serial-data standards for Internet of Things (IoT) devices, let's now turn to three more popular protocols: Ethernet, SATA, and PCIe. Ethernet is found in computer networking applications, while the Serial Advanced Technology Attachment (SATA) connects host bus adapters to mass-storage devices. The Peripheral Component Interconnect Express (PCI Express or PCIe) handles communication between root complexes (motherboards) and expansion-card interfaces.

An Under-The-Hood View of PCIe 3.0 Link Training (Part II)

Figure 1: A diagrammatic view of the
PCIe 3.0 dynamic link training process

Our last post in this series began examining the recovery.equalization process of PCIe 3.0 dynamic link training, beginning with Phases 0 and 1 of the process (Figure 1). Next, we will move on to take a closer look at Phases 2 and 3, where we'll see what can happen with devices in which the algorithms are not up to par. Namely, issues such as packet errors, dropped packets, and link retraining at lower data rates than 8 Gb/s.

PCIe 4.0 Transmitter Electrical Testing (Part II)

Figure 1: With an add-in card as our DUT, we will measure
the transmit signal at the root complex on the system board

With PCIe 4.0 compliance workshops close at hand, let's get familiar with the compliance test process. We've set the stage for electrical transmitter tests by describing the PCIe 4.0 nominal channel and also reviewed the test-equipment requirements; now we'll begin examining the tests in some detail. The two basic transmitter tests are the preset test and signal-quality test.

PCIe 4.0 Transmitter Electrical Testing (Part I)

Figure 1: The two basic PCIe 4.0 transmitter tests
are shown above outlined in green

You've been introduced to some of the background and history that has brought the PCI Express protocol standard to its fourth generation, and we've discussed the test-equipment requirements for PCIe 4.0 electrical compliance testing. Let's begin examining the compliance testing, beginning with transmitter electrical tests.

The Periodic Table of Oscilloscope Tools: Analyze (Part III)

We're nearing the end of our tour of the Periodic Table of Oscilloscope Tools, our way of presenting our broad palette of oscilloscope tools in a concise, clear fashion. In this installment, we'll finish up the Analyze grouping, by far the largest on the Periodic Table.

An Under-the-Hood View of PCIe 3.0 Link Training (Part I)

Figure 1: An overview of the elements of PCIe 3.0
dynamic link equalization

Now that we've looked at the basics of PCIe 3.0 dynamic link equalization and at some of the particulars of de-emphasis and preshoot, it's time to dive a little deeper into what actually happens in the link training process. It all happens in the blink of an eye but there's enough going on to warrant some dissection.

PCIe 3.0 Dynamic Link EQ: De-Emphasis, Preshoot, Cursors, and Presets

Figure 1: De-emphasis, a key transmit-side equalization
technique for PCIe 3.0, boosts high-frequency content

In an earlier post, we looked at some of the basics of dynamic link equalization for PCIe 3.0, and in particular the reasons why it's not only necessary but mandated by the PCI-SIG for compliance testing. Essentially, the boost in data rates from 5 GT/s in PCIe 2.0 to 8 GT/s in PCIe 3.0 wreaked havoc in terms of signal integrity in the channel. The solution is found in equalization both before (TxEQ) and after (RxEQ) the channel.

The Hows and Whys of PCIe 3.0 Dynamic Link Equalization

Figure 1: SI problems are the root cause for
dynamic link equalization in PCIe 3.0

If you're designing a computer peripheral these days, chances are that you'll use the Peripheral Component Interconnect Express (PCIe) protocol for communication between the device and the host system. With the emergence of PCIe, a bunch of older bus standards were kicked to the curb. PCIe itself became the basis for more specialized standards, most notably ExpressCard for laptop expansion cards and SATA Express for storage interfaces.