Synopsys Cloud
Cloud native EDA tools & pre-optimized hardware platforms
Request a Free Trial →
Choosing the Right IP for Die-to-Die Connectivity
Manuel Mota, Sr. Product Marketing Manager, Staff, Synopsys
System-on-Chip (SoC) designers for hyperscale data centers, artificial intelligence (AI), and networking applications are facing a challenge that has been trending since the advent of big data. Because of workload demands and the need to move data faster, such SoCs have become more complex with advanced functionalities and reached maximum reticle sizes. For this reason, designers are partitioning SoCs in smaller modules in a multi-chip module (MCM) packaging. These disaggregated chips require ultra- and extra-short reach links to enable inter-die connectivity with high data rates. In addition to bandwidth, the die-to-die connectivity must ensure reliable links with extremely low latency and power efficiency. This article describes several different use cases for die-to-die connectivity and essential considerations to make when searching for high-speed PHY IP for die-to-die connectivity.
Die-to-Die Connectivity Use Cases
New use cases are emerging for die-to-die connectivity in MCMs, some of which include:
1. High-performance computing and server SoCs approaching maximum reticle sizes
2. Ethernet switches and networking SoCs exceeding maximum reticle sizes
3. Artificial intelligence (AI) SoCs with distributed SRAMs to scale complex algorithms
High-performance computing and server SoCs are becoming large in size, reaching 550 squared millimeters (mm2) to 800 mm2, diminishing the SoC yield and increasing the cost per die. A better approach to optimizing SoC yield is to split the SoC into two or multiple equivalent homogenous dies, as shown in Figure 1, and connecting the dies using a die-to-die PHY IP. The key requirements in such a use case are extremely low latency and zero bit-error rate since the smaller dies must depict and behave as a single die.
Figure 1: Example of a high-performance computing and server SoC requiring die-to-die connectivity
Ethernet switch SoCs are at the core of the data center and must move data at faster rates from 12 Tbps to 25 Tbps which require 256 lanes of 100G SerDes interfaces, making it impossible to fit the SoC in an 800 mm2 reticle area. To overcome such a challenge, designers are splitting the SoC into a configuration where the core die is surrounded by I/O dies, as seen in Figure 2. The core die is then connected to the I/O dies using a die-to-die transceiver.
In this use case, die splitting is only effective if the bandwidth density by the die-to-die transceiver is much superior to the long reach SerDes in the I/O die. The key parameter is therefore the die-edge bandwidth density per millimeter.
Figure 2: Example of an Ethernet switch SoC requiring die-to-die connectivity
In an AI SoC, each die incorporates intelligence processing units (IPUs) and distributed SRAMs localized in the vicinity of each IPU. In such use case, the IPU in one die may need to access data located in a SRAM in another die relying on an extremely low-latency short-reach die-to-die link.
Figure 3: Example of an AI SoC requiring die-to-die connectivity
In all of these use cases, the ideal high-speed PHY for die-to-die connectivity can simplify the MCM packaging requirements. Due to the high throughput per lane, up to 112 Gbps, very high aggregate throughputs can be achieved with a relatively limited number of lanes. In this case, the package trace pitch and stack can be conservative (L/S of 10u/10u is common). In these use cases, traditional, low cost, organic substrate-based packaging can also be used.
High-Speed PHY IP Requirements for Die-to-Die Connectivity
The Optical Internetworking Forum (OIF) is defining electrical I/O standards for transmission with data rates up to 112 Gbps over ultra-short reach (USR) and extra-short reach (XSR) links. These specifications define die-to-die links (i.e: within package) and die-to-die to optical engine that sits in the same package as the SoC) with significantly reduced power and complexity as well as very high throughput density.
When researching high-speed PHY IP solutions for die-to-die connectivity in MCMs, SoC designers must take several essential features into consideration, including data throughput or bandwidth measured in Giga- or Terabits per second (Gbps or Tbps), energy efficiency measured in picojoules per bit (pJ/bit), latency measured in nanoseconds (ns), maximum link reach measured in millimeter (mm), and bit error rate (unit-less).
Data Throughput or Bandwidth
To enable interoperability with other transceivers, the die-to-die PHY IP must ensure compliance with the relevant OIF electrical specifications for USR and XSR links. Supporting Pulse Amplitude Modulation (PAM-4) and Non-Return to Zero (NRZ) signaling is essential to meet the requirements of both types of links and achieve the maximum 112 Gbps bandwidth per lane. Such signaling leads to very high-bandwidth efficiency, which is a critical requirement due to the very large amount of data traveling between the dies in the MCM. The data movement is often in the range of Terabits per second, which imposes limits to the size of the chip edge (beach front) allocated to USR and XSR links.
However, equally important is support for a wide range of data rates. Often, it is desirable to implement the die-to-die link assuming a data rate that is aligned with the data rate used in the internal fabric or supports all the data rates required by the chip-to-chip protocol. For example, PCI Express, even at high speeds such as 32 Gbps, must support data rates down to 2.5 Gbps for protocol initialization.
Link Reach
In a die-to-die implementation, large amounts of data must travel through short data paths across the gap between dies. In order to guarantee the maximum flexibility of placement of the dies over the package substrate, it is necessary for the PHY IP to support distances between TX and RX of up to 50 millimeters.
Energy Efficiency
Energy efficiency becomes an important factor especially in use cases where the SoC functionality is split into several homogeneous dies. In such cases, designers look for ways to push large amounts of data between the dies without impacting the SoC’s total power budget. An ideal die-to-die PHY IP offers energy efficiency better than one picojoule per bit (1pJ/bit), or equivalently, 1mW/Gbps.
Latency and BER
In order to make the connection between dies “transparent”, the latency must be extremely low, while the bit-error rate (BER) must be optimized. Due to the simplified architecture, the die-to-die PHY IP achieves ultra-low latency by itself, with a BER better than 10e-15. Depending on the link reach, the interconnect may need to be protected with a Forward Error Correction (FEC) mechanism to achieve such low BERs. The FEC latency impacts the overall latency of the solution.
Macro Placement
In addition to these performance related parameters, the PHY IP must support placement in all sides of the die to enable efficient floorplanning of the die as well as the MCM. An optimized placement of the macros enables efficient inter-die routing with low coupling, optimizing the die and MCM area and ultimately improving power efficiency.
There are many other considerations when selecting a die-to-die PHY IP, including the inclusion of testability features to enable production test of the dies before packaging, but the above are the most essential.
Conclusion
Higher data rates and more complex functionalities are increasing the SoC size for hyperscale data center, AI, and networking applications. As SoC sizes are approaching the reticle size, designers are forced to split the SoC into smaller dies which are packaged in multi-chip modules (MCMs) to achieve high production yield and reduce overall cost. The smaller dies in an MCM are then linked via die-to-die interconnects featuring very low-power consumption and high bandwidth per die-edge. In high-performance computing and AI applications, large SoCs are split into two or multiple homogenous dies, and in networking applications I/Os and networking cores are split into separate dies. The die-to-die interconnect in such SoCs must not impact overall system performance and demand low-latency, power-efficiency, and high throughput. These requirements are driving the need for high-throughput die-to-die PHYs such as Synopsys’ DesignWare® USR/XSR PHY IP which enables die-to-die connectivity in MCM designs for data rates up to 112 Gbps per lane with very high energy efficiency. The DesignWare USR/XSR PHY IP is compliant with the OIF CEI-112G and CEI-56G standards for ultra-short reach (USR) and extra-short reach (XSR) links.