Synopsys Cloud
Cloud native EDA tools & pre-optimized hardware platforms
Request a Free Trial →
Table of Contents
This article was originally published on EE Times.
What have you asked ChatGPT to do lately? Chances are, if you haven’t tried the AI chatbot yourself, you know someone who has. From writing essays and code to explaining complex concepts, ChatGPT is blowing minds around the world with its speed and human-sounding prose. It’s also another example of how AI is becoming more accessible and pervasive in our smart everything world.
As compute-intensive applications like AI and machine learning (ML) continue to become more ingrained in our lives, it’s worth considering the underlying infrastructure that makes these innovations possible. Simply put, these applications are demanding a heavy load from the hardware that processes the algorithms, runs the models, and keeps the data flowing where it needs to go.
Hyperscale data centers with very high-performing compute resources have emerged to tackle the workloads of AI, high-performance computing, big data analytics, and the like. However, it is becoming increasingly clear that the traditional copper interconnects that bring together different components inside these data centers are hitting a limit in delivering the bandwidth demanded. This is where photonic integrated circuits (PICs), which use the power of light, can play a pivotal role.
In this article, we’ll explore how photonics can provide an avenue not only for a higher level of performance but also greater energy efficiency as well as support for miniaturization. Given the power impact of today’s computing workloads, it’s imperative for the industry to seek ways to minimize its power footprint as it creates the next ChatGPT.
The Power Price of Executing AI Models
The authenticity of ChatGPT lies in its use of the Generative Pre-trained Transformer 3 (GPT-3) autoregressive language model from OpenAI, which uses deep learning to produce text. At 175 billion parameters, the architecture for this model surpasses the level of processing exhibited by the human brain (which is equivalent to a model of 100 billion parameters).
AI models such as these are putting huge demands on hardware components such as memory, GPUs, CPUs, and accelerators that process them. A hardware foundation comprised of large arrays of GPUs and high-bandwidth optical connectivity is needed to execute AI models, but all of this comes with some serious power (and cost) considerations.
Hyperscale data centers—which typically feature at least 5,000 servers managing petabytes or more of data in 100,000s of square feet of space—provide efficiency in their ability to quickly process voluminous amounts of data. However, this capacity and capability comes with a huge power penalty: data center energy use was in the 220-to-330 Terawatt-hours (TWh) range in 2021, representing roughly 0.9% to 1.3% of global final electricity demand, according to the International Energy Agency. To put things into perspective, this is more energy than some countries consume in one year.
In many data centers, the hardware components are typically connected via copper interconnects, while the connections between the racks in the centers often use optical fiber. The trend to use optical connections for increasingly shorter distances is now reaching a point where optical I/Os for core silicon such as switches, CPUs, GPUs, and die-to-die interconnects using photonics are quickly emerging as an inevitable solution for next-generation data centers. By using the properties of light, photonic ICs can enable, extend, and increase data transmission. From a physics standpoint, nothing else can do what photonics can do to increase bandwidth and speed while also reducing latency and power consumption. This is just what data centers—and the AI chatbots that rely on them—need.
“As we drive discovery of the most optimal AI and quantum systems for an array of industries, including healthcare, finance, and industrial, we experience the real benefits of using photonics for a substantial uplift in bandwidth and speed,” said Dr Nagendra Nagaraja, founder and CEO of QpiAI. “Synopsys photonics solutions enable us to infuse our technologies with the speed of light to help our customers enhance their business outcomes.”
Photonics: The Optical Highway for Disaggregated Data Centers
We are already seeing a shift in data center architectures toward disaggregation, where homogeneous resources such as storage, compute, and networking are separated in different boxes and connected optically. This type of architecture wastes no resources; instead, a central intelligence unit determines and takes what’s needed from each of the boxes, with the data traversing optical interconnects. Remaining resources can then be used for other workloads.
Aside from their use in rack-to-rack, room-to-room, and building-to-building configurations, optical interconnects could also become predominant at the CPU and GPU levels and take care of the data I/O using optical signals. The desire to replace many parallel and high-speed serial electrical I/O lanes with optical high-bandwidth connections is driving the need for near-packaged optics, which use a high-performance PCB substrate (an interposer) on the host board, and co-packaged optics, a single package integration of electrical and photonic dies. While photonics likely won’t replace traditional electronic semiconductor components in the short term, they clearly have a place at the table to address the bandwidth and latency requirements. Meanwhile, research is underway to understand the value of analog and digital computing in the optical domain.
Photonics Break Through Bandwidth Barriers
The PIC market is anticipated to grow with a CAGR of 21.5% between 2022 and 2032, according to Future Market Insights. With their high-speed data transfer and low energy consumption, it’s obvious that PICs offer a way to break through bandwidth barriers while minimizing energy impacts.
Companies like LightMatter, with its photonic AI computing platform, and Ayar Labs, which develops optical interconnect chiplets, are among those who are at the forefront of developing new technologies to address bandwidth demands while reducing environmental impact. In addition, several companies are devising analog and digital compute solutions using photons instead of electrons to function as the arithmetic core. Indeed, the field of photonics is ripe for continued innovation.
However, PIC design is not as straightforward as designing traditional ICs. The performance of these circuits is tied to the material as well as optical properties that are, in turn, tied to geometrical shapes. Achieving success in this realm calls for knowledge of the latest research, tools, and techniques, along with a depth of understanding of quantum and physical optics.
Synopsys, which provides the industry’s only seamless design flow for photonic devices, systems, and ICs, can deliver a path toward photonic design success. We also collaborate closely with major foundries on process design kits (PDKs) that streamline PIC development, and we work with governmental organizations on photonic education initiatives. Synopsys also invests in the technology, as exemplified by our collaboration with Juniper Networks in creating OpenLight, which introduced the world’s first open silicon photonics platform with integrated lasers.
From ChatGPT to the IoT and beyond, bandwidth-hungry applications are driving the need for higher speed data transfer. Photonic-based chips answer the call, taking advantage of the speed of light to deliver greater bandwidth, lower latency, and lower power. With continued development of photonic ICs, the future is indeed bright for the electronics industry.