Optical Tech Transforms Data Centers

Despite optical technology having been used in data centers for decades, it is now increasingly penetrating the core of data centers—the processors and accelerators that run data center workloads. So, how did we get to this point? What will the future look like?

Optical communication offers the advantages of high bandwidth, low latency, and low energy loss. Although initially used to facilitate traffic to and from data centers, it was later used within the data centers themselves—typically at the top of the rack (TOR) to create high-speed links between top-of-rack switches. The large data flows used in artificial intelligence, where low latency and high bandwidth interconnects are crucial, will only accelerate the use of optical communication.

As an example of the importance of optics, Google even developed its own optical switch, which uses free-space optical components to reduce the latency and power consumption of its network. By using free-space optics, Google avoids the need to convert signals between the optical and electrical domains at each switch.

Google claims that its custom network can increase throughput by 30%, reduce power consumption by 40%, reduce capital expenditure by 30%, decrease process completion rate by 10%, and reduce network downtime by 50 times. This means that in the event of a node failure or maintenance, a quick switch can be made.

The rise of optical interconnects

Recently, there has been a growing trend in the use of optical interconnects within the rack. Driven by the high bandwidth and low latency requirements of artificial intelligence (since AI models are distributed across dozens of processing nodes), optical interconnects are helping these multi-node systems run as fast as possible. Speed is as crucial as ever.

Optical interconnects are a field full of innovation. Some startups are developing all-optical packet switching—eliminating the need to convert signals between the electrical and optical domains, thereby saving a lot of power and latency.

Other companies are applying optical technology to the next layer, developing all-optical chip-to-chip and even silicon-to-silicon interconnects. Here, the communication bandwidth is even higher. To achieve this advancement, co-packaged optics are essential. Foundries and chip packaging companies are heavily investing in this feature.

From switching to processing

Optics is not only used for switching—it can also be used for processing. It has similar advantages in providing speed, power, and efficiency.

Why use optics for processing? The demand for artificial intelligence processing is growing at an astonishing rate, and the silicon roadmap cannot keep up. Silicon developers are chasing diminishing returns; investing more and more time, effort, and money to achieve marginal performance improvements.

The current industry answer is to increase the size of the silicon chip, use multiple silicon chips within the chip, and use complex packaging to address the challenges this brings. But this comes at a great cost, including money and power consumption.

The huge monetary cost is driven by the substantial capital expenditure required to purchase the latest and best AI accelerators, as well as the increased infrastructure needed to supply and cool these power-hungry devices; not to mention the increased cost of energy consumption.

Optics is very suitable for artificial intelligence, especially because the core of artificial intelligence uses vector-matrix multiplication, which can be very effectively solved in light. Over the past few years, there has been a hope to apply integrated photonics to AI processing, however, due to the limitations of integrated photonics in large-scale matrix processing required by modern AI, the technology is currently mainly focused on interconnect or switching applications.

Using 3D (free-space) optical devices for processing overcomes these challenges—it uses only a fraction of the power but achieves a leap in performance. Using 3D optical devices means that very wide vectors can be used, maximizing performance and energy efficiency. Compared with today's GPU solutions, AI accelerators using 3D optical devices are expected to provide low-power, high-performance solutions at very low capital and operating costs.

As data centers increasingly use optical devices, 3D optical devices for AI processing will inevitably be the next step.