Optical interconnects refers to any system of transmitting signals from one part of an integrated circuit to another using light.

Along with getting faster and spanning greater distances, interconnects will dramatically increase in number. Chat GPT3 was trained on a cluster with approximately 1,000 accelerators that required 2,000 optical interconnects. Chat GPT4 was trained on a 25,000-accelerator cluster with about 75,000 optical interconnects.

“100K clusters will be available soon and that may require five layers of switching and 500,000 optical interconnects. People are talking about a million (accelerator) clusters, I can’t even imagine that, but those are the kind of numbers that people are talking about today and that may require millions of optical interconnects in a single AI cluster,” Nguyen added. “These numbers should only be used as a guide…(but) no matter how you look at it, the optical interconnects will grow faster than the accelerator growth in an AI cluster.”

https://www.marvell.com/blogs/scaling-ai-means-scaling-interconnects.html

Broadcom leads the industry in three core optical interconnect technologies essential to building large-scale AI networks.

AI server clusters are large and are only going to get larger. Optical links provide the connections needed to allow these clusters to scale at distance. Optical technology can support that growth in terms of both scale and cost.

The Core Optical Interconnects for AI

There are three core optical interconnects:

• Vertical-Cavity Surface-Emitting Lasers, or VCSELs, are a workhorse for optical AI interconnect technology across the industry. Its low power and low cost make it ideal for data communications and sensing applications. The only limitation is that it operates best at shorter link distances.
• Electro-absorption Modulated Lasers, or EMLs, are ideal for AI systems that scale to greater distances and hundreds of thousands, or even millions of units. This technology provides better performance at very high bandwidth and typically are first to reach volume deployments at next generation data rates.
• Co-Packaged Optics, or CPO, is an advanced heterogeneous integration of high-speed silicon photonics onto application-specific integrated circuits for addressing next-generation bandwidth and power challenges. We see this new technology providing the power and cost leadership for future generations of AI systems and enabling infrastructures supporting large-scale AI networks.
• https://www.broadcom.com/blog/optical-interconnects-enabling-large-scale-ai-clusters