Computing is at an inflection point. Moore’s Law, which predicts that the variety of transistors on an electronic chip will double annually, is slowing down on account of the physical limits of fitting more transistors on inexpensive microchips. These increases in computer power are slowing down because the demand grows for high-performance computers that may support increasingly complex artificial intelligence models. This inconvenience has led engineers to explore recent methods for expanding the computational capabilities of their machines, but an answer stays unclear.
Photonic computing is one potential treatment for the growing computational demands of machine-learning models. As an alternative of using transistors and wires, these systems utilize photons (microscopic light particles) to perform computation operations within the analog domain. Lasers produce these small bundles of energy, which move on the speed of sunshine like a spaceship flying at warp speed in a science fiction movie. When photonic computing cores are added to programmable accelerators like a network interface card (NIC, and its augmented counterpart, SmartNICs), the resulting hardware may be plugged in to turbocharge a regular computer.
MIT researchers have now harnessed the potential of photonics to speed up modern computing by demonstrating its capabilities in machine learning. Dubbed “Lightning,” their photonic-electronic reconfigurable SmartNIC helps deep neural networks — machine-learning models that imitate how brains process information — to finish inference tasks like image recognition and language generation in chatbots equivalent to ChatGPT. The prototype’s novel design enables impressive speeds, creating the primary photonic computing system to serve real-time machine-learning inference requests.
Despite its potential, a serious challenge in implementing photonic computing devices is that they’re passive, meaning they lack the memory or instructions to manage dataflows, unlike their electronic counterparts. Previous photonic computing systems faced this bottleneck, but Lightning removes this obstacle to make sure data movement between electronic and photonic components runs easily.
“Photonic computing has shown significant benefits in accelerating bulky linear computation tasks like matrix multiplication, while it needs electronics to deal with the remaining: memory access, nonlinear computations, and conditional logics. This creates a major amount of knowledge to be exchanged between photonics and electronics to finish real-world computing tasks, like a machine learning inference request,” says Zhizhen Zhong, a postdoc within the group of MIT Associate Professor Manya Ghobadi on the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “Controlling this dataflow between photonics and electronics was the Achilles’ heel of past state-of-the-art photonic computing works. Even when you could have a super-fast photonic computer, you wish enough data to power it without stalls. Otherwise, you’ve got a supercomputer just running idle without making any reasonable computation.”
Ghobadi, an associate professor at MIT’s Department of Electrical Engineering and Computer Science (EECS) and a CSAIL member, and her group colleagues are the primary to discover and solve this issue. To perform this feat, they combined the speed of photonics and the dataflow control capabilities of electronic computers.
Before Lightning, photonic and electronic computing schemes operated independently, speaking different languages. The team’s hybrid system tracks the required computation operations on the datapath using a reconfigurable count-action abstraction, which connects photonics to the electronic components of a pc. This programming abstraction functions as a unified language between the 2, controlling access to the dataflows passing through. Information carried by electrons is translated into light in the shape of photons, which work at light speed to help with completing an inference task. Then, the photons are converted back to electrons to relay the knowledge to the pc.
By seamlessly connecting photonics to electronics, the novel count-action abstraction makes Lightning’s rapid real-time computing frequency possible. Previous attempts used a stop-and-go approach, meaning data could be impeded by a much slower control software that made all the choices about its movements. “Constructing a photonic computing system with no count-action programming abstraction is like attempting to steer a Lamborghini without knowing how you can drive,” says Ghobadi, who’s a senior creator of the paper. “What would you do? You almost certainly have a driving manual in a single hand, then press the clutch, then check the manual, then let go of the brake, then check the manual, and so forth. It is a stop-and-go operation because, for each decision, you could have to seek the advice of some higher-level entity to inform you what to do. But that is not how we drive; we learn how you can drive after which use muscle memory without checking the manual or driving rules behind the wheel. Our count-action programming abstraction acts because the muscle memory in Lightning. It seamlessly drives the electrons and photons within the system at runtime.”
An environmentally-friendly solution
Machine-learning services completing inference-based tasks, like ChatGPT and BERT, currently require heavy computing resources. Not only are they expensive — some estimates show that ChatGPT requires $3 million monthly to run — but they’re also environmentally detrimental, potentially emitting greater than double the common person’s carbon dioxide. Lightning uses photons that move faster than electrons do in wires, while generating less heat, enabling it to compute at a faster frequency while being more energy-efficient.
To measure this, the Ghobadi group compared their device to straightforward graphics processing units, data processing units, SmartNICs, and other accelerators by synthesizing a Lightning chip. The team observed that Lightning was more energy-efficient when completing inference requests. “Our synthesis and simulation studies show that Lightning reduces machine learning inference power consumption by orders of magnitude in comparison with state-of-the-art accelerators,” says Mingran Yang, a graduate student in Ghobadi’s lab and a co-author of the paper. By being a more cost effective, speedier option, Lightning presents a possible upgrade for data centers to cut back their machine learning model’s carbon footprint while accelerating the inference response time for users.
Additional authors on the paper are MIT CSAIL postdoc Homa Esfahanizadeh and undergraduate student Liam Kronman, in addition to MIT EECS Associate Professor Dirk Englund and three recent graduates throughout the department: Jay Lang ’22, MEng ’23; Christian Williams ’22, MEng ’23; and Alexander Sludds ’18, MEng ’19, PhD ’23. Their research was supported, partially, by the DARPA FastNICs program, the ARPA-E ENLITENED program, the DAF-MIT AI Accelerator, the USA Army Research Office through the Institute for Soldier Nanotechnologies, National Science Foundation (NSF) grants, the NSF Center for Quantum Networks, and a Sloan Fellowship.
The group will present their findings on the Association for Computing Machinery’s Special Interest Group on Data Communication (SIGCOMM) this month.