Photonic matrix multiplication lights up photonic accelerator and beyond — part 4/4

Cleaned-text code points [43852, 58739). Bibliography omitted here; source.md includes it.

The scale-out issue can also be solved by optimizing and improving optical components. For example, the number of neurons can be further expanded utilizing spectrum reuse strategies for the WDM scheme 82 , and the topology structures of neuron cluster, small-world neural network, and interconnected SNN PICs were proposed to build larger-scale on-chip photonic neurons 28 . As the scale of MRR array becomes larger, the controlling technique would be paramount, integrated photoconductive heaters enable control of large-scale silicon photonic MRR array without requiring additional components, complex tuning algorithms, or additional electrical I/Os 131 . The electro-optical modulators using lithium niobate and barium titanate integrated with silicon photonics offer high-speed phase modulation and low operating voltage, making these devices very attractive for PICs designed for photonic computing 132 . The maturity of state-of-the-art silicon nitride platform has enabled low-loss waveguides (<1 dB/m), thus reducing energy consumption and cost compared with current digital electronics, and provided opportunities for the practical application of photonic accelerators to SOI and III–V PICs especially when computation bandwidth and modulation rates continue to increase rapidly 133 . Challenges arise in scaling to larger matrices, since phase shifters in MZI mesh scheme typically consume 10 mW to 20 mW per unit for thermal tuning 134 , and thermal power consumption accumulation for thousands of phase shifter units will deteriorate the competitiveness of the photonic accelerator. Nano-optical-electro mechanical system (NOEMS) technology can be applied to replace traditional thermal phase shifters to reduce the power consuming of maintaining the status of MZIs 135 . Compared to thermal phase shifters, the static power dissipation of NOEMS components is nearly zero because mechanical displacements only require a small amount of energy to move the waveguide back and forth.

To form a scalable neural network, optical neurons should be able to excite with a certain strength to evoke at least an equivalent response in a downstream neuron 82 . To construct a cascadable neuron, the first step is to use an active amplifier, which provides energy gain in the optical or electrical domain 136 . The second step is to improve the efficiency of optoelectronic devices, which can be achieved by enhancing the interaction between the active materials and propagating waveguide mode (i.e., light-matter interaction) with nanoscale devices and novel materials 137 , 138 , 139 , 140 . And hybrid integration technology is significant for integrating the low-loss passive silicon or silicon nitride waveguides with the active amplifiers and lasers 141 , 142 , 143 . These promising technologies pave the way for cascadable photonic neurons.

### Activation functions

MVMs and activation functions are two basic elements of perceptrons 94 . Photonic MVMs show significant advantages on signal rate, latency, computing density and power consumption compared to electrical neurons, while photonic activation functions are still not mature. The implementation of photonic neurons relies on the nonlinear response of optical devices. Based on the physical representation of signals inside a neuron, the techniques are divided into two primary categories: optical-electrical-optical (OEO) and all-optical activation functions. OEO neurons convert optical power into an electrical current and then back into the signal pathway. Their nonlinearities manifest themselves in the electrical domain as well as during the EO conversion step, in which lasers 144 , 145 , 146 or saturation modulators 147 , 148 are employed. Using foundry-compatible silicon-on-insulator (SOI) technology, OEO neurons were demonstrated by Tait et al. using a high-speed silicon MRR modulator 147 and by Williamson et al. with a Mach–Zehnder-type modulator 149 . All-optical neurons depend on semiconductor carriers, reverse saturated absorption, or optical susceptibility, which can be found in a variety of materials 150 . All-optical neuron implementations are thought to be faster than the OEO techniques. All-optical neurons have been proven using optical nonlinearities, such as the carrier effect in MRRs 151 , 152 , 153 and the alteration of a material state 89 , 154 . Generally, for different AI applications, activation functions need to be chosen dependent on particular tasks. Due to the weak optical nonlinearity, the resonant devices were used to reduce the threshold and simultaneously enhance the phase sensitivity 89 , 152 . Huang et al. proposed using multiple coupled cavity devices to optimize different activation functions for different machine-learning tasks 152 , followed by an experimental demonstration 153 . And the microring resonators with PCMs were also demonstrated as effective all-optical activation functions 89 . The active optical devices are also promising candidates for activation functions 144 , 155 , 156 , 157 . A reconfigurable photonic activation function was also demonstrated using injection-locked Fabry–Perot semiconductor lasers 155 . The neuronlike excitable behavior in a micropillar laser with saturable absorber was experimentally demonstrated by introducing optical perturbations 144 . And the vertical-cavity surface-emitting laser with an embedded saturable absorber was employed as a spiking neuron 156 , 157 . The semiconductor optical amplifiers were also demonstrated for all-optical activation functions 158 , 159 , 160 , 161 .

### Optoelectronic-hybrid AI

The activation function can be realized by using either electronic or photonic methods. The optical activation function is still in the preliminary research stage, and there is no mature scheme since the efficiency of optical nonlinearity is rather low. The realization of an all-optical activation function with a low loss and a high nonlinear effect remains a key issue in the entire optical network. On the other hand, all-optical cascaded ONNs are still difficult to achieve due to the accumulative loss of optical networks. In fact, only ANNs with quite simple structures or without activation functions were all-optical, such as the SNNs with PCMs 89 , reservoir computing using optical amplifiers or passive silicon circuits 31 , 162 , 163 , and D 2 NN with passive phase masks 46 , 47 , 54 . On the contrary, most previous works of deep ANNs were implemented based on optoelectronic-hybrid hardware 3 , 16 , 18 , 58 , 90 . Before the all-optical ANNs are mature, especially in optical nonlinear effect and optical cascade, optoelectronic-hybrid AI is a more practical and more competitive candidate for deep ANNs. Therefore, the development of a highly efficient and dedicated optoelectronic-hybrid AI hardware chip system is one of the core research routes of photonic AI.

Photonic matrix multiplication has revealed great potential for optical signal processing and AI acceleration. It can greatly reduce the power consumption and signal delay. In the future, the photonic matrix core would be more comprehensive and cover richer functions. Figure 14 shows a possible route for the optoelectronic-hybrid AI computing chip framework. It mainly contains three layers: the bottom hardware layer, the algorithm layer and the top application layer.

Schematic diagram of the optoelectronic-hybrid AI computing chip framework.

Electronic computing has become quite mature, and it has outstanding advantages in terms of data storage and flow control, which are basically unknown for photonic computing. The computing capacity and speed of photons are superior to those of electronic computing, which can be improved by several orders of magnitude 23 , 164 . By combining the advantages of electronic and photonic systems, the performance in terms of the power consumption, computing capacity, computing speed, etc., can be improved by orders of magnitude compared with that of traditional electronic methods 3 , 16 , 58 . The hardware layer is mainly regarded as the photonic AI hardware system built on photoelectric devices. The electronic part of the hardware layer completes the data storage, data write/read, flow control and slight computations. The optical part executes the matrix computation operators, which take up most of the computing tasks 58 .

The algorithm layer is used to develop universal algorithm frameworks for the photonic AI hardware system, such as linear regression and gradient descent 165 , or to develop computing models, such as the feedforward neural network and convolutional neural network 7 . These algorithms can be efficiently executed in the physical layer. Different algorithms can be combined with photoelectric AI hardware depending on the type of problem. For example, the linear regression algorithm is often used in prediction, and logistic regression is often used to solve the problem of binary classification 165 . Neural network algorithms are the most widely used machine-learning methods and can significantly improve deep learning based on text, images, and voice 7 . In addition, based on the activation function, various logical computing functions can be developed as the basic unit of the optoelectronic-hybrid digital computer 166 . The algorithm framework can be learned from the mature AI algorithms of electronic computing, but it should be adjusted appropriately considering the hardware differences.

The application layer is a user-oriented interface based on the entire AI hardware system and algorithm frameworks. Users can develop various applications, such as channel equalization 69 , 103 , Google PageRank 104 , image recognition 16 , 90 , and voice recognition 3 . For example, the linear part of the optical computing core can be directly used in image sharpening, smoothing, etc., as well as in all-optical signal processing (such as channel equalization) 167 . Neural network algorithms can be employed for image recognition and voice recognition 3 , 16 , 90 . In addition, multiple algorithms can be combined to jointly address optimization and decision issues, such as NP-hard problems and high-speed tracking problems 51 , 74 , 168 . An optical computing system based on digital logic can also be built with all-optical or optoelectronic-hybrid logic computing functions 166 , 169 .

In summary, photonic matrix multiplication has been applied in many areas, such as optical signal processing in optical communications and AI accelerators. Numerous promising applications established based on matrix multiplication computation provide a complementary opportunity to expand the domain of photonic accelerators. We have reviewed the recent progress in photonic matrix multiplication with various methods and applications. A perspective for photonic matrix multiplication was further discussed, which might be extended to an easy-to-operate minicomputer for different photonic accelerator applications.

## Supplementary information

### Copyright for Fig.12(a) (download PDF )

### Copyright for Fig.12(b) (download PDF )

### Copyright for Fig.12(c) (download PDF )

### Copyright for Fig.12(d) (download PDF )

### Copyright for Fig.13(a) (download PDF )

### Copyright for Fig.13(b) (download PDF )

### Copyright for Fig.3(a) (download PDF )

### Copyright for Fig.3(b) (download PDF )

### Copyright for Fig.3(c) (download PDF )

### Copyright for Fig.4(b,c) (download PDF )

### Copyright for Fig.4(d) (download PDF )

### Copyright for Fig.5(a) (download PDF )

### Copyright for Fig.5(c) (download PDF )

### Copyright for Fig.6(c) (download PDF )

### Copyright for Fig.6(e) (download PDF )

### Copyright for Fig.7(a,b) (download PDF )

### Copyright for Fig.8(a) (download PDF )

### Copyright for Fig.8(b) (download PDF )

### Copyright for Fig.8(c) (download PDF )

### Copyright for Fig.8(d) (download PDF )

### Copyright for Fig.8(e) (download PDF )

### Copyright for Fig.9 (download PDF )

## Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

## About this article

### Cite this article

Zhou, H., Dong, J., Cheng, J. et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci Appl 11 , 30 (2022). https://doi.org/10.1038/s41377-022-00717-8

Download citation

Received : 28 July 2021

Revised : 07 January 2022

Accepted : 13 January 2022

Published : 03 February 2022

Version of record : 03 February 2022

DOI : https://doi.org/10.1038/s41377-022-00717-8

### Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

## Associated content

Collection

### 10th Anniversary of LSA: Collection on Integrated and Optoelectronic Devices

## Search

### Quick links

Explore articles by subject

Find a job

Guide to authors

Editorial policies

Light: Science & Applications ( Light Sci Appl )

ISSN 2047-7538 (online)

## nature.com footer links

### About Nature Portfolio

About us

Press releases

Press office

### Discover content

Journals A-Z

Articles by subject

protocols.io

Nature Index

### Publishing policies

Nature portfolio policies

Open access

### Author & Researcher services

Reprints & permissions

Research data

Language editing

Scientific editing

Nature Masterclasses

Research Solutions

### Libraries & institutions

Librarian service & tools

Librarian portal

Open research

Recommend to library

### Advertising & partnerships

Advertising

Partnerships & Services

Media kits

Branded content

### Professional development

Nature Awards

Nature Careers

Nature Conferences

### Regional websites

Nature Africa

Nature China

Nature India

Nature Japan

Nature Middle East

Use of cookies

Your privacy choices/Manage cookies

Legal notice

Accessibility statement

Terms & Conditions

Your US state privacy rights