The rapid development of artificial neural networks (ANNs) in the past decade has empowered artificial intelligence in many applications such as image recognition, natural language processing, autonomous driving, and security, etc [1]. While feedforward neural networks, in which neurons are separated into layers and the signal only goes forward, are widely used to recognize static images, neural networks with recurrent connections (e.g., recurrent neural network, or RNN) are usually needed to process temporal signals. However, the training of RNN is usually very difficult and requires extensive computational power, mainly due to the problem of exploding or vanishing gradients in recurrent structures.

Derived from standard RNN, reservoir computing (RC) is designed as a highly efficient network suitable for processing temporal signals with low training cost and simple hardware implementation. RC has been widely used in dynamic system identification, time-series forecasting and many other applications. Recent research has revealed that a recurrent network with complex connections can be equivalently replaced by a nonlinear dynamic system, which enables a complete RC system to be realized using just a few components.

Memristor, as a new type of information processing device, has the advantages of analog resistive switching and in-memory computing. In recent years, significant progress has been made in the implementations of memristor-based ANNs, mostly feedforward neural networks such as multi-layer perceptron (MLP) and convolutional neural network (CNN) [2], [3]. Interestingly, there is a special class of memristors, called dynamic memristor [4], whose inherent dynamic properties and nonlinearity make it have great potential to realize compact and efficient RC systems.

In this work, a single dynamic memristor, with a device structure of Ti/TiO_x/TaO_y/Pt, is employed to realize the complete reservoir function. By developing a controllable mask process to generate rich reservoir states, a parallel RC system based on multiple dynamic memristors is further designed. The state richness and feedback strength of the RC system can be well controlled by adjusting the mask length, so the system performance can be optimized. In order to verify the feasibility of the RC system, temporal signal processing tasks including spoken-digit recognition and time-series prediction task of the Hénon map are demonstrated with extremely low word error rate (WER = 0.4%) and prediction error (NRMSE = 0.046) respectively. Compared with previous work [5], the operating power of our memristor-based RC system is much lower owing to the mask process, and the energy consumption can be further reduced by reducing the input voltage pulse width.

It should be noted that the current RC system is based on a single reservoir layer as a proof-of-concept demonstration. Nevertheless, it points out a feasible pathway to construct a high-efficiency memristor-based RC system that can handle complex temporal tasks in real time. In the future, a more sophisticated RC system based on multiple reservoir layers can be constructed, which would have much improved performance because of its richer reservoir states and stronger memory capacity.

These results were recently published in Nature Communications:

"Dynamic Memristor-based Reservoir Computing for High-Efficiency Temporal Signal Processing", Nature Communications XXX, XXX–XXX (2021).

 https://www.nature.com/articles/s41467-020-20692-1

 Reference:

[1]         LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436-444, (2015).

[2]         Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61-64, (2015).

[3]         Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577, 641-646, (2020).

[4]         Li, X. et al. Power-efficient neural network with artificial dendrites. Nature Nanotechnology 15, 776-782, (2020).

[5]         Moon, J. et al. Temporal data classification and forecasting using a memristor-based reservoir computing system. Nature Electronics 2, 480-487, (2019).