Journal Article

Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing

Sun, Bai, Zhou, Guangdong, Xu, Gaobo, Cheng, Lan, Hu, Xiaofang, Wang, Lidan, Zhou, Feichi, Ren, Zhijun, Song, Qunliang, Li, Jie, Wang, Wenhua, Duan, Shukai, Chen, Xiaodie

2023

Overview

In-sensor and near-sensor computing are becoming the next-generation computing paradigm for high-density and low-power sensory processing. To fulfil a high-density and efficient neuromorphic visual system with fully hierarchical emulation of the retina and visual cortex, emerging multimodal neuromorphic devices for multi-stage processing and a fully hardware-implemented system with versatile image processing functions are still lacking and highly desirable. Here we demonstrate an emerging multimodal-multifunctional resistive random-access memory (RRAM) device array based on modified silk fibroin protein (MSFP), exhibiting both optoelectronic RRAM (ORRAM) mode featured by unique negative and positive photoconductance memory and electrical RRAM (ERRAM) mode featured by analogue resistive switching. A full hardware implementation of the artificial visual system with versatile image processing functions is realised for the first time, including ORRAM mode array for the in-sensor image pre-processing (contrast enhancement, background denoising, feature extraction) and ERRAM mode array for near-sensor high-level image recognition, which hugely improves the integration density, and simply the circuit design and the fabrication and integration complexity. In-sensor and near-sensor computing are emerging as the next-generation computing paradigm, for high-density and low-power sensory processing. Here, the authors report a fully hardware-implemented artificial visual system for versatile image processing based on multimodal-multifunctional optoelectronic resistive memory devices with optical and electrical resistive switching modes.

• Publisher: Nature Publishing Group UK,Nature Publishing Group,Nature Portfolio
• Subject: 639/301/1005/1007 / 639/301/1005/1008 / Circuit design / Fabrication / Feature extraction / Hardware / High density / Humanities and Social Sciences / Image contrast / Image enhancement / Image processing / Information processing / Memory devices / multidisciplinary / Neuromorphic computing / Optical memory (data storage) / Optoelectronic devices / Power management / Random access memory / Science / Science (multidisciplinary) / Sensor arrays / Sensors / Sensory integration / Sensory perception / Silk fibroin / Somatosensory cortex / Switching / Visual cortex / Visual system