Bio-inspired cross-modal super-additive plasticity for seamless visual processing-in-sensory and -in-memory

Bio-inspired cross-modal visual perception hardware offers potential for edge intelligence. However, physical implementation of such hardware by conventional optoelectronics typically results in linear function combinations, lacking super-additive integration. Here, inspired by the primary cortex of the biological brain, we design a hardware platform based on molybdenum disulfide channel for processing-in-sensory and -in-memory. Cross-modal correlation photoelectric signals processing is demonstrated by utilizing electric field-assisted photogenerated carrier tunneling based on a floating gate photoelectric device array. The devices exhibit high synergistic paradigm super-additive behavior up to 103 times and significant time-dependent plasticity for visual encoding and perception enhancement. After sensory preprocessing, patterns are accurately routed and recognized by a non-volatile four-transistor ternary content-addressable memory circuit array. The cell maintains a large resistance ratio of 105 and high lookup durability of 1012. The hardware platform of cross-modal visual perception empowers seamless visual process-in-sensory and -in-memory, providing potential for ubiquitous visual edge intelligent systems.