  Current large scale implementations of deep learning and data mining require
thousands of processors, massive amounts of off-chip memory, and consume
gigajoules of energy. Emerging memory technologies such as nanoscale
two-terminal resistive switching memory devices offer a compact, scalable and
low power alternative that permits on-chip co-located processing and memory in
fine-grain distributed parallel architecture. Here we report first use of
resistive switching memory devices for implementing and training a Restricted
Boltzmann Machine (RBM), a generative probabilistic graphical model as a key
component for unsupervised learning in deep networks. We experimentally
demonstrate a 45-synapse RBM realized with 90 resistive switching phase change
memory (PCM) elements trained with a bio-inspired variant of the Contrastive
Divergence (CD) algorithm, implementing Hebbian and anti-Hebbian weight
updates. The resistive PCM devices show a two-fold to ten-fold reduction in
error rate in a missing pixel pattern completion task trained over 30 epochs,
compared to untrained case. Measured programming energy consumption is 6.1 nJ
per epoch with the resistive switching PCM devices, a factor of ~150 times
lower than conventional processor-memory systems. We analyze and discuss the
dependence of learning performance on cycle-to-cycle variations as well as
number of gradual levels in the PCM analog memory devices.

Neural and Evolutionary Computing;Distributed, Parallel, and Cluster Computing;Emerging Technologies