Neuromorphic Computing

Summary

Neuromorphic computing promises to dramatically improve the efficiency of important computational tasks, such as perception and decision making. While software and specialized hardware implementations of neural networks have made tremendous accomplishments, both implementations are still many orders of magnitude less energy efficient than the human brain. Researchers in the Spin Electronics Group at NIST are working on several novel, bio-inspired, hardware implementations of these types of networks. One is based on high-frequency room-temperature nanoscale oscillators based on the spin-torque effect and the other is based on dynamically reconfigurable magnetic Josephson junctions operating at liquid-helium temperature.

Description

Spin Torque Oscillators: The research at the heart of this effort is to better understand and control mutual synchronization of arrays of spintronic nanoscale oscillators operating in the range of 10 GHz to 40 GHz. The devices under study are well suited to neuromorphic applications because they are intrinsically nonlinear and strongly interact with high-frequency injected signals. Because of these properties they can, under certain circumstances, be frequency-locked together, although their relative phase can be changed. This makes these nanoscale (50 nm to 75 nm) devices excellent candidates for implementation in approximate computing architectures that have been developed over the last half-decade. NIST researchers are presently working on this approach in two-terminal “spin-transfer torque” oscillators and three-terminal “spin-orbit torque” oscillators. The goal is to understand how the oscillators can be most efficiently coupled together, either through spinwaves, magnetic fields, or electrical currents.

Magnetic Josephson Junction Devices: The focus of this research is to develop magnetic Josephson junctions (MJJs) as both synaptic and neuronal elements for neuromorphic circuitry. Josephson junctions intrinsically operate at frequencies of 100 GHz or more, meaning that they operate much faster than modern-day semiconductor devices and therefore can potentially perform computations at much higher speeds. Researchers in the Spin Electronics Group are developing essential components of an energy-efficient neuromorphic processor and have demonstrated a new form of an artificial synapse based on their newly developed MJJ barriers. Like the brain, these devices communicate via voltage spikes, but with zeptojoule pulse energies instead of the brain’s femtojoule pulse energies. The present research in the group is focused on how to couple these artificial synapses and neurons together.

Major Accomplishments

Spintronic Nanodevices for Non-Boolean Computation

For the past several years, the Spin Electronics Group has been interested in making measurements to aid in the application of spintronic nanodevices to non-Boolean computation. The essential idea is to use the analog nonlinearities inherent in the physics of spintronic nanodevices to perform some type of difficult computation: one that would normally require a large amount of processor power or take a significant amount of time using conventional digital schemes.

Phase-locked spin-torque oscillators (STOs) have shown promise as potential non-Boolean or neuromorphic computational devices. For example, the nonlinear process of frequency-pulling and phase-locking can be used as a measure of how “close” a test value is to a reference value, the essential calculation for “degree of match.” The degree of phase coherence can be mapped on to a “distance” measure. In an STO, a DC current bias through a nanoscale ferromagnetic multilayered element induces a torque—either through a “spin filtering” or a “spin-orbit” process—that counteracts damping, resulting in harmonic oscillation of the magnetization and an AC voltage out of the device. The frequency of precession is a function of the net effective field seen by the magnetization and the current through the device, resulting in a tunable nonlinear oscillator, with frequencies in the 5-30 GHz range for the typical ferromagnetic materials in the device. This nonlinearity allows the oscillators to frequency-pull and phase-lock to AC currents, AC magnetic fields, and to spin waves propagating in the magnetic medium surrounding the STO.

We have been working to fabricate and measure the response of larger arrays of STOs to injected AC fields, because this phase locking response can be mapped to a “degree of match” function for pattern matching. The measurement challenges of this project include: the measurement of statistically significant distributions of AC outputs of devices, and connecting these variations to ferromagnet properties; making phase-sensitive measurements of multiple devices simultaneously, and determining the degree phase uniformity; measuring the phase noise of STOs when locked and unlocked, as well as for the array. For example, we have designed and built a phase-sensitive detection circuit that injection locks 16 devices to an AC field at a common phase, and measures the phase-sensitive output as a function of current bias through the devices (see Fig. 1a). The variation in phase is detected by interference with a reference signal in a homodyne process. (see Fig. 1b). Our main goal is to measure a range of oscillators with different magnetic properties and differing inter-device coupling, and determine the impact of these properties on the nonlinear response to determine their potential suitability for computation.

In addition to periodic (harmonic) STOs, we are also interested in using STOs that operate near the thermal limit and act as two-state stochastic fluctuators. Such oscillators also exhibit nonlinear coupling, and theoretical work indicates that they can be arrayed into so-called Boltzmann machines. Such machines can be constructed to find the solutions of mathematical problems (such as factorization) via a process akin to energy minimization rather than Boolean-based arithmetic, potentially at lower energy cost. The measurement challenges of these involve correlating nanoscale structure with function: how do the magnetic normal modes of the devices affect the device fluctuation distributions, the shape of which are critical to the computation.

(a) Schematic of detection circuit and layout of STO chip. ( b) Detected power vs. bias current for two STOs, f_inj = 7.5 GHz.

Publications

• E. Jué, M. R. Pufall, and W. H. Rippard,  "Asymmetric and partial injection locking of a three-terminal spin-torque oscillator," Appl. Phys. Lett. 112, 102403 (Mar 2018); doi: 10.1063/1.5019729.

• W. H. Rippard, M. R. Pufall, and A. B. Kos, "Time required to injection-lock spin torque nanoscale oscillators," Appl. Phys. Lett. 103, 182403 (Oct 2013); doi: 10.1063/1.4821179.
• M. R. Pufall, W. H. Rippard, S. E. Russek, and E. R. Evarts, "Anisotropic frequency response of spin-torque oscillators with applied field polarity and direction," Phys. Rev. B 86, 094404 (Sep 2012); doi: 10.1103/PhysRevB.86.094404.
• M. R. Pufall, W. H. Rippard, G. Csaba, D. E. Nikonov, G. I. Bourianoff, and W. Porod, "Physical implementation of coherently coupled oscillator networks," IEEE J. Explor. Solid-State Comp. Dev. Circuits, 1, 76-84 (Oct 2015), doi: 10.1109/JXCDC.2015.2468070

Ultra-Low Power, Artificial Synapses to Enable Large-Scale Neuromorphic Computing

Neuromorphic computing promises to dramatically improve the efficiency of important computational tasks, such as perception and decision making. While software and specialized hardware implementations of neural networks have made tremendous accomplishments, both implementations require many orders of magnitude more energy than equivalent processes in the human brain.

Researchers in PML’s Spin Electronics Group, in collaboration with the Superconductive Electronics Group and the Magnetic Imaging Group, are developing essential components of an energy-efficient neuromorphic processor, an outgrowth of their work in the IARPA Cryogenic Computing Complexity (C3) program. They have demonstrated a new form of artificial synapse based on dynamically reconfigurable superconducting Josephson junctions with magnetic nanoclusters in the tunneling barriers.

Although the neural spiking energy per pulse varies with the magnetic configuration, the devices’ spiking energy is always less than 1 aJ (10^-18 J). This is four orders of magnitude less than the roughly 10 fJ per synaptic event in the human brain. The critical current of each synapse junction, which is analogous to the synaptic weight, can be tuned using input current spikes that change the spin alignment of the barriers’ manganese nanoclusters. Synaptic weight training was demonstrated in these devices with electrical pulses as small as 3 aJ.

These new artificial synapses provide a significant step toward a neuromorphic processor that is faster, more energy efficient, and thus can attain far greater complexity than has been demonstrated with any other technology. Lead researcher Michael Schneider presented the results at the November 2017 IEEE International Conference on Rebooting Computing, which was summarized in an IEEE Spectrum article (https://spectrum.ieee.org/nanoclast/computing/hardware/4-strange-new-wa…). NIST has filed for a patent based on the work.

Publication

• M. L. Schneider, C. A. Donnelly, S. E. Russek, B. Baek, M. R. Pufall, P. F. Hopkins, P. D. Dresselhaus, S. P. Benz, and W. H. Rippard, "Ultralow power artificial synapses using nanotextured magnetic Josephson junctions," Science Advances, 4, e1701329 (Jan 2018); doi: 10.1126/sciadv.1701329.