Home Career Engineers working on ‘analog deep learning’ find way to propel protons through...

Engineers working on ‘analog deep learning’ find way to propel protons through solids at unprecedented speeds – ScienceDaily


As scientists push the boundaries of machine learning, the amount of time, energy, and money required to train increasingly complex neural network models is skyrocketing. A new field of artificial intelligence called “analog deep learning” promises faster computations with less power consumption.

Programmable resistors are the key building blocks in analog deep learning, just as transistors are the basic building blocks for digital processors. By repeating arrays of programmable resistors in complex layers, researchers can create a network of analog artificial “neurons” and “synapses” that perform calculations just like a digital neural network. This network can then be trained to perform complex artificial intelligence tasks such as image recognition and natural language processing.

An interdisciplinary team of MIT researchers set out to push the speed limits of the human analog synapse they had previously developed. They used a practical inorganic material in the manufacturing process that allows their devices to work 1 million times faster than previous versions, which is also about 1 million times faster than the synapses in the human brain.

In addition, this inorganic material also makes the resistor extremely energy efficient. Unlike the materials used in an earlier version of their device, the new material is compatible with silicon manufacturing methods. This change has made it possible to fabricate devices at the nanometer scale and may pave the way for integration into commercial computing hardware for deep learning applications.

“With this key insight and the very powerful nanofabrication techniques we have at MIT.nano, we were able to put these pieces together and demonstrate that these devices are inherently very fast and operate at reasonable voltages,” says senior author Jesus A .del Alamo, Donner Professor in the Department of Electrical Engineering and Computer Science at the Massachusetts Institute of Technology (EECS). “This work has really put these devices at a point where they now look very promising for future applications.”

“The working mechanism of the device is to electrochemically introduce the smallest ion, a proton, into the insulating oxide to modulate its electronic conductivity. Because we are working with very thin devices, we could accelerate the movement of this ion using a strong electric field and bring these ion devices into the nanosecond mode of operation,” explains senior author Bilge Yildiz, Bryn M. Kerr Professor in the Departments of Nuclear Science and Engineering and materials science and engineering.

“Action potentials in biological cells rise and fall with time in milliseconds because a voltage difference of about 0.1 volts is limited by the stability of water,” says senior author Ju Li, Battelle Energy Alliance Professor of Nuclear Science and Engineering and Professor of Materials Science and Engineering. Here, we apply up to 10 volts through a special hard glass film of nanoscale thickness that conducts protons without permanently damaging it. And the stronger the field, the faster the ion devices.”

These programmable resistors greatly increase the speed of neural network learning while dramatically reducing the cost and energy required to perform that learning. This could help scientists develop deep learning models much faster, which could then be applied to purposes such as self-driving cars, fraud detection or medical image analysis.

“Once you have an analog processor, you’re no longer training networks that everyone else is working on. You will train networks with an unprecedented level of complexity that no one else can afford, thus far surpassing them all. In other words, it’s not a faster car, it’s a spaceship,” adds lead author and MIT postdoctoral researcher Murat Onen.

Co-authors include Francis M. Ross, Ellen Swallow Richards Professor in the Department of Materials Science and Engineering; postdoctoral fellows Nicholas Emond and Baoming Wang; and Difei Zhang, EECS graduate student. The study was published today in Science.

Accelerating deep learning

Analog deep learning is faster and more energy efficient than digital for two main reasons. “First, the computations are done in memory, so huge data loads aren’t being transferred back and forth from memory to the processor.” Analog processors also perform operations in parallel. As the size of the matrix increases, the analog processor does not need more time to perform new operations because all the calculations are done simultaneously.

A key element of MIT’s new analog processor technology is known as a proton programmable resistor. These resistors, which are measured in nanometers (one nanometer is one billionth of a meter), are arranged in an array like a checkerboard.

In the human brain, learning occurs through the strengthening and weakening of connections between neurons, called synapses. Deep neural networks have long adopted this strategy, where network weights are programmed using learning algorithms. In the case of this new processor, increasing and decreasing the electrical conductivity of proton resistors is enabled by analog machine learning.

Conduction is controlled by the movement of protons. To increase conductivity, more protons are pushed into the resistor channel, and to decrease conductivity, protons are removed. This is achieved using an electrolyte (similar to a battery) that conducts protons but blocks electrons.

To develop an ultra-fast and highly energy-efficient programmable proton resistor, researchers looked for different electrolyte materials. While other devices used organic compounds, Onen focused on inorganic phosphosilicate glass (PSG).

PSG is basically silicon dioxide, which is a powdered desiccant that comes in tiny packets that come with new furniture to remove moisture. It is also the most famous oxide used in silicon processing. To make PSG, a small amount of phosphorus is added to the silicon to give it special proton conduction characteristics.

Onen suggested that the optimized PSG could have high proton conductivity at room temperature without the need for water, making it an ideal solid electrolyte for this application. He was right.

Amazing speed

PSG enables ultrafast proton movement because it contains many nanometer-sized pores whose surfaces provide pathways for proton diffusion. It can also withstand very strong pulsed electric fields. This is important, Onen explains, because applying more voltage to the device allows the protons to move at breakneck speeds.

“The speed was definitely amazing. Normally, we wouldn’t apply such extreme fields to devices, lest we turn them to ash. But instead, the protons ended up moving at enormous speeds through the device stack, specifically a million times faster than what we had before. And this movement does not damage anything, thanks to the small size and low mass of protons. It’s almost like teleportation,” he says.

“The nanosecond time means that we are close to the regime of ballistic or even quantum tunneling of the proton in such an extreme field,” Lee adds.

Because the protons do not damage the material, the resistor can operate for millions of cycles without failure. This new electrolyte incorporated a programmable proton resistor that works a million times faster than their previous device and can operate efficiently at room temperature, which is important for its inclusion in computing equipment.

Due to the insulating properties of PSG, almost no electric current passes through the material when protons move. This makes the device extremely energy efficient, Onen adds.

Now that they’ve demonstrated the effectiveness of these programmable resistors, the researchers plan to reengineer them for high-volume production, del Alamo says. They can then study the properties of the resistors and scale them up so they can be incorporated into systems.

At the same time, they plan to explore materials to remove the bottlenecks that limit the voltage required to efficiently transport protons into, through, and out of an electrolyte.

“Another interesting direction that these ion devices can enable is energy-efficient hardware for emulating neural circuits and synaptic plasticity rules derived from neuroscience, beyond analog deep neural networks,” Yildiz adds.

“Our collaboration will be essential for innovation in the future. The path forward will still be very challenging, but at the same time very exciting,” says del Alamo.

This research is funded in part by the MIT-IBM Watson AI Laboratory.

Source link

Previous articleEditor’s Note – Debtfree Magazine
Next articleHUD, Citi, Anchor Loans and Doma make leadership announcements