HAMUX: The Future of Hopfield Networks

Do we really understand what Hopfield Networks are? How do they relate to Deep Learning today? In this video, we discuss a new way to understand the math behind Hopfield Networks that enables us to make Hopfield Networks deep and connected in a hypergraph. Our novel architecture is called “Hierarchical Associative Memory” (HAM), and it has the same guarantees as the original, biologically plausible Hopfield Network:

- It is a dynamical system
- With a tractable energy function
- That converges to a fixed point

We additionally release an accompanying deep learning library called HAMUX (Hierarchical Associative Memory User eXperience), written in JAX, that enables anyone to easily build and use these architectures. HAMUX is:

- A new class of deep learning library built around energy
- Software to describe any associative memory using a hypergraph
- A way to integrate familiar deep learning operations into an associative memory framework

Link to code: [ Ссылка ]

Accepted as a poster to NeurIPS 2022 Memari workshop and a spotlight poster to NeurIPS 2022 DLDE workshop.

Work done in collaboration with the MIT-IBM Lab and the PoloClub at Georgia Tech.

0:00 - Introduction
0:23 - Hierarchical Associative Memories (HAMs) vs Hopfield Networks 1.0
1:02 - From Hopfield Networks to HAMs
1:34 - The Energy of Neuron Layers
1:49 - The Energy of Synapses
2:06 - Hopfield Networks are Ensembles of Energies
2:28 - HAMs: Generalized Hopfield Networks
2:51 - Introducing HAMUX: A Deep Learning Framework Built around Energy

свернуть