Literature and Nobel records checked through 2026-10-25. Formal equivalence, historical influence, physical implementation and universality are kept separate; the essay does not infer one from another.

On 8 October 2024, the Royal Swedish Academy of Sciences announced that the Nobel Prize in Physics would go to John Hopfield and Geoffrey Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.” Within hours, the physics corner of the internet had an episode. Thermodynamics Twitter — yes, that is a thing — asked whether gradient descent is really physics in the sense that the Higgs mechanism is physics. The condensed matter community, who have been doing disordered systems since before most ML practitioners were born, oscillated between pride (“finally, they noticed us”) and bafflement (“why is Hinton here and not Parisi?”). There were takes. There were dunks. Someone made a graph of Nobel prizes versus average journal impact factor and it was not flattering to this year’s winner.

I understand the irritation. I do not share it.

The argument I can support is stronger than a loose metaphor but weaker than one unbroken identity. Hopfield’s binary symmetric energy has Ising form. Boltzmann machines define a probabilistic model using a Boltzmann distribution and a statistical-mechanical learning derivation. Different couplings, dynamics, objectives and implementations still matter. The path from those models to modern protein prediction mixes formal analogy, historical influence and later engineering; it is not a sequence of mathematical identities.

Let me trace it properly.

The 2021 Nobel: Parisi and the frozen magnet

Before we get to 2024, we need 2021. Giorgio Parisi received half the Nobel Prize in Physics that year, “for the discovery of the interplay of disorder and fluctuations in physical systems from atomic to planetary scales”; the spin-glass work at its centre was done between 1979 and 1983. The other half went to Syukuro Manabe and Klaus Hasselmann for climate modelling — an interesting pairing that provoked its own set of takes, though rather fewer.

A spin glass is a disordered magnetic system. The canonical physical realisation is a dilute alloy: a small concentration of manganese atoms dissolved in copper. Each manganese atom carries a magnetic moment — a spin. In the alloy the moments can point in any direction; the models below keep only two, which we label $\sigma_i \in \{-1, +1\}$. The spins interact with each other via exchange interactions mediated by the conduction electrons. The crucial feature is that these interactions are random: some spin pairs prefer to align (ferromagnetic coupling, $J_{ij} > 0$) and others prefer to anti-align (antiferromagnetic coupling, $J_{ij} < 0$), and because the manganese atoms sit at random positions there is no regular pattern to which is which.

The Hamiltonian of the system is

$$H = -\sum_{i < j} J_{ij} \sigma_i \sigma_j \tag{1}$$

where the $J_{ij}$ are random variables drawn from some distribution. Edwards and Anderson wrote this model in 1975 with random couplings between nearby spins; the same year, Sherrington and Kirkpatrick made it infinite-ranged (Nobel Committee for Physics, 2021). In the Sherrington-Kirkpatrick (SK) model (Sherrington & Kirkpatrick, 1975), all $N$ spins interact with all other spins — a mean-field model — and the couplings are drawn independently from a Gaussian distribution, here with mean zero and variance $J^2/N$:

$$J_{ij} \sim \mathcal{N}\!\left(0,\, \frac{J^2}{N}\right) \tag{2}$$

(Sherrington and Kirkpatrick also allowed a positive mean, which gives their ferromagnetic phase.) The factor of $1/N$ is essential for extensivity: without it, the ground-state energy would scale as $N^{3/2}$ rather than $N$, which is unphysical.

Now here is the key phenomenon. At high temperature, the spins fluctuate freely and the system is paramagnetic. Cool it below the glass transition temperature $T_g$, and the system “freezes” — but not into a ferromagnet with all spins aligned, and not into a simple antiferromagnet. It freezes into one of an astronomically large number of disordered, metastable states. The system is not in its true ground state; it is trapped. It cannot find its way down because the energy landscape is rugged: the paths toward lower energy run over barriers.

In mean-field descriptions the rugged landscape has many metastable states and history-dependent dynamics. Which barriers scale how, and how this picture maps to a finite-dimensional alloy, depend on the model; “glass” is not named from one memory property alone.

Computing thermodynamic quantities in this system requires averaging over the disorder (the random $J_{ij}$), which means computing the quenched average of the free energy:

$$\overline{F} = -T\, \overline{\ln Z}$$

The overline denotes an average over the distribution of couplings. The problem is that $\ln Z$ is hard to average because $Z$ is a sum of exponentially many terms. Parisi’s solution rests on the replica trick, which Edwards and Anderson had introduced for exactly this average — a mathematical device worth describing, because it is beautifully strange.

The formal replica step uses $\ln Z = \lim_{n \to 0}(Z^n-1)/n$: calculate $\overline{Z^n}$ for positive integers, then analytically continue toward zero. That continuation is the heuristic step, not an automatic consequence of the integer calculation. Averaging $Z^n$ couples the $n$ copies, or replicas, of the system, and the order parameters of the calculation are their overlaps, $q^{ab} = N^{-1}\sum_i \sigma_i^a \sigma_i^b$ for $a \neq b$: roughly, how far two copies of the same sample, cooled independently, agree spin by spin.

The naive assumption is replica symmetry: all $q^{ab}$ are equal. This assumption turns out to be wrong. Sherrington and Kirkpatrick’s own replica-symmetric solution has a negative entropy at low temperature, which no system of Ising spins can have. Parisi showed that the correct solution breaks replica symmetry in a hierarchical way — the overlap matrix $q^{ab}$ has a nested structure, described by a function $q(x)$ for $x \in [0,1]$. This is replica symmetry breaking (RSB).

Full RSB has an ultrametric, hierarchical interpretation in the mean-field SK model. The replica derivation was not rigorous — the continuation to $n \to 0$ is not a theorem — but it was not a lucky guess either. Guerra (2003) proved that Parisi’s expression bounds the free energy, Talagrand (2006) proved that it is the free energy of the SK model at every temperature, and Panchenko (2013) proved the ultrametric structure for mean-field models of this kind. The solution is Parisi’s; the proofs came later, and from others.

RSB gives exact mean-field predictions and a powerful language for experiment. Finite-dimensional alloys are not thereby proven identical to the full SK hierarchy.

Three years after Parisi solved the SK model, a physicist at Caltech and Bell Labs wrote a paper about memory.

Hopfield (1982): memory as energy minimisation

John Hopfield was a condensed matter physicist who had drifted toward biophysics — electron transfer in proteins, neural computation. In 1982 he published a paper in PNAS with the title “Neural networks and physical systems with emergent collective computational abilities” (Hopfield, 1982). Most biologists read it as a neuroscience paper. It is a statistical mechanics paper.

Hopfield’s neurons are binary in the computer’s sense, not the magnet’s: each has $V_i \in \{0, 1\}$, “not firing” or “firing at maximum rate”. Neuron $i$ receives input $\sum_{j \neq i} T_{ij} V_j$ through connections of strength $T_{ij}$, compares it with a threshold $U_i$ (zero unless stated otherwise), and switches on above it and off below it; each neuron does this at random times, asynchronously. To store states $V^s$, $s = 1, \ldots, n$, Hopfield set $T_{ij} = \sum_s (2V_i^s - 1)(2V_j^s - 1)$ with $T_{ii} = 0$. For symmetric $T_{ij}$ he defined the energy

$$E = -\frac{1}{2} \sum_{i \neq j} T_{ij}\, V_i V_j \tag{3}$$

Hopfield’s comment on this energy: “This case is isomorphic with an Ising model. $T_{ij}$ provides the role of the exchange coupling, and there is also an external local field at each site.”

The local field is what the change of variables leaves behind. With $s_i = 2V_i - 1 \in \{-1, +1\}$, Eq. (3) becomes

$$E = -\frac{1}{8} \sum_{i \neq j} T_{ij}\, s_i s_j - \frac{1}{4} \sum_i s_i \sum_{j \neq i} T_{ij} - \frac{1}{8} \sum_{i \neq j} T_{ij},$$

the pairwise energy of Eq. (1) with couplings $T_{ij}/4$, plus a field $\frac{1}{4}\sum_{j \neq i} T_{ij}$ on spin $i$ and a constant. Hopfield also noted that a judicious choice of thresholds doubles what can be stored at a given accuracy, and that “this choice is equivalent to using variables $\mu_i = \pm 1$, $T_{ij} = \sum_s \mu_i^s \mu_j^s$, and a threshold level of 0.” In his variables that choice is $U_i = \frac{1}{2}\sum_{j \neq i} T_{ij}$, which cancels the field. From here on I use that $\pm 1$ form: spins $s_i$, patterns $\xi^\mu = 2V^\mu - 1 \in \{-1, +1\}^N$, weights $W_{ij} = T_{ij}/N$, and

$$E = -\frac{1}{2} \sum_{i \neq j} W_{ij}\, s_i s_j, \qquad s_i \leftarrow \operatorname{sgn}\!\Big(\sum_{j \neq i} W_{ij} s_j\Big),$$

where a neuron whose input is exactly zero keeps its state; the factor $1/N$ changes no update.

With asynchronous updates, symmetric weights, a zero diagonal and that tie rule, the Lyapunov energy does not increase and the finite network reaches a fixed point. “Gradient descent” is shorthand, not a differentiable gradient step.

The innovation is in how Hopfield chose the weights. To store $p$ patterns $\xi^\mu$ (for $\mu = 1, \ldots, p$), his rule reads, in the $\pm 1$ form, as Hebb’s rule:

$$W_{ij} = \frac{1}{N} \sum_{\mu=1}^{p} \xi^\mu_i\, \xi^\mu_j \quad (i \neq j), \qquad W_{ii} = 0 \tag{4}$$

This is the outer product rule. Each stored pattern contributes a rank-1 matrix to $W$. You can verify that if $s = \xi^\mu$, then the local field at neuron $i$ is

$$h_i = \sum_{j \neq i} W_{ij} \xi^\mu_j = \frac{N-1}{N}\,\xi^\mu_i + \frac{1}{N}\sum_{\nu \neq \mu} \xi^\nu_i \underbrace{\sum_{j \neq i} \xi^\nu_j \xi^\mu_j}_{\text{cross-talk}}$$

The first term reinforces pattern $\mu$; the rest is cross-talk. For random unbiased patterns its standard deviation is close to $\sqrt{\alpha}$, where $\alpha = p/N$ is the load.

At sufficiently low load, then, retrieval states can be stable. A corrupted cue may converge to a stored attractor, but not necessarily the nearest pattern under every metric or initial condition: in Hopfield’s runs with 30 neurons, starts within five flipped bits of a memory reached the nearest memory more than 90 percent of the time.

Same form, different couplings

The Hopfield energy and the SK Hamiltonian share their form exactly. Both are quadratic energies in $\pm 1$ variables with symmetric couplings between every pair of units, Eq. (1) with $\sigma \to s$ and $J \to W$, and both are mean-field models in the physicist’s sense. What differs is the couplings. In SK they are independent Gaussians, Eq. (2). The Hebbian couplings of Eq. (4) are built from $p$ patterns, and the energy shows it: with $m_\mu = N^{-1}\sum_i \xi^\mu_i s_i$, the overlap of the state with pattern $\mu$,

$$E = -\frac{N}{2} \sum_{\mu=1}^{p} m_\mu^2 + \frac{p}{2} \tag{5}$$

The energy is lowest where the state lines up with a stored pattern: the minima are planted. Entry by entry the two ensembles look alike — a Hebbian coupling between random patterns has mean zero and variance $p/N^2 = \alpha/N$, the scaling of Eq. (2) with $J^2 = \alpha$ — but the entries are not independent. In matrix form $W = N^{-1}\Xi^\top\Xi - (p/N)\,I$, where the rows of $\Xi$ are the patterns: rank at most $p$ up to the diagonal shift, with Eq. (5) saying where its low-energy states are. A Hopfield network is not an SK spin glass with memories added on top, and the two phase diagrams should not be called identical. The 2021 and 2024 prizes concern related statistical-mechanical ideas, not the same model or theorem.

How much it can hold

Hopfield’s own simulations, with 30 and 100 neurons, gave the first answer: “About 0.15 N states can be simultaneously remembered before error in recall is severe.” That figure belongs to his $\{0, 1\}$ network with threshold zero, which on his own remark stores half as much at a given accuracy as the $\pm 1$ form, so it is not an earlier estimate of the number below.

The sharp answer came from the methods of the previous section. Amit, Gutfreund and Sompolinsky (1985) treated the model in the limit $N \to \infty$ at fixed $\alpha$ with the mean-field theory of spin glasses and found that it “exhibits associative memory for $\alpha < \alpha_c$, $\alpha_c \gtrsim 0.14$”. Their longer 1987 paper, which I know only through Steffan and Kühn (1994), does the replica calculation under replica symmetry; the overlaps $m_\mu$ of Eq. (5) join the replica overlaps $q^{ab}$ as its order parameters. At zero temperature its retrieval state satisfies

$$m = \operatorname{erf}(y), \qquad y\left(\sqrt{2\alpha} + \frac{2}{\sqrt{\pi}}\, e^{-y^2}\right) = \operatorname{erf}(y) \tag{6}$$

which has a solution only for $\alpha \le \alpha_c \approx 0.138$. Eq. (6) is my zero-temperature reduction of the replica-symmetric equations as Steffan and Kühn print them; it reproduces their $\alpha_c$, and at the threshold it gives $m = 0.967$ before the retrieval state disappears. Breaking replica symmetry moves $\alpha_c$ only in the fourth decimal place, to 0.138186 at one step. The popular “0.14 N” is this number rounded.

The same 1985 letter contains a less quoted result: the retrieval states are ground states only for $\alpha < 0.05$. Between 0.05 and $\alpha_c$ the memories are metastable, and the network still works because they are dynamically stable: started near a pattern, it stays there instead of finding the lower states elsewhere. A Hopfield network at moderate load runs on metastability.

Fig. 1 shows the loss in finite networks. Up to $\alpha = 0.12$ every run, at every size, ends with $m > 0.9$, and the mean overlap stays above 0.99. Beyond that the mean falls as a rounded slope, not a step: in my runs half the starts still retrieve their pattern ($m > 0.9$) at $\alpha \approx 0.16$ with 1000 and 2000 neurons and 0.15 with 4000. Amit, Gutfreund and Sompolinsky’s own simulations gave $0.145 \pm 0.009$ (as Steffan and Kühn report it). Finite networks hold a little more than the infinite one, and in these runs the excess shrinks slowly with size.

Figure 1. Retrieval fails near the replica-symmetric threshold, but finite networks round the failure off and push it to slightly higher load. Each line averages 192 (1000 neurons), 96 (2000) or 48 (4000) runs started on a stored pattern. Highlighted: the retrieval branch of Eq. (6), ending at $\alpha_c$ with $m = 0.967$. My simulation, seeded and reproducible.

The number belongs to the standard Hebbian model with random unbiased patterns in the thermodynamic limit. It is not universal for correlated data, finite networks or other learning rules, and spurious attractors also occur below it.

Boltzmann machines (Ackley, Hinton & Sejnowski, 1985)

The Hopfield model is deterministic and shallow — one layer of visible neurons, no hidden structure. Geoffrey Hinton and Terry Sejnowski, in a collaboration that, in Hinton’s account, began at a small workshop in Rochester in the summer of 1982, where Hopfield presented his network, and culminated in a 1985 paper (Ackley, Hinton, & Sejnowski, 1985; Hinton, 2025), added two things: hidden units and stochastic dynamics.

Hidden units can represent latent structure. The units are binary in Hopfield’s original sense, $s_i \in \{0, 1\}$ for off and on, and the 1985 paper’s energy is $E = -\sum_{i

with $v_i, h_j \in \{0, 1\}$. This is still an Ising-type energy, by the same change of variables as for Hopfield’s Eq. (3). The $W_{ij}$ are now inter-layer weights.

The stochastic dynamics replace deterministic gradient descent with a Markov chain. Each unit is switched on with probability

$$P(s_k = 1 \mid \text{rest}) = \sigma\!\left(\beta\, \Delta E_k\right), \qquad \Delta E_k = \sum_j w_{kj} s_j - \theta_k$$

where $\Delta E_k$ is the energy gap between unit $k$ off and on, $\sigma(x) = 1/(1 + e^{-x})$ is the logistic sigmoid, and $\beta = 1/T$ is an inverse temperature. Run long enough at fixed $T$, the network visits each complete configuration with probability

$$P(\mathbf{v}, \mathbf{h}) = \frac{1}{Z}\, e^{-\beta E(\mathbf{v}, \mathbf{h})}$$

This is the Boltzmann distribution. The machine is named after Ludwig Boltzmann because the equilibrium distribution of its states is the Boltzmann distribution. Not analogously. Literally.

Learning amounts to adjusting the weights to make the model distribution $P(\mathbf{v}, \mathbf{h})$ match the data distribution $P_{\text{data}}(\mathbf{v})$. The objective is to minimise the Kullback-Leibler divergence:

$$\mathcal{L} = D_{\mathrm{KL}}(P_{\text{data}} \| P_{\text{model}}) = \sum_{\mathbf{v}} P_{\text{data}}(\mathbf{v}) \ln \frac{P_{\text{data}}(\mathbf{v})}{P_{\text{model}}(\mathbf{v})}$$

The gradient with respect to the weight $W_{ij}$ is

$$\frac{\partial \mathcal{L}}{\partial W_{ij}} = -\beta\left(\langle v_i h_j \rangle_{\text{data}} - \langle v_i h_j \rangle_{\text{model}}\right)$$

For $\{0, 1\}$ units, $\langle v_i h_j \rangle$ is the probability that both units are on; this is the 1985 paper’s gradient, $-(p_{ij} - p'_{ij})/T$. The first term is the empirical correlation between visible unit $i$ and hidden unit $j$ when the visible units are clamped to data. The second term is the correlation in the model’s free-running equilibrium. The learning rule says: increase $W_{ij}$ if the data sees these two units co-active more than the model does, and decrease it otherwise. This is Hebbian learning with a contrastive correction — the physics of equilibration drives the learning.

The model expectation is generally intractable exactly, and Markov-chain mixing can be very slow; it is not proven exponentially slow for every instance. Contrastive divergence (Hinton, 2002) replaces equilibrium sampling with a short chain and therefore gives a generally biased approximation to the likelihood gradient.

That approximation is not unusual in physics, but “works” is problem-dependent.

From Boltzmann machines to transformers

The Boltzmann machine was computationally difficult but conceptually foundational. The restricted Boltzmann machine (RBM) — with no within-layer connections, so that hidden units are conditionally independent given the visible units and vice versa — made training via contrastive divergence practical.

Hinton, Osindero and Teh (2006) showed effective greedy pretraining of deep belief networks with RBMs. It was one influential step in deep learning’s revival alongside backpropagation, larger datasets, compute, convolutional networks and other work. The counterfactual that modern language models could not have emerged without RBMs is not established.

The connection between Hopfield networks and modern attention mechanisms is more recent and more surprising, and it came in steps. Krotov and Hopfield (2016) replaced the square in Eq. (5) by a power $x^n$, with $n = 2$ the classical network, and Demircigil et al. (2017) by an exponential, both for binary states. Ramsauer et al. (2020) made the states continuous; the capacity of their modern Hopfield network grows exponentially with the dimension of the pattern space. For stored patterns $\mathbf{x}^1, \ldots, \mathbf{x}^p \in \mathbb{R}^d$, the columns of a matrix $\mathbf{X}$, and a state $\mathbf{s} \in \mathbb{R}^d$, the energy is

$$E = -\frac{1}{\beta} \ln \sum_{\mu=1}^{p} e^{\beta\, \mathbf{x}^\mu \cdot \mathbf{s}} + \frac{1}{2}\, \mathbf{s} \cdot \mathbf{s} + \text{const}$$

Compare Eq. (5): the classical energy sums squared overlaps; this one sums exponentials and takes a logarithm, and the quadratic term keeps the state bounded. More strikingly, the update rule of the modern Hopfield network, which converges to stationary points of this energy and, in their analysis, retrieves a well-separated pattern after one update, is:

$$\mathbf{s}^{\text{new}} = \mathbf{X}\, \text{softmax}\!\left(\beta \mathbf{X}^\top \mathbf{s}\right)$$

where $\mathbf{X}$ stores patterns and $\mathbf{s}$ is a query. Under the paper’s parameterisation this update matches a form of attention: map the patterns and the query through learned key and query matrices, set $\beta = 1/\sqrt{d_k}$, and it is the softmax attention of transformers. Standard multi-head attention includes learned projections, values, masks, residual paths and other components; a transformer as a whole is not therefore proved to minimise one Hopfield energy.

I do not want to overstate this. The connection is formal and the interpretation is contested. But it is not nothing. The physicist who built the Hopfield network in 1982 was not working on the object that now processes language, images, and protein sequences at industrial scale. He was working on its quadratic ancestor, and in 2016, with Krotov, took the first step away from the square. Replace the square in Eq. (5) by a higher power or an exponential, let the states be continuous, and one update of the result is an attention step.

The protein folding connection

The 2024 Nobel Prize in Chemistry went to Demis Hassabis, John Jumper, and David Baker for computational protein design (Baker) and protein structure prediction (Hassabis and Jumper) — the latter specifically for AlphaFold2 (Jumper et al., 2021). This made October 2024 a remarkable month for Nobel Prizes in fields adjacent to artificial intelligence, and it is not a coincidence.

Protein folding also uses energy-landscape language, including ruggedness and funnels, but a protein is not simply an SK spin glass. Sequences can populate ensembles, have disordered regions or multiple functional conformations; solvent, kinetics and free energy matter. “Native structure equals one global minimum” is an approximation, not a universal law.

Levinthal’s paradox, formulated in 1969, makes the absurdity quantitative. A modest protein of 100 amino acids might have $3^{100} \approx 5 \times 10^{47}$ possible conformations (allowing three dihedral angle states per residue). Random search of this space, at the rate of one conformation per picosecond, would take about $10^{28}$ years — some $10^{18}$ times the age of the universe. Yet proteins fold in milliseconds to seconds. They do not search randomly; the energy landscape is funnel-shaped, channelling the dynamics toward the native state. But predicting which state is the native one from sequence alone remained one of the hard problems of structural biology for fifty years.

AlphaFold2 combines Evoformer blocks, built partly from attention, with a geometric structure module, and was trained on known structures and sequence information. It does not simulate folding dynamics, but neither does that prove it learned or minimises a physical folding free energy. Calling it a descendant of Boltzmann machines is a retrospective narrative, not a documented exclusive architecture genealogy.

The prizes make a compelling thematic sequence for this essay: disordered systems, neural computation and protein prediction. The later achievements are not mathematical consequences of the earlier Nobel work, and the Chemistry prize also included David Baker’s computational protein design.

The controversy: did the committee err?

I said I understand the irritation. Here is what is right about it.

Hinton’s work after the Boltzmann machine — backpropagation, dropout, convolutional networks, deep learning at ImageNet scale — is primarily engineering and empirical machine learning. The 2012 AlexNet result that restarted the field was not a theoretical physics contribution; it was a demonstration that known methods work very well on very large datasets with very large GPUs. The fact that it works is not explained by statistical mechanics. The scaling laws of neural networks (loss scales as a power law with compute, parameters, and data) are empirical observations that physicists have tried to explain with renormalisation group arguments with mixed success.

If the Nobel Prize in Physics were awarded for “the work that most influenced technology in the past decade,” the case for Hinton is strong. If it were awarded for “the most important contribution to the science of physics,” the case is weaker. There is a version of the Nobel announcement that emphasises the Boltzmann machine specifically — the 1985 paper that is literally named after a physicist and uses his distribution — and that version sits cleanly within physics. There is a broader version that encompasses all of Hinton’s career, and that version includes a great deal of empirical machine learning that the physics community is reasonably reluctant to claim. The Academy’s press release is mostly the first version: it describes the Hopfield network and the Boltzmann machine by name and only then says that Hinton “has built upon this work, helping initiate the current explosive development of machine learning” (Royal Swedish Academy of Sciences, 2024). The one-line citation reads more like the second.

My view, for what it is worth from someone who has been thinking about AI ethics and consequences for rather longer than feels comfortable: the Nobel correctly identifies that the foundational conceptual contributions — the Ising Hamiltonian as associative memory, the Boltzmann distribution as a learning target, the connection between statistical mechanics and computation — are physics. They came from physicists, they use physics mathematics, they extend physics intuition into a new domain. The subsequent scaling of these ideas using TPUs and transformer architectures is engineering. Valuable engineering, world-changing engineering, but engineering. The Nobel is for the former. If the citation had been more specific — “for the Boltzmann machine and its demonstration that physical principles govern neural computation” — the physics community would have been less irritated and equally correct.

What the irritation reveals is something slightly uncomfortable about disciplinary identity. Physicists are proud of universality, in the loose sense of the word: the idea that the same mathematical structures appear in wildly different physical systems. RSB in spin glasses, replica methods in random matrices, the Parisi–Sourlas correspondence between disordered systems and supersymmetric field theories — the joy of physics is precisely that these deep structural similarities cross domain boundaries. When that habit of mind reaches into machine learning and says “your transformer attention layer is a Hopfield retrieval step,” under the right parameterisation, physicists should be delighted, not affronted.

The agentic systems that are being built right now on top of transformer architectures are doing something that looks, from a sufficiently abstract distance, like what the Hopfield network was designed to do: find stored patterns that match a query, and use them to generate a response. An earlier version of this paragraph called the failures of grounding that I have written about elsewhere failures of the energy landscape: the model finds a metastable state that is not the correct minimum, and the dynamics cannot escape. That was an analogy dressed as a mechanism, for two reasons.

First, the sentence borrowed “metastable” with the wrong meaning. In Ramsauer et al.’s analysis a metastable state is a fixed point of one attention head that averages several stored patterns, not a trap, and they found the heads of pre-trained BERT models predominantly in such states. Metastability in that sense is how trained attention normally works.

Second, a transformer applies each layer once, to keys recomputed from the context, rather than descending one energy I could point to, and I know of no demonstration that a grounding failure is a trapped state of a defined energy. Spin glass physics does not explain these failures; it gives a language for thinking about them. That is what physics is for.

The universality argument

Let me make the deeper claim explicit. Why should disordered magnets, associative memory networks, and protein folding all live in the same mathematical family?

They share useful motifs: many interacting degrees of freedom, competing constraints and large configuration spaces. Shared motifs do not establish a renormalisation-group universality class; that requires specified scaling, symmetries, dimensionality and critical behaviour.

Fig. 2 sorts the links this essay has drawn by kind.

Figure 2. Identities of form are the minority, and each joins energies with different couplings or, for attention, holds only under Ramsauer et al.’s parameterisation; the other links carry a method across, record historical influence, or are analogies and retrospective narratives (highlighted). Nodes and links are the claims made in this essay, not a complete history.

Parisi supplied the mean-field SK solution. Spin-glass methods transfer fruitfully to related random optimisation and neural models, but that transfer is neither one exact solution for all systems nor a proof of common universality.

The Kuramoto model also has mean-field limits and collective transitions, but that does not make ordinary Kuramoto synchronisation an SK/Hopfield universality-class member. The analogy stops at shared tools unless a specific disordered oscillator model is analysed.

This framework is exact for specified energy-based models and suggestive for others. A software neural network has a physical implementation, but a generic transformer inference is not defined by a thermodynamic Hamiltonian; its failures are not automatically phase transitions and its successes are not automatically energy minimisation. Statistical mechanics is one analytical language, not the unique literal description of all AI.

The Nobel committee noticed. They were right to notice.


The 2021 and 2024 Nobel Prizes in Physics have now officially bridged the gap between condensed matter physics and machine learning in the public record. For anyone who wants to understand either field more deeply than the press releases suggest, the SK model and the Hopfield network are the right place to start. The papers are short by modern standards — Sherrington and Kirkpatrick’s letter is five pages, Hopfield’s 1982 PNAS paper five, and Parisi’s 1979 letter three — and all three repay close reading.

References

  • Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9(1), 147–169. DOI: 10.1207/s15516709cog0901_7

  • Amit, D. J., Gutfreund, H., & Sompolinsky, H. (1985). Storing infinite numbers of patterns in a spin-glass model of neural networks. Physical Review Letters, 55(14), 1530–1533. DOI: 10.1103/PhysRevLett.55.1530

  • Demircigil, M., Heusel, J., Löwe, M., Upgang, S., & Vermet, F. (2017). On a model of associative memory with huge storage capacity. Journal of Statistical Physics, 168(2), 288–299. DOI: 10.1007/s10955-017-1806-y

  • Guerra, F. (2003). Broken replica symmetry bounds in the mean field spin glass model. Communications in Mathematical Physics, 233(1), 1–12. DOI: 10.1007/s00220-002-0773-5

  • Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771–1800. DOI: 10.1162/089976602760128018

  • Hinton, G. E. (2025). Nobel Lecture: Boltzmann machines. Reviews of Modern Physics, 97(3), 030502. DOI: 10.1103/revmodphys.97.030502

  • Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527–1554. DOI: 10.1162/neco.2006.18.7.1527

  • Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8), 2554–2558. DOI: 10.1073/pnas.79.8.2554

  • Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., … Hassabis, D. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583–589. DOI: 10.1038/s41586-021-03819-2

  • Krotov, D., & Hopfield, J. J. (2016). Dense associative memory for pattern recognition. In Advances in Neural Information Processing Systems 29 (pp. 1172–1180). arXiv:1606.01164

  • Nobel Committee for Physics. (2021). Scientific background on the Nobel Prize in Physics 2021: “For groundbreaking contributions to our understanding of complex physical systems”. The Royal Swedish Academy of Sciences. Retrieved from https://www.nobelprize.org/uploads/2021/10/sciback_fy_en_21.pdf

  • Nobel Committee for Physics. (2024). Scientific background to the Nobel Prize in Physics 2024: “For foundational discoveries and inventions that enable machine learning with artificial neural networks”. The Royal Swedish Academy of Sciences. Retrieved from https://www.nobelprize.org/prizes/physics/2024/advanced-information/

  • Panchenko, D. (2013). The Parisi ultrametricity conjecture. Annals of Mathematics, 177(1), 383–393. DOI: 10.4007/annals.2013.177.1.8

  • Parisi, G. (1979). Infinite number of order parameters for spin-glasses. Physical Review Letters, 43(23), 1754–1756. DOI: 10.1103/PhysRevLett.43.1754

  • Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., Gruber, L., Holzleitner, M., Pavlović, M., Sandve, G. K., Greiff, V., Kreil, D., Kopp, M., Klambauer, G., Brandstetter, J., & Hochreiter, S. (2020). Hopfield networks is all you need. arXiv:2008.02217. Retrieved from https://arxiv.org/abs/2008.02217

  • Royal Swedish Academy of Sciences. (2024, 8 October). The Nobel Prize in Physics 2024 [Press release]. Retrieved from https://www.nobelprize.org/prizes/physics/2024/press-release/

  • Sherrington, D., & Kirkpatrick, S. (1975). Solvable model of a spin-glass. Physical Review Letters, 35(26), 1792–1796. DOI: 10.1103/PhysRevLett.35.1792

  • Steffan, H., & Kühn, R. (1994). Replica symmetry breaking in attractor neural network models. Zeitschrift für Physik B, 95(2), 249–260. DOI: 10.1007/BF01312198. Preprint: arXiv:cond-mat/9404036

  • Talagrand, M. (2006). The Parisi formula. Annals of Mathematics, 163(1), 221–263. DOI: 10.4007/annals.2006.163.221


Changelog

  • 2026-07-11: Separated Ising-form equations from SK-model identity; bounded Hopfield capacity and Boltzmann sampling; corrected the history of rigorous RSB results; removed a causal RBM-to-transformer-to-AlphaFold lineage, a shared universality-class claim, and the claim that generic AI failures are phase transitions.
  • 2026-10-03: Corrected the Levinthal estimate ($3^{100} \approx 5 \times 10^{47}$ conformations at one per picosecond is about $10^{28}$ years, not $10^{35}$); corrected the unnormalised SK energy scaling from $N^2$ to $N^{3/2}$; attributed the 2024 Chemistry prize correctly (Baker for design, Hassabis and Jumper for structure prediction); gave Hopfield’s 1982 affiliation as Caltech and Bell Labs; and reworded the sentence that equated Parisi’s solution with the replica trick. Replaced the unsupported “Cognitive Science summer school in Pittsfield in 1983” origin of the Hinton–Sejnowski collaboration with Hinton’s own account (Nobel Lecture): a small workshop in Rochester in summer 1982 where Hopfield presented, followed by their joint work in the second half of 1982.
  • 2026-10-25: Corrected Hopfield’s 1982 variables: his neurons are $V_i \in \{0, 1\}$ with weights $T_{ij}$ and thresholds, not $\pm 1$ spins; added the change of variables $s = 2V - 1$ (which leaves a local field) here and for the Boltzmann machine. Stated what the Hopfield and SK energies share (the pairwise form) and what differs (Hebbian couplings plant the minima, Eq. (5)), and replaced “the same mathematical object” with the actual relation to the modern Hopfield energy, naming the steps between (Krotov & Hopfield, 2016; Demircigil et al., 2017). Added Hopfield’s 0.15 N estimate (for his threshold-zero $\{0, 1\}$ network, not comparable with the $\pm 1$ value), Amit, Gutfreund and Sompolinsky’s $\alpha_c \gtrsim 0.14$ and ground states only below 0.05, the replica-symmetric equations behind 0.138 (Eq. (6); Steffan & Kühn, 1994) and a finite-size simulation (Fig. 1). Moved the sentence on the replica method’s rigour into the replica section and named what Guerra, Talagrand and Panchenko proved. Corrected “each manganese atom … can point in one of two directions”: the Ising spin is the model’s simplification. Softened “every path toward lower energy is blocked by a barrier” to paths that run over barriers. Added the temperature to the Boltzmann unit update and $1/T$ to its gradient (Ackley et al., Eq. 9), and attributed the 1985 paper to all three authors in the heading. Rewrote the paragraph that called grounding failures “failures of the energy landscape”, which contradicted the article’s own disclaimer, as an explicit analogy. Added the press release’s own emphasis on the Hopfield network and the Boltzmann machine, qualified “universality” as the loose sense, and added Fig. 2. Corrected the closing note’s page counts, the title of the 2024 scientific background, and the missing entry for Hinton, Osindero and Teh (2006).