Literature and Nobel records checked through 2026-07-11. Formal equivalence, historical influence, physical implementation and universality are kept separate; the essay does not infer one from another.
On October 8, 2024, the Royal Swedish Academy of Sciences announced that the Nobel Prize in Physics would go to John Hopfield and Geoffrey Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.” Within hours, the physics corner of the internet had an episode. Thermodynamics Twitter — yes, that is a thing — asked whether gradient descent is really physics in the sense that the Higgs mechanism is physics. The condensed matter community, who have been doing disordered systems since before most ML practitioners were born, oscillated between pride (“finally, they noticed us”) and bafflement (“why is Hinton here and not Parisi?”). There were takes. There were dunks. Someone made a graph of Nobel prizes versus average journal impact factor and it was not flattering to this year’s winner.
I understand the irritation. I do not share it.
The argument I can support is stronger than a loose metaphor but weaker than one unbroken identity. Hopfield’s binary symmetric energy has Ising form. Boltzmann machines define a probabilistic model using a Boltzmann distribution and a statistical-mechanical learning derivation. Different couplings, dynamics, objectives and implementations still matter. The path from those models to modern protein prediction mixes formal analogy, historical influence and later engineering; it is not a sequence of mathematical identities.
Let me trace it properly.
The 2021 Nobel: Parisi and the frozen magnet
Before we get to 2024, we need 2021. Giorgio Parisi received half the Nobel Prize in Physics that year for work done between 1979 and 1983 on spin glasses. The other half went to Syukuro Manabe and Klaus Hasselmann for climate modelling — an interesting pairing that provoked its own set of takes, though rather fewer.
A spin glass is a disordered magnetic system. The canonical physical realisation is a dilute alloy: a small concentration of manganese atoms dissolved in copper. Each manganese atom carries a magnetic moment — a spin — that can point in one of two directions, which we label $\sigma_i \in \{-1, +1\}$. The spins interact with each other via exchange interactions mediated by the conduction electrons. The crucial feature is that these interactions are random: some spin pairs prefer to align (ferromagnetic coupling, $J_{ij} > 0$) and others prefer to anti-align (antiferromagnetic coupling, $J_{ij} < 0$), and there is no spatial pattern to which is which.
The Hamiltonian of the system is
$$H = -\sum_{i < j} J_{ij} \sigma_i \sigma_j$$where the $J_{ij}$ are random variables drawn from some distribution. In the Sherrington-Kirkpatrick (SK) model (Sherrington & Kirkpatrick, 1975), all $N$ spins interact with all other spins — a mean-field model — and the couplings are drawn from a Gaussian distribution with mean zero and variance $J^2/N$:
$$J_{ij} \sim \mathcal{N}\!\left(0,\, \frac{J^2}{N}\right)$$The factor of $1/N$ is essential for extensivity: without it, the energy would scale as $N^2$ rather than $N$, which is unphysical.
Now here is the key phenomenon. At high temperature, the spins fluctuate freely and the system is paramagnetic. Cool it below the glass transition temperature $T_g$, and the system “freezes” — but not into a ferromagnet with all spins aligned, and not into a simple antiferromagnet. It freezes into one of an astronomically large number of disordered, metastable states. The system is not in its true ground state; it is trapped. It cannot find its way down because the energy landscape is rugged: every path toward lower energy is blocked by a barrier.
In mean-field descriptions the rugged landscape has many metastable states and history-dependent dynamics. Which barriers scale how, and how this picture maps to a finite-dimensional alloy, depend on the model; “glass” is not named from one memory property alone.
Computing thermodynamic quantities in this system requires averaging over the disorder (the random $J_{ij}$), which means computing the quenched average of the free energy:
$$\overline{F} = -T\, \overline{\ln Z}$$The overline denotes an average over the distribution of couplings. The problem is that $\ln Z$ is hard to average because $Z$ is a sum of exponentially many terms. Parisi’s solution — the replica trick — is a mathematical device worth describing, because it is beautifully strange.
The formal replica step uses $\ln Z = \lim_{n \to 0}(Z^n-1)/n$: calculate $\overline{Z^n}$ for positive integers, then analytically continue toward zero. That continuation is the heuristic step, not an automatic consequence of the integer calculation. The resulting overlap parameters compare replicas.
The naive assumption is replica symmetry: all $q^{ab}$ are equal. This assumption turns out to be wrong. Parisi showed that the correct solution breaks replica symmetry in a hierarchical way — the overlap matrix $q^{ab}$ has a nested structure, described by a function $q(x)$ for $x \in [0,1]$. This is replica symmetry breaking (RSB).
Full RSB has an ultrametric, hierarchical interpretation in the mean-field SK model. Parisi proposed the solution; later rigorous work by Guerra, Talagrand, Panchenko and others established the Parisi formula and major structural results. It is incorrect to attribute all later proof to Parisi alone.
RSB gives exact mean-field predictions and a powerful language for experiment. Finite-dimensional alloys are not thereby proven identical to the full SK hierarchy.
Three years after Parisi solved the SK model, a physicist at Bell Labs wrote a paper about memory.
Hopfield (1982): memory as energy minimisation
John Hopfield was a condensed matter physicist who had drifted toward biophysics — electron transfer in proteins, neural computation. In 1982 he published a paper in PNAS with the title “Neural networks and physical systems with emergent collective computational abilities” (Hopfield, 1982). Most biologists read it as a neuroscience paper. It is a statistical mechanics paper.
Hopfield defined a network of $N$ binary “neurons” $s_i \in \{-1, +1\}$ with symmetric weights $W_{ij} = W_{ji}$, and an energy function:
$$E = -\frac{1}{2} \sum_{i \neq j} W_{ij}\, s_i s_j$$The Hopfield energy has the same pairwise Ising form after identifying neuron states with spins and weights with couplings. The coupling ensemble and purpose differ: Hebbian weights are structured by stored patterns, whereas SK couplings are drawn from a specified random ensemble.
The dynamics: at each step, choose a neuron $i$ at random and update it according to
$$s_i \leftarrow \text{sgn}\!\left(\sum_{j} W_{ij} s_j\right)$$With asynchronous updates, symmetric weights and usual diagonal/tie conventions, the Lyapunov energy does not increase and the finite network reaches a fixed point. “Gradient descent” is shorthand, not a differentiable gradient step.
The innovation is in how Hopfield chose the weights. To store a set of $p$ binary patterns $\xi^\mu \in \{-1,+1\}^N$ (for $\mu = 1, \ldots, p$), use Hebb’s rule:
$$W_{ij} = \frac{1}{N} \sum_{\mu=1}^{p} \xi^\mu_i\, \xi^\mu_j$$This is the outer product rule. Each stored pattern contributes a rank-1 matrix to $W$. You can verify that if $s = \xi^\mu$, then the local field at neuron $i$ is
$$h_i = \sum_j W_{ij} s_j = \frac{1}{N}\sum_j \sum_{\nu} \xi^\nu_i \xi^\nu_j \xi^\mu_j = \xi^\mu_i + \frac{1}{N}\sum_{\nu \neq \mu} \xi^\nu_i \underbrace{\left(\sum_j \xi^\nu_j \xi^\mu_j\right)}_{\text{cross-talk}}$$The first term reinforces pattern $\mu$; the rest is cross-talk. For random unbiased patterns at sufficiently low load, retrieval states can be stable. A corrupted cue may converge to a stored attractor, but not necessarily the nearest pattern under every metric or initial condition.
For the standard Hebbian model with random unbiased patterns in the thermodynamic limit, analyses place the zero-temperature retrieval threshold near $p/N\approx0.138$. It is not universal for correlated data, finite networks or other learning rules, and spurious attractors also occur below it.
Amit, Gutfreund and Sompolinsky analysed the Hopfield model using spin-glass methods (1985), including retrieval and glassy regimes. Its pattern-correlated Hebbian couplings are not simply an SK Gaussian ensemble with planted minima, and the two phase diagrams should not be called identical.
The 2021 and 2024 prizes concern related statistical-mechanical ideas, not the same model or theorem.
Boltzmann machines (Hinton & Sejnowski, 1985)
The Hopfield model is deterministic and shallow — one layer of visible neurons, no hidden structure. Geoffrey Hinton and Terry Sejnowski, in a collaboration that began at the Cognitive Science summer school in Pittsfield in 1983 and culminated in a 1985 paper (Ackley, Hinton, & Sejnowski, 1985), added two things: hidden units and stochastic dynamics.
Hidden units can represent latent structure. The bipartite energy written below is the later restricted Boltzmann machine form; the original 1985 Boltzmann machine allowed more general symmetric connections:
$$E(\mathbf{v}, \mathbf{h}) = -\sum_{i,j} W_{ij}\, v_i h_j - \sum_i a_i v_i - \sum_j b_j h_j$$where $v_i$ are the visible (data) units, $h_j$ are the hidden units, $a_i$ and $b_j$ are biases. Note that this is still an Ising-type energy; the $W_{ij}$ are now inter-layer weights.
The stochastic dynamics replace deterministic gradient descent with a Markov chain. Each unit is updated probabilistically:
$$P(s_k = 1 \mid \text{rest}) = \sigma\!\left(\sum_j W_{kj} s_j + \text{bias}_k\right)$$where $\sigma(x) = 1/(1 + e^{-x})$ is the logistic sigmoid. At inverse temperature $\beta = 1/T$, the probability of any complete configuration is
$$P(\mathbf{v}, \mathbf{h}) = \frac{1}{Z}\, e^{-\beta E(\mathbf{v}, \mathbf{h})}$$This is the Boltzmann distribution. The machine is named after Ludwig Boltzmann because the equilibrium distribution of its states is the Boltzmann distribution. Not analogously. Literally.
Learning amounts to adjusting the weights to make the model distribution $P(\mathbf{v}, \mathbf{h})$ match the data distribution $P_{\text{data}}(\mathbf{v})$. The objective is to minimise the Kullback-Leibler divergence:
$$\mathcal{L} = D_{\mathrm{KL}}(P_{\text{data}} \| P_{\text{model}}) = \sum_{\mathbf{v}} P_{\text{data}}(\mathbf{v}) \ln \frac{P_{\text{data}}(\mathbf{v})}{P_{\text{model}}(\mathbf{v})}$$The gradient with respect to the weight $W_{ij}$ is
$$\frac{\partial \mathcal{L}}{\partial W_{ij}} = -\langle v_i h_j \rangle_{\text{data}} + \langle v_i h_j \rangle_{\text{model}}$$The first term is the empirical correlation between visible unit $i$ and hidden unit $j$ when the visible units are clamped to data. The second term is the correlation in the model’s free-running equilibrium. The learning rule says: increase $W_{ij}$ if the data sees these two units co-active more than the model does, and decrease it otherwise. This is Hebbian learning with a contrastive correction — the physics of equilibration drives the learning.
The model expectation is generally intractable exactly, and Markov-chain mixing can be very slow; it is not proven exponentially slow for every instance. Contrastive divergence replaces equilibrium sampling with a short chain and therefore gives a generally biased approximation to the likelihood gradient.
That approximation is not unusual in physics, but “works” is problem-dependent. The replica derivation was historically non-rigorous; later rigorous results for the Parisi formula should not be erased by calling it merely a lucky guess.
From Boltzmann machines to transformers
The Boltzmann machine was computationally difficult but conceptually foundational. The restricted Boltzmann machine (RBM) — with no within-layer connections, so that hidden units are conditionally independent given the visible units and vice versa — made training via contrastive divergence practical.
Hinton, Osindero and Teh (2006) showed effective greedy pretraining of deep belief networks with RBMs. It was one influential step in deep learning’s revival alongside backpropagation, larger datasets, compute, convolutional networks and other work. The counterfactual that modern language models could not have emerged without RBMs is not established.
The connection between Hopfield networks and modern attention mechanisms is more recent and more surprising. Ramsauer et al. (2020) showed that modern Hopfield networks — a generalisation of the original with continuous states and a different energy function — have exponential storage capacity (Ramsauer et al., 2020). More strikingly, the update rule of the modern Hopfield network is:
$$\mathbf{s}^{\text{new}} = \mathbf{X}\, \text{softmax}\!\left(\beta \mathbf{X}^\top \mathbf{s}\right)$$where $\mathbf{X}$ stores patterns and $\mathbf{s}$ is a query. Under the paper’s parameterisation this update matches a form of attention. Standard multi-head attention includes learned projections, values, masks, residual paths and other components; a transformer as a whole is not therefore proved to minimise one Hopfield energy.
I do not want to overstate this. The connection is formal and the interpretation is contested. But it is not nothing. The physicists who built the Hopfield network in 1982 were working on the same mathematical object that is now used to process language, images, and protein sequences at industrial scale.
The protein folding connection
The 2024 Nobel Prize in Chemistry went to Demis Hassabis, John Jumper, and David Baker for computational protein structure prediction — specifically for AlphaFold2 (Jumper et al., 2021). This made October 2024 a remarkable month for Nobel Prizes in fields adjacent to artificial intelligence, and it is not a coincidence.
Protein folding also uses energy-landscape language, including ruggedness and funnels, but a protein is not simply an SK spin glass. Sequences can populate ensembles, have disordered regions or multiple functional conformations; solvent, kinetics and free energy matter. “Native structure equals one global minimum” is an approximation, not a universal law.
Levinthal’s paradox, formulated in 1969, makes the absurdity quantitative. A modest protein of 100 amino acids might have $3^{100} \approx 10^{47}$ possible conformations (allowing three dihedral angle states per residue). Random search of this space, at the rate of one conformation per picosecond, would take $10^{35}$ years — somewhat longer than the age of the universe. Yet proteins fold in milliseconds to seconds. They do not search randomly; the energy landscape is funnel-shaped, channelling the dynamics toward the native state. But predicting which state is the native one from sequence alone remained one of the hard problems of structural biology for fifty years.
AlphaFold2 combines attention-based Evoformer blocks with geometric structure modules and was trained on known structures and sequence information. It does not simulate folding dynamics, but neither does that prove it learned or minimises a physical folding free energy. Calling it a descendant of Boltzmann machines is a retrospective narrative, not a documented exclusive architecture genealogy.
The prizes make a compelling thematic sequence for this essay: disordered systems, neural computation and protein prediction. The later achievements are not mathematical consequences of the earlier Nobel work, and the Chemistry prize also included David Baker’s computational protein design.
The controversy: did the committee err?
I said I understand the irritation. Here is what is right about it.
Hinton’s work after the Boltzmann machine — backpropagation, dropout, convolutional networks, deep learning at ImageNet scale — is primarily engineering and empirical machine learning. The 2012 AlexNet result that restarted the field was not a theoretical physics contribution; it was a demonstration that known methods work very well on very large datasets with very large GPUs. The fact that it works is not explained by statistical mechanics. The scaling laws of neural networks (loss scales as a power law with compute, parameters, and data) are empirical observations that physicists have tried to explain with renormalisation group arguments with mixed success.
If the Nobel Prize in Physics were awarded for “the work that most influenced technology in the past decade,” the case for Hinton is strong. If it were awarded for “the most important contribution to the science of physics,” the case is weaker. There is a version of the Nobel announcement that emphasises the Boltzmann machine specifically — the 1985 paper that is literally named after a physicist and uses his distribution — and that version sits cleanly within physics. There is a broader version that encompasses all of Hinton’s career, and that version includes a great deal of empirical machine learning that the physics community is reasonably reluctant to claim.
My view, for what it is worth from someone who has been thinking about AI ethics and consequences for rather longer than feels comfortable: the Nobel correctly identifies that the foundational conceptual contributions — the Ising Hamiltonian as associative memory, the Boltzmann distribution as a learning target, the connection between statistical mechanics and computation — are physics. They came from physicists, they use physics mathematics, they extend physics intuition into a new domain. The subsequent scaling of these ideas using TPUs and transformer architectures is engineering. Valuable engineering, world-changing engineering, but engineering. The Nobel is for the former. If the citation had been more specific — “for the Boltzmann machine and its demonstration that physical principles govern neural computation” — the physics community would have been less irritated and equally correct.
What the irritation reveals is something slightly uncomfortable about disciplinary identity. Physicists are proud of universality: the idea that the same mathematical structures appear in wildly different physical systems. RSB in spin glasses, replica methods in random matrices, the Parisi–Sourlas correspondence between disordered systems and supersymmetric field theories — the joy of physics is precisely that these deep structural similarities cross domain boundaries. When that universality reaches into machine learning and says “your transformer attention layer is a Hopfield retrieval step,” physicists should be delighted, not affronted.
The agentic systems that are being built right now on top of transformer architectures are doing something that looks, from a sufficiently abstract distance, like what the Hopfield network was designed to do: find stored patterns that match a query, and use them to generate a response. The failures of grounding that I have written about elsewhere are, in this view, failures of the energy landscape — the model finds a metastable state that is not the correct minimum, and the dynamics cannot escape. Spin glass physics does not explain these failures in detail, but it gives a language for thinking about them. That is what physics is for.
The universality argument
Let me make the deeper claim explicit. Why should disordered magnets, associative memory networks, and protein folding all live in the same mathematical family?
They share useful motifs: many interacting degrees of freedom, competing constraints and large configuration spaces. Shared motifs do not establish a renormalisation-group universality class; that requires specified scaling, symmetries, dimensionality and critical behaviour.
Parisi supplied the mean-field SK solution. Spin-glass methods transfer fruitfully to related random optimisation and neural models, but that transfer is neither one exact solution for all systems nor a proof of common universality.
The Kuramoto model also has mean-field limits and collective transitions, but that does not make ordinary Kuramoto synchronisation an SK/Hopfield universality-class member. The analogy stops at shared tools unless a specific disordered oscillator model is analysed.
This framework is exact for specified energy-based models and suggestive for others. A software neural network has a physical implementation, but a generic transformer inference is not defined by a thermodynamic Hamiltonian; its failures are not automatically phase transitions and its successes are not automatically energy minimisation. Statistical mechanics is one analytical language, not the unique literal description of all AI.
The Nobel committee noticed. They were right to notice.
The 2021 and 2024 Nobel Prizes in Physics have now officially bridged the gap between condensed matter physics and machine learning in the public record. For anyone who wants to understand either field more deeply than the press releases suggest, the SK model and the Hopfield network are the right place to start. Both papers are short by modern standards — Parisi’s 1979 letter is three pages; Hopfield’s 1982 PNAS paper is five — and both repay close reading.
References
Sherrington, D., & Kirkpatrick, S. (1975). Solvable model of a spin-glass. Physical Review Letters, 35(26), 1792–1796. DOI: 10.1103/PhysRevLett.35.1792
Parisi, G. (1979). Infinite number of order parameters for spin-glasses. Physical Review Letters, 43(23), 1754–1756. DOI: 10.1103/PhysRevLett.43.1754
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8), 2554–2558. DOI: 10.1073/pnas.79.8.2554
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9(1), 147–169. DOI: 10.1207/s15516709cog0901_7
Amit, D. J., Gutfreund, H., & Sompolinsky, H. (1985). Storing infinite numbers of patterns in a spin-glass model of neural networks. Physical Review Letters, 55(14), 1530–1533. DOI: 10.1103/PhysRevLett.55.1530
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., … Hassabis, D. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583–589. DOI: 10.1038/s41586-021-03819-2
Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., Gruber, L., Holzleitner, M., Pavlović, M., Sandve, G. K., Greiff, V., Kreil, D., Kopp, M., Klambauer, G., Brandstetter, J., & Hochreiter, S. (2020). Hopfield networks is all you need. arXiv:2008.02217. Retrieved from https://arxiv.org/abs/2008.02217
Nobel Prize Committee. (2024). Scientific background: Machine learning and physical systems. The Royal Swedish Academy of Sciences. Retrieved from https://www.nobelprize.org/prizes/physics/2024/advanced-information/
Talagrand, M. (2006). The Parisi formula. Annals of Mathematics, 163(1), 221–263. DOI: 10.4007/annals.2006.163.221
Changelog
- 2026-07-11: Separated Ising-form equations from SK-model identity; bounded Hopfield capacity and Boltzmann sampling; corrected the history of rigorous RSB results; removed a causal RBM-to-transformer-to-AlphaFold lineage, a shared universality-class claim, and the claim that generic AI failures are phase transitions.