Linking to the paper: https://arxiv.org/pdf/2605.01172 which is also a fantastic read, the application to deep learning is good. It does a lot of cross-mapping and highlighting a bunch of old stuff that is named differently in this paper and worth calling out for those with those backgrounds:
"Cumulative Dissipation Gramian" Ws = Observability Gramian (from Control Theory). For example the spectral cutoff is exactly the Hankel singular value truncation from model reduction.
"Signal Channel" / "Reservoir" is Controllable/Observable vs. Uncontrollable/Unobservable Subspaces. Using Adamjan-Arov-Krein (AAK) theory gives the optimal nonlinear reduced model answering the optimal compression question.
"Drift–Diffusion Separation" is Freidlin-Wentzell Large Deviation Theory. They can predict "grokking" time from the FW action.
"Population-Risk Gate" is Quantum Weak Value / Postselection (Aharonov)
So for the follow-up problems
Control theory gives the truncation error bounds for model compression. Large deviation theory gives the grokking time predictions. Quantum measurement theory gives the imaginary preconditioners. Information geometry gives the optimal continuous relaxation of the gate.
Some nice implications of new ways of doing stuff which are nice to see formalized here:
Old: Pick architecture, hope it generalizes
New: Design architecture to maximize observability Gramian rank (Honestly we pull a lot from control theory here)
Old: Use validation set to detect overfitting
New: Monitor λ(Ws) spectrum during training; no validation needed
Old: Prune post-hoc based on magnitude
New: Prune during training based on ker(Ws) membership
This is a beautifully written way of saying “Some parts of what the network memorizes affect test behavior, and some don’t.” But that’s not a theory of deep learning, the grand unified theory would explain that.
We're given a signal channel and a reservoir. Signal lives in the channel, noise lives in the reservoir, and the reservoir supposedly doesn’t show up at test time.
Okay, but then we have: why would SGD put the right things in the right bucket?
If the answer is “because the reservoir is defined as the stuff that doesn’t transfer to test,” then this is close to circular.
The Borges/Lavoisier stuff is a tell. "We have unified the field” rhetoric should come after nontrivial predictions and results. Claiming to solve benign overfitting, double descent, grokking, implicit bias, risk of training on population, how to avoid a validation set, and last but not least, skipping training by analytically jumping to the end is 6 theory papers, 3 NeurIPS winners, and a $10B startup. Let's get some results before we tell everyone we unified the field. :) I hope you're right.
> why would SGD put the right things in the right bucket?
Think of it as a best fit curve and exceptions to that curve. The noise is essentially this set of exceptions that move points away from where they would otherwise fall on the curve.
Gradient descent wants to be able to make the smallest change that moves the most data points towards the curve. To do this it learns an arrangement where it can change, say, one parameter and have a bunch of points move at once. What does this correspond to? The big common patterns shared by many data points.
Most of the capacity gets soaked up modelling these sorts of common patterns, and after they have been learned the model starts adding exceptions that allow individual points to deviate from the curve.
Because they’re exceptions, they must not impact neighbouring points, or at least only ones within a very short distance from them. Otherwise they’re now driving the error higher by impacting more points than they should. So you end up with very narrow ranges of features that are able to trigger different sorts of noise.
How narrow they are is shaped by the training data, they’re exactly as narrow as needed not to raise the error, so assuming the total population has the same distribution, they don’t get hit. Much.
Admittedly probably some aggrandized boasting here, but I think empirical verification of that Adam modification alone would be a meaningful contribution, unless that's prior work?
A theory that skips the parameter space, and understands grokking theory, comes up with an unexplained update rule, which notably works on a per-parameter level by dropping the updates for most parameters.
I suspect there is going to be a lot of handwaving to actually go from eNTK to that new update rule.
I also doubt it helps in the non-grokking regime, given the focus of the theory, which is where all the practical applications I have ever heard from live.
Don't get me wrong, I did enjoy reading this essay. It's well written and reasonably argumented without going into details.
The handwaving required is just to assume a diagonal preconditioner, and the optimal preconditioner under that constraint corresponds to the new update rule. (See section F of the paper.) And of course a diagonal preconditioner works on the per-paramer level.
These are the same complaints I had. Also felt like it was high quality ai writing, possibly because of the style choices like "Benign overfitting is noise sitting in the reservoir at interpolation. XYZ is ..." and because of the similarity it has to the times I ended up with chatgpt or gemini creating very detailed and plausible reports about my own crackpot or vague-enough-to-be-useless ideas.
If that's the case, a way to test the theory and understanding (assuming some parts of reservoir and signal channel can be reliably identified) would be to prune the high-confidence reservoir significantly reducing the model size while still getting good predictions. I don't believe the authors mention this (though I skimmed and didn't read the full paper in detail so I may be wrong)
Nah, the softer stuff seems like valuable outreach / good science communication for people that aren't up for the math. Including probably lots of software engineers who are sick of dumb debates in forums, and starting to dip into the real literature and listen to better authorities. More people should do this really, since it's the only way to see past the marketing and hype from fully entrenched AI boosters or detractors. Neither of those groups is big on critical thinking, and they dominate most conversation.
Time/effort coming from experts who want to make things accessible is a gift! The paper is linked elsewhere in the thread if you want no-frills.
I don't know the math, but this point was clear to me and it screamed, "crank" but not being sure of that because I am not learned enough to understand the math... but even I could tell the magnitude of the claim. Even just the removing the need for validation sets would have epic consequences across many fields.
I interpreted the kernel K of this paper as the BRDF in Rendering Equation[0] and its familiar diffusion process (from light transport simulation, or really any integro-differential equation system); together with https://en.wikipedia.org/wiki/Neural_tangent_kernel I hope this paper might be accessible with some study
Idk to me this is just redescribing what deep neural networks do without actually explaining why anything happens. I guess it "unifies" things but I am kinda over most unifying theories. Everything is Bayesian, everything is a graph or a group or some other fancy geometric structure, everything is a category. Ultimately the best framework is whatever is useful enough to explain what's happening in such a way that a practitioner can manipulate the model towards a desired outcome. In other words, where is the knob? The tool they share may be interesting and I hope to play with it to see what happens at different levels of noise applied to the labels.
Lastly, we derive an exact population-risk objective from a single training run with no validation data, for any architecture, loss, or optimizer, and prove that it measures precisely the noise in the signal channel. This objective reduces in practice to an SNR preconditioner on top of Adam, adding one state vector at no extra cost; it accelerates grokking by 5x, suppresses memorization in PINNs and implicit neural representations, and improves DPO fine-tuning under noisy preferences while staying 3x closer to the reference policy. [1]
This is a fascinating mathematical framework, but the post title might be a bit of an overreach. I often wonder if "a theory of deep learning" could exist that could be stated succinctly and that could predict (1) scaling laws and (2) the surprising reliability of gradient descent.
Note that I said "predict" not "describe". It feels like we're still in the era of Kepler, not Newton.
I dunno... gradient descent is only really reliable with a big bag of tricks. Knowing good initializations is a starting point, but recurrent connections and batch/layer normalization go a very long way towards making it reliable.
I agree, this is the correct way to see it IMO. Instead of designing better optimizers, we designed easier parameterizations to optimize. The surprising part is that these parameterizations exist in the first place.
Gradient descent is mathematically the most efficient optimization strategy (safe for some special functions) in high dimensions. This goes so far that people nowadays even believe it has to be used in the human brain [1], if only because every other method of updating the brain would be way too energy inefficient. From that perspective, finding the right parameterization was all we ever needed to achieve AI.
Even in supervised ML, pure gradient descent is not the most efficient optimization strategy. E.g., momentum is ubiquitous, and the updates it induces cannot be expressed as a gradient of some scalar loss. But the rotational non-gradient component of its updates substantially improves performance and convergence on the architectures we use.
The brain probably primarily uses something like TD for task learning, which is also not expressible as a gradient of any objective function. And, though the paper mentions Hebbian learning, it's only very particular network architectures (e.g. single neuron; symmetric connections) that you can treat its updates as a gradient of some energy function; these architectures aren't anything close to what we see in the brain.
Pure gradient descent is not what happens in either field, but e.g. momentum is just another parameter constructed from historic gradients. While it is unlikely that the brain runs backpropagation the way you see it implemented in modern ML (same goes for TD btw), the core principle kind of needs to be the same from a pure large scale, high dimensional network efficiency POV. On top of that, adaptive plasticity is almost by definition about estimating useful directions of change. The key insight here would be that the brain does gradient estimation quite cheap and we can probably still learn a thing or two about modern ML from it.
Their claim isn't that the brain uses gradient descent, but that the direction of updates has (on average) positive inner product with the gradient. I expect this would also be true for (say) simulated annealing, yet we don't say that simulated annealing is gradient descent.
There's also a discussion of loss functions and how they relate to the update missing - as far as I know, there's still no great notion of how the brain picks a global loss function, and no mechanism for backprop. In this paper, looking at a specific learning task you can define a loss function extrinsically allowing us to talk about the gradient, but how that relates to things happening in the brain is a big big mystery.
My intuitive understanding about double descent is that
1. Older ML models encoded in their architecture and lack of expressivity a bias to simplicity; which aided interpolation.
2. Overparameterized models instead use regularization to nudge parameters to simpler and more robust representations, while still memorizing the noise. In this manner, we still achieve generalization performance OOD. Moreover, the softer nudging and fundamental architectural expressivity allows for "data-specific" generalizations and representations that may be impossible to represent in small models.
3. At the critical point between the two regimes, the model is expressive enough to memorize; but not expressive enough to simultaneously both do that and encode general patterns.
I wonder how this understanding translates to these researchers' models of deep learning.
Interesting read. I remember the grokking paper when it came out but I don't think I've ever seen that classic grokking loss curve in my own hands on real data. Curious if others have seen it more often in practice
To get pure grokking, you need a model large enough to easily memorize the entire training data and keep training for a long time after memorization. In practice, you'll probably use a more realistically-sized model that might grok on some subset of the data, but not so strongly that it's extremely obvious.
I think I trained models with #params >> #training examples for hundreds of epochs, but still don't recall seeing that loss curve on real data. Curious if others have seen it with larger models or much longer runs
> This exact characterization is possible because in output space, training dynamics can be understood through a locally linear differential equation along the realized path, where dominant eigenmodes of the evolving kernel equilibrate exponentially fast. Forcing an optimizer to slowly step through these solved directions is highly inefficient and suggests a path to analytically jump to the final network state.
> That is, if the batch signal on a parameter exceeds its leave-one-out noise, update it; if not, skip it. This is a one-line change to Adam that accelerates grokking by 5x, suppresses memorization in PINNs, and improves DPO fine-tuning, eliminating the need for validation sets entirely.
Does anyone understand the formula they expressed above this sentence? is this just the classic "skip updating parameters with high gradient/loss variance in multiple batches/samples" ?
What is classic about "skip updating parameters with high gradient/loss variance in multiple batches/samples"? Do you have a particular algorithm in mind that uses this heuristic?
Theres been multiple papers discussing how only updating parameters that have high agreement in update direction leads to less overfitting and better generalization. Lemme see if I can find em.
https://arxiv.org/abs/2411.16085 - set updates to 0 where theres disagreement in the sign of the parameter update - got accepted!
https://arxiv.org/pdf/2412.18052 - discard gradient updates from batches/minibatches that disagree where disagree means cosine distance threshold (they solved for 0.97 or something being optimal)
this landed precisely on like 3 weird bugs ive been hitting and solving in different stupid ways for dealing with things like sgd collapsing too many good answers into one bad answer, and gave me a real direction to try to fix the link missing in my own ml stuff. what timing. i have tried analytic solutions too and they're useful for like mapping prompts into memory geometry but from there ive ended up still having to use sgd. cause i think what happens is, sgd teaches the neural net both the geometry and how to navigate it. if you just teleport to the answer it doesnt learn how to walk.
Looks like a typical machine learning paper to me. It cannot be understood unless you already kind of understand it. That is OK for communication with peers, but eventually I expect a "theory of" to be readable by anyone with a math degree.
This looks like excellent work, it's reminding me of things I learned from Welch Labs vidoes. Given the amount of time I budget for keeping up with this stuff (regrettably too low) I'll wait until Welch Labs presents something on this.
So, this is either the paper of the year, or ... definitely not the paper of the year.
https://arxiv.org/pdf/2605.01172 is the current version. The money graphs are page 8 and on where they show (some weirdly thick) line charts with loss results reached in roughly 1/5 the number of steps that Adam takes, just what the blog post mentions.
They also claim holding back test data is not needed, also with more graphs.
I'm not an ML scientist, and I did not attempt to seriously parse the math. It reads to me as something precisely in that liminal space some math papers do where there's enough new terminology that actually parsing through it all is going to take real, concerted effort, possibly with mild brain damage as a risk.
Their 3d graphs of "kernel eigenstructure" also do double duty for me as totally impenetrable and possibly part of an April fool's ML paper that's hilarious to insiders. Or maybe they show something really amazing; they definitely seem to converge into a shape...What does that shape mean??? Why??? What is an eigenstructure? Is it just 3D eigenvectors of some matrices? Is it natural to have a 3D shape representing these large matrices? If not, how and why were these projected down? And why are they different colors in the paper?? You get the feel for my level of understanding.
I think it would frankly just be easier to validate this claim than parse the whole paper. If only I could understand
> Each one-step kernel increment ηKMtSS integrates into WMS , so a sequence of one-step rate-maximizers is the greedy policy whose integral is the signal-channel content of the trajectory through G, exactly as plain SGD is the greedy step whose integral is empirical-risk descent through D. The diagonal cutoff µ2 k >σ2 k/(b−1) is the optimal first-order preconditioner for population risk on any diagonal base, and a streaming variance EMAˆst of squared gradient deviations realizes it as a one-line change to AdamW: one extra parameter-sized state vector and a per parameter gate that multiplies the standard moment update
Well enough to implement the one line update to Adam in python. I have not asked codex or claude to assist yet.
Also of note to me, they talk about grokking which I found SUUUPER fascinating when it was first reported, and have never heard about since. So I was really glad to read about it and read that there has been a little academic work on the phenomenon.
Finally, of the three models they repot results on, two are extremely tiny, the last is a DPO round on Qwen 0.5B -- if the code for that is published, I imagine it would be easy to adapt and evaluate in other regimes.
I think it's a solid theoretical contribution, but it might nonetheless fail to have practical relevance if some of their assumptions and approximations turn out to be too unrealistic. One way this could happen, for example, would be if typical training batches get gradients with a high-enough signal-to-noise ratio that their optimizer tweak ends up not tweaking much. Their somewhat unusual selection of experiments makes me suspect that this might be the case.
I read the paper earlier when it showed up on https://news.ycombinator.com/from?site=arxiv.org and the writing style of the blog post turned me off so I didn't bother to check how much it overhypes the results compared to the paper, but certainly a lot of people seem to have gotten the idea that this must be big if true, whereas I think it's better classified as neat, but not revolutionary.
Thanks. I found the DPO test very interesting where they intentionally twiddled the dataset to create 'noise'. That would I think back your concern. But on the other hand a method that allows noisier data in or does better with data sets that are low signal across bands would be nice to have.
Where’s the theory of how the human brain does what it does? Maybe these high dimensional structures don’t have a nice compact “theory”. Trying to fit these systems into a nice compact theory is a very human thing, but not everything works like that.
Linking to the paper: https://arxiv.org/pdf/2605.01172 which is also a fantastic read, the application to deep learning is good. It does a lot of cross-mapping and highlighting a bunch of old stuff that is named differently in this paper and worth calling out for those with those backgrounds:
"Cumulative Dissipation Gramian" Ws = Observability Gramian (from Control Theory). For example the spectral cutoff is exactly the Hankel singular value truncation from model reduction.
"Signal Channel" / "Reservoir" is Controllable/Observable vs. Uncontrollable/Unobservable Subspaces. Using Adamjan-Arov-Krein (AAK) theory gives the optimal nonlinear reduced model answering the optimal compression question.
"Drift–Diffusion Separation" is Freidlin-Wentzell Large Deviation Theory. They can predict "grokking" time from the FW action.
"Population-Risk Gate" is Quantum Weak Value / Postselection (Aharonov)
So for the follow-up problems
Control theory gives the truncation error bounds for model compression. Large deviation theory gives the grokking time predictions. Quantum measurement theory gives the imaginary preconditioners. Information geometry gives the optimal continuous relaxation of the gate.
Some nice implications of new ways of doing stuff which are nice to see formalized here:
Old: Pick architecture, hope it generalizes New: Design architecture to maximize observability Gramian rank (Honestly we pull a lot from control theory here)
Old: Use validation set to detect overfitting New: Monitor λ(Ws) spectrum during training; no validation needed
Old: Prune post-hoc based on magnitude New: Prune during training based on ker(Ws) membership
Old: Fixed learning rate New: Spectral learning rate
> We present a non-asymptotic theory of generalization
what is a non-elephant animal (to paraphrase stan ulam)?
This is a beautifully written way of saying “Some parts of what the network memorizes affect test behavior, and some don’t.” But that’s not a theory of deep learning, the grand unified theory would explain that.
We're given a signal channel and a reservoir. Signal lives in the channel, noise lives in the reservoir, and the reservoir supposedly doesn’t show up at test time.
Okay, but then we have: why would SGD put the right things in the right bucket?
If the answer is “because the reservoir is defined as the stuff that doesn’t transfer to test,” then this is close to circular.
The Borges/Lavoisier stuff is a tell. "We have unified the field” rhetoric should come after nontrivial predictions and results. Claiming to solve benign overfitting, double descent, grokking, implicit bias, risk of training on population, how to avoid a validation set, and last but not least, skipping training by analytically jumping to the end is 6 theory papers, 3 NeurIPS winners, and a $10B startup. Let's get some results before we tell everyone we unified the field. :) I hope you're right.
> why would SGD put the right things in the right bucket?
Think of it as a best fit curve and exceptions to that curve. The noise is essentially this set of exceptions that move points away from where they would otherwise fall on the curve.
Gradient descent wants to be able to make the smallest change that moves the most data points towards the curve. To do this it learns an arrangement where it can change, say, one parameter and have a bunch of points move at once. What does this correspond to? The big common patterns shared by many data points.
Most of the capacity gets soaked up modelling these sorts of common patterns, and after they have been learned the model starts adding exceptions that allow individual points to deviate from the curve.
Because they’re exceptions, they must not impact neighbouring points, or at least only ones within a very short distance from them. Otherwise they’re now driving the error higher by impacting more points than they should. So you end up with very narrow ranges of features that are able to trigger different sorts of noise.
How narrow they are is shaped by the training data, they’re exactly as narrow as needed not to raise the error, so assuming the total population has the same distribution, they don’t get hit. Much.
At least, that’s what I take away from it.
Admittedly probably some aggrandized boasting here, but I think empirical verification of that Adam modification alone would be a meaningful contribution, unless that's prior work?
A theory that skips the parameter space, and understands grokking theory, comes up with an unexplained update rule, which notably works on a per-parameter level by dropping the updates for most parameters.
I suspect there is going to be a lot of handwaving to actually go from eNTK to that new update rule.
I also doubt it helps in the non-grokking regime, given the focus of the theory, which is where all the practical applications I have ever heard from live.
Don't get me wrong, I did enjoy reading this essay. It's well written and reasonably argumented without going into details.
The handwaving required is just to assume a diagonal preconditioner, and the optimal preconditioner under that constraint corresponds to the new update rule. (See section F of the paper.) And of course a diagonal preconditioner works on the per-paramer level.
These are the same complaints I had. Also felt like it was high quality ai writing, possibly because of the style choices like "Benign overfitting is noise sitting in the reservoir at interpolation. XYZ is ..." and because of the similarity it has to the times I ended up with chatgpt or gemini creating very detailed and plausible reports about my own crackpot or vague-enough-to-be-useless ideas.
If that's the case, a way to test the theory and understanding (assuming some parts of reservoir and signal channel can be reliably identified) would be to prune the high-confidence reservoir significantly reducing the model size while still getting good predictions. I don't believe the authors mention this (though I skimmed and didn't read the full paper in detail so I may be wrong)
I find the landscape perspective very valuable when trying to understand NN. Why SGD finds the right buckets? I am not super current, but this looked like the right trail to me -- https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.12...
> The Borges/Lavoisier stuff is a tell.
Nah, the softer stuff seems like valuable outreach / good science communication for people that aren't up for the math. Including probably lots of software engineers who are sick of dumb debates in forums, and starting to dip into the real literature and listen to better authorities. More people should do this really, since it's the only way to see past the marketing and hype from fully entrenched AI boosters or detractors. Neither of those groups is big on critical thinking, and they dominate most conversation.
Time/effort coming from experts who want to make things accessible is a gift! The paper is linked elsewhere in the thread if you want no-frills.
I don't know the math, but this point was clear to me and it screamed, "crank" but not being sure of that because I am not learned enough to understand the math... but even I could tell the magnitude of the claim. Even just the removing the need for validation sets would have epic consequences across many fields.
The relevant paper: "A Theory of Generalization in Deep Learning". https://arxiv.org/abs/2605.01172
I interpreted the kernel K of this paper as the BRDF in Rendering Equation[0] and its familiar diffusion process (from light transport simulation, or really any integro-differential equation system); together with https://en.wikipedia.org/wiki/Neural_tangent_kernel I hope this paper might be accessible with some study
[0] https://en.wikipedia.org/wiki/Rendering_equation
Idk to me this is just redescribing what deep neural networks do without actually explaining why anything happens. I guess it "unifies" things but I am kinda over most unifying theories. Everything is Bayesian, everything is a graph or a group or some other fancy geometric structure, everything is a category. Ultimately the best framework is whatever is useful enough to explain what's happening in such a way that a practitioner can manipulate the model towards a desired outcome. In other words, where is the knob? The tool they share may be interesting and I hope to play with it to see what happens at different levels of noise applied to the labels.
We're still in the era of room-sized-computers-only-scientists-understand era of the neural networks. Knobs and buttons for nerds are slowly coming.
I agree, which is why it’s too early to make such grandiose claims about deep learning theory.
A real theory would predict phenomena thus far unseen. We already know about this 4 part taxonomy.
Did you also know about this?
Lastly, we derive an exact population-risk objective from a single training run with no validation data, for any architecture, loss, or optimizer, and prove that it measures precisely the noise in the signal channel. This objective reduces in practice to an SNR preconditioner on top of Adam, adding one state vector at no extra cost; it accelerates grokking by 5x, suppresses memorization in PINNs and implicit neural representations, and improves DPO fine-tuning under noisy preferences while staying 3x closer to the reference policy. [1]
[1] https://arxiv.org/abs/2605.01172
[dead]
This is a fascinating mathematical framework, but the post title might be a bit of an overreach. I often wonder if "a theory of deep learning" could exist that could be stated succinctly and that could predict (1) scaling laws and (2) the surprising reliability of gradient descent.
Note that I said "predict" not "describe". It feels like we're still in the era of Kepler, not Newton.
I dunno... gradient descent is only really reliable with a big bag of tricks. Knowing good initializations is a starting point, but recurrent connections and batch/layer normalization go a very long way towards making it reliable.
I agree, this is the correct way to see it IMO. Instead of designing better optimizers, we designed easier parameterizations to optimize. The surprising part is that these parameterizations exist in the first place.
Gradient descent is mathematically the most efficient optimization strategy (safe for some special functions) in high dimensions. This goes so far that people nowadays even believe it has to be used in the human brain [1], if only because every other method of updating the brain would be way too energy inefficient. From that perspective, finding the right parameterization was all we ever needed to achieve AI.
[1] https://physoc.onlinelibrary.wiley.com/doi/full/10.1113/JP28...
Even in supervised ML, pure gradient descent is not the most efficient optimization strategy. E.g., momentum is ubiquitous, and the updates it induces cannot be expressed as a gradient of some scalar loss. But the rotational non-gradient component of its updates substantially improves performance and convergence on the architectures we use.
The brain probably primarily uses something like TD for task learning, which is also not expressible as a gradient of any objective function. And, though the paper mentions Hebbian learning, it's only very particular network architectures (e.g. single neuron; symmetric connections) that you can treat its updates as a gradient of some energy function; these architectures aren't anything close to what we see in the brain.
Pure gradient descent is not what happens in either field, but e.g. momentum is just another parameter constructed from historic gradients. While it is unlikely that the brain runs backpropagation the way you see it implemented in modern ML (same goes for TD btw), the core principle kind of needs to be the same from a pure large scale, high dimensional network efficiency POV. On top of that, adaptive plasticity is almost by definition about estimating useful directions of change. The key insight here would be that the brain does gradient estimation quite cheap and we can probably still learn a thing or two about modern ML from it.
Taking a quick look at the paper...
Their claim isn't that the brain uses gradient descent, but that the direction of updates has (on average) positive inner product with the gradient. I expect this would also be true for (say) simulated annealing, yet we don't say that simulated annealing is gradient descent.
There's also a discussion of loss functions and how they relate to the update missing - as far as I know, there's still no great notion of how the brain picks a global loss function, and no mechanism for backprop. In this paper, looking at a specific learning task you can define a loss function extrinsically allowing us to talk about the gradient, but how that relates to things happening in the brain is a big big mystery.
[flagged]
This essay seems to be related to the paper "There Will Be a Scientific Theory of Deep Learning" [1] which was discussed here recently [2].
[1] https://arxiv.org/pdf/2604.21691
[2] https://news.ycombinator.com/item?id=47893779
My intuitive understanding about double descent is that
1. Older ML models encoded in their architecture and lack of expressivity a bias to simplicity; which aided interpolation.
2. Overparameterized models instead use regularization to nudge parameters to simpler and more robust representations, while still memorizing the noise. In this manner, we still achieve generalization performance OOD. Moreover, the softer nudging and fundamental architectural expressivity allows for "data-specific" generalizations and representations that may be impossible to represent in small models. 3. At the critical point between the two regimes, the model is expressive enough to memorize; but not expressive enough to simultaneously both do that and encode general patterns.
I wonder how this understanding translates to these researchers' models of deep learning.
Interesting read. I remember the grokking paper when it came out but I don't think I've ever seen that classic grokking loss curve in my own hands on real data. Curious if others have seen it more often in practice
To get pure grokking, you need a model large enough to easily memorize the entire training data and keep training for a long time after memorization. In practice, you'll probably use a more realistically-sized model that might grok on some subset of the data, but not so strongly that it's extremely obvious.
I think I trained models with #params >> #training examples for hundreds of epochs, but still don't recall seeing that loss curve on real data. Curious if others have seen it with larger models or much longer runs
> This exact characterization is possible because in output space, training dynamics can be understood through a locally linear differential equation along the realized path, where dominant eigenmodes of the evolving kernel equilibrate exponentially fast. Forcing an optimizer to slowly step through these solved directions is highly inefficient and suggests a path to analytically jump to the final network state.
But at what computational cost?
> That is, if the batch signal on a parameter exceeds its leave-one-out noise, update it; if not, skip it. This is a one-line change to Adam that accelerates grokking by 5x, suppresses memorization in PINNs, and improves DPO fine-tuning, eliminating the need for validation sets entirely.
Does anyone understand the formula they expressed above this sentence? is this just the classic "skip updating parameters with high gradient/loss variance in multiple batches/samples" ?
What is classic about "skip updating parameters with high gradient/loss variance in multiple batches/samples"? Do you have a particular algorithm in mind that uses this heuristic?
Theres been multiple papers discussing how only updating parameters that have high agreement in update direction leads to less overfitting and better generalization. Lemme see if I can find em.
https://arxiv.org/abs/2411.16085 - set updates to 0 where theres disagreement in the sign of the parameter update - got accepted!
https://arxiv.org/pdf/2412.18052 - discard gradient updates from batches/minibatches that disagree where disagree means cosine distance threshold (they solved for 0.97 or something being optimal)
Reminded me strongly of the paper "Deep Learning is Not So Mysterious or Different" from a year ago: https://arxiv.org/abs/2503.02113
Unrelated to the contents, but WOW your blog styling is gorgeous. Incredible
A very fascinating read.
As a fellow tufte css enjoyer, Why is user select turned off on the sidenotes? I would like to be able to copy paste them quite badly.
Layout is fine but font is atrocious.
Uppercase letters have different stroke width than lowercase ones — it’s like they are *B*old *L*ike this.
Not only that: tracking, kerning is basically non-existent.
Please don’t use that open-source font
You need real Bembo, not that piece of shit
Are you on Windows? I patched the original tufte fonts to fix these crazy kerning issues, but i have indeed noticed them.
What a beautifully written article. It's extremely that I favourite an article but this is one.
Very extremely. Quite a lovely presentation. I'm definitely having a Patrick Bateman-esque appreciation for that delicate cream background.
The Hidden Physics of LLMs: Retrieval as Thermodynamics
https://www.youtube.com/watch?v=ppCZfjLdSY8
I found this video to be illustrative as well. Simple and anyone can understand.
this landed precisely on like 3 weird bugs ive been hitting and solving in different stupid ways for dealing with things like sgd collapsing too many good answers into one bad answer, and gave me a real direction to try to fix the link missing in my own ml stuff. what timing. i have tried analytic solutions too and they're useful for like mapping prompts into memory geometry but from there ive ended up still having to use sgd. cause i think what happens is, sgd teaches the neural net both the geometry and how to navigate it. if you just teleport to the answer it doesnt learn how to walk.
Looks like a typical machine learning paper to me. It cannot be understood unless you already kind of understand it. That is OK for communication with peers, but eventually I expect a "theory of" to be readable by anyone with a math degree.
This looks like excellent work, it's reminding me of things I learned from Welch Labs vidoes. Given the amount of time I budget for keeping up with this stuff (regrettably too low) I'll wait until Welch Labs presents something on this.
So, this is either the paper of the year, or ... definitely not the paper of the year.
https://arxiv.org/pdf/2605.01172 is the current version. The money graphs are page 8 and on where they show (some weirdly thick) line charts with loss results reached in roughly 1/5 the number of steps that Adam takes, just what the blog post mentions.
They also claim holding back test data is not needed, also with more graphs.
I'm not an ML scientist, and I did not attempt to seriously parse the math. It reads to me as something precisely in that liminal space some math papers do where there's enough new terminology that actually parsing through it all is going to take real, concerted effort, possibly with mild brain damage as a risk.
Their 3d graphs of "kernel eigenstructure" also do double duty for me as totally impenetrable and possibly part of an April fool's ML paper that's hilarious to insiders. Or maybe they show something really amazing; they definitely seem to converge into a shape...What does that shape mean??? Why??? What is an eigenstructure? Is it just 3D eigenvectors of some matrices? Is it natural to have a 3D shape representing these large matrices? If not, how and why were these projected down? And why are they different colors in the paper?? You get the feel for my level of understanding.
I think it would frankly just be easier to validate this claim than parse the whole paper. If only I could understand
Well enough to implement the one line update to Adam in python. I have not asked codex or claude to assist yet.Also of note to me, they talk about grokking which I found SUUUPER fascinating when it was first reported, and have never heard about since. So I was really glad to read about it and read that there has been a little academic work on the phenomenon.
Finally, of the three models they repot results on, two are extremely tiny, the last is a DPO round on Qwen 0.5B -- if the code for that is published, I imagine it would be easy to adapt and evaluate in other regimes.
You don't need to understand that part of the derivation to implement it. You just need Algorithm 1 on page 33 of the paper. Or look at the author's implementation: https://github.com/elonlit/PopRiskMinimization/blob/main/pop...
Thanks for the link - I did not see a GitHub.
So, your thoughts on the paper?
I think it's a solid theoretical contribution, but it might nonetheless fail to have practical relevance if some of their assumptions and approximations turn out to be too unrealistic. One way this could happen, for example, would be if typical training batches get gradients with a high-enough signal-to-noise ratio that their optimizer tweak ends up not tweaking much. Their somewhat unusual selection of experiments makes me suspect that this might be the case.
I read the paper earlier when it showed up on https://news.ycombinator.com/from?site=arxiv.org and the writing style of the blog post turned me off so I didn't bother to check how much it overhypes the results compared to the paper, but certainly a lot of people seem to have gotten the idea that this must be big if true, whereas I think it's better classified as neat, but not revolutionary.
Thanks. I found the DPO test very interesting where they intentionally twiddled the dataset to create 'noise'. That would I think back your concern. But on the other hand a method that allows noisier data in or does better with data sets that are low signal across bands would be nice to have.
[flagged]
Does anyone happen to know what font this site is using? It looks really elegant.
It is a modified version of ET_Book called ET_Bembo:
https://github.com/DavidBarts/ET_Bembo
I love u. thanks!
apparently its the font used in Edward Tufte's books. Its on github: https://edwardtufte.github.io/et-book/
The "Quantitative Display of Information", which I just checked, is using Monotype Bembo. So still Bembo, but a different version.
Font is atrocious.
Uppercase letters have different stroke width than lowercase ones — it’s like they are *B*old *L*ike this.
Not only that: tracking, kerning is basically non-existent.
Please don’t use that open-source font
You need real paid Bembo, not that piece of shit.
Where’s the theory of how the human brain does what it does? Maybe these high dimensional structures don’t have a nice compact “theory”. Trying to fit these systems into a nice compact theory is a very human thing, but not everything works like that.