Matrix denoising is central to signal processing and machine learning. Its statistical analysis when the matrix to infer has a factorised structure with a rank growing proportionally to its dimension remains a challenge, except when it is rotationally invariant. The reason is that the model is not a usual spin system because of the growing rank dimension, nor a matrix model due to the lack of rotation symmetry, but rather a hybrid between the two. I will discuss recent findings on Bayesian matrix denoising when the hidden signal XX^t is not rotationally invariant. I will discuss the existence of a « universality breaking » phase transition separating a regime akin to random matrix theory with strong universality properties, from one of the mean-field type as in spin models, treatable by spin glass techniques.
In the second part, I will connect this model and phenomenology to learning in neural networks. We will see how these findings allow to analyse neural networks with an extensively large hidden layer trained near their interpolation threshold, a model that has been resisting for a long time. I will show that the phase transition in matrix denoising translates in this context into a sharp learning transition. The related papers are: https://arxiv.org/pdf/2411.01974 \DSEMIC https://arxiv.org/pdf/2501.18530