We study the distribution of singular values for the product of random matrices related to the analysis of deep neural networks. The matrices are similar to the product of sample covariance matrices of statistics, but an important difference is that in statistics the population covariance matrices are assumed to be non-random or random but independent of the random data matrix, while now they are certain functions of the random data matrices (matrices of synaptic weights in the terminology of deep neural networks). The problem was treated recently by J. Pennington et al. assuming that the weight matrices are Gaussian and using the methods of free probability theory. Since, however, free probability theory deals with population covariance matrices that do not depend on data matrices, its applicability to this case must be justified. We use a version of the random matrix theory technique to prove the results of J. Pennington et al. in the general case where the entries of weight matrices are independent identically distributed random variables with zero mean and finite fourth
moment. This, in particular, extends the property of the so-called macroscopic universality to the random matrices in question.

Lecture 3 in the minicourse by Prof. Leonid Pastur

see

https://dsadvancedlectures.weebly.com/

We consider a model of the evolution of one and two qubits embedded in an environment. In contrast to the well-known spin-boson model, used to model a translation invariant and macroscopic environment, we model the environment by random matrices of large size that are widely used to describe multi-connected disordered environments of mesoscopic and even nanoscopic size. An important property of the model is that it incorporates non-Markovian evolution allowing for the backflow of energy and information from the environment to the qubits.
We obtain an asymptotically exact in size of the environment expression for the reduced density matrix of qubits valid for all typical realizations of the disordered environment. By using detailed analytical and numerical analysis of expressions, we find several interesting patterns of the qubit evolution, including the disappearance of entanglement in the finite moments and, especially, its subsequent reappearance. These patterns are known in quantum information science as the sudden death and the sudden birth of entanglement. They were found earlier in special versions of the spin-boson model. Our results obtained for a non microscopic and disordered environment demonstrate the robustness and universality of the patterns. When combined with certain tools of quantum information science (e.g., entanglement distillation), the results can lead to a much slower decay entanglement down to its asymptotic persistence.