Date/Time : 23 Apr 2025 15:00
Venue : S16 #05-17 (ACTIVE LEARNING ROOM 1)
Speaker : Jared P. Whitehead, Brigham Young University

In classical statistics, predictive error is understood as a trade-off between the bias and the variance that balances model simplicity with its ability to fit complex functions. However, over-parameterized models exhibit counterintuitive behaviors, such as “double descent” in which models of increasing complexity exhibit decreasing generalization error. Others may exhibit more complicated patterns of predictive error with multiple peaks and valleys. Neither double descent nor multiple descent phenomena are well explained by the bias-variance decomposition.

We introduce a novel decomposition that we call the generalized aliasing decomposition (GAD) to explain the relationship between predictive performance and model complexity. The GAD decomposes the predictive error into three parts: 1) model insufficiency, which dominates when the number of parameters is much smaller than the number of data points, 2) data insufficiency, which dominates when the number of parameters is much greater than the number of data points, and 3) generalized aliasing, which dominates between these two extremes.

We demonstrate the applicability of the GAD to diverse applications, showing that it accurately captures the non-monotonic nature of generalization curves in several different settings. GAD can be explicitly calculated from the relationship between model class and samples without seeing any data labels, so that this decomposition can answer questions related to experimental design and model selection before collecting data or performing experiments.

Add to calendar :

Date/Time : 15 Apr 2025 15:00
Venue : S17 #04-05 (SR2)
Speaker : Hemant Tyagi, Nanyang Technological University

We consider the problem of finite-time identification of linear dynamical systems from $T$ samples of a single trajectory. Recent results have predominantly focused on the setup where either (a) no structural assumption is made on the system matrix $A^*$, or (b) specific structural assumptions (e.g. sparsity, low-rankness etc) are made on $A^*$. We assume prior structural information on $A^*$ is available, which can be captured in the form of a convex set $\mathcal{K}$ . For the solution of the ensuing constrained least squares estimator, we derive non-asymptotic error bounds in the Frobenius norm that depend on the local size of $\mathcal{K}$ at $A^*$. The results are instantiated for four examples, namely when (i) $A^*$ is sparse and $\mathcal{K}$ is a suitably scaled $\ell_1$ ball; (ii) $\mathcal{K}$ is a subspace; (iii) $\mathcal{K}$ consists of matrices each of which is formed by sampling a bivariate convex function on a uniform $n \times n$ grid (convex regression); (iv) $\mathcal{K}$ consists of matrices each row of which is formed by uniform sampling (with step size $1/T$) of a univariate Lipschitz function. In all these situations, we show that $A^*$ can be reliably estimated for values of $T$ much smaller than what is needed for the unconstrained setting.

Add to calendar :

Date/Time : 04 Mar 2025 15:00
Venue : S17 #04-05 (SR2)
Speaker : Anastasia Borovykh, Imperial College London

State-of-the-art foundation models are often seen as black boxes: we send a prompt in and we get out our – often useful – answer. But what happens inside the system as the prompt gets processed remains a bit of a mystery & our ability to control the processing into specific directions is limited.

In this walk we will discuss two concepts that can help us add additional controls into these models. First up, LogitLens, a method to gain more insight into the representations of hidden nodes. Second, steering vectors: by computing a vector that represents a particular feature or concept, we can use this to steer the model to include any property in the output we want. We end the talk by discussing whether the approach of ‘build a good model, then interpret it’ can work or whether we should shift our focus to ‘build an a priori interpretable model’ and discuss ways of potentially achieving this.

Add to calendar :

Date/Time : 19 Feb 2025 15:00
Venue : S17 #05-12 (SR4)
Speaker : Yue Xie, University of Hong Kong

The two-metric projection method is a simple yet elegant algorithm proposed by Bertsekas to address bound/box-constrained optimization problems. The algorithm’s low per-iteration cost and potential for using Hessian information make it a favorable computation method for this problem class. Inspired by this algorithm, we propose a two-metric adaptive projection method for solving the $\ell$_1-norm minimization problem that inherit these advantages. We demonstrate that the method is theoretically sound – it has global convergence. Furthermore, it is capable of manifold identification and has superlinear convergence rate under the error bound condition and strict complementarity. Therefore, given sparsity in the solution, the method enjoys superfast convergence in iteration while maintaining scalability, making it desirable for large-scale problems. We also equip the algorithm with competitive complexity to solve nonconvex problems. Numerical experiments are conducted to illustrate the advantages of this algorithm implied by the theory compared over other competitive methods, especially in large-scale scenarios. Compared with the two-metric projection method, our algorithm directly solves the l1-norm minimization problem without resorting to the intermediate reformulation as a bound-constrained problem, so it circumvents the issue of numerical instability.

Add to calendar :

Date/Time : 21 Jan 2025 14:00
Venue : S17 #06-11 (SR6)
Speaker : Johannes Lederer, University of Hamburg

Artificial intelligence is arguably the hottest topic in all of science. Computer science and engineering currently set the agenda in this field, sidelining mathematics to a large extent. This talk, however, will highlight that mathematics has a lot to offer. We will introduce mathematical guarantees that provide deep insights into the inner workings of AI, and we will show how statistical principles can make AI more efficient. More generally, we will discuss the role of mathematics, especially statistics, in AI and data science.

Add to calendar :

Date/Time : 21 Jan 2025 16:00
Venue : S17 #04-05 (SR2)
Speaker : Yunpeng Shi, University of California, Davis

In this talk, we explore two classical image processing tasks motivated by cryo-electron microscopy imaging: tomographic image denoising and rigid image registration. Both tasks inherently involve operations of 2D rotations, where leveraging specific transforms can significantly enhance the speed and robustness of the algorithms.

In the first part of the talk, I will introduce an unsupervised image denoiser based on estimating the covariance matrix of clean images. A key insight is that if the image manifold is invariant under global in-plane rotations, this symmetry can be exploited to accelerate computations and reduce dimensionality. I will discuss recent advances in fast expansion into steerable bases that allow us to efficiently utilize this rotational symmetry, leading to a thousand fold improvement in the speed of covariance estimation over existing methods. This technique has been successfully applied to joint deconvolution and denoising of large-scale, real-world cryo-EM images.

In the second part, I will present a fast algorithm for aligning images using optimal transport. Our method leverages the sliced Wasserstein distance by computing the 1D Wasserstein distance between radial line projections of input images. By applying a special transform to the frequency domain, we achieve efficient alignment of two L by L images in O(L^2 log L) operations — matching the complexity of alignment using Euclidean distance. I will demonstrate the robustness of our method to translations and deformations.

Finally, I will comment on the practical limitations of both methods and discuss open questions that remain, highlighting areas for future exploration.

Add to calendar :

Date/Time : 13 Jan 2025 11:00
Venue : S17 #05-11 (SR5)
Speaker : Stephen Wright, University of Wisconsin-Madison

Complexity analysis in optimization seeks upper bounds on the amount of work required to find approximate solutions of optimization problems in a given class with a given algorithm. There is also study of lower bounds, usually in the form of a worst-case example from a given problem class for a given algorithm class. The relationship between theoretical complexity bounds and practical performance of algorithms on “typical” problems varies widely across problem and algorithm classes, and relative interest among researchers between the theoretical and practical aspects of algorithm design and analysis has waxed and waned over the years. This talk surveys complexity analysis and its relationship to practice in optimization, with an emphasis on linear programming and convex and nonconvex nonlinear optimization, providing historical (and cultural) perspectives on research in these areas.

Add to calendar :

Date/Time : 10 Jan 2025 16:00
Venue : S17 #04-04 (SR3)
Speaker : Yufeng Wu, University of Connecticut

Suppose a person collected his/her own genome and wants to learn something about his/her ancestry. Further suppose this person’s genome originates from multiple ancestral populations, a phenomenon called population admixture in genetics. The genome of an individual from an admixed population consists of segments originated from different ancestral populations. Most existing ancestry inference approaches can provide an estimate of the percentage (called admixture proportion) of the genome originated from a specific ancestral population. Now what if someone wants to estimate admixture proportions of recent ancestors from his/her own genome? For example, let us say an individual has 50% European and 50% Asian ancestry. This individual’s parents can be also 50-50 European/Asian ancestry, but it is also possible one is 100% European and the other is 100% Asian. More generally, can one learn more about the ancestry of recent ancestors from his/her own genome?

In this talk, I will present research results on ancestry inference for estimating admixture proportions of recent ancestors from an extant genome. I will first briefly describe PedMix, one of the first methods for such inference. A major downside of PedMix is that it can only estimate admixture proportions for parents and grandparents, and cannot scale for more distantly related ancestors. I will then focus on PedMix2, a general approach that can estimate admixture proportions of all recent ancestors of this individual. This approach was published recently in PNAS (2024). To the best of our knowledge, there is no other method that can practically infer ancestors beyond grandparents from an extant individual’s genome. I will present results on both simulated and real data show that PedMix2 performs well in ancestry inference.

Add to calendar :

Date/Time : 10 Jan 2025 14:00
Venue : S17 #04-06 (SR1)
Speaker : Govind Menon, Brown University

We study the training dynamics of the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning. The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions.

Add to calendar :