Date/Time : 30 Apr 2025 14:00
Venue : S17 #04-05 (SR2)
Speaker : Zak Shumaylov,

In the last five years, neural solvers have emerged as powerful tools for learning solution operators of partial differential equations (PDEs). Paralleling the central role of symmetries in classical numerical analysis – where they guide the development of stable and structure-preserving algorithms, these principles are now proving equally vital in machine learning. Incorporation of symmetries often leads to faster learning, better generalization, with solutions respecting fundamental physical laws. This talk explores how similar principles can be be integrated into modern neural PDE solvers, focusing on Lie point symmetries, generalized symmetries, and Hamiltonian structures. These approaches demonstrate how revisiting classical symmetry ideas leads to more robust, data-efficient, and physically consistent learning frameworks

Add to calendar :

Date/Time : 30 Apr 2025 16:30
Venue : S17 #04-06 (SR1)
Speaker : Grigorios Chrysos , University of Wisconsin-Madison

Activation functions play a pivotal role in deep neural networks, enabling them to tackle complex tasks like image recognition. However, activation functions also introduce significant challenges for deep learning theory, network dynamics analysis, and properties such as interpretability and privacy preservation. In this talk, we revisit the necessity of activation functions across various scenarios. Specifically, we explore expressing network outputs through high-order interactions among input elements using multilinear algebra. This approach allows us to attain the necessary expressivity via these high-order interactions. Yet, the question remains: Is this expressivity alone sufficient for effective learning? Our recent research, presented at ICLR’24, unveils meticulously designed networks, which achieve strong performance even in demanding tasks, such as ImageNet image recognition.

Add to calendar :

Date/Time : 28 Apr 2025 15:00
Venue : S17 #04-05 (SR2)
Speaker : Lin, Youzuo, University of North Carolina at Chapel Hill

Computational Wave Imaging (CWI) enables the reconstruction of hidden structures and the estimation of physical properties within a medium by analyzing propagating wave signals. This technique has critical applications in fields such as seismic exploration of the Earth’s subsurface and ultrasound computed tomography in medical imaging. Traditional approaches to CWI are broadly categorized into physics-based methods and machine learning-based techniques. Physics-based methods deliver high-resolution, quantitatively accurate reconstructions of acoustic properties but are often computationally intensive and susceptible to issues of ill-posedness and nonconvexity. In contrast, machine learning approaches, particularly those leveraging deep learning, have recently emerged as powerful tools to address these challenges by offering faster, data-driven solutions. In this presentation, we introduce our recent advancements in hybrid CWI methodologies that combine fundamental wave physics with state-of-the-art machine learning techniques. We focus on the development of self-supervised learning strategies that significantly reduce computational costs while maintaining accuracy. We will demonstrate the effectiveness of these approaches through applications in subsurface geophysics and medical ultrasound imaging, highlighting their potential for broad, real-world impact.

Add to calendar :

Date/Time : 25 Apr 2025 16:00
Venue : S17 #04-04 (SR3)
Speaker : Youngjoon Hong, Seoul National University

We propose a novel approach for solving parametric partial differential equations (PDEs) using an operator network. This method combines the strengths of deep learning with traditional numerical techniques, specifically the finite element or spectral element methods, to address parametric PDEs ranging from singularly perturbed convection-diffusion equations to the Navier-Stokes equations, all without requiring paired input-output training data. Through extensive experiments on various benchmark problems, our approach has demonstrated excellent performance across diverse settings, showcasing its versatility in terms of accuracy, generalization, and computational efficiency. The proposed framework holds significant potential for applications in fields where PDEs are critical for modeling complex domains with diverse boundary conditions and singular behavior. Additionally, we provide a rigorous theoretical convergence analysis, grounded in numerical analysis, to further support our approach.

Add to calendar :

Date/Time : 24 Apr 2025 16:00
Venue : S17 #06-11 (SR6)
Speaker : John Harlim, Penn State University

In this talk, I will present two recent topics on learning vector fields of dynamical systems. For the first part, I will discuss the problem of learning vector fields of dynamical systems that lie on a Riemannian submanifold of R^n, with applications ranging from dissipative dynamics that exhibit inertial manifold structure to differential equations with algebraic constraints. I will present an operator-valued kernel constructed by geometrical constraints estimated from the time series and an ODE solver that guarantees the prediction to lie on the manifold in the limit of large data. I will demonstrate the numerical methods on various examples of various ambient dimensions of order 1 to order $10^4$ and compare them to existing neural network approaches. For the second part, I will discuss the problem of learning coarse-grained dynamics on graphs with Graph Neural Network. We systematically determine the GNN architecture by inspecting how the leading term of the Mori-Zwanzig memory term depends on the coarse-grained interaction coefficients that encode the graph topology. I will present numerical demonstrations on two examples, a heterogeneous Kuramoto oscillator model and a power system, which shows that the proposed GNN architecture can predict the coarse-grained dynamics under fixed and time-varying graph topologies.

Add to calendar :

Date/Time : 23 Apr 2025 15:00
Venue : S16 #05-17 (ACTIVE LEARNING ROOM 1)
Speaker : Jared P. Whitehead, Brigham Young University

In classical statistics, predictive error is understood as a trade-off between the bias and the variance that balances model simplicity with its ability to fit complex functions. However, over-parameterized models exhibit counterintuitive behaviors, such as “double descent” in which models of increasing complexity exhibit decreasing generalization error. Others may exhibit more complicated patterns of predictive error with multiple peaks and valleys. Neither double descent nor multiple descent phenomena are well explained by the bias-variance decomposition.

We introduce a novel decomposition that we call the generalized aliasing decomposition (GAD) to explain the relationship between predictive performance and model complexity. The GAD decomposes the predictive error into three parts: 1) model insufficiency, which dominates when the number of parameters is much smaller than the number of data points, 2) data insufficiency, which dominates when the number of parameters is much greater than the number of data points, and 3) generalized aliasing, which dominates between these two extremes.

We demonstrate the applicability of the GAD to diverse applications, showing that it accurately captures the non-monotonic nature of generalization curves in several different settings. GAD can be explicitly calculated from the relationship between model class and samples without seeing any data labels, so that this decomposition can answer questions related to experimental design and model selection before collecting data or performing experiments.

Add to calendar :

Date/Time : 15 Apr 2025 15:00
Venue : S17 #04-05 (SR2)
Speaker : Hemant Tyagi, Nanyang Technological University

We consider the problem of finite-time identification of linear dynamical systems from $T$ samples of a single trajectory. Recent results have predominantly focused on the setup where either (a) no structural assumption is made on the system matrix $A^*$, or (b) specific structural assumptions (e.g. sparsity, low-rankness etc) are made on $A^*$. We assume prior structural information on $A^*$ is available, which can be captured in the form of a convex set $\mathcal{K}$ . For the solution of the ensuing constrained least squares estimator, we derive non-asymptotic error bounds in the Frobenius norm that depend on the local size of $\mathcal{K}$ at $A^*$. The results are instantiated for four examples, namely when (i) $A^*$ is sparse and $\mathcal{K}$ is a suitably scaled $\ell_1$ ball; (ii) $\mathcal{K}$ is a subspace; (iii) $\mathcal{K}$ consists of matrices each of which is formed by sampling a bivariate convex function on a uniform $n \times n$ grid (convex regression); (iv) $\mathcal{K}$ consists of matrices each row of which is formed by uniform sampling (with step size $1/T$) of a univariate Lipschitz function. In all these situations, we show that $A^*$ can be reliably estimated for values of $T$ much smaller than what is needed for the unconstrained setting.

Add to calendar :

Date/Time : 04 Mar 2025 15:00
Venue : S17 #04-05 (SR2)
Speaker : Anastasia Borovykh, Imperial College London

State-of-the-art foundation models are often seen as black boxes: we send a prompt in and we get out our – often useful – answer. But what happens inside the system as the prompt gets processed remains a bit of a mystery & our ability to control the processing into specific directions is limited.

In this walk we will discuss two concepts that can help us add additional controls into these models. First up, LogitLens, a method to gain more insight into the representations of hidden nodes. Second, steering vectors: by computing a vector that represents a particular feature or concept, we can use this to steer the model to include any property in the output we want. We end the talk by discussing whether the approach of ‘build a good model, then interpret it’ can work or whether we should shift our focus to ‘build an a priori interpretable model’ and discuss ways of potentially achieving this.

Add to calendar :

Date/Time : 19 Feb 2025 15:00
Venue : S17 #05-12 (SR4)
Speaker : Yue Xie, University of Hong Kong

The two-metric projection method is a simple yet elegant algorithm proposed by Bertsekas to address bound/box-constrained optimization problems. The algorithm’s low per-iteration cost and potential for using Hessian information make it a favorable computation method for this problem class. Inspired by this algorithm, we propose a two-metric adaptive projection method for solving the $\ell$_1-norm minimization problem that inherit these advantages. We demonstrate that the method is theoretically sound – it has global convergence. Furthermore, it is capable of manifold identification and has superlinear convergence rate under the error bound condition and strict complementarity. Therefore, given sparsity in the solution, the method enjoys superfast convergence in iteration while maintaining scalability, making it desirable for large-scale problems. We also equip the algorithm with competitive complexity to solve nonconvex problems. Numerical experiments are conducted to illustrate the advantages of this algorithm implied by the theory compared over other competitive methods, especially in large-scale scenarios. Compared with the two-metric projection method, our algorithm directly solves the l1-norm minimization problem without resorting to the intermediate reformulation as a bound-constrained problem, so it circumvents the issue of numerical instability.

Add to calendar :