Date/Time : 06 Oct 2022 10:00
Venue : via Zoom
Speaker : Ruqi Zhang, Purdue University

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract:  Probabilistic modeling provides a principled framework for learning from data, with the key advantage of offering rigorous solutions for uncertainty quantification. In the era of big and complex data, there is an urgent need for new inference methods in probabilistic modeling to extract information from data effectively and efficiently.

This talk will show how to do theoretically guaranteed scalable and reliable inference for modern machine learning. First, I will introduce a general and theoretically grounded framework to enable fast and asymptotically correct inference, with applications to Metropolis-Hastings and Gibbs sampling. Then, I will highlight the key challenges of probabilistic inference in deep learning, and present a novel approach for fast posterior inference of neural network weights. This method has achieved state-of-the-art results and has been regarded as one of the benchmarks in deep probabilistic modeling.

Add to calendar :

Date/Time : 29 Sep 2022 10:00
Venue : via Zoom
Speaker : He Haiyun, National University of Singapore

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract:  Using information-theoretic principles, we consider the generalization error (gen-error) of iterative semi-supervised learning (SSL) algorithms that iteratively generate pseudo-labels for a large amount of unlabelled data to progressively refine the model parameters. In contrast to most previous works that bound the gen-error, we provide an exact expression for the gen-error and particularize it to the binary Gaussian mixture model. Our theoretical results suggest that when the class conditional variances are not too large, the gen-error decreases with the number of iterations, but quickly saturates. On the flip side, if the class conditional variances (and so amount of overlap between the classes) are large, the gen- error increases with the number of iterations. To mitigate this undesirable effect, we show that regularization can reduce the gen-error. The theoretical results are corroborated by extensive experiments on the MNIST and CIFAR datasets in which we notice that for easy- to-distinguish classes, the gen-error improves after several pseudo-labelling iterations, but saturates afterwards, and for more difficult-to-distinguish classes, regularization improves the generalization performance.

Add to calendar :

Date/Time : 22 Sep 2022 10:00
Venue : S17 #05-11 (SR5)
Speaker : Kengo Kamatani, Institute of Statistical Mathematics

Abstract:  We construct a class of non-reversible Metropolis kernels as a multivariate extension of the guided-walk kernel proposed by Gustafson 1998. The main idea of our method is to introduce a projection that maps a state space to a totally ordered group. By using Haar measure, we construct a novel Markov kernel termed Haar-mixture kernel, which is of interest in its own right. This is achieved by inducing a topological structure to the totally ordered group. Our proposed method, the Delta-guided Metropolis–Haar kernel, is constructed by using the Haar-mixture kernel as a proposal kernel. The proposed non-reversible kernel is at least 10 times better than the random-walk Metropolis kernel and Hamiltonian Monte Carlo kernel for the logistic regression and a discretely observed stochastic process in terms of effective sample size per second.

Add to calendar :

Date/Time : 15 Sep 2022 10:00
Venue : via Zoom
Speaker : Chen Xinyun, The Chinese University of Hong Kong

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract:  Motivated by the emerging needs of personalized intervention in many healthcare applications, we consider a class of multi-stage decision-making problems formulated by Markov Decision Process (MDP) with unknown model parameters. To deal with the pervasive issue of small sample size in personalized intervention, we develop a novel data-pooling estimator and the corresponding reinforcement learning (RL) algorithm based on a general perturbed value iteration framework. The model-free nature of our algorithm has the built-in benefits of being robust to model-misspecification and only requiring data-sharing via aggregate statistics. We establish a theoretical performance guarantee for our algorithm and show that it achieves strictly smaller regret bound than RL algorithms that use only target data or other data-pooling estimators. Furthermore, we substantiate our theoretical development with numerical results in a specific healthcare context: post-discharge follow-up intervention to prevent unplanned readmissions. We demonstrate the empirically better performance of our algorithm over a number of competitive benchmarks using a real hospital dataset. The talk is based on a joint work with Pengyi Shi and Shanwen Pu.

Add to calendar :

Date/Time : 01 Sep 2022 10:00
Venue : via Zoom
Speaker : Dr. Pallavi Basu, Indian School of Business

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract: In large-scale multiple hypothesis testing problems, the false discovery exceedance (FDX) provides a desirable alternative to the widely used false discovery rate (FDR) when the false discovery proportion (FDP) is highly variable. We develop an empirical Bayes approach to control the FDX. We show that, for independent hypotheses from a two-group model and dependent hypotheses from a Gaussian model fulfilling the exchangeability condition, an oracle decision rule based on ranking and thresholding the local false discovery rate (lfdr) is optimal in the sense that the power is maximized subject to the FDX constraint. We propose a data-driven FDX procedure that uses carefully designed computational shortcuts to emulate the oracle rule. We investigate the empirical performance of the proposed method using both simulated and real data and study the merits of FDX control through an application for identifying abnormal stock trading strategies. This is a joint work with Luella Fu, Alessio Saretto, and Wenguang Sun.

Add to calendar :

Date/Time : 22 Apr 2022 14:00
Venue : via Zoom
Speaker : Zhou Pan, Sea Limited

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract:  To train networks, lookahead algorithm updates its fast weights k times via an inner-loop optimizer before updating its slow weights once by using the latest fast weights. Any optimizer, e.g. SGD, can serve as the inner-loop optimizer, and the derived lookahead generally enjoys remarkable test performance improvement over the vanilla optimizer. But theoretical understandings on the test performance improvement of lookahead remain absent yet. To solve this issue, we theoretically justify the advantages of lookahead in terms of the excess risk error which measures the test performance. Specifically, we prove that lookahead using SGD as its inner-loop optimizer can better balance the optimization error and generalization error to achieve a smaller excess risk error than vanilla SGD on (strongly) convex problems and nonconvex problems with Polyak-Łojasiewicz condition which has been observed/proved in neural networks. Moreover, we show the stagewise optimization strategy which decays the learning rate several times during training can also benefit lookahead in improving its optimization and generalization errors on strongly convex problems. Finally, we propose a stagewise locally-regularized lookahead (SLRLA) algorithm which sums up the vanilla objective and a local regularizer to minimize at each stage and provably enjoys optimization and generalization improvement over the conventional (stagewise) lookahead. Experimental results on CIFAR10/100 and ImageNet testify its advantages.

Add to calendar :

Date/Time : 08 Apr 2022 10:00
Venue : via Zoom
Speaker : Gianmarco Mengaldo, National University of Singapore

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract: We provide a perspective on the challenges and opportunities in computational modeling for industry-relevant applications towards the Exascale Era. As an example, we present the successful deployment of high-fidelity Large-Eddy Simulation (LES) technologies based on spectral/hp element methods to simulate the aerodynamics of a real automotive car, namely the Elemental Rp1 model. The simulation presents the common challenges of an industry-relevant problem, namely high Reynolds number and complex geometry. To the best of the authors’ knowledge, this simulation represents the first fifth-order accurate transient LES of an entire real car geometry using spectral element methods. Moreover, this constitutes a key milestone towards considerably expanding the computational design envelope currently allowed in industry, where steady-state modelling remains the standard. We also draw some parallels with other fields, namely numerical weather prediction, where large scale simulations play a central role and have been used since the 1950s.

Add to calendar :

Date/Time : 01 Apr 2022 10:00
Venue : via Zoom
Speaker : Rafael Peres Da Silva, National University of Singapore

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Large-scale cancer omics studies have highlighted the diversity of patient molecular profiles and the importance of leveraging this information to deliver the right drug to the right patient at the right time. Key challenges in learning predictive models for this include the high-dimensionality of omics data and heterogeneity in biological and clinical factors affecting patient response. The use of multi-task learning techniques has been widely explored to address dataset limitations for in vitro drug response models, while domain adaptation (DA) has been employed to extend them to predict in vivo responses. In both of these transfer learning settings, noisy data for some tasks (or domains) can substantially reduce the performance for others compared to single-task (domain) learners, i.e. lead to negative transfer (NT).

This talk introduces TUGDA (ISMB 2021), a novel multi-task unsupervised DA method, that addresses these limitations in a unified framework by quantifying uncertainty in predictors and weighting their influence on shared feature representations. TUGDA’s ability to rely more on predictors with low-uncertainty allowed it to notably reduce cases of NT for in vitro models (94% overall) compared to state-of-the-art methods. For DA to in vivo settings, TUGDA improved over previous methods for patient-derived xenografts (9 out of 14 drugs) as well as patient datasets (significant associations in 9 out of 22 drugs). TUGDA’s ability to avoid NT thus provides a key capability as we try to integrate diverse drug-response datasets to build consistent predictive models with in-vivo utility.

Add to calendar :

Date/Time : 24 Mar 2022 15:00
Venue : via Zoom
Speaker : Patrick Rebeschini, Oxford University

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

The local Rademacher complexity framework is one of the most successful general-purpose toolboxes for establishing sharp excess risk bounds for statistical estimators based on the framework of empirical risk minimization. Applying this toolbox typically requires using the Bernstein condition, which often restricts applicability to convex and proper settings. Recent years have witnessed several examples of problems where optimal statistical performance is only achievable via non-convex and improper estimators originating from aggregation theory, including the fundamental problem of model selection. These examples are currently outside of the reach of the classical localization theory.

In this work, we build upon the recent approach to localization via offset Rademacher complexities, for which a general high-probability theory has yet to be established. Our main result is an exponential-tail excess risk bound expressed in terms of the offset Rademacher complexity that yields results at least as sharp as those obtainable via the classical theory. However, our bound applies under an estimator-dependent geometric condition (the “offset condition”) instead of the estimator-independent (but, in general, distribution-dependent) Bernstein condition on which the classical theory relies. Our results apply to improper prediction regimes not directly covered by the classical theory. (Based on joint work with V. Kanade and T. Vaškevičius)

Add to calendar :