Date/Time : 24 Mar 2022 15:00
Venue : via Zoom
Speaker : Patrick Rebeschini, Oxford University

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

The local Rademacher complexity framework is one of the most successful general-purpose toolboxes for establishing sharp excess risk bounds for statistical estimators based on the framework of empirical risk minimization. Applying this toolbox typically requires using the Bernstein condition, which often restricts applicability to convex and proper settings. Recent years have witnessed several examples of problems where optimal statistical performance is only achievable via non-convex and improper estimators originating from aggregation theory, including the fundamental problem of model selection. These examples are currently outside of the reach of the classical localization theory.

In this work, we build upon the recent approach to localization via offset Rademacher complexities, for which a general high-probability theory has yet to be established. Our main result is an exponential-tail excess risk bound expressed in terms of the offset Rademacher complexity that yields results at least as sharp as those obtainable via the classical theory. However, our bound applies under an estimator-dependent geometric condition (the “offset condition”) instead of the estimator-independent (but, in general, distribution-dependent) Bernstein condition on which the classical theory relies. Our results apply to improper prediction regimes not directly covered by the classical theory. (Based on joint work with V. Kanade and T. Vaškevičius)

Add to calendar :

Date/Time : 18 Mar 2022 10:00
Venue : via Zoom
Speaker : Yian Ma, University of California at San Diego

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract:  I will introduce some recent progress towards understanding the scalability of Markov chain Monte Carlo (MCMC) methods and their comparative advantage with respect to variational inference. I will discuss an optimization perspective on the infinite dimensional probability space, where MCMC leverages stochastic sample paths while variational inference projects the probabilities onto a finite dimensional parameter space. Three ingredients will be the focus of this discussion: non-convexity, dimension dependence, and stochasticity. This line of work is motivated by epidemic prediction, where we need uncertainty quantification for credible predictions and informed decision making with complex models and evolving data.

Add to calendar :

Date/Time : 11 Mar 2022 10:00
Venue : via Zoom
Speaker : Jean Honorio, Purdue University

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Several problems in machine learning are combinatorial (e.g., clustering, learning sparse representations.) Convex relaxations have been used in the past, in order to circumvent the NP-hardness of the combinatorial problems, albeit being approximate solutions. My long-term research goal is to uncover large classes of continuous relaxations (beyond convexity) of worst-case NP-hard problems, that admit exact solutions.

In this talk, I will introduce a unifying framework, which uses the power of continuous relaxations (beyond convexity), Karush-Kuhn-Tucker conditions, primal-dual certificates and concentration inequalities. This framework has produced novel algorithms not only for classical worst-case NP-hard problems, e.g., Bayes nets, structured prediction, clustering, but also for aspects of fairness and meta learning.

As a first example, we study the problem of fair sparse regression on a biased dataset where bias depends upon a hidden binary attribute. The presence of a hidden attribute adds an extra layer of complexity to the problem by combining sparse regression and clustering with unknown binary labels. The corresponding optimization problem is combinatorial but we propose an invex relaxation, a subclass of nonconvex relaxations. We then provide exact recovery guarantees.

As a second example, we study the problem of exact partitioning of high-order models. Exact partitioning can be formulated as a tensor optimization problem. We relax this high-order combinatorial problem to a convex conic form problem. To this end, we carefully define the Caratheodory symmetric tensor cone, and show its convexity, and the convexity of its dual cone. We also provide exact recovery guarantees.

Add to calendar :

Date/Time : 04 Mar 2022 10:00
Venue : via Zoom
Speaker : Yunlong Hou, National University of Singapore

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract: We design and analyze VA-LUCB, a parameter-free algorithm, for identifying the best arm under the fixed-confidence setup and under a stringent constraint that the variance of the chosen arm is strictly smaller than a given threshold. An upper bound on VA-LUCB’s sample complexity is shown to be characterized by a fundamental variance-aware hardness quantity H_VA. By proving a lower bound, we show that sample complexity of VA-LUCB is optimal up to a factor logarithmic in H_VA. Extensive experiments corroborate the dependence of the sample complexity on the various terms in H_VA. By comparing VA-LUCB’s empirical performance to a close competitor RiskAverse-UCB-BAI by David et al. (2018), our experiments suggest that VA-LUCB has the lowest sample complexity for this class of risk-constrained best arm identification problems, especially for the riskiest instances.

Add to calendar :

Date/Time : 18 Feb 2022 10:00
Venue : via Zoom
Speaker : Yang Haizhao, Purdue University

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract: Learning operators between infinitely dimensional spaces is an important learning task arising in wide applications in machine learning, imaging science, mathematical modeling and simulations, etc. This talk introduces the nonparametric estimation of Lipschitz operators using deep neural networks. Non-asymptotic upper bounds are derived for the generalization error of the empirical risk minimizer over a properly chosen network class. Under the assumption that the target operator exhibits a low dimensional structure, our error bounds decay as the training sample size increases, with an attractive fast rate depending on the intrinsic dimension in our estimation. Our assumptions cover most scenarios in real applications and our results give rise to fast rates by exploiting low dimensional structures of data in operator estimation. We also investigate the influence of network structures (e.g., network width, depth, and sparsity) on the generalization error of the neural network estimator and propose a general suggestion on the choice of network structures to maximize the learning efficiency quantitatively.

Add to calendar :

Date/Time : 11 Feb 2022 10:00
Venue : via Zoom
Speaker : Soufiane Hayou, National University of Singapore

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract: The theoretical analysis of infinite-width neural networks has led to many interesting practical results (choice of initialization schemes, choice of Bayesian priors etc.). However, the traditional infinite-width framework focuses on fixed depth networks and omits the large depth behavior of these models. In this talk, I will cover different topics on the infinite-width-then-infinite-depth networks and show how we can leverage some theoretical results to obtain principled algorithms. The following papers will be covered in this talk:

• On the Impact of the activation function on deep neural networks training. ICML 2019. (Joint work with A. Doucet and J. Rousseau)
• Stable ResNet. AISTATS 2021. (Joint work with E. Clerico, B. He, G. Deligiannidis, A. Doucet, and J. Rousseau)

Add to calendar :

Date/Time : 12 Jan 2022 09:00
Venue : via Zoom
Speaker : Cheng Mao, Georgia Tech

Join Zoom Meeting
https://nus-sg.zoom.us/j/85910065038?pwd=ay9nT0hzR2ROeUs4U3d5dTBoMGNBdz09

Meeting ID: 859 1006 5038
Password: 753065

Abstract: Finding latent structures in networks is an important task in many fields of science and engineering.  For a variety of problems, the task can be formulated as inferring a latent permutation of vertices of a network from noisy data. Problems in this class are often computationally infeasible in the worst case due to the combinatorial nature of permutations, so statistical methods are essential for inference in large-scale networks. This talk will provide an overview of two problems: ranking items from comparisons and matching correlated networks. For each problem, I will introduce an average-case model and the associated task of learning a latent permutation. Then I will discuss recent development of efficient algorithms that come with theoretical guarantees.

Add to calendar :

Date/Time : 07 Jan 2022 10:00
Venue : via Zoom
Speaker : Pavitra Krishnaswamy, I2R, A*STAR

Join Zoom Meeting
https://nus-sg.zoom.us/j/89775139658?pwd=b3VxL0pDT0ZNTnIrUUV6MWNZK28xQT09

Meeting ID: 897 7513 9658
Passcode: 238865

Abstract:  Real-world health data from clinical records, disease registries, consumer health devices, and mobile applications offer great potential for a number of health intelligence applications. However, effectively learning from these observational datasets remains challenging as they lack detailed annotations, exhibit significant irregularities, and are often of variable quality. In this talk, I will discuss our research efforts in developing machine learning capabilities to address these challenges for targeted screening, allied health and clinical decision support applications.

Add to calendar :

Date/Time : 17 Dec 2021 10:00
Venue : via Zoom
Speaker : Masashi Sugiyama, RIKEN and University of Tokyo

Join Zoom Meeting
https://nus-sg.zoom.us/j/9775606531?pwd=cVZ6eTRYS3ZzVUlFNXRuSlhMVjJSUT09

Meeting ID: 977 560 6531
Passcode: 123321

Abstract: Supervised machine learning based on labeled big data has been highly successful in a wide range of applications. Nevertheless, there are still many challenging situations where only limited labeled data is available, e.g., due to high labeling costs, rareness of events, and privacy concerns. In this talk, I will give an overview of our recent advances in weakly supervised classification based on empirical risk minimization. Our framework contains various important weakly supervised classification problems such as positive-unlabeled classification, positive-confidence classification, similar-dissimilar classification, and unlabeled-unlabeled classification for binary problems, and complementary-label classification, partial-label classification, and single-class-confidence classification for multi-class problems. This framework allows us to estimate the classification risk only from weakly supervised data, provides theoretical convergence guarantee, and is compatible with any loss function, classifier, and optimizer.

Add to calendar :