Closing date: 25 August 2023
Campus: Curtin University; Centre for Optimisation and Decision Science
Remuneration: $60,000 – $70,000 per annum
Status: 3 years
Title: Dynamics-driven operator splitting methods for data science, machine learning, and engineering problems
In data science, machine learning, and engineering, many problems take the form of finding a solution that minimizes a cost, subject to constraints on allowable solutions. Some examples of costs include expected financial losses, model prediction errors, and energy used. Some examples of constraints include resource limitations, minimum requirements on what is produced, and so forth.
These problems are solved with operator splitting methods, a modern class of non-linear optimisation algorithms that allow the constraint structure and cost structure to be treated as two components of a single unifying function. These algorithms were independently discovered by mathematicians working on physics and imaging problems, and they have been developed and improved with the powerful machinery of convex analysis.
For many important problems, we desire to make these algorithms go faster, either to find solutions within the maximum time allowable (for example: balancing power flow in electricity grids) or to make better data science models computationally tractable for large data sets. Researchers have recently turned to studying the dynamical systems associated with operator splitting methods. This research is allowing us to prove results in nonconvex settings and build new algorithms. Dr. Scott Lindstrom recently introduced a meta-algorithm that uses operator dynamics to suggest alternative algorithm updates. The intent of this meta-algorithm is to solve surrogates for something called a Lyapunov function, which is an object that describes the dynamics. This meta-algorithm has already become state-of-the-art for finding wavelets with structural constraints (an imaging sciences problem).
Scientific Aims
The scientific aim of this project is to identify classes of problems in data science, machine learning, and engineering for which meta-algorithms—such as the one described above—may be deliver superior performance. The approach will be multi-faceted, combining both computational experiment and rigorous proof. The results will be communicated in articles and submitted to peer-reviewed publications.
Upskilling Aims
The upskilling aims for the selected candidate are as follows (in no particular order). The candidate will build expertise in the algorithms that make it possible to solve many modern data science models and engineering problems, understanding both how the algorithms are designed, how geometry informs model selection, and what the outstanding challenges are. At the project’s completion, the candidate will be competent to rigorously conduct both experimental and theoretical mathematics research, and to communicate the results of their discoveries to others in the field.
In the literature review component, you will learn the fundamental theory—convex analysis—of operator splitting and learn how operator splitting algorithms are formulated for solving various classes of problems. Some examples of the types of problems you will study are as follows: (1) least absolute deviations for outlier-resistant linear regression (a data science modelling problem), (2) progressive hedging for maximizing expected earnings (a finance problem), (3) computation of a one-norm centroid (a statistics problem), and (4) phase retrieval (a signal processing problem).
In the experimental component, you will apply Lyapunov surrogate methods to solve those problems. You will build performance profiles, which are visualizations that allow researchers to compare the speeds of different algorithms.
In the theoretical component, you will formally analyse the dynamical systems associated with operator splitting methods when they are applied to these problem classes. Particular emphasis will be placed on the duality of algorithms; duality is a fundamental concept in convex analysis.
You will document and communicate the findings in written articles.
Context
As described in the overview, in the here and now, faster operator splitting methods will allow us to obtain better solutions to important problems in data science and energy. On a ten year horizon, this research advances an emerging new paradigm in problem solving, where artificial intelligence will observe an algorithm’s performance and suggest on-the-fly parameter adjustments and alternative updates for the iterates. Finally, the project builds fundamental knowledge in the mathematical sciences and equips the selected candidate with a skill set of extreme contemporary demand.
To apply
For more information, please copy and paste the following link into your web browser: http://staff.curtin.edu.au/Scholarship/?id=6730
