Workshop: Multifidelity Methods for Stochastic and Uncertain Problems

Monday

Aretha Teckentrup

Tommaso Vanzan

Steffen W.R. Werner

Tuesday

Giovanni Samaey

Josef Martínek

Tùng Lê

Wednesday

Sebastian Krumscheid

David Schneiderhan

Dániel Hajnal

Thursday

Elisabeth Ullmann

Elisa Iacomini

Niklas Baumgarten

Friday

Maarten Volkaerts

Emil Løvbak

Aretha Teckentrup (University of Edinburgh): Multilevel Monte Carlo Methods with Smoothing

Parameters in mathematical models are often impossible to determine fully or accurately, and are hence subject to uncertainty. By modelling the input parameters as stochastic processes, it is possible to quantify the uncertainty in the model outputs.

In this talk, we employ the multilevel Monte Carlo (MLMC) method to compute expected values of quantities of interest related to partial differential equations with random coefficients. We make use of the circulant embedding method for sampling from the coefficient, and to further improve the computational complexity of the MLMC estimator, we devise and implement the smoothing technique integrated into the circulant embedding method. This allows to choose the coarsest mesh on the  first level of MLMC independently of the correlation length of the covariance function of the random  field, leading to considerable savings in computational cost.

Tommaso Vanzan (Politecnico di Torino): Sparse and multi-level approximations for PDE-constrained optimization under uncertainty

In this talk we are concerned with the minimization of the expected value of a functional constrained by a random partial differential equation. The solution of such problems requires an extremely high computational cost, and thus motivates a very active research area.

Several works proposed multilevel/sparse approximations of the expectation operator, which however involve negative quadrature weights which may destroy the (possible) convexity of the continuous optimization problems.

We here present a novel and alternative framework for using multilevel and sparse quadrature formulae that still preserves the properties of the original problem. Our approach consists in solving a sequence of optimization problems, each discretized with different levels of accuracy of the physical and probability spaces. The final approximation of the control is obtained in a postprocessing step, by suitably combining the adjoint variables computed on the different levels.

We will discuss a complete convergence analysis for multilevel quadrature formulae, and present numerical experiments confirming the better computational complexity of our multilevel approach, even beyond the theoretical assumptions.

Steffen W.R. Werner (Virginia Polytechnic Institute and State University): Interpolatory Model Reduction for Structured Stochastic and Nonlinear Systems

For the accurate modeling of time-dependent real-world phenomena, high-dimensional nonlinear stochastic dynamical systems are indispensable. Thereby, many physical properties are encoded in the internal differential structure of these systems. Typical differential structures are second-order time derivatives in mechanical systems or time-delay terms. When using such models in computational settings, the high-dimensional nature represented by a large number of differential equations that describe the dynamics becomes the main computational bottleneck. A remedy to this problem is model order reduction, which is concerned with the construction of cheap-to-evaluate surrogate models that are described by a significantly smaller number of differential equations while accurately approximating the input-to-output behavior of the original high-dimensional systems. It has been shown that many nonlinear and stochastic phenomena can be equivalently modeled using only bilinear and quadratic terms. Dynamical systems with such terms can be represented in the Laplace domain using multivariate rational functions. In this work, we present a structure-preserving model reduction framework for nonlinear dynamical systems via multivariate rational function interpolation. This new approach allows the simulation-free construction of cheap-to-evaluate surrogate models for nonlinear dynamical systems with internal structures.

Giovanni Samaey (KU Leuven): Multilevel Markov chain and interacting particle methods for Bayesian inverse problems

This talk presents computational strategies for Bayesian inversion based on multilevel methods. We study Multilevel Markov Chain Monte Carlo (MLMCMC) for high-resolution observations and reduce the cost of evaluating large data sets through level-dependent data resolution and likelihood scaling using weighted subsets of observations at coarser levels. We compare parameter representations for spatially varying random fields using Karhunen–Loève expansions, wavelet expansions, and Local Average Subdivision (LAS), and identify LAS as the most efficient option on very fine grids. We also analyze subsampling in the MLMCMC hierarchy and show that independent coarse-level proposals are not required for valid multilevel acceptance probabilities. We derive an adaptive criterion for selecting subsampling rates that balance sampling error and per-level cost, yielding order-of-magnitude reductions in computational effort.

We then consider multilevel interacting particle methods for Bayesian inversion. We focus on a single-ensemble MLMC formulation for Ensemble Kalman methods (EnKF, EKI, EKS), where MLMC is applied at each time step to estimate interaction terms within a globally coupled ensemble. This construction improves asymptotic cost-to-error scaling for computationally expensive forward models. To address non-Gaussian target measures, we introduce localized Consensus-Based Sampling (CBS). This derivative-free, affine-invariant method approximates the potential using a Moreau envelope and its gradient via a proximal operator, leading to dynamics driven by local particle interactions and a correction term that improves performance in multimodal settings.

Josef Martínek (Heidelberg University): Multilevel Sequential Monte Carlo for Bayesian Inverse Problems with Random Likelihoods

In this talk we focus on Bayesian inverse problems in which the forward parameter-to-observable map is approximated by Monte Carlo (MC) simulations. Such problems often arise, for example, in uncertainty quantification of particle transport, where the parameter-to-observable map is defined through the solution of the Boltzmann equation. Approximating the likelihood with high accuracy requires MC simulations with many samples, which makes sampling from the posterior distribution expensive. We present an efficient method for sampling from the posterior based on pseudo-marginal sequential Monte Carlo (SMC) using tempering. To accelerate sampling from the posterior the method adopts a multilevel approach. A significant speedup is achieved compared to a single-level SMC using high-fidelity MC simulations while targeting the same posterior, which is demonstrated by numerical experiments.

Tùng Lê (Oldenburg University): Parametric regularity for non-linear PDEs

While the parametric regularity of linear PDEs has been extensively studied, many fundamental models—including incompressible fluid mechanics and biological pattern formation—are inherently nonlinear. These nonlinear systems are essential for accurately capturing complex real-world phenomena, yet understanding their parametric regularity remains a significant challenge.

In this talk, we investigate the regularity of solutions with respect to parameters for three classical nonlinear PDEs: elliptic eigenvalue problems, semilinear reaction-diffusion equations, and the incompressible Navier-Stokes equations.

Prior research in this area has largely been restricted to affine parameter dependencies, often yielding sub-optimal regularity results or implicit derivative bounds. To overcome these limitations, we introduce a novel proof technique, termed the “alternative-to-factorial” approach, to effectively manage nonlinear terms. This technique enables us to extend the parametric regularity of solutions to the class of analytic functions and even to non-analytic Gevrey classes. Finally, we present an error analysis for the QMC integration of high-dimension applied to these nonlinear problems, specifically for cases involving high-dimensional analytic coefficients.

(The recording is not available)

Sebastian Krumscheid (Karlsruhe Institute of Technology): Beyond Expectations: Multilevel Monte Carlo Methods for Comprehensive Uncertainty Quantification

Multilevel Monte Carlo (MLMC) methods have emerged as powerful tools for efficient uncertainty quantification in computationally expensive simulations. While traditionally focused on estimating expected values, recent advances extend MLMC principles to capture richer information about system outputs under uncertainty. This talk presents a systematic progression in MLMC methodology: beginning with estimators for central moments (variance, skewness, and kurtosis) of model outputs, we extend the MLMC framework to function approximations via hierarchical models, and finally discuss measures of distributional behavior and robustness. By leveraging multilevel hierarchies not only for mean estimation but also for comprehensive distributional characterization, we demonstrate how MLMC can provide practitioners with deeper insight into system uncertainty at a competitive computational cost. We illustrate these extensions through theoretical analysis and applications, highlighting how problem-adapted approximation hierarchies enable efficient uncertainty quantification beyond first-moment statistics.

David Schneiderhan (Karlsruhe Institute of Technology): Multilevel Stochastic Gradient Descent for Risk-Averse PDE Constraint Optimization

We introduce a multilevel stochastic gradient descent framework for the optimization of partial differential equation–constrained systems with uncertain input data, including recent extensions to risk‑averse objectives.

The method employs a parallel multilevel Monte Carlo estimator to approximate stochastic gradients, providing explicit control over the bias, introduced by numerical discretization and the sampling error arising from incomplete gradient information, while optimally managing computational resources. We show that the method exhibits linear convergence in the number of optimization steps while avoiding the cost of sample average approximation to compute the full gradient.

Numerical experiments demonstrate substantial gains in convergence speed and accuracy compared to standard (mini‑batch) stochastic gradient descent with respect to computational resources. The approach is especially well‑suited for high‑dimensional control problems, leveraging parallel computing architectures and a distributed multilevel data structure.

Dániel Hajnal (University of Bath): Multifidelity Monte Carlo framework for photon dose uncertainty quantification

During the presentation, we will look at a two-level multifidelity Monte Carlo estimator for photon dose uncertainty that couples a deterministic algorithm with a high-fidelity Monte Carlo (MC) model via an optimal control variate. We will quantify the sensitivity to correlation mis-specification and, using Fisher’s transform, derive pilot-sample prescriptions that bound expected variance inflation together with guarded allocations based on lower confidence bounds. Throughout, the target of inference is the MC expectation, and the deterministic algorithm enters solely through correlation. The framework yields substantial variance reduction at fixed computational expenditure; as we will see by applying it to a clinically relevant case.

(The recording has not been edited, yet)

Elisabeth Ullmann (Technical University of Munich): Uncertainty quantification analysis of bifurcations of the Allen-Cahn equation with random coefficients

In this work we consider the Allen-Cahn equation, a prototypical model problem in nonlinear dynamics that exhibits bifurcations corresponding to variations of a deterministic bifurcation parameter. Going beyond the state-of-the-art, we introduce a random coefficient in the linear reaction part of the equation, thereby accounting for random, spatially-heterogeneous effects. Importantly, we assume a spatially constant, deterministic mean value of the random coefficient. We show that this mean value is in fact a bifurcation parameter in the Allen-Cahn equation with random coefficients. Moreover, we show that the bifurcation points and bifurcation curves become random objects. We consider two distinct modelling situations: (i) for a spatially homogeneous coefficient we derive analytical expressions for the distribution of the bifurcation points and show that the bifurcation curves are random shifts of a fixed reference curve; (ii) for a spatially heterogeneous coefficient we employ a generalized polynomial chaos expansion to approximate the statistical properties of the random bifurcation points and bifurcation curves. Our exposition addresses both, dynamical systems and uncertainty quantification, highlighting how analytical and numerical tools from both areas can be combined efficiently for the challenging uncertainty quantification analysis of bifurcations in random differential equations. This is joint work with Chiara Piazzola (TUM) and Christian Kuehn (TUM).

(The recording has not been edited, yet)

Elisa Iacomini (University of Ferrara): Investigating Uncertainty in Traffic Flow Modeling: Multi-Level Monte Carlo and Multi-Fidelity Approaches

Modeling traffic flow plays a crucial role in designing, managing, and optimizing transportation systems. Partial differential equations (PDEs) are a powerful mathematical tool in this area, providing a solid framework to describe traffic dynamics, including key aspects like density, speed, and flux across time and space. Through PDEs, researchers can predict and analyze complex traffic patterns, supporting the development of effective traffic management strategies and infrastructure improvements.

However, real-world traffic is inherently uncertain due to various factors such as fluctuating demand, unpredictable incidents, and varying driver behaviors. Such uncertainty can significantly impact the accuracy and reliability of traffic flow models. Therefore, incorporating uncertainty into traffic models is crucial for creating more realistic and trustful solutions.

Investigating the propagation of uncertainties in traffic flow models is indeed the aim of this talk. Several approaches to quantify uncertainty are presented in the literature and can be classified in non-intrusive and intrusive methods.

In this talk we will focus on the former approach, in particular we investigate multi fidelity control variate method and multi level Monte Carlo approach.

The first exploits the multiscale nature of the problem to reduce the variance in Monte Carlo simulations. Specifically, we exploit the hierarchical relationship between different model scales: high-fidelity models, such as kinetic models, offer high accuracy but are computationally expensive, while low-fidelity models, such as macroscopic models, are less precise but computationally efficient.

The crucial aspect is that the low fidelity model must be an approximation of the high fidelity model. By performing a limited number of high-fidelity evaluations and numerous low-fidelity ones, we can improve accuracy without significantly increasing computational costs.

Instead of performing all simulations at a single resolution, multi level Monte Carlo combines simulations at multiple levels of accuracy, where coarse simulations are inexpensive and provide broad trends, while fine simulations are used to correct errors. The key insight is that by carefully balancing the number of simulations across different levels, the computational cost of achieving a given accuracy can be significantly reduced compared to standard Monte Carlo methods. This makes this approach especially powerful in high-dimensional problems, as in the uncertainty quantification framework.

Numerical simulations demonstrate that the multi-fidelity framework achieves significant improvements in accuracy compared to standard Monte Carlo methods while keeping computational costs manageable. These results underscore the potential of multi-fidelity approaches for enhancing model accuracy in traffic flow simulations, paving the way for more reliable and effective traffic management solutions.

Niklas Baumgarten (Heidelberg University): Distributed Multilevel Sequential Monte Carlo applied to PDE-based Bayesian Inverse Problems

We present ideas on distributed multilevel sequential Monte Carlo method for Bayesian inverse problems to identify input parameters of models based on partial differential equations. The method is implemented on a distributed multilevel data structure to divide the computational load across multiple processing devices and to control the sampling and discretization errors introduced by Monte Carlo estimation and finite element approximation.

By formulating these errors as the minimization target in a knapsack problem, with available computational resources as constraints, we control the error with respect to a given computational budget. Based on this framework, we discuss properties and present preliminary experimental results.

Maarten Volkaerts (KU Leuven): MCMC for Bayesian inference of the non-conducting region in intra-atrial reentrant tachycardia

Cardiac arrhythmia arise from abnormal electrical activity in the heart and are associated with a large number of deaths. The underlying disease condition varies between individuals, necessitating patient-specific treatment. In this context, personalized cardiac models or cardiac digital twins offer a promising new approach. However, current imaging techniques limit the fidelity with which individual cardiac anatomy and function can be captured, making uncertainty quantification (UQ) during model calibration essential.

We employ Bayesian inference for parameter estimation in a cardiac electrophysiology model and for UQ that accounts for measurement and discretization errors. In a two-dimensional cardiac tissue slab, we simulate the electrical signal propagation that coordinates heart contraction. From mapping catheter data, we aim to estimate the shape parameters of electrically inactive regions, or scars, which disrupt signal propagation and can lead to arrhythmia.

In our talk, we propose a Markov chain Monte Carlo method that requires a minimal number of samples for posterior estimation by defining a likelihood based on summary statistics and by using an adaptive proposal distribution. To account for the discretization error, we inflate the likelihood variance and we tailor the meshing strategies used for the forward simulations in the accept-reject step. Further computational efficiency is achieved through a multilevel method that substitutes expensive simulations with cheaper approximations.

Emil Løvbak (Karlsruhe Institute of Technology): Multilevel Monte Carlo simulation of diffusively scaled kinetic equations

Kinetic equations model the dynamics of particles subject to transport and collision dynamics as a time-dependent probability density f(x,v,t) in a position-velocity phase space. Such equations play a key role in multiple application domains, including fusion research, radiation therapy and medical imaging. Often, one is not interested in the full density f(x,v,t), rather in lower dimensional quantities of interest, such as the particle mass density or flux, only depending on position and time. Hence, particle-based Monte Carlo simulations become an attractive approach to avoid constructing grids in phase space.

In many applications, kinetic equations also exhibit multiscale behavior due to collision time scales being significantly shorter than those at which the quantities of interest evolve. This scale separation leads to a blowup in computational cost when using classical explicit time-integration methods to simulate particle trajectories in the Monte Carlo simulations. However, in the infinite scale-separation limit, well-known macroscopic limiting models exist, that can be simulated without such issues. Numerical schemes for kinetic equations that can resolve the dynamics of these limiting models without a blowup in computational cost are called asymptotic-preserving.

In this talk, I discuss a combination of the frameworks of asymptotic-preserving Monte Carlo schemes and the multilevel Monte Carlo method, to perform efficient simulations of diffusively scaled Boltzmann-BGK equations. The core idea here is to construct a hierarchy of simulations where particles simulated at coarser levels in the hierarchy follow macroscopic dynamics based on the limiting dynamics, while particles at finer levels resolve the true high-collisional kinetic dynamics. This hierarchy makes use of correlated particle trajectories to produce estimates of the kinetic dynamics, at significantly reduced computational effort. I will introduce the framework, discuss how one produces correlated simulations throughout the multilevel hierarchy and present some numerical results.