Neural network-based approaches have recently shown significant promise in solving partial differential equations (PDEs) in science and engineering, especially in scenarios featuring complex domains or incorporation of empirical data. One advantage of the neural network methods for PDEs lies in its automatic differentiation (AD), which necessitates only the sample points themselves, unlike traditional finite difference (FD) approximations that require nearby local points to compute derivatives. In this paper, we quantitatively demonstrate the advantage of AD in training neural networks. The concept of truncated entropy is introduced to characterize the training property. Specifically, through comprehensive experimental and theoretical analyses conducted on random feature models and two-layer neural networks, we discover that the defined truncated entropy serves as a reliable metric for quantifying the residual loss of random feature models and the training speed of neural networks for both AD and FD methods. Our experimental and theoretical analyses demonstrate that, from a training perspective, AD outperforms FD in solving PDEs.

In this paper, we present a novel training approach called the Homotopy Relaxation Training Algorithm (HRTA), aimed at accelerating the training process in contrast to traditional methods. Our algorithm incorporates two key mechanisms: one involves building a homotopy activation function that seamlessly connects the linear activation function with the activation function; the other technique entails relaxing the homotopy parameter to enhance the training refinement process. We have conducted an in-depth analysis of this novel method within the context of the neural tangent kernel (NTK), revealing significantly improved convergence rates. Our experimental results, especially when considering networks with larger widths, validate the theoretical conclusions. This proposed HRTA exhibits the potential for other activation functions and deep neural networks.

The numerical solution of differential equations using machine learning-based approaches has gained significant popularity. Neural network-based discretization has emerged as a powerful tool for solving differential equations by parameterizing a set of functions. Various approaches, such as the deep Ritz method and physics-informed neural networks, have been developed for numerical solutions. Training algorithms, including gradient descent and greedy algorithms, have been proposed to solve the resulting optimization problems. In this paper, we focus on the variational formulation of the problem and propose a Gauss-Newton method for computing the numerical solution. We provide a comprehensive analysis of the superlinear convergence properties of this method, along with a discussion on semi-regular zeros of the vanishing gradient. Numerical examples are presented to demonstrate the efficiency of the proposed Gauss-Newton method.

We present a higher-order space-time adaptive method for the numerical solution of the Richards equation that describes a flow motion through variably saturated media. The discretization is based on the space-time discontinuous Galerkin method, which provides high stability and accuracy and can naturally handle varying meshes. We derive reliable and efficient a posteriori error estimates in the residual-based norm. The estimates use well-balanced spatial and temporal flux reconstructions which are constructed locally over space-time elements or space-time patches. The accuracy of the estimates is verified by numerical experiments. Moreover, we develop the -adaptive method and demonstrate its efficiency and usefulness on a practically relevant example.

This paper is devoted to the numerical solution of constrained energy minimization problems arising in computational physics and chemistry such as the Gross-Pitaevskii and Kohn-Sham models. In particular, we introduce Riemannian Newton methods on the infinite-dimensional Stiefel and Grassmann manifolds. We study the geometry of these two manifolds, its impact on the Newton algorithms, and present expressions of the Riemannian Hessians in the infinite-dimensional setting, which are suitable for variational spatial discretizations. A series of numerical experiments illustrates the performance of the methods and demonstrates their supremacy compared to other well-established schemes such as the self-consistent field iteration and gradient descent schemes.

We propose a novel Model Order Reduction framework that is able to handle solutions of hyperbolic problems characterized by multiple travelling discontinuities. By means of an optimization based approach, we introduce suitable calibration maps that allow us to transform the original solution manifold into a lower dimensional one. The novelty of the methodology is represented by the fact that the optimization process does not require the knowledge of the discontinuities location. The optimization can be carried out simply by choosing some reference control points, thus avoiding the use of some implicit shock tracking techniques, which would translate into an increased computational effort during the offline phase. In the online phase, we rely on a non-intrusive approach, where the coefficients of the projection of the reduced order solution onto the reduced space are recovered by means of an Artificial Neural Network. To validate the methodology, we present numerical results for the 1D Sod shock tube problem, for the 2D double Mach reflection problem, also in the parametric case, and for the triple point problem.

We present a new approach for nonlinear dimensionality reduction, specifically designed for computationally expensive mathematical models. We leverage autoencoders to discover a one-dimensional (NeurAM) capturing the model output variability, through the aid of a simultaneously learnt surrogate model with inputs on this manifold. Our method only relies on model evaluations and does not require the knowledge of gradients. The proposed dimensionality reduction framework can then be applied to assist outer loop many-query tasks in scientific computing, like sensitivity analysis and multifidelity uncertainty propagation. In particular, we prove, both theoretically under idealized conditions, and numerically in challenging test cases, how NeurAM can be used to obtain multifidelity sampling estimators with reduced variance by sampling the models on the discovered low-dimensional and shared manifold among models. Several numerical examples illustrate the main features of the proposed dimensionality reduction strategy and highlight its advantages with respect to existing approaches in the literature.

We develop new multilevel Monte Carlo (MLMC) methods to estimate the expectation of the smallest eigenvalue of a stochastic convection-diffusion operator with random coefficients. The MLMC method is based on a sequence of finite element (FE) discretizations of the eigenvalue problem on a hierarchy of increasingly finer meshes. For the discretized, algebraic eigenproblems we use both the Rayleigh quotient (RQ) iteration and implicitly restarted Arnoldi (IRA), providing an analysis of the cost in each case. By studying the variance on each level and adapting classical FE error bounds to the stochastic setting, we are able to bound the total error of our MLMC estimator and provide a complexity analysis. As expected, the complexity bound for our MLMC estimator is superior to plain Monte Carlo. To improve the efficiency of the MLMC further, we exploit the hierarchy of meshes and use coarser approximations as starting values for the eigensolvers on finer ones. To improve the stability of the MLMC method for convection-dominated problems, we employ two additional strategies. First, we consider the streamline upwind Petrov-Galerkin formulation of the discrete eigenvalue problem, which allows us to start the MLMC method on coarser meshes than is possible with standard FEs. Second, we apply a homotopy method to add stability to the eigensolver for each sample. Finally, we present a multilevel quasi-Monte Carlo method that replaces Monte Carlo with a quasi-Monte Carlo (QMC) rule on each level. Due to the faster convergence of QMC, this improves the overall complexity. We provide detailed numerical results comparing our different strategies to demonstrate the practical feasibility of the MLMC method in different use cases. The results support our complexity analysis and further demonstrate the superiority over plain Monte Carlo in all cases.

We introduce an -version discontinuous Galerkin finite element method (DGFEM) for the linear Boltzmann transport problem. A key feature of this new method is that, while offering arbitrary order convergence rates, it may be implemented in an almost identical form to standard multigroup discrete ordinates methods, meaning that solutions can be computed efficiently with high accuracy and in parallel within existing software. This method provides a unified discretisation of the space, angle, and energy domains of the underlying integro-differential equation and naturally incorporates both local mesh and local polynomial degree variation within each of these computational domains. Moreover, general polytopic elements can be handled by the method, enabling efficient discretisations of problems posed on complicated spatial geometries. We study the stability and -version a priori error analysis of the proposed method, by deriving suitable -approximation estimates together with a novel inf-sup bound. Numerical experiments highlighting the performance of the method for both polyenergetic and monoenergetic problems are presented.

We develop a sparse hierarchical -finite element method (-FEM) for the Helmholtz equation with variable coefficients posed on a two-dimensional disk or annulus. The mesh is an inner disk cell (omitted if on an annulus domain) and concentric annuli cells. The discretization preserves the Fourier mode decoupling of rotationally invariant operators, such as the Laplacian, which manifests as block diagonal mass and stiffness matrices. Moreover, the matrices have a sparsity pattern independent of the order of the discretization and admit an optimal complexity factorization. The sparse -FEM can handle radial discontinuities in the right-hand side and in rotationally invariant Helmholtz coefficients. Rotationally anisotropic coefficients that are approximated by low-degree polynomials in Cartesian coordinates also result in sparse linear systems. e consider examples such as a high-frequency Helmholtz equation with radial discontinuities and rotationally anisotropic coefficients, singular source terms, țhe time-dependent Schrödinger equation, and an extension to a three-dimensional cylinder domain, with a quasi-optimal solve, via the Alternating Direction Implicit (ADI) algorithm.

This paper describes novel algorithms for the identification of (almost-)resonant behavior in scattering problems. Our methods, relying on rational approximation, aim at building surrogate models of what we call "field amplification", defined as the norm of the solution operator of the scattering problem, which we express through boundary-integral equations. To provide our techniques with theoretical foundations, we first derive results linking the field amplification to the spectral properties of the operator that defines the scattering problem. Such results are then used to justify the use of rational approximation in the surrogate-modeling task. Some of our proposed methods apply rational approximation in a "standard" way, building a rational approximant for either the solution operator directly or, in the interest of computational efficiency, for a randomly "sketched" version of it. Our other "hybrid" approaches are more innovative, combining rational-approximation-assisted root-finding with approximation using radial basis functions. Three key features of our methods are that (i) they are agnostic of the strategy used to discretize the scattering problem, (ii) they do not require any computations involving non-real wavenumbers, and (iii) they can adjust to different settings through the use of adaptive sampling strategies. We carry out some numerical experiments involving 2D scatterers to compare our approaches. In our tests, two of our approaches (one standard, one hybrid) emerge as the best performers, with one or the other being preferable, depending on whether emphasis is placed on accuracy or efficiency.

We present and analyze a discontinuous Galerkin method for the numerical modeling of a Kelvin-Voigt thermo/poro-viscoelastic problem. We present the derivation of the model and we develop a stability analysis in the continuous setting that holds both for the full inertial and quasi-static problems and that is robust with respect to most of the physical parameters of the problem. For spatial discretization, we propose an arbitrary-order weighted symmetric interior penalty scheme that supports general polytopal grids and is robust with respect to strong heterogeneities in the model coefficients. For the semi-discrete problem, we prove the extension of the stability result demonstrated in the continuous setting and we provide an a-priori error estimate. A wide set of numerical simulations is presented to assess the convergence and robustness properties of the proposed method. Moreover, we test the scheme with literature and physically sound test cases for applications in the geophysical context.

Free-stream preservation is an essential property for numerical solvers on curvilinear grids. Key to this property is that the metric terms of the curvilinear mapping satisfy discrete metric identities, i.e., have zero divergence. Divergence-free metric terms are furthermore essential for entropy stability on curvilinear grids. We present a new way to compute the metric terms for discontinuous Galerkin spectral element methods (DGSEMs) that guarantees they are divergence-free. The proposed mimetic approach uses projections that fit within the de Rham Cohomology.

In this paper, we derive a Fast Reflected Forward-Backward (Fast RFB) algorithm to solve the problem of finding a zero of the sum of a maximally monotone operator and a monotone and Lipschitz continuous operator in a real Hilbert space. Our approach extends the class of reflected forward-backward methods by introducing a Nesterov momentum term and a correction term, resulting in enhanced convergence performance. The iterative sequence of the proposed algorithm is proven to converge weakly, and the Fast RFB algorithm demonstrates impressive convergence rates, achieving as for both the discrete velocity and the tangent residual at the . When applied to minimax problems with a smooth coupling term and nonsmooth convex regularizers, the resulting algorithm demonstrates significantly improved convergence properties compared to the current state of the art in the literature. For convex optimization problems with linear cone constraints, our approach yields a fully splitting primal-dual algorithm that ensures not only the convergence of iterates to a primal-dual solution, but also a convergence rate of as for the objective function value, feasibility measure, and complementarity condition. This represents the most competitive theoretical result currently known for algorithms addressing this class of optimization problems. Numerical experiments are performed to illustrate the convergence behavior of Fast RFB.

We propose two implicit numerical schemes for the low-rank time integration of stiff nonlinear partial differential equations. Our approach uses the preconditioned Riemannian trust-region method of Absil, Baker, and Gallivan, 2007. We demonstrate the efficiency of our method for solving the Allen-Cahn and the Fisher-KPP equations on the manifold of fixed-rank matrices. Our approach allows us to avoid the restriction on the time step typical of methods that use the fixed-point iteration to solve the inner nonlinear equations. Finally, we demonstrate the efficiency of the preconditioner on the same variational problems presented in Sutti and Vandereycken, 2021.

This paper studies two hybrid discontinuous Galerkin (HDG) discretizations for the velocity-density formulation of the compressible Stokes equations with respect to several desired structural properties, namely provable convergence, the preservation of non-negativity and mass constraints for the density, and gradient-robustness. The later property dramatically enhances the accuracy in well-balanced situations, such as the hydrostatic balance where the pressure gradient balances the gravity force. One of the studied schemes employs an -conforming velocity ansatz space which ensures all mentioned properties, while a fully discontinuous method is shown to satisfy all properties but the gradient-robustness. Also higher-order schemes for both variants are presented and compared in three numerical benchmark problems. The final example shows the importance also for non-hydrostatic well-balanced states for the compressible Navier-Stokes equations.

In this paper, we introduce a quasi-Newton method optimized for efficiently solving quasi-linear elliptic equations and systems, with a specific focus on GPU-based computation. By approximating the Jacobian matrix with a combination of linear Laplacian and simplified nonlinear terms, our method reduces the computational overhead typical of traditional Newton methods while handling the large, sparse matrices generated from discretized PDEs. We also provide a convergence analysis demonstrating local convergence to the exact solution under optimal choices for the regularization parameter, ensuring stability and efficiency in each iteration. Numerical experiments in two- and three-dimensional domains validate the proposed method's robustness and computational gains with tensor product implementation. This approach offers a promising pathway for accelerating quasi-linear elliptic equations and systems solvers, expanding the feasibility of complex simulations in physics, engineering, and other fields leveraging advanced hardware capabilities.

We present an algorithm for fast generation of quasi-uniform and variable-spacing nodes on domains whose boundaries are represented as computer-aided design (CAD) models, more specifically non-uniform rational B-splines (NURBS). This new algorithm enables the solution of partial differential equations within the volumes enclosed by these CAD models using (collocation-based) meshless numerical discretizations. Our hierarchical algorithm first generates quasi-uniform node sets directly on the NURBS surfaces representing the domain boundary, then uses the NURBS representation in conjunction with the surface nodes to generate nodes within the volume enclosed by the NURBS surface. We provide evidence for the quality of these node sets by analyzing them in terms of local regularity and separation distances. Finally, we demonstrate that these node sets are well-suited (both in terms of accuracy and numerical stability) for meshless radial basis function generated finite differences discretizations of the Poisson, Navier-Cauchy, and heat equations. Our algorithm constitutes an important step in bridging the field of node generation for meshless discretizations with isogeometric analysis.

The computational effort entailed in the discretization of fluid-poromechanics systems is typically highly demanding. This is particularly true for models of multiphysics flows in the brain, due to the geometrical complexity of the cerebral anatomy-requiring a very fine computational mesh for finite element discretization-and to the high number of variables involved. Indeed, this kind of problems can be modeled by a coupled system encompassing the Stokes equations for the cerebrospinal fluid in the brain ventricles and Multiple-network Poro-Elasticity (MPE) equations describing the brain tissue, the interstitial fluid, and the blood vascular networks at different space scales. The present work aims to rigorously derive a posteriori error estimates for the coupled Stokes-MPE problem, as a first step towards the design of adaptive refinement strategies or reduced order models to decrease the computational demand of the problem. Through numerical experiments, we verify the reliability and optimal efficiency of the proposed a posteriori estimator and identify the role of the different solution variables in its composition.

We introduce two new lowest order methods, a mixed method, and a hybrid discontinuous Galerkin method, for the approximation of incompressible flows. Both methods use divergence-conforming linear Brezzi-Douglas-Marini space for approximating the velocity and the lowest order Raviart-Thomas space for approximating the vorticity. Our methods are based on the physically correct viscous stress tensor of the fluid, involving the symmetric gradient of velocity (rather than the gradient), provide exactly divergence-free discrete velocity solutions, and optimal error estimates that are also pressure robust. We explain how the methods are constructed using the minimal number of coupling degrees of freedom per facet. The stability analysis of both methods are based on a Korn-like inequality for vector finite elements with continuous normal component. Numerical examples illustrate the theoretical findings and offer comparisons of condition numbers between the two new methods.

We propose and study conformal integrators for linearly damped stochastic Poisson systems. We analyse the qualitative and quantitative properties of these numerical integrators: preservation of dynamics of certain Casimir and Hamiltonian functions, almost sure bounds of the numerical solutions, and strong and weak rates of convergence under appropriate conditions. These theoretical results are illustrated with several numerical experiments on, for example, the linearly damped free rigid body with random inertia tensor or the linearly damped stochastic Lotka-Volterra system.