approximation

Deep Neural Network Approximation via Function Compositions

-

KAUST

Abstract Deep neural networks have brought transformative changes in areas such as image recognition and healthcare, yet their underlying theoretical principles remain somewhat elusive. The main emphasis of this presentation is to explore the theoretical dimensions of deep neural networks, with a special focus on evaluating their approximation capabilities. We begin by assessing the approximation errors in ReLU networks when they attempt to approximate various target functions, including (Lipschitz) continuous functions, polynomials, and smooth functions. Utilizing the concept of Vapnik

Approximation and Generalization Errors in Deep Neural Networks for Sobolev Spaces measured by Sobolev Norms

-

KAUST

Abstract In this presentation, we initially discuss the approximation capabilities of deep neural networks (DNNs) with ReLU and the square of ReLU as activation functions for Sobolev functions measured by the Sobolev norms \(W^{m,p}\) where \(m \ge 1\). Subsequently, we consider how to address the issue of the curse of dimensionality for DNNs’ approximation. Finally, we analyze the generalization errors associated with DNNs using such Sobolev loss functions. Additionally, we provide recommendations on when to opt for deeper NNs versus wider NNs, considering factors such as the number of sample