Topics

<> machine learning (15)

David Gomez-Cabrero

Associate Professor, Bioscience
Professor Cabrero's research focuses on developing statistics and machine learning tools for understanding biological systems that address health-related applications, advance biomedical research and enhance our understanding of the molecular mechanisms of diseases with the potential to uncover novel therapeutic approaches.

Hayssam Dahrouj

Senior Research Scientist at KAUST (King Abdullah University of Science and Technology)

Karen Sanchez

Postdoctoral Research Fellow
Karen Sanchez is a Postdoctoral Researcher in the IVUL lab at KAUST, specializing in deep learning, machine learning, and artificial intelligence (AI) for healthcare applications. Her research focuses on video understanding, domain adaptation, generative AI, and methods for preserving patient privacy.

Hani A. Al Majed

M.S. Student
Electrical and Computer Engineering Master's student at KAUST under the supervision of Professor Bernard Ghanem, with expertise in machine learning workflow automation, physics-informed neural operators, and AI for Science, particularly in chemistry applications. Passionate about leveraging AI to solve complex scientific problems.

Ahmad Sait

M.S. Student, Computer Science
Ahmad Sait is a Master's student in the Computer Science program at KAUST. Before joining KAUST, Ahmad obtained his Bachelor's degree in Computer Science from King Abdulaziz University, Saudi Arabia. Ahmad is also a KAUST Academy graduate from the AI track, having been under the supervision of Prof. Naeemullah Khan. The Academy provided him with seven opportunities to assist in teaching students enrolled in subsequent cohorts.

Saleh AlJazzar

Research Fellow at King Abdullah University of Science and Technology (KAUST) .

Muhammad Esmat

Visiting student at, King Abdullah University of Science and Technology (KAUST).

Yasser Jaber

Visiting Student Researcher ,Machine Learning Engineer at KAUST (King Abdullah University of Science and Technology).
Syed Abidullah Fareed

Syed Abidullah Fareed

Visiting Student (former), Information Science Lab

Ahsan Mehmood

Intern (former), Information Science Lab
Visiting student at King Abdullah University of Science and Technology (KAUST).

Abdulkarem Badiea Alshaibani

Visiting Student at the Information Science Lab at the King Abdullah University of Science and Technology (KAUST).

On the resolution of a theoretical question related to the nature of local training in federated learning

-

B1 L3 R3119

In this talk, I will explain the problem, its solution, and some subsequent work generalizing, extending and improving the ProxSkip method in various ways. We study distributed optimization methods based on the local training (LT) paradigm - achieving improved communication efficiency by performing richer local gradient-based training on the clients before parameter averaging - which is of key importance in federated learning. Looking back at the progress of the field in the last decade, we identify 5 generations of LT methods: 1) heuristic, 2) homogeneous, 3) sublinear, 4) linear, and 5) accelerated. The 5th generation, initiated by the ProxSkip method of Mishchenko et al (2022) and its analysis, is characterized by the first theoretical confirmation that LT is a communication acceleration mechanism.

Models, Methods, Applications: HPC in Medicine, Science, Engineering, and AI

-

B3 L5 R5209

Abstract The accurate numerical prediction of complex processes in medicine, science, and engineering requires a well balanced interplay of mathematical models, numerical methods, and scientific software. In this talk, we discuss along different examples from engineering, earth-sciences, and medicine the various challenges in building predictive models. Our focus will be on the algorithmic aspects, i.e. on the numerical methods, -may it be finite elements or neural networks in the context of massive parallelism. We will present in particular parallel preconditioners for non-linear problems

Course: A Mathematical Introduction to Deep Learning

1 min read ·
I will give an elementary introduction of basic deep learning models and training algorithms from a mathematical viewpoint. In particular, I will relate some basic deep learning models with finite element and multigrid methods. I will also touch on some advanced topics to demonstrate the potential of new mathematical insight and analysis for improving the efficiency of deep learning technologies and, in particular, for their application to numerical solution of partial differential equations. Course Schedule View the full course schedule here.

Juncai He

I am currently a research scientist in Computer, Electrical and Mathematical Science and Engineering Division (CEMSE) at King Abdullah University of Science and Technology (KAUST). I received a B.S. degree in Mathematics and Applied Mathematics from Sichuan University in 2014. In the Summer of 2019, I received my Ph.D. degree in Computational Mathematics under the supervision of Prof. Jinchao Xu and Prof. Jun Hu at Peking University in Beijing, China. From 2019 to 2020, I worked as a Postdoctoral Scholar supervised by Prof. Jinchao Xu in The Center for Computational Mathematics and Application
<> Deep learning (9) <> Computer Vision (6)

Abdelrahman Eldesokey

Postdoctoral Research Fellow
I am a Postdoctoral Fellow at KAUST, Saudi Arabia, specializing in Generative AI and conducting research on advanced topics in Generative and Large Language Models (LLMs). I bring over 11 years of combined academic and industrial experience in Computer Vision and Deep Learning across Sweden, Egypt, and Saudi Arabia.
Najmeddine Dhieb

Najmeddine Dhieb

Karen Sanchez

Postdoctoral Research Fellow
Karen Sanchez is a Postdoctoral Researcher in the IVUL lab at KAUST, specializing in deep learning, machine learning, and artificial intelligence (AI) for healthcare applications. Her research focuses on video understanding, domain adaptation, generative AI, and methods for preserving patient privacy.

Aznaur Aliev

M.S. Student

Hani A. Al Majed

M.S. Student
Electrical and Computer Engineering Master's student at KAUST under the supervision of Professor Bernard Ghanem, with expertise in machine learning workflow automation, physics-informed neural operators, and AI for Science, particularly in chemistry applications. Passionate about leveraging AI to solve complex scientific problems.

Ahmad Sait

M.S. Student, Computer Science
Ahmad Sait is a Master's student in the Computer Science program at KAUST. Before joining KAUST, Ahmad obtained his Bachelor's degree in Computer Science from King Abdulaziz University, Saudi Arabia. Ahmad is also a KAUST Academy graduate from the AI track, having been under the supervision of Prof. Naeemullah Khan. The Academy provided him with seven opportunities to assist in teaching students enrolled in subsequent cohorts.
<> artificial intelligence (5)
Najmeddine Dhieb

Najmeddine Dhieb

Yasmeen Alsaedy

M.S. Student, Computer Science
She is an AI researcher specializing in generative AI and deep learning. With a strong academic foundation and hands-on experience in machine learning, data science and artificial intelligence, she is passionate about advancing AI applications in real-world scenarios. She has contributed to cutting-edge projects, published research, and collaborated on innovative solutions that bridge the gap between AI theory and practice. Her expertise includes designing and implementing advanced data models, AI-driven visual/text computing, and deep learning architectures.

Yasser Jaber

Visiting Student Researcher ,Machine Learning Engineer at KAUST (King Abdullah University of Science and Technology).
Syed Abidullah Fareed

Syed Abidullah Fareed

Visiting Student (former), Information Science Lab

Ahmed Youssef Ragab Radwan

Visiting Student (former), Information Science Lab
Visiting student at the Information Science Lab, in the Electrical Engineering Department, CEMSE, King Abdullah University of Science and Technology (KAUST)
<> AI (4) <> Signal processing (4) <> haps (3)

Bang Huang

Postdoctoral Research Fellow, Electrical and Computer Engineering
Bang Huang works as a postdoctoral fellow at the Computer Electrical and Mathematical Sciences and Engineering Division (CEMSE), King Abdullah University of Science and Technology (KAUST), under the supervision of Prof. Mohamed-Slim Alouini in the Communication Theory Lab (CTL). Education and Early Career Bang Huang received the M.S. degree (with honors) in electrical engineering and Ph.D degree in information and communication engineering from the School of Information and Communication Engineering (SICE), University of Electronic Science and Technology of China (UESTC), Chengdu, China, in

Xiaowei Pang

Postdoctoral Research Fellow, Electrical and Computer Engineering
Xiaowei Pang is currently a Postdoctoral Research Fellow at the Computer Electrical and Mathematical Sciences and Engineering Division (CEMSE), King Abdullah University of Science and Technology (KAUST), working with Professor Mohamed-Slim Alouini in the Communication Theory Lab (CTL). Education and Early Career Xiaowei Pang received her B.S. degree in Telecommunication Engineering from Northwestern Polytechnical University, Xi'an, China, in 2018, and her Ph.D. degree in Information and Communication Engineering from Dalian University of Technology, Dalian, China, in 2024. From 2021 to 2022
H

Hussein Al-Shatri

I chose KAUST because of its reputation for excellence in research and its recent emphasis on applied research. With my extensive background in both academic research and industrial development, I am excited to contribute my experience and ambition. My goal is to foster new ideas and develop innovative prototypes in the field of wireless communications, which will undoubtedly enhance people's lives.
<> mathematical optimization (3)

On the resolution of a theoretical question related to the nature of local training in federated learning

-

B1 L3 R3119

In this talk, I will explain the problem, its solution, and some subsequent work generalizing, extending and improving the ProxSkip method in various ways. We study distributed optimization methods based on the local training (LT) paradigm - achieving improved communication efficiency by performing richer local gradient-based training on the clients before parameter averaging - which is of key importance in federated learning. Looking back at the progress of the field in the last decade, we identify 5 generations of LT methods: 1) heuristic, 2) homogeneous, 3) sublinear, 4) linear, and 5) accelerated. The 5th generation, initiated by the ProxSkip method of Mishchenko et al (2022) and its analysis, is characterized by the first theoretical confirmation that LT is a communication acceleration mechanism.

Optimization for Deep Learning

-

B5 L5 R5209

Abstract The field of optimization for machine learning has undergone significant changes in recent years with deep learning models increasing in scale and fine-tuning taking a more prominent role. In this presentation, I will share a perspective on the direction of changes in the field and highlight interesting research directions. I will provide real-world examples of what practitioners want from optimization methods to train deep networks at scale. I will then present my recent work on adaptive methods, such as Adam and Adagrad, and explain how we can estimate the learning rate for these

Jongho Park

I am a research scientist in Computer, Electrical and Mathematical Science and Engineering Division (CEMSE) at King Abdullah University of Science and Technology (KAUST). I received a B.S. degree in Mathematical Sciences/Computer Science in 2013, and M.S. and Ph.D. degrees in Mathematical Sciences in 2015 and 2019, respectively (Advisor: Chang-Ock Lee), all from KAIST. From 2019 to 2020, I was a postdoctoral researcher in Department of Mathematical Sciences at KAIST (Mentor: Chang-Ock Lee). From 2020 to 2023, I was a research assistant professor in Natural Science Research Institute at KAIST
<> MATLAB (3) <> Multigrid (3)

Neural Operators: Theory, Architecture, and Applications for PDEs

-

B1 L3 R3119

Abstract Neural operator methods provide a novel approach for solving or learning the complex mappings from parameters to solutions arising from intricate physical systems. In this talk, I will cover the foundational aspects of neural operators, encompassing both theoretical frameworks and algorithmic developments, including some well-known neural operator architectures. Additionally, I will share our recent work on applying the neural operator method to multiscale partial differential equations (PDEs). To tackle the challenges of multiscale PDEs, we have developed a neural operator with a

Multigrid Monte Carlo Revisited: Theory and Bayesian Inference

-

B1 L3 R3119

Abstract The fast simulation of Gaussian random fields plays a pivotal role in many research areas such as Uncertainty Quantification, data science and spatial statistics. In theory, it is a well understood and solved problem, but in practice the efficiency and performance of traditional sampling procedures degenerates quickly when the random field is discretised on a grid with spatial resolution going to zero. Most existing algorithms, such as Cholesky factorisation samplers, do not scale well on large-scale parallel computers. On the other hand, stationary, iterative approaches such as the

Cell-centered extrapolation cascadic multigrid method for solving diffusion equation with discontinuous coeffcients

Xiaoxin Wu, Central South University, China

-

https://meeting.tencent.com/dm/bpUiLe2AlgBC

This seminar is an online interview for a postdoctoral position in KAUST.
<> numerical analysis (3)
Georgios Grekas

Georgios Grekas

Juncai He

I am currently a research scientist in Computer, Electrical and Mathematical Science and Engineering Division (CEMSE) at King Abdullah University of Science and Technology (KAUST). I received a B.S. degree in Mathematics and Applied Mathematics from Sichuan University in 2014. In the Summer of 2019, I received my Ph.D. degree in Computational Mathematics under the supervision of Prof. Jinchao Xu and Prof. Jun Hu at Peking University in Beijing, China. From 2019 to 2020, I worked as a Postdoctoral Scholar supervised by Prof. Jinchao Xu in The Center for Computational Mathematics and Application

Jongho Park

I am a research scientist in Computer, Electrical and Mathematical Science and Engineering Division (CEMSE) at King Abdullah University of Science and Technology (KAUST). I received a B.S. degree in Mathematical Sciences/Computer Science in 2013, and M.S. and Ph.D. degrees in Mathematical Sciences in 2015 and 2019, respectively (Advisor: Chang-Ock Lee), all from KAIST. From 2019 to 2020, I was a postdoctoral researcher in Department of Mathematical Sciences at KAIST (Mentor: Chang-Ock Lee). From 2020 to 2023, I was a research assistant professor in Natural Science Research Institute at KAIST
<> PDEs (3) <> Reinforcement Learning (3) <> adaptive methods (2) <> AI4Science (2) <> applied mathematics (2) <> approximation (2)

Deep Neural Network Approximation via Function Compositions

-

KAUST

Abstract Deep neural networks have brought transformative changes in areas such as image recognition and healthcare, yet their underlying theoretical principles remain somewhat elusive. The main emphasis of this presentation is to explore the theoretical dimensions of deep neural networks, with a special focus on evaluating their approximation capabilities. We begin by assessing the approximation errors in ReLU networks when they attempt to approximate various target functions, including (Lipschitz) continuous functions, polynomials, and smooth functions. Utilizing the concept of Vapnik

Approximation and Generalization Errors in Deep Neural Networks for Sobolev Spaces measured by Sobolev Norms

-

KAUST

Abstract In this presentation, we initially discuss the approximation capabilities of deep neural networks (DNNs) with ReLU and the square of ReLU as activation functions for Sobolev functions measured by the Sobolev norms \(W^{m,p}\) where \(m \ge 1\). Subsequently, we consider how to address the issue of the curse of dimensionality for DNNs’ approximation. Finally, we analyze the generalization errors associated with DNNs using such Sobolev loss functions. Additionally, we provide recommendations on when to opt for deeper NNs versus wider NNs, considering factors such as the number of sample
<> Arabic natural language processing (2) <> deep neural networks (2)

Deep Neural Network Approximation via Function Compositions

-

KAUST

Abstract Deep neural networks have brought transformative changes in areas such as image recognition and healthcare, yet their underlying theoretical principles remain somewhat elusive. The main emphasis of this presentation is to explore the theoretical dimensions of deep neural networks, with a special focus on evaluating their approximation capabilities. We begin by assessing the approximation errors in ReLU networks when they attempt to approximate various target functions, including (Lipschitz) continuous functions, polynomials, and smooth functions. Utilizing the concept of Vapnik

Approximation and Generalization Errors in Deep Neural Networks for Sobolev Spaces measured by Sobolev Norms

-

KAUST

Abstract In this presentation, we initially discuss the approximation capabilities of deep neural networks (DNNs) with ReLU and the square of ReLU as activation functions for Sobolev functions measured by the Sobolev norms \(W^{m,p}\) where \(m \ge 1\). Subsequently, we consider how to address the issue of the curse of dimensionality for DNNs’ approximation. Finally, we analyze the generalization errors associated with DNNs using such Sobolev loss functions. Additionally, we provide recommendations on when to opt for deeper NNs versus wider NNs, considering factors such as the number of sample
<> Generative AI and LLMs (2) <> gradient methods (2)

Unveiling Insights from "Gradient Descent Converges Linearly for Logistic Regression on Separable Data"

-

B1 L0 R0118

Abstract In this presentation, I will share a paper titled "Gradient Descent Converges Linearly for Logistic Regression on Separable Data", a work highly related to my ongoing research. I will explore its relevance to my current research topic and discuss the inspiration for our future works. Abstract of the paper: We show that running gradient descent with variable learning rate guarantees loss f(x) \leq 1.1f(x^*)+\epsilon for the logistic regression objective, where the error \epsilon decays exponentially with the number of iterations and polynomially with the magnitude of the entries of an

On the Natural Gradient Descent

-

KAUST

Abstract Numerous problems in scientific computing can be formulated as optimization problems of suitable parametric models over parameter spaces. Neural network and deep learning methods provide unique capabilities for building and optimizing such models, especially in high-dimensional settings. Nevertheless, neural networks and deep learning techniques are often opaque and resistant to precise control of their mathematical properties in terms of architectures, hyperparameters, etc. Consequently, optimizing neural network models can result in a laborious hyperparameter tuning process that
<> healthcare (2) <> IoT (2) <> ISAC (2) <> LiDAR (2) <> Localization (2) <> Network Design (2) <> Non-Terrestrial Networks (2) <> numerical methods (2) <> optimization (2) <> Physics-informed Neural Networks (2)

A Combination of Physics-informed Neural Networks with the Fixed-stress Splitting Iteration for Solving Biot’s Model

-

KAUST

Abstract In this study, we aim to solve Biot’s consolidation models by employing physics-informed neural networks. Based on the fixed-stress splitting method, loss functions are designed for the displacement variable and the pressure variables separately. Such a strategy enables us to train two independent and smaller neural networks. The coupling of different variables is realized via the fixed-stress iterative algorithm. Numerical experiments are provided to show that the fixed-stress iterative algorithm can improve the performance of neural network approximations of the solution of Biot’s

Physics-informed Neural Networks for Learning the Homogenized Coefficients of Multiscale Elliptic Equations

-

KAUST

Abstract Multiscale elliptic equations with scale separation are often approximated by the corresponding homogenized equations with slowly varying homogenized coefficients (the G-limit). The traditional homogenization techniques typically rely on the periodicity of the multiscale coefficients, thus finding the G-limits often requires sophisticated techniques in more general settings even when multiscale coefficient is known, if possible. Alternatively, we propose a simple approach to estimate the G-limits from (noisy-free or noisy) multiscale solution data, either from the existing forward
<> Programming language (2) <> quantum communication (2) <> Wireless Communications (2) <> 5G networks (1) <> 6G On-chip reconfigurable intelligent surface (1) <> 6G On-chip reconfigurable transmitarray and phased array (1) <> 6g wireless systems (1) <> Action Recognition (1) <> Action understanding (1) <> adaptive control (1) <> Adaptive Optics (1) <> Additive Schwarz methods (1) <> Aerial Robotics (1) <> AI for healthcare (1) <> algorithms (1) <> Applied Machine Learning (1) <> Applied Partial Differential Equations (1) <> Arbitrageurs (1) <> beamforming (1) <> biological models (1) <> biomechanics (1) <> Biomedical Signal Processing (1) <> bioscience (1) <> biotechnology (1) <> BLE communication (1) <> Boundaries (1) <> Cahn-Hilliard equations (1) <> calculus of variations (1) <> Cellular Networks (1) <> Chaos Shift Keying (1) <> cloud- and fog-radio access networks (1) <> communication (1) <> communication networks (1) <> communication systems (1) <> communications (1) <> Compressed Sensing Approach (1) <> computational bioscience (1) <> Computational mathematics (1) <> computational methods (1) <> computer networks (1) <> Continual Learning (1) <> convex optimization (1) <> Cooperative networks (1) <> Critical Phenomena (1) <> cross-layer optimization (1) <> cybersecurity (1) <> data analysis (1) <> Data Analyst (1) <> Deep generative models (1) <> development (1) <> Diffusion Models (1) <> Digital Communications (1) <> Digital signal processing (1) <> Dynamical Systems (1) <> elasticity (1) <> Electron Transport (1) <> Elgenvalue (1) <> embedded systems (1) <> Embedded systems. (1) <> energy (1) <> energy efficiency (1) <> epigenetics (1) <> FDA-MIMO Radar (1) <> FEMSimulation (1) <> Financial (1) <> Finite elements (1) <> fluid dynamics (1) <> Free (1) <> free boundary problems (1)