Advanced Deep Learning

Artificial intelligence (AI) and deep learning applications have proliferated and are having an increasing impact on industry, sciences, and engineering. This course expounds on those trends and enables students to engage in advanced research and development in AI and deep learning. We investigate essential concepts and topics such as large language models (LLMs), generative adversarial networks (GANs, stable diffusion, or text-to-speech), graph neural networks (GNNs), and differentiable applications in natural science. For important classes of neural networks, we explore the fundamental mechanisms behind their operations and provide practical illustrations of their uses. For example, we review the structure of transformer-based pretrained LLMs, the principles of attention, and their use in applications such as ChatGPT, with a focus on understanding prompt programming. For GANs, we examine the generation of realistic representations of people, speech, paintings, and music. For GNNs, we dive into the analysis of chemical molecules, proteins, and drugs and quantitative structure property relationships in physical systems. We learn how to impose constraints that are reflections of physical or geometric laws governing physical systems. Concepts introduced in every lecture are illustrated by practical examples. Code samples used in lectures and homework assignments are written in PyTorch and occasionally in Keras 3.