Deep Learning

Objectives

This course aims at introducing artificial neural networks as well as their extension known as deep learning and providing practical skills to tackle a machine learning task using artificial neural networks. Beforehand, real-world applications of deep learning are presented, and the differences between artificial intelligence, machine learning, and deep learning are discussed. The main concepts of an artificial neural network (layers, activation functions, architectures) and how to train one (stochastic gradient descent, batches, momentum) are then defined. The following most common architecture families are presented: multilayer perceptrons, convolutional neural networks, recurrent neural networks, (variational) autoencoders, generative adversarial networks, transformers, and diffusion models. The introduction of each architecture family is motivated by concrete examples of relevant machine learning tasks. During practical sessions, such concrete examples of relevant machine learning tasks will be addressed using artificial neural networks. The materials for the practical sessions are Jupyter notebooks, the programming language that will be used is Python and the Python packages that will be used are PyTorch and its ecosystem.

Course outline

rn – Illustrative examples of deep learning – Differences between artificial intelligence, machine learning and deep learning – Basic concepts of artificial neural networks: layers, activation functions, architectures – Training an artificial neural network: stochastic gradient descent and its variants – Common neural network architecture families: multilayer perceptrons, convolutional neural networks, recurrent neural networks, (variational) autoencoders, generative adversarial networks, transformers, diffusion models – Applications: computer vision, signal processing, natural language processing

Prerequisites

Basic linear algebra (vectors, matrices, matrix multiplication), gradient descent, basic Python (object-oriented programming in Python would be a plus but is not required)