Dimension Reduction and Matrix Completion
Objectives
n modern machine learning, data is increasingly high-dimensional (images, text, sensor streams) and increasingly unlabeled, so two questions become central: how to extract a compact, meaningful structure from raw data, and how to learn useful representations without relying on costly human annotation. Dimension reduction addresses the first question, moving from linear projections such as PCA, which capture directions of maximum variance but fail when the underlying structure is nonlinear, to neighborhood-embedding methods such as t-SNE and UMAP, which preserve local similarity structure and are the standard tools for visualizing and exploring high-dimensional datasets. Representation learning addresses the second question: rather than relying on labels, self-supervised methods define pretext tasks directly from the data itself, and this course focuses on contrastive approaches such as SimCLR, together with the alignment and uniformity framework that explains why such methods avoid trivial solutions, and on sparse autoencoders, which learn compact and disentangled features through a reconstruction objective. This course will state the main principles behind these techniques while emphasizing their methodological and algorithmic aspects.
Course outline
– Understand the curse of dimensionality and the motivation for both linear and nonlinear dimension reduction.rn- Know PCA, its assumptions, and its limitations on nonlinear data.rn- Understand neighborhood embedding methods (t-SNE, UMAP) and their algorithmic principles.rn- Understand the principles of self-supervised and contrastive representation learning.rn- Know the SimCLR framework and the alignment/uniformity theoretical perspective.rn- Understand sparse autoencoders as a non-contrastive route to interpretable representations.
Prerequisites
Basic statistics, linear algebra and probability; basic notions of neural networks;.