Machine learningLesson 1 of 1 in the course Unsupervised learning

Autoencoder Architectures, Training, Regularization, and Applications

5 flashcards · 5 quiz questions · 5 match pairs · 5 fill-in sentences · English

Save course
Lesson progress
0 / 5

Your progress is kept in this browser only.Sign up to save it

Summary

This lesson introduces autoencoders, their two-part architecture, and the compressed representations they learn. These ideas are the foundation for understanding how autoencoders are trained and used.

An autoencoder is a neural network that learns to represent input data and then reconstruct it. It is typically trained on unlabeled examples, so it is commonly associated with unsupervised learning. The network is given an input, such as an image, and learns a representation that preserves information useful for recreating that input.

All flashcards

Autoencoder
A neural network that encodes an input into a representation and decodes it to reconstruct the input.
Undercomplete autoencoder
An autoencoder whose code space has fewer dimensions than its input space.
Sparse autoencoder
An autoencoder that encourages most entries in an activation vector to be near zero.
Denoising autoencoder
An autoencoder trained to reconstruct clean examples from corrupted inputs.
Semantic hashing
Using low-dimensional binary codes to index items for efficient retrieval.

Quiz questions

  1. What reconstruction measure is used by L2 reconstruction loss?Answer: The squared Euclidean distance between the input and its reconstruction.
  2. How does a denoising autoencoder train to reconstruct an example?Answer: It receives a corrupted version and is trained to reproduce the clean example.
  3. How does a variational autoencoder represent latent variables?Answer: With probability distributions in a probabilistic generative model.
  4. What is a concrete autoencoder designed to learn?Answer: A user-specified number of selected input features.
  5. Under squared reconstruction error, what is the relationship between an optimally trained linear autoencoder with bottleneck size pp and PCA?Answer: Its bottleneck spans the subspace of the first pp principal components.

Based on the Wikipedia article Autoencoder, licensed CC BY-SA 4.0. Adapted into a summary, flashcards and quizzes.

More in Machine learning

All Machine learning lessons
Machine learningSupport Vector Machines: Margins, Kernels, Optimization, and Extensions5 flashcards · 5 quiz questions · 4 gamesPlay free →Machine learningNeural-Network Activation Functions: Properties and Families5 flashcards · 5 quiz questions · 4 gamesPlay free →