A standard VAE learns a single latent distribution over all images. For controllable generation we need to steer outputs toward a specific class—e.g. produce new “frog” or “truck” images on demand. The challenge is conditioning both encoding and decoding while balancing reconstruction fidelity against a well-regularized latent space.
CIFAR-10 training set (50,000 images, 32×32 RGB, 10 classes). Images are resized to 32×32 and scaled to [0, 1]. Class labels are converted to 10-dimensional one-hot vectors used as conditioning signals in both the encoder and decoder paths.
I built ImprovedConvCVAE with latent dimension 128. Conditioning works by broadcasting the one-hot label to 32×32 and concatenating it as extra input channels on the encoder (13 channels total). In the decoder, the latent vector and label are concatenated before the fully connected expansion. The encoder uses four stride-2 conv blocks (13→64→128→256→512) with BatchNorm and LeakyReLU, outputting μ and log σ² for the reparameterization trick. The decoder mirrors this with transposed convolutions back to 3×32×32. Loss is β-VAE: BCE reconstruction (summed over pixels) plus β times KL divergence.
Adam, learning rate 1e-3, batch size 128, 200 epochs (~391 steps/epoch). β = 0.1 to prioritize reconstruction over latent regularization—higher β values produced blurrier outputs in early experiments. Average combined loss tracked per epoch.
- Designed ImprovedConvCVAE with deeper conv stacks and BatchNorm after an initial architecture produced poor samples
- Implemented one-hot label conditioning via channel concat (encoder) and vector concat (decoder)
- Applied the reparameterization trick z = μ + σ ⊙ ε for differentiable sampling
- Tuned β from higher values down to 0.1 to sharpen reconstructions while keeping latents usable
- At inference, sampled z ~ N(0, I) per class and decoded with fixed one-hot labels to generate 10 images × 10 classes
- 100 class-conditional samples: 10 images per CIFAR-10 class from random latent draws z ~ N(0, I)
- Visual check that decoded outputs reflect the requested class label across all 10 categories
- Training loss: 1852.05 (epoch 1) → 1732.95 (epoch 200)
- Generated 100 class-conditional samples (10 per CIFAR-10 class) from random latent draws
- Qualitative: distinct class cues (vehicles, animals, scenes) visible though fine detail remains limited at 32×32
Conditioning turns a VAE from a generic image sampler into a steerable generator—but the β knob is real. Lower β gave sharper, more recognizable class outputs; higher β pushed latents toward a smoother prior at the cost of detail. I left with a concrete feel for the reconstruction–regularization trade-off that papers describe abstractly.
Work completed during Oxford summer coursework. This page shows only my own implementations and outputs; course materials are not redistributed.