Deep Unsupervised Learning

CycleGAN

CycleGANU-NetPatchGANInstanceNorm
CycleGAN
Problem

Real-world domain translation often lacks paired examples—you have selfies and anime portraits, but not pixel-aligned pairs. CycleGAN asks: can two generators learn A→B and B→A mappings such that round-tripping preserves content, without ever seeing matched training pairs?

Dataset

Selfie2Anime dataset: unpaired domains with trainA/trainB (3,400 images) and testA/testB (100 images). Domain A = real selfies, domain B = anime portraits. Images resized to 128×128, tensorized, and normalized to [-1, 1]. Each batch samples random images from each domain independently—no alignment required.

Approach

Two U-Net generators (G_AB: selfie→anime, G_BA: anime→selfie) with InstanceNorm and LeakyReLU, plus two PatchGAN discriminators (D_A, D_B) using MSE (LSGAN-style) targets. Generator loss combines adversarial terms, cycle-consistency L1 (λ_cycle=10), and identity L1 (λ_id=5) so that G_BA(G_AB(x)) ≈ x and generators leave in-domain images unchanged. InstanceNorm replaces BatchNorm for better GAN stability on small batches.

Training

Batch size 16, 100 epochs (~213 steps/epoch). Adam lr=0.0002, β1=0.5, β2=0.999. λ_cycle=10, λ_identity=5. Discriminator and generator updated alternately each step. Training was noted as unstable with unusual loss curves—consistent with mode-collapse risk in adversarial unpaired setups.

My Work
  • Built UnpairedImageDataset sampling random images from domains A and B per batch
  • Implemented GeneratorUNet with 7 encoder blocks, bottleneck, and 6 decoder blocks with skip connections
  • Implemented dual PatchGAN discriminators with InstanceNorm
  • Wired cycle-consistency and identity losses into the generator objective
  • Evaluated on 10 test-set pairs and 3 internet selfies + 3 internet anime images outside the training distribution
  • Plotted generator and discriminator loss curves over 100 epochs
Evaluation
  • 100-image held-out test split: bidirectional selfie ↔ anime translation
  • 6 out-of-distribution internet images (3 selfies + 3 anime) not seen during training
  • Sample strip of translated outputs to inspect style transfer across diverse inputs
CycleGAN translations on held-out test set
Test set — selfie → anime and anime → selfie
CycleGAN translations on internet-sourced images
Out-of-distribution internet selfies and anime portraits
CycleGAN translation sample strip
Extended translation samples across inputs
Results
  • 3,400 unpaired training images per direction; 100-image held-out test split
  • Bidirectional translations on test set and out-of-distribution web images
  • Training instability observed—loss curves suggest possible mode collapse during adversarial optimization
  • Qualitative: anime style transfer preserves face pose and composition in many samples
CycleGAN generator and discriminator loss curves
Generator and discriminator losses over 100 epochs
Takeaway

Cycle consistency is the glue that makes unpaired translation possible—it penalizes content drift when you cannot supervise with aligned pairs. The selfie↔anime results were the most visually striking demo of the course, even though GAN training instability reminded me these systems need careful monitoring. Unpaired translation trades data convenience for optimization fragility.

Work completed during Oxford summer coursework. This page shows only my own implementations and outputs; course materials are not redistributed.