Overview of GAN Structure

A generative adversarial network (GAN) has two parts:

  • The generator learns to generate plausible data. The generated instances become negative training examples for the discriminator.
  • The discriminator learns to distinguish the generator's fake data from real data. The discriminator penalizes the generator for producing implausible results.

When training begins, the generator produces obviously fake data, and the discriminator quickly learns to tell that it's fake:

Three columns are labeled 'Generated Data', 'Discriminator', and
          Real Data'. Under 'Generated Data' a blue rectangle contains
          a squiggle and a circle. The rectangle is the
          generator's first, bad attempt
          to draw a dollar bill. Under 'Real Data' there's a picture of a real
          ten dollar bill. Under 'Discriminator' are the words 'FAKE' and
          'REAL'. An arrow points from the word 'FAKE' to the picture under
          'Generated Data'. Another arrow points from the word 'REAL' to the
          picture under 'Real Data'.

As training progresses, the generator gets closer to producing output that can fool the discriminator:

This image adds a new row under the 'Generated Data', 'Discriminator',
          and 'Real Data' headings in the previous image. Under 'Generated Data'
          there is a green rectangle with the number 10 in the upper left corner
          and a simple drawing of a face. Under 'Real Data' there's a picture of
          a real 100 dollar bill. Under 'Discriminator' is the word 'FAKE' with
          an arrow pointing to the picture under 'Generated Data' and the word
          'REAL' with an arrow pointing to the picture under 'Real Data'.

Finally, if generator training goes well, the discriminator gets worse at telling the difference between real and fake. It starts to classify fake data as real, and its accuracy decreases.

This image adds a new row under the 'Generated Data', 'Discriminator',
          and 'Real Data' headings in first previous image. Under 'Generated
          Data' there is a picture of a twenty dollar bill. Under 'Real Data
          there's a picture of a twenty dollar bill. Under 'Discriminator'
          is the word 'REAL' with
          an arrow pointing to the picture under 'Generated Data' and the word
          'REAL' with an arrow pointing to the picture under 'Real Data'.

Here's a picture of the whole system:

A diagram of a generative adversarial network. At the center of the
          diagram is a box labeled 'discriminator'. Two branches feed into this
          box from the left.  The top branch starts at the upper left of the
          diagram with a cylinder labeled 'real world images'. An arrow leads
          from this cylinder to a box labeled 'Sample'. An arrow from the box
          labeled 'Sample' feeds into the 'Discriminator' box. The bottom branch
          feeds into the 'Discriminator' box starting with a box labeled 'Random
          Input'. An arrow leads from the 'Random Input' box to a box labeled
          'Generator'. An arrow leads from the 'Generator' box to a second
          'Sample' box. An arrow leads from the 'Sample' box to the
          'Discriminator box. On the right side of the Discriminator box, an
          arrow leads to a box containing a green circle and a red circle. The
          word 'Real' appears in green text above the box and the word 'False'
          appears in red below the box. Two arrows lead from this box to two
          boxes on the right side of the diagram. One arrow leads to a box
          labeled 'Discriminator loss'. The other arrow leads to a box labeled
          'Generator loss'.

Both the generator and the discriminator are neural networks. The generator output is connected directly to the discriminator input. Through backpropagation, the discriminator's classification provides a signal that the generator uses to update its weights.

Let's explain the pieces of this system in greater detail.