![Generative vs Discriminative models](Images/Generative_vs_Discriminative_models.png)
- Discriminative models -> classifiers
- The noise is larger to ensure that what's generated isn't actually the same dog each time
- if you're only generating one class, one Y of a dog, then you probably don't need this conditioning on Y and instead it's just the probability over all the features X
![two of the most popular generative model architectures](Images/Generative_Models.png)
The two models in a GAN compete with each other and reach a point where realistic examples are produced by the generator.
In the end, the Generator is not needed anymore, and only some random noise is used as input to generate some realistic image.
Only the parameters of the Discriminator are updated!!
The discriminator looks at real and fake images over time, makes guesses, and gets feedback on whether its guess was right or wrong.
Over time, it learns to discern real from fake better, but note that since the generator is also learning, the fake images get more realistic and harder to discern. This cat and mouse game enables both models to learn in tandem.
With feedback from the discriminator on whether a fake image looks real or fake, the generator starts producing fake images that are more and more realistic (that tries to fool the discriminator). Images that look “fake” to the discriminator are discarded in favor of those that look “real” to the discriminator; since the discriminator is improving over time as it sees more reals and fakes, the generator needs to continually learn too.
So as you alternate their training, only **one model is trained at a time**, while the other one is held constant. So in training GANs in this alternating fashion, it's important to keep in mind that **both models should improve together and should be kept at similar skill levels** from the beginning of training.
One common issue is having a **superior discriminator**, having this discriminator learn too quickly. And when it learns too quickly and it suddenly looks at a fake image and says, this is 100% fake. This 100% is not useful for **the generator at all because it doesn't know which way to grow and learn**