Tangents - Deep Learning Methodologies

I attended an extremely well presented talk on Generative Adversarial Networks (GANs) yesterday, an approach for deep learning in the area of machine learning and artificial intelligence. This area of science is of use to the computational anthropologist as a means of better understanding large data sets. These data sets include the mixed media of images, text, music and movies that are increasingly critical to understanding modern culture and communication. There is very impressive work in this area that could eventually open up these media to anthropological analysis.
 

The talk was given by Jennifer Sleeman from the University of Maryland, Baltimore Campus, at George Mason University's Foundation Hall and hosted by Data Science DC, https://www.meetup.com/Data-Science-DC/. Her presentation was an excellent introduction to Generative Adversarial Networks for the layman. The methodology was first introduced by Ian Goodfellow and his colleagues, http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf 

GANs are two neural networks, discriminatory and generative, that compete to produce ‘real’ images. They are set up in a game theory dynamic and play a minimax zero-sum game. The Discriminator, starting with real training images, maps input images from the generator network to a desired output and is provides probabilistic feedback. The Generator, starting with an image of random noise, creates an image and compares it with feedback from the discriminator. Over time the Generator produces better images based on the feedback. In effect the Discriminator trains the Generator to produce the best image. A nice example of this can be found here: https://oshearesearch.com/index.php/2016/07/01/mnist-generative-adversarial-model-in-keras/

GAN Architecture, Source: O'Shea Research

I am struck by how creatively this methodology utilizes game theory to improve the machine learning process in neural networks. It is also an impressive example of how computers can learn and provides us with a model of how learning can occur in humans. Humans use creativity and innovation to produce new ideas and devices, and then we compare it to our concrete knowledge and understanding of the world. Over time the continuous generation of new ideas leads to novel human behavior that changes culture.

GANs can currently be used to classify images and create new data sets. It is limited to smaller image resolutions, 64x64, and images, and has problems with animal body parts, 3-D images and global structure. However work in this area continues with application to text and music data. 

More example code:

http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/

Useful Python libraries for deep learning and GANs include Keras, Torch, and TensorFlow.

Finally, for those interested in getting up to speed on GANs, here is a link to a free online book that provides the basics on deep learning: http://www.deeplearningbook.org/