Ai in art, 2 approaches / two perspectives

 

Ai in art, 2 approaches / two perspectives

The expression or application of human creativity and imagination, usually in visual form such as painting or sculpture, to create works valued primarily for their beauty or emotional power, often how people think of art. In this article, we describe two approaches to art creation based entirely on AI participation.

    Mostly emerging in the mid-2010s, the most common types of AI art revolve around processing images, recognizing aspects like color, texture, and text. There are many types of AI that can create Art. Some of them would be: General Adversarial Network (GAN), Convolutional neural networks (CNN), Neural style transfer (NST). We will talk more on GAN and NST as they are two different methods which offer a whole different perspective on how images are created.

Generative Adversarial Network (GAN)

    A generative adversarial network (GAN) is a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in June 2014. Two neural networks contest with each other in the form of a zero-sum game, where one agent's gain is another agent's loss.The algorithm finishes training in the moment when the predictor has a 50% rate in discovering if the generating image is from the data set or not. 

    The generator model takes a fixed-length random vector as input and generates a sample in the domain. The vector is drawn randomly from a Gaussian distribution, and the vector is used to seed the generative process.

    After training, points in this multidimensional vector space will correspond to points in the problem domain, forming a compressed representation of the data distribution. This vector space is referred to as a latent space, or a vector space composed of latent variables. Latent variables, or hidden variables, are those variables that are important for a domain but are not directly observable

    The two models, generator and discriminator, are trained together. The generator generates a sequence of samples which, along with real samples in the domain, are fed to the discriminator and classified as real or fake. The discriminator is then updated to better distinguish between real and fake samples in the next round, and importantly, the generator is updated according to how well the generated samples fool the discriminator

Neural style transfer (NST)

    Neural style transfer (NST) refers to a class of software algorithms that manipulate digital images, or videos, in order to adopt the appearance or visual style of another image. NST algorithms are characterized by their use of deep neural networks for the sake of image transformation.

   Common uses for NST are the creation of artificial artwork from photographs, for example by transferring the appearance of famous paintings to user-supplied photographs. Several notable mobile apps use NST techniques for this purpose, including DeepArt and Prisma.

    This method has been used by artists and designers around the globe to develop new artwork based on existent style(s). NST is an example of image stylization, a problem studied for over two decades within the field of non-photorealistic rendering. The first two example-based style transfer algorithms were image analogies  and image quilting. Both of these methods were based on patch-based texture synthesis algorithms. 

    

To represent the style of an input image, we utilize a feature space originally intended to represent texture information. This area is built on top of the filter responses in each layer of the network. It's composed of the relationships between the different filter responses across the spatial extent of the feature maps (see Methods for more information). By including the correlations of multiple layers, we obtain a stationary, multi-scale representation of the input image that captures its texture information but not the overall structure.

Each layer of units can be understood as a collection of image filters, each of which extracts a certain feature from the input image. Thus, the output of a given layer consists of so-called feature maps: differently filtered versions of the input image. When Convolutional Neural Networks are trained on object recognition, they develop a representation of the image that makes object

Given a training pair of images–a photo and an artwork depicting that photo–a transformation could be learned and then applied to create new artwork from a new photo, by analogy. If no training photo was available, it would need to be produced by processing the input artwork; image quilting did not require this processing step, though it was demonstrated on only one style. 

When Convolutional Neural Networks are trained on object recognition, they develop a representation of the image that makes object information increasingly explicit along the processing hierarchy. Therefore, along the processing hierarchy of the network, the input image is transformed into representations that increasingly care about the actual content of the image compared to its detailed pixel values. We can directly visualize the information each layer contains about the input image by reconstructing the image only from the feature maps in that layer. Higher layers in the network capture the high-level content in terms of objects and their arrangement in the input image but do not constrain the exact pixel values of the reconstruction. In contrast, reconstructions from the lower layers simply reproduce the exact pixel values of the original image. When synthesizing an image that combines the content of one image with the style of another, there usually does not exist an image that perfectly matches both constraints at the same time.  

Thus, we can smoothly adjust the emphasis on refactoring of content or style. Strong emphasis causes the image to match the look of the artwork, effectively giving it a textured version, but when placed, the content of the photo rarely shows the strong emphasis and you can clearly identify the photo, but the style doesn't match. For a given pair of source images, the content-versus-style trade-off can be adjusted to create visually pleasing images.                                                                                                               

 

Conclusion

These are two methods through which is generating art.  The first one generates new pictures based on an existing set of data while the second one combines photos with real life paintings in order to offer a  new version of your photo.

Bibleography:

https://arxiv.org/pdf/1508.06576.pdf

https://www.domestika.org/en/blog/10352-what-is-ai-art-how-artists-use-ai-and-how-to-generate-your-own