Thoughts and Explorations
Barbara Schussmann

Abstract by AI

Written on October 18, 2021

Over the past few weeks, I've been exploring artificial intelligence and machine learning techniques. In this text, I write about a brief experiment on generating abstract art using StyleGAN.

As both an designer and newcomer to the field of machine learning I am fascinated by what the technology can achieve yet somewhat overwhelmed by its complexity. Therefore, I used RunwayML, an a tool that aims to make Machine Learning accesible to everyone without the need to write code.

I get very excited about how to train algorithms to generate new text, images, and videos, which is why I've been looking into machine learning a bit. I sometimes feel uninspired, especially when it comes to being creative on the fly. Unfortunately, my creative skills regarding creating art are not very high, so I was thinking about how to use machine learning to generate abstract art. With this in mind, I wondered if ML could be used to spark creativity?

Technology has influenced art a lot in the last few years, especially lately, where terms such as NFT and machine learning are increasingly finding their place in the context of art. It is clear that technologies will influence art to an increasing extent. But in the context of machine learning, I often think to myself, what will a mix between Van Goth and Mark Rothko look like? Or whether everyone will soon be able to put their own computer-generated Picasso in the living room.

Working with a Generative Adversarial Network (GAN)

What I needed were some
(1) source images,
(2) an ML model (StyleGAN), and
(3) a lot of time to train the model (4 hours).

Input Images

As input I used my Pinterest folder and various images I found on Instagram under the hashtag #abstractart. After that, I had to curate the images further, based on whether they had a frame, a wrong format etc. Over the past few years, I've created folders on various platforms where I collect abstract art. I took a total of 300 images as input - which is actually not much to train a StyleGAN model.

Running model.
Conclusion: It's obvious that pink is my favorite color.

Background: Machine Learning class

A few weeks ago we had a machine learning class at UID where we got introduced to AI, object recognition, and RunwayML. We started training our model with Google's teachable machine, explored ml5js and existing models and libraries. After this class, I got curious about machine learning and decided to explore it a bit further.

Early exploration: Experimenting with COCO-SSD, a pre-trained multiple object detector.
Short exploration: A tangible mood tracker I created using Teachable machine and p5.js. Tracking colors with Teachable Machine was quite a challenge due to the constantly changing lighting conditions.

Conclusion

It was fun to explore machine learning and especially learn about StyleGAN, which I was curious about since I've seen it on many platforms these days. I thought it was much more complicated to use and understand than it actually is (on a surface level of course). However, for me, I haven't really found a use for StyleGAN yet, aside for rather artistic purposes.