Animating with Artificial Intelligence

Over the past few years, we’ve seen artificial intelligence progress to the point where it is not only helping with logical processes involving numbers and predictive outcomes, but now exploring the world of creativity, from musical scores to digital artworks with fascinating results. This was originally thought to be the one area whereby artificial intelligence could not compete with humans and, although it’s not perfected yet, it’s evolving and improving to the point where it’s starting to influence the creative industry. This raises questions around how much further it can go and if it will eventually become commercialised. Could an artist simply fill out a few lines of code and let AI create the masterpiece? Who gets creative and intellectual rights? Will we reach a point where AI generated art is indistinguishable from a human’s?

Google and NVIDIA have really been pushing forward with this technology, with various applications which you can try out yourself such as does this person exist and Jukebox AI, but the applications I have been interested in specifically is the creation of images from text prompts. I originally started playing around with simple programs like Dream from Wombo Art, which produces images from short, basic text prompts. The outcome is very abstract but you can see how the AI tries to create the image from the input.

Image created by Wombo Art with the text prompt "English Countryside"

However, more recently I discovered Disco Diffusion which offers more user customisation, producing images from very specific text prompts, to the point where you can define the artist you want to take inspiration from. The program was created by Somnai_dreams as part of an ever-evolving, open-source collaboration, allowing the user to get under the hood and see how it works.

For my first attempt, I asked the AI to create an image from the text input “a beautiful painting of rustic flowers by James Jean, with an orange colour scheme”. The outcome showed how the AI had pieced together the elements of my request and combined it with the style of the artist James Jean, with hints of facial features matching his style. As you inspect the images further, you can see that these facial features are just abstract shapes and our brains are filling in the gaps. It’s almost as if a snapshot of a dream had been created.

The next step was to see how this would look as an animation. Disco Diffusion offers inputs to provide simple transitioning, such as rotating, zooming and panning and creates a series of images as frames from the text prompt which can be compiled to create an animation. Starting with some tests, the results are even more dream-like.

Applications for AI Digital Art

We're really only seeing the birth of this technology coming into the hands of users who are just playing around with the tools. There isn't much practical application for this yet, outside of artistic output, and the process can be quite long and arduous, but as the tech improves and evolves, we could start to see entire animations created by AI that follow a story. Could a user eventually input an entire script for the AI to create? How detailed could the images be and how much control would the user have? Considering how this technology has improved over the course of a few years, we can most likely expect this growth to continue, becoming more user-friendly and real practical use for marketing, media and art.