AIC 53rd Annual Meeting

How is machine learning used to create works of art? How do machine learning technologies work? What are the various software tools and programming languages that are available to artists? What are the conservation problems that arise with each of these techniques?

The software applications that artists use for creating works of art which integrate or are based on machine learning fall into several categories. For artists who do not know how to program, or prefer not to program, and/or do not have opportunities to collaborate with programmers, there are text-to-image applications in which an artist creates images generated through textual description. Examples from the New York City bitforms gallery exhibition DALL·E: Artificial Imagination (October 26–Dec 29, 2022) demonstrate this approach.[1] 

Newly created images using text-to-image techniques can be loosely based on predefined styles provided by the software authors or company; or the artist can “train” a model to use style-transfer based on the artist’s own original digital-born images or digital surrogates of physical artworks in order to instruct the software to computationally mimic the artist’s own or another style. The Whitney Museum of American Art’s xhairymutantx Embedding 2024 by Holly Herndon and Mat Dryhurst (2024) is an example of style-transfer.[2]

With advanced beginner or intermediate programming skills, artists may prefer writing original code such as Python scripts to generate new images based on style-transfer and other techniques. An artist at this level of programming skill can also write scripts to programmatically download images from the web that meet specific textual criteria, e.g., “watercolors of pink roses.” 

Building an original machine learning application requires great resources and advanced computational and programming skills. The artist Refik Anadol, in his talk at the Institute of Fine Arts in New York City on June 3, 2024, said that it can take over six months of teamwork at his studio to compile data and build the application for a work such as Unsupervised, which was exhibited at the Museum of Modern Art in New York City.[3] Managing a team to retrieve and prepare data sets, as well as collaborating with programmers to run and train machine learning models, requires extensive studio resources including hardware for data storage and processing, bespoke software that addresses the artist’s vision, and a staff with appropriate expertise. 

Each of these approaches brings up a specific set of questions regarding acquisition practices, documentation practices, preparation for future re-exhibition, and other conservation concerns. Answering these and other questions, focusing on the collaboration between institutions and collectors with artists and engineers, leads to conservation strategies for these fragile and complex artworks, as artists continue to explore the use of machine learning as an artistic medium.

[1] https://bitforms.art/exhibition/dall%C2%B7e-artificial-imagination/ 

[2] https://whitney.org/exhibitions/xhairymutantx

[3] https://www.moma.org/calendar/exhibitions/5535