My AI project: Wait... what? Why?

I've kicked off the beginning of a generative image model (AKA AI image maker) that is trained off all 210+ of my works. I'm so excited to introduce this project to you, but also want to touch on the "AIness" of it all.

So wait... what is this?

The idea for this project started at the end of 2023 and was a huge part of why I built Prolific (I'll do a blog post about that at some point)! I've titled it "gen.em.otion" (said broken up like gen em otion) and honestly, I really just wanted to play! The inspiration for this project came in two parts:

1. As a technologist, developer, and general nerd I want to understand what goes into crafting a generative model and gain some basic understanding of how “AI” works behind the scenes. I strongly believe that, while this "bubble" might burst soon, "AI" in general isn't going anywhere.
2. As an artist I want to make a statement about “AI art” and I want to feel the emotions of training this model and watching it output in the style I’ve refined over the last 5+ years. A style I’ve poured so much of my life, time, and heart into. I don’t know what the statement is yet, that’s part of the journey.

The final piece will be a local (no cloud or internet needed) custom application that's powered by this model I will have made from my art. People come, type a prompt, and it generates art in my style that will vanish after a few minutes.

This model/application won't be public. (If you want to copy my style, go for it! Buy you gotta work for it 😉) But I plan to demo it here and there and everywhere, and show it in future gallery shows! I'm also not going to use this in lieu of putting in the time and emotional work into my art. This model/app is the art. The project of building it is the art. I have no interest in using it to make pieces.

But like... isn't AI 🤮?

Okay stay with me here! Like broadcast television, the internet, smartphones, or the introduction of microcomputers to the home, I strongly believe that AI will transform the way we live and work. For better or worse. It may not look like it does right now in this AI bubble/boom we’re in, but it isn’t going away.

The elephant, is she in the room with us right now? 🐘

Here's a potentially spicy take: Not all AI is good. Not all AI is used for good. For example, the internet is full of lifeless, meaningless AI slop you can’t escape, not to mention how AI-powered tech is used by governments worldwide for unspeakable acts.

But AI isn’t going anywhere. We either learn how to use it, tolerate it, work with it, et cetera or we risk being left behind. I really believe that. The environmental impacts are also a huge concern to me. The model I am making is a type of generative image model called a “LoRA” model. These models are smaller and focus on something specific (like an artistic style). They sit on top of an existing, bigger model. My estimates are that training this model will require <24 hours of compute time when running on an Nvidia RTX 4000 GPU (a graphics card you can buy for content creation and daily use). Compared to ChatGPT for example, my model’s impact will be… nearly nonexistent.

I want you to know I'm super excited about this project. It's a journey, it's learning, it's feeling the big feelings. Rest assured, I’m weighing pros and cons, thinking about impact, and working with the emotions. This isn’t a frivolous project. I hope you’ll follow along.

If you do want to follow this journey, be it for the tech or the emotional art or both, please subscribe to my brand new newsletter! I have many posts planned where I'll share the process of brainstorming, progress I'm making in getting my training data squared away, scripts I've written to help me out, and everything else along to way. (The newsletter is only 1-2 times per month and is free!)

Some nerdy details for my nerds

I’m building my training dataset with the help of my art management software, Prolific. It keeps track of all the training descriptions and what pieces have and haven’t been processed in a MySQL database. Lots of Node.js scripts are in the mix too, helping with things like image conversion, error catching, and SQL-to-TXT exporting. After the training data is ready, I’ll use DigitalOcean/Paperspace GPUs (specifically the Nvidia RTX 4000) to train the model (I don’t have a device that will train a model in any reasonable amount of time and after researching I think this is the most environmentally friendly way to do it). I’ll likely use kohya_ss to build the model and then Automatic1111 to act as a backend for the generation of images. The gen.em.otion model will likely sit on top of Stable Diffusion XL 1.0 base. Finally I am building an Electron frontend that will communicate with the local Automatic1111 backend. The project will be designed to run on an Apple Silicon Mac (likely an M1 Pro Mac mini in gallery settings).