Perfect face matching with Midjourney AI

Perfect face matching with Midjourney AI

Creating Matching Faces with AI: Exploring Midjourney, InsightFace, and Adobe Generative Fill

image

Image generation using AIs like Midjourney is shockingly good. But the AIs also have a few weaknesses. In this article, I’ll show you how you can work around one of those weaknesses. We’re going to learn how to make two or more characters with matching faces.

Face matching is surprisingly inconsistent, and there are a few different techniques that can be used. When I created my music promotion poses, I had to use some special techniques to get the graphics to work.

How we use Midjourney, InsightFace, and Adobe Generative Fill to create this look

In my recent article about using AI to create an album, I glossed over the techniques I used, giving a thousand-foot view of what they were. But in this article, I’m going to dive in and show you how to make it all happen, step by step.

First, I’ll show you what I did to create my images, and then I’ll show you some additional techniques.

Step one: Use a starter image

This technique uses an existing photograph as part of the image prompt, which gives Midjourney some guidance on what to create. Since I wanted my main album promotion images to look like me, I gave Midjourney a starter image.

To do this, double click on the plus icon at the bottom of the Discord screen (remember, Midjourney runs in Discord), choose your image, and then hit return.

Once it’s been brought into Midjourney, right click on the image and select Copy Link. This link is what you’ll use in preparing your prompt. Give Midjourney the /IMAGINE prompt, followed by the URL, and then your spec.

Please note that this set of images was most definitely not my first “roll” on Midjourney. Expect to do five, 10, even 20 different tries before you get something you like.

That said, you can see how the generated version looks reasonably close to what I look like. More interestingly, the leather jacket in the image looks weirdly like the one I’ve been wearing for the past decade or so. How does it know?

image

They’re all fine-looking gentlemen, but they look nothing like me.

You can use the starter-image trick for anything you want. Here’s what Midjourney thinks my little Yorkie Poo pup Pixel might have looked like as a Scottish warrior prince:

image

Step two: Get a seed ID

Each quad of images generated by Midjourney has its own ID, called a seed. If you want to create future images that are similar to an existing Midjourney image, you can try referencing the seed ID. It doesn’t always work, but it’s worth a try.

To get a seed ID, click the half-moon-plus icon at the upper right of the image where you want to get the seed. Then, in the search box, type in :envelope:. Then click on the envelope.

I know it’s completely intuitively unobvious, but this process will convince Midjourney to send you a message that contains the seed ID for the quad of images you’re working with.

image

It’s unfortunate that the seed is for the quad of images, because that makes it harder to drill into the one image you like. But it’s a starting point.

Step three: Create a second image

For my promotion, I wanted an image of me standing in front of my car. I drive a red Dodge Challenger and I thought that would be cool for my bio page.

This time, instead of starting with the actual photo of me, I used the generated profile image as my starting point. Then, I combined the profile image with the previously retrieved seed value and a prompt, and after about 20 spins, I got this quad.

Notice the /imagine prompt. The URL is at the beginning, and the seed prompt at the end had my seed number. That’s how I tied the two images together.

While the image on the lower left was absolutely terrible, the one on the right wasn’t bad. Strictly speaking, it didn’t look entirely like me. But it did capture the same thematic style as the original profile image, it was close enough, and the car looked good.

But this was not my face and most definitely not my hair.

Step four: Swap faces

To get the faces right, I used a tool called InsightFace, which also runs in Discord. Getting InsightFace set up is its own project, and rather than taking you down that rabbit hole here, I’ll point you to the tutorial I used to get started.

Once you have InsightFace set up, we can go to town. The process involves two steps: uploading the face you want to use, and uploading the image you want it applied to.

To upload the image you want to use, use the /saveid command. When you type /saveid, you’ll be given a place to upload an image. Then you need to give that image a name ID. I called mine “daviddoor” because I’m standing in front of a door. Remember that ID because you’ll use it in the next step.

Now, it’s time for the face swap. Type /swapid: Upload the image you want the face to be applied to. Then type in the ID you created earlier. If you’re lucky, once you hit return, you’ll get an image with the new face.

Of course, the hair still isn’t right.

Step five: Fix the hair

Finally, I needed to fix the hair. For this, I dropped into the Adobe Photoshop beta and used Generative Fill. Midjourney is great for creating entire images. Photoshop Generative Fill is ideal for placing new imagery on top of existing images and having it all blend together seamlessly.

The key problem was all the extra hair below the ears. To eliminate that, I first brought the Dave Head to something nobody had ever seen before: a bald me. I selected the space just where I wanted the hair removed, generated “bald,” and Photoshop did the work.

Then I selected a smaller area and generated “curly hair.” This took a few tries, but it worked out.

And, after all the various AI effects, the final picture is ready.

FAQs

Here are a few questions I’ve been asked since I published the original article.

Can I still use Midjourney for free?

No. Midjourney no longer has a free tier. The plan I use costs $8 a month. So far, I haven’t hit any limits with it, although it sometimes can take a little while for images to be created.

What about InsightFace? Is it free?

InsightFace does offer some free credits. I’ve done four or five face swaps and haven’t used them up. It appears to take about three “credits” per face swap, and you get 50 free credits.

Where does Picsi.ai fit into all this?

When you run out of face swap credits, you’ll need to subscribe to a Patreon account for Picsi.ai. There are two plans. The base $10/month plan looks like it will do fine for most people. As to the relationship between InsightFace and Picsi.ai, it appears that InsightFace is an open-source project, and Picsi.ai has created a usable Discord client for it. But I’m sure more will be revealed over time.

Have you used any of these AI image tools? What do you think of the face swap technique? Does this spark any ideas for you? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter, Facebook, Instagram, and YouTube.