Thanks so much for this! This was an exact problem my team had at a recent hackathon, and we decided to use diffusion instead. We theorised that the initial image parameter might help, but we didn't have time to test it. Documenting this was really helpful. If you have any other articles or advice on using VQGAN (or other generative models) with CLIP (or other language embeddings) I'd love to see them.

Replying again as I think my previous message didn't appear: Glad you found the article useful. While Diffusion can produce more coherent images, I prefer VQGAN as it's much more malleable. I'm writing a whole series of posts on AI generative art, so let me know if there's anything it would be useful to cover.

Thanks, I'm glad you found it useful. I'm working on a series of posts about VQGAN+CLIP, so if there's anything you'd like me to cover please do let me know.

Using objects on white background as init image is also quite helpful to isolate the subject you want to generate.

That's what I did here with sea slugs and sex toys mix: https://www.instagram.com/p/CWqvXHXj4sM/

That’s a great looking blog - what’s it built with?

Why is the author's sibling comment[0] that answers the parent's question [dead]?

> Hey thanks! It's built with Jekyll and Netlify. The galleries are all driven from a Google Sheet as a sort of DIY database.

Is there some context I'm missing, or was it killed by the system?

[0]: Might need to enable show dead: https://news.ycombinator.com/item?id=30943675

Thanks for reposting! I created an account to reply, but I seem to have been shadowbanned. Not sure why, I've emailed the mods to ask what's up.

Thanks for re-posting, much appreciated.

Hey thanks! It's built with Jekyll and Netlify. The galleries are all driven from a Google Sheet as a sort of DIY database.

Now do conditioning GLIDE with initial images. :P