BICYCLEDAYDESIGN.COM (Proof I made one and it works!)
Here is how I installed Stable Diffusion running ControlNet on my Mac. I am running an M2 processor and I would suggest checking the requirements here before getting in too deep. There are other ways to run stable diffusion and I chose the Automatic1111 version because it is the most complicated to install but has the most capabilities. The install process was the most difficult part. I am still very new when it comes to stable diffusion but I'm learning and everything makes more sense now that I have it on my computer. I am going to try and explain the process by sharing the links I used and hopefully it is useful. If you have any questions feel free to reach out and I will try my best to help but I'm very new to using it myself.
You have to install things via your terminal and run code that is completely foreign to me. I would be lying if I said that I wasn't a little paranoid about getting a virus or getting hacked. As I was trying to figure this out I ran across a few sites that asked for money or info which is completely unnecessary. I have CleanMyMac installed on my computer and I used it to run malware scans on everything I'm posting here and so far so good. Maybe the malware is writing this to give you a false sense of security and I'm furiously trying to delete my website as you read this to stop the hackers - proceed at your own risk!
The first thing you need to do is install Python on your computer which will install a Python launcher.
Python Virtual Machine (PVM) first understands the operating system and processor in the computer and then converts it into machine code. Further, these machine code instructions are executed by a processor and the results are displayed in Terminal. The download for mac is located here: https://www.python.org/downloads/macos/
I then installed Homebrew which supplements the missing package manager for macOS (or Linux) that comes standard on a PC to the best of my understanding. You need to open Terminal on your Mac which can be found searching "Terminal" using finder (⌘f) and then follow the instructions from the following site. You will have to scroll down until you get to "Install AUTOMATIC1111 on Mac instructions" and follow the instructions to install ControlNet: https://stable-diffusion-art.com/install-mac/
The site points you to the Homebrew site and once there it will explain what will happen and also mentions it will pause in Terminal before each execution which is true. Follow all of the instructions for "Install AUTOMATIC1111" until you get to the part where you generate a cat as a test (do this and it will generate a cat picture). You now have stable diffusion on your computer and hopefully a picture of a generated cat similar to Midjourney. It will tell you to close terminal but you will want to install some more stuff first.
It will explain on the site where to put the model file which you have hopefully done but just to clarify it will be in stable-diffusion-webui > models > Stable-diffusion
Now that you have Stable Diffusion on your computer you will need to install models that have been trained to achieve certain results. There is actually a model made specifically to create QR Codes. I installed that one first and then installed the more common one you see people post QR codes from. Focusing on QR Codes turned out to be a great way for me to wrap my brain around the Stable Diffusion concept but if you aren't as interested in that you can get other trained models that will create portraits, etc. and it's pretty much the same process.
You will need to download a pre-trained ControlNet model. I got all of my files from this point on from: https://huggingface.co/
You will then need to put the files in the right folder on your computer for Stable Diffusion to access them. I personally decided to tag them with a color code just to make it easier in case I accidentally put them in the wrong place and needed to find them quickly. I will post the video I followed but if you want to skip ahead these are the names you can search on huggingface.co. Once you watch the video and use them it will start to make more sense and you will understand all of the different ControlNet models that are out there. Some of the files the following YouTube videos will reference:
Then you will need to put them in your models folder which will be stable-diffusion-webui > extensions > sd-webui-controlnet > models
Here is a rough overview of the Stable Diffusion interface. All of the callouts are meant to help but it also makes it look more confusing than it actually is so you'll have to check it out for yourself:
Before you attempt to create a QR Code with stable diffusion set yourself up for success with the best code possible. https://qrcode.antfu.me/ is a site where you can adjust the error correction to high (H) and also boost the error correction which are good things for Stable diffusion. You can also add a margin of 4 which will help it read better.
You really have to play around with the Stable Diffusion settings because a great looking QR Code might not scan and you have to find the happy balance. The codes are also harder to scan if you are too close or zoomed in too much compared to traditional QR Codes.
Below is a video that explains how to use the QR Code Monster ControlNet model which has been created specifically for QR Codes. It is good at creating these photorealistic QR Codes. In the video there is added post-production of steam and extending the image around the original image.
Also to get an idea of how prompts work inside Stable Diffusion they shared this link: https://pastebin.com/nqxCh4hP You can see examples of positive and negative prompts used with QR Code Monster. I will note that there are some spelling mistakes that are easy to fix but something to be aware of.
After messing around with the above version I had a better idea of how Stable Diffusion works and I watched this video to create more of the anime-style QR codes.
I'm not sure if this is "the right way" like the thumbnail states but I thought this creator had an interesting take on creating the QR Codes:
Hopefully this page helped you get Stable Diffusion up and running. It's super confusing to install so feel free to reach out with any questions if this doesn't make sense and hopefully it can get to a place that makes Stable Diffusion easier to understand.
QR Codes don't even begin to scratch the surface of what's possible but it's a good entry point and who knows what it will lead to in this wild AI journey we're all on together.
