Yeah, the neural network "diffusion models" are not very well named. If you have background in natural sciences, you would understand diffusion to mean, well, diffusion. Whereas there generative neural networks are about (1) blurring data by Gaussian noise, (2) teaching a NN to denoise the noised data, and finally (3) with Gaussian noise as input, let the denoiser NN to generate new data. So it's not so much about diffusion as it's about reversing the diffusion. And it's not really (smooth) diffusion, but Gaussian noise.
"Denoising autoencoder" is already used for processes that reconstruct partially corrupted input. So what name to suggest for a process that reconstructs data from nothing but noise?
It still doesn’t work because even the latest AI tech is unable to understand the complex rules of what problematic content is.
How can you identify socially acceptable bias. For example, you’d expect it to be biased towards cars with 4 wheels vs rare 3 wheeled cars, but how does it know that bias is ok but being biased to male lawyers isn’t.
It's an almost impassible task. ML is reflecting the world and the data in the world. If you ask for an anime style man, they will pretty much universally generate white men because the dataset of anime characters almost universally contains white characters. The model isn't wrong, its generating exactly what exists in the world already. And there are an infinite number of scenarios and biases that it reflects which you will never be able to manually flag.
It reminds me a lot of the early self driving car debate where there were endless surveys asking if the car should run over the 2 old ladies or the one child studying medicine. And in the end we decided it was an unreasonable burden and just accepted that ML doesn't need to make impossible moral judgements.
Exactly! I’m being downvoted, but I’m sincerely suggesting that we train a full spectrum of ideologically biased censorship engines and then let people pick which ones they want to use.
https://distill.pub/2020/growing-ca