I have found that I have my best (and worst) ideas by changing mental states. For example I’m usually sober so when I’m intoxicated I usually have garbage ideas, but every now and then I think of something truly incredible/useful that I would never have had insight into sober. The same applies to being sleep deprived, it leads me into a different mental headspace where I can think differently than normal (though I try to avoid being sleep deprived cause it’s miserable)
There is something about sleep deprivation that makes me so vividly visually creative, things and image flow in my mind like they otherwise never do. Unfortunately I haven't found a way to trigger this any other way, so my only real creative moments tend to happen in the middle of the night where they aren't much use to anyone.
I'm guessing the added visuals come from hypnagogia, the state between waking and sleep where you can see imagery related to dreaming. I'm especially aware of this state because I have aphantasia and thus have no other method of seeing imagery in my head that doesn't pass through my retinas first. I've seen it when I thought I was awake, but later realize I wasn't fully awake. I also used to dabble in lucid dreaming, and spent a lot of time trying to ride the line between being awake and falling asleep. I'd suggest learning how to lucid dream, because that's another environment where you can be visually creative.
AFAIK sleep deprivation is discussed as a short term treatment of depression. I don't have any exclusive papers on hand (I'm not an expert), but if you are interested, please do your own research.
Recently I had to wake up at 3am and drive for a couple of hour. Something I never do. I had the strangest and most interesting insights! It was very similar to what you describe, a different mental state.
However, unfortunately I forgot many of the insights since I was so endlessly tired! Should have taken notes while driving with just 3h of sleep…
I've seen some people using a dictaphone for this kind of thing. I'm sure these days you can get a phone app that would even auto-transcribe your 3AM ramblings. :)
Very interesting you said that here. I regurarly walk aimlessly, I regurarly have this impression that i have to skip a day of sleep to be exceptionally productive, and i also used to be obseessed with digging large holes especially when i was younger and able to dig like that in my parents garden.
As a person that often exhibits characteristics and symptoms of depression (but has never been officially diagnosed), I find that staying awake for well over a day can clear my symptoms and reset my brain. I Scott Alexander has written an essay on this [1].
