I'm surprised, I found clojure even while I was a pretty much total beginner programmer fairly easy to get into, via clojure for the brave and true at least was very good for getting the basics.
Clojure for the brave and true shows the exact problem.
Chapter 1: install and learn the JVM
Chapter 2: Emacs
Chapter 3, finally start learning Clojure if you made through the first two chapters. Those first two chapters are going to filter out quite a few potential Clojure learners.
