I don't know what is it about math -- especially when it involves manipulation of symbols as opposed to pictures or lay language -- that turns off so many people.
I can tell you at least part of it, from my subjective perspective. I tend to "think" in a very verbal fashion and I instinctively try to sub-vocalize everything I read. So when I see math, as soon as I see a symbol that I can't "say" to myself (eg, a greek letter that I don't recognize, or any other unfamiliar notation) my brain just tries to short-circuit whatever is going on, and my eyes want to glaze over and jump to the stuff that is familiar.
OTOH, with written prose, I might see a word I don't recognize, but I can usually work out how to pronounce it (at least approximately) and I can often infer the meaning (at least approximately) from context. So I can read prose even when bits of it are unfamiliar.
There's also the issue that math is so linear in terms of dependencies, and it's - in my experience - very "use it or lose it" in terms of how quickly you forget bits of it if you aren't using it on day-in / day-out basis.
My way of dealing with this is to treat symbols as proxies for the verbal concept, rather than just some letters. As an example, when I see "E = mc^2" I read (energy-of-object) = (mass-of-object) * (speed-of-light)^2 and not "Eee equals Em Cee Square". Another great idea I use a lot when writing/reading is David Mermin's 2nd rule (verbalize the damn equation!) [1].
It's sad that many mathematical resources do not make a careful effort of helping someone reason verbally. I guess this is partly due to the fact that most people who are skilled in the subject and write about it prefer equational reasoning (for lack of a better word) to verbal reasoning! In my experience as a physics instructor for non-STEM majors, this might be one of the biggest impediments for otherwise intelligent people trying to learn math/physics.
Very good point about reading/verbalizing symbols and notation. Once you know what they mean, they are super useful for expressing complex concepts precisely and compactly, but when you're getting started they look like an alien language...
This is why I'm using Anki to memorize the Greek alphabet, and to keep basic algebraic (h.s. algebra that is) stuff in mind. It might seem like a small thing, but remembering the various rules for factoring, working with fractions, dealing with exponents / root, etc. is not easy when you don't do math all the time.
I can tell you at least part of it, from my subjective perspective. I tend to "think" in a very verbal fashion and I instinctively try to sub-vocalize everything I read. So when I see math, as soon as I see a symbol that I can't "say" to myself (eg, a greek letter that I don't recognize, or any other unfamiliar notation) my brain just tries to short-circuit whatever is going on, and my eyes want to glaze over and jump to the stuff that is familiar.
OTOH, with written prose, I might see a word I don't recognize, but I can usually work out how to pronounce it (at least approximately) and I can often infer the meaning (at least approximately) from context. So I can read prose even when bits of it are unfamiliar.
There's also the issue that math is so linear in terms of dependencies, and it's - in my experience - very "use it or lose it" in terms of how quickly you forget bits of it if you aren't using it on day-in / day-out basis.