I don't have a great way to put it in words, but the intuition is that you are putting enormous mental energy into something that doesn't give anything back. Being able to solve heat equations doesn't make you warm.
Other subjects are the same, obviously, but when writing a history essay you can BS and not think too hard and skate by with a C. With a math problem, if it's difficult for you, you need to really put your whole self into solving it to make any progress at all. And I think that can be painful, in a certain way.
I was studying calculus not too long ago and arrived at the conclusion that drawing is actually way harder than math. In math, you learn how something works, and then you can do it. In drawing, you can literally see the "solution" right in front of you (e.g. your own hand that you are drawing) and still get it wrong for years (and even then, only asymptotically approach perfection).
That discrepancy between the goal (photorealism, or at least beauty) and the result (ugly art) is very painful for me, to such a degree that it
discouraged me from putting much effort into art, despite my love for it.
Drawing is similar (and a lifelong source of frustration for me). I can see the same experience playing out in different disciplines.
I would say that in the kind of math you study after calculus it becomes more common to have that "I know what the solution should look like but I can't seem to get there" feeling.
Don't try for photorealism straight away. Pick a more abstract style, and work on that: change things around until you find it fun. When you learn to follow your whims, that's when you'll start getting better (perhaps even quite rapidly).
This isn't to say give up on photorealistic drawing: but don't bludgeon yourself with it. If you don't find it fun, you're probably not at a point where practice will make perfect.