They may certainly be counterproductive at times, but if they truly served no purpose it seems unlikely we'd have them? Also there's at least mixed evidence for the existence of similar states in animals[1].
Hmm, that feels a bit like the difference between verifiability and falsifiability to me. It doesn't seem like adaptations need to be goal-driven, but if they were flat-out bad/destructive/unhelpful to the species, I would tend to expect them to fade over time? Pointing to any specific trait and saying it's useless, though, opens you up to arguments that you just haven't found the use yet.
There are a lot of accidents of evolution that don't have a "purpose". The shape/trajectory of the vagus nerve, the appendix, the human tailbone, etc...
This goes to the root of what we mean when we say "purpose". Are we saying the trait is "helping" to "achieve" some "goal", or are we really just saying that there's some relationship between the trait and the individual's environment, or that the existence of the trait can be explained by properties of the environment? If the latter, then are we stretching the meaning of the word "purpose"?
It's a bit like how some people say "everything happens for a reason" (in this sense reason points to a purpose), whereas I'm more of the mind that no, most things more or less happen for no particular reason (ie.: serving no particular purpose).
I would argue that it's not exactly clear whether you could even say that evolution itself has a "purpose". It's just something that happens, given a bunch of organisms competing for resources.
1: https://en.wikipedia.org/wiki/Emotion_in_animals