Mostly you only concern about the criticism from within scientific community. Since your result is published, generally, people with background will spend time to figure out how to use it by themselves. And their criticism is only relevant when it gets published. At that point, you are surely obligated to response such criticism. Most of the time, it is really not relevant that some random people send you email and have such "eclipse plug-in" problem.
"some random people send you email and have such "eclipse plug-in" problem" -> They are not random people, they are Ph.D. students working on their thesis and you can be assured that they will find a way to publish their comparison one way or the other. Some Ph.D. students will go at great length to figure out other's code before asking a question, but some will suck your energy big time and still do a poor job at representing your work. Now compare this to those who do not release their code: they can only be evaluated on their published results and they can spend their time on their next study/research work.
My point is that there is not a lot of incentive to release code: it requires a lot of your time, it is not always clear that granting agencies/employers consider this as being more positive than publications, and it exposes your work to all sorts of unfair replication/comparison.
Now, if research work without released code were never cited, that would be a terrific incentive to release your code :-)
P.S. I speak from experience and I release the code of my most important research work.
"Mostly you only concern about the criticism from within scientific community"
Ha. That doesn't apply to climate scientists. And there was the case of Andy Schlafly of Conservapedia (a creationist lawyer) demanding all sorts of nonsense from a scientist who documented evolution through many, many generations of bacteria.
At its worst, you get ideological cranks ganging up and demanding information, possibly through FOIA requests, pretty much for the sole purpose of using up all the scientist's time responding. Then when they don't respond, and/or it turns out they aren't subject to FOIA requests, the cranks freak out, break into a server, steal emails, and declare a conspiracy.
