I agree. A lot of that is because no one has taken the lessons hard learned from every other numerical language and just replicated it (hardware acceleration,BLAS,ndarrays,..)
The JVM has a lot of matrix library fragmentation.
Without easy access to SIMD or value types (and JNI does not count as easy), the JVM's performance for regular computations on large datasets is still pretty lacking. Quoting http://cr.openjdk.java.net/~jrose/values/values-0.html, "Numeric types like complex numbers, extended-precision or unsigned integers, and decimal types are widely useful but can only be approximated (to the detriment of type safety and/or performance) by primitives or object classes."
And the JVM's costs in terms of deployment and startup times aren't very attractive for scientific workloads. You wouldn't gain as much in a migration from Python or Matlab to Java (or any other JVM language) as you would to C++ (or increasingly Julia), since you'll likely still need access to the same native libraries and JNI is kind of a pain.
Having done both Java and scientific programming I'd mostly say that the verbosity of Java really distracts away from the task of writing scientific code. An academic really doesn't usually care to learn about singletons and classes to write really fast for-loops. Also while the speed of Java now is awesome, floating point performance is key for those scientists who do know more systems level programming (memory management is a big issues). Actually GC in Julia can be a pain also.
I have a passing knowledge of Julia (I've written a simple Lorentz Force[0] stepper in it) is there a way to avoid GC in the middle of the calculation, ie., a performance critical loop or other?
Which is why a c++ underbelly is so important. The computations shouldn't happen at the java level if a scientific lib is being done right. I'd say scala isn't a bad alternative to that (which we have nd4s which is a wrapper over nd4j)
