It was multiple things, really. I would attribute ute some of it to Swig, Perl attrition, SCons/Software Carpentry, integration with GUI libraries, good documentation, and various other efforts in the mid 2000s. A lot of those things were solving research problems simply, and Python’s use just kept expanding.
Python was already taking over in many use cases by late 2000s.
Ruby was known, but it didn’t have the following at multiple levels in academia like Python did
You describe what happened, which I saw happen too. The question I have is why though. Right, why did python's use in scientific computing keep expanding, and not ruby's? Why was python already taing over many use cases by the 2000s, but not ruby? Why did python develop the following at multiple levels in academia, and not ruby? (Why is Perl attrition relevant, when ruby was in fact explicitly based on Perl?)
That's the question, not the answer!
It seems like a lot of the answer is NumPy, which makes the question -- why did NumPy happen on python, not ruby?
Certainly one answer could be "nothing having to do with the features of the language, it's just a coincidence, they chose to write it in Python, if those working on numpy had chosen to use ruby instead, history would be different."
But one hypothesis is that maybe NumPy wouldn't have been as easy in ruby as python.
Someone else suggested the first numpy release happened before the first ruby release, so that could also be an answer.
I think the difference is in the community. I've used both Python (extensively) and Ruby (a little bit). While the capacities of the languages are relatively similar, the people around the languages, at least the ones creating packages and driving the discussion in conferences are actually quite different, for some reason.
People attracted to Ruby are mostly of an "artistic mindset", they want to be expressive, write code that doesn't look like programming code and using "magic" like dynamically created methods, monkey-patching, etc is accepted or even encouraged.
On the other hand, Python attracts more people with "engineering mindset", they like straight forward code that's readable, clear and understandable, even if it's not as expressive. "Magic" elements are frowned upon: for example, imports are explicit and always included in each file.
Obviously, I'm exaggerating it, but I think is a clear differentiation between the communities.
My guess is that the "Python mindset" got into creating better integrations for "engineering applications", like NumPy or SciPy, and that created some positive feedback in certain environments. The main strength of Python is its rich ecosystem of third party packages. There's a compounding effect, making it grow faster and faster.
I think that’s exactly it, and that there’s much less understanding required to start reading and writing Python code. Ruby has some beautiful features, but they make it much less clear to newbies who are trying to figure out what on earth’s going on.
Ruby makes it easy to do "magic". Which is fun to write, but painful to read for others.
I've encountered real cases of ruby code where a simple code snippet behaves differently with and without a `require` (IIRC, some utility function added to a class with monkey patching). In another case I've also had to modify (and to some extent maintain) a codebase that relied on overriding `method_missing` in the happy case / normal flow. I was trying to find out where some method was being defined by grepping the whole codebase. It probably cost me half a day of unabridged profanity.
In theory you can do the same thing with python -- thing is it usually doesn't happen for some reason (likely the ones you mentioned). Something about the language features and the culture in the community lead to devs doing different things with the different languages. But the effect is real, and I know which language to avoid if I had the choice.
I was using Perl and Python in the 1990s for scientific work.
Around 1993 I got hooked on Perl. I read the Perl book and it was great. But 1) I couldn't figure out how to handle complex data structures (this was Perl 4), and 2) I couldn't embed it into other projects.
More specifically, worked on a molecular visualization program called VMD. It had its own scripting language. I wanted a language to embed in VMD that was usable by my grad student users. This is when I first learned about Python, but I chose Tcl because it fit the existing command language almost perfectly.
At around the same time, UCSF started embedding Python for their molecular visualization package, Chimera, so it was already making in-roads in structural biology.
I later (1997) went into more bioinformatics-oriented work, where I did a lot of Perl. I tried out one implementation (a Prosite pattern matcher) in Perl - which took me reading an advanced Perl book to learn how Perl 5 objects worked. I then tried the same in Python, a language I wasn't as familiar with. And it was just so much easier!
At this time Perl was THE language for bioinformatics, but I thought it was a difficult language for complex data structures. (Bioinformatics at that time was mostly string related, plus CGI and databases - Perl was a great fit.)
I then moved over (1998) to cheminformatics, working more directly on molecular graphs. Python was a much better fit for those data structures than Perl. I started using Python full-time, and it's been that way since.
We used a third-party commercial package for the underlying cheminformatics called the Daylight toolkit. It had C and Fortran bindings. Someone else had already written the SWIG configuration to generate Perl, Python, and Tcl bindings, but these still meant manual garbage collection.
I was able to use __getattr__, __setattr__, and __del__ to turn these into a natural-feeling high-level API, hooked into (C)Python's reference-counted garbage collector.
I presented a couple of talks about this work, got an article in Dr. Dobb's (!) and got consulting work helping companies which either had existing Python work, or were moving to Python.
By contrast, I don't think I heard about Ruby until 2000 or so, years after Python started entering structural biology/cheminformatics. [1]
I wasn't particularly cutting edge - others had already developed tool like SWIG, which was because Beazley and others were using Python at LANL. Numeric Python started in part because of work at LLNL and other research organizations. The concept already firmly established was that Python would be used to "steer" a high-performance kernel.
And Python in turn changed, to better reflect the needs of numeric computing, in particular, the "..." notation in array slices was added to make matrix operations easier. (This was 20 years before '@@' was added to simplify matrix multiplication.) I believe the needs of numeric computing also influenced the changed to "rich" comparisons.
This all took place around the time Matz started developing Ruby. Python had a clear head-start. And except for bioinformatics, Perl never had much presence in the fields I worked in.
So:
> why did python's use in scientific computing keep expanding, and not ruby's?
Because Python was in-use several years before Ruby, and already rather visible as one of the three main languages to consider in that space (Tcl and Perl being the other two).
> Why was python already taing over many use cases by the 2000s, but not ruby?
Because people didn't really know about Ruby, while Python already had a pretty large user community. Probably also because Python's work was all in English, while a lot of the Ruby community was using Japanese.
> Why is Perl attrition relevant, when ruby was in fact explicitly based on Perl?
Perl attrition started before Ruby was much known. The complexity of the language, and the cumbersome need to roll-your-own OO, made it difficult for me to recommend to the typical software developers I work with - grad students and researchers in the physical sciences with little formal training in CS. Python by comparison which easier to pick.
So a language which explicitly based on Perl also picks up that negative impression.
(FWIW, I think Tcl is an easier language to start with than Python.)
"""In 1995 the special interest group (SIG) matrix-sig was founded with the aim of defining an array computing package; among its members was Python designer and maintainer Guido van Rossum, who extended Python's syntax (in particular the indexing syntax[8]) to make array computing easier."""
"""The first public release of Ruby 0.95 was announced on Japanese domestic newsgroups on December 21, 1995. ... In 1997, the first article about Ruby was published on the Web. ... In 1999, the first English language mailing list ruby-talk began, which signaled a growing interest in the language outside Japan."""
""" I think my criteria for
selecting Python over Perl is still true for Python over Ruby,
in that it has too many special characters (like @ and the
built-in regexpes), features (like continuations and code
blocks) which are hard to explain well (I didn't understand
continuations until the Houston IPC), and 'best practices'
(like modifying base classes like strings and numbers)
which aren't appropriate for large-scale software
development."""
Nitpick: Numpy is the newest, revised and reconciled vector library for Python; The first one was called “Numeric”; then there was “Numarray” which was not fully compatible, which caused a bifurcated ecosystem; and then IIRC it was Travis Oliphant who decided enough is enough, created Numpy which was somehow magically backward compatible with both, and reunited the community.
> It seems like a lot of the answer is NumPy, which makes the question -- why did NumPy happen on python, not ruby?
One of Python's original use cases was as a macro/script language you could import into your C application. Adding python to your C app took a day or so, and a side effect often was you'd make a Python library out of your app's library and suddenly, you could write standalone Python that called your app's code. Because it was so easy to write a python wrapper for an existing C library, by 1995/96 when Ruby hit the scene, Python already had quite a bit of importable functionality. The first serious web framework for Python was Zope, and it came out in 1998. I think Rails was around 2005/2006, and it was really cool, but one of the rubs was Ruby didn't have the libraries that Python did. In reality, it's amazing how good Ruby and Python have been.
Python was already taking over in many use cases by late 2000s.
Ruby was known, but it didn’t have the following at multiple levels in academia like Python did