As a rubyist, it makes me sad that python ended up here rather than ruby. And I sometimes wonder why.
> As the name suggests, numeric data is manipulated through this package, not in plain Python, and behind the scenes all the heavy lifting is done by C/C++ or Fortran compiled routines.
So I wonder, was it easier to write C/C++ or fortran compiled extensions in python than it was in ruby?
Readability, 100%. I have programmed in large projects in both Python and Ruby.
Ruby is very productive to write, because everything and the kitchen sink is at your fingertips at all times.
But because of Ruby's many ways to skin a cat, everyone's code is very different. Add to that the penchant for domain-specific sub-languages in Ruby: new syntaxes that you might have to learn half a dozen of to integrate a large project, all of which end up being more limiting than if you could just, you know, write Ruby.
Contrast with Python, which goes so far at normalizing as to have a language-wide coding standard in PEP8. Python has its problems, package management and distribution is still ugly for example. But I can read any project I find and understand it without loads of context.
My impression is that Perl, Ruby, and Lisp all suffer from this issue.
Even proponents will say things like "this language is so expressive, I feel so productive in it, but people do such idiosyncratic and clever things that it's hard for anyone else but the author to understand".
That sort of "solo rockstar" programming culture doesn't really lend itself to large-scale FOSS projects, which need to be inviting to wide participation.
A language like Ruby can be very productive for someone who has climbed the learning curve to learn all its ins and outs. However, in my experience, this turns into a large productivity drain the moment someone else (who is less of an expert) has to touch it.
For large projects with multiple developers, readability should win over writability every time, most code are read more than they are written (my hypothesis). You can see evidence of this, given the success of languages like Python and Go.
That said, for scientific compute, in a lot of cases, writability matters way more, as your job as a scientist is to produce results as fast as possible, code quality be damned. However, only a small number of scientists are expert developers and have climb the learning curve and can write code with ease. The vast majority of them are junior at best, and Python's approachability (which is rooted from its readability of course), wins. With most of the people using Python, the ecosystem develops and there are no other viable alternatives. In the long run, I suspect even languages like MATLAB and Mathematica will die out as the open source stack becomes more mature and (eventually, if not already) significantly more capable. Julia might be a wildcard due to its (potential) performance advantages, but the aesthetics of the programming language is simply not in the minds of 99% of the scientific compute users out there.
I am totally interested in hearing opinions from people who have done serious hours of programming in both ruby and python, as to readabilty comparison.
If it's just people who have done a lot of work in ruby and little in python saying ruby is more readable, and vice versa, I don't find it very useful even anecdotally.
As someone that have spent a lot of time in both Ruby and Python (and specifically a lot of hours in the Python scientific compute stack), I would say that Python is significantly more readable. Python is also significantly easier to teach as opposed to Ruby, especially if the target audience already has a bit of programming experience (from MATLAB, or other courses).
I suspect the main reasons are:
1. Python's guiding philosophy of "There should be one-- and preferably only one --obvious way to do it". With later additions to the language, this is getting less true (3 different ways to format strings, asyncio, type hinting, etc). Some libraries also don't conform to this (matplotlib). That said, it's a lot better than the Ruby code I've encountered, which is like the wild west.
2. Python's syntax is reasonably simple to teach. The object model could be condensed into something very simple if you don't need a lot. With very basic knowledge, you can go a long way. Ruby's a bit more chaotic with things like inheritance, extend, and include; proc, block, lambda; having to use attr_accessor; syntax things like a = b could be a function call or not; if/unless; and many more things that are confusing.
3. Even basic things like loops in Ruby is not idiomatic as it wants to apply a function/block instead. Beginners, especially those with a bit of background, like their loops better than functional programming.
As I've spent many years working on Ruby code base I still get lost all the time. Python in my experience has been a lot better, although recent Python versions have regressed a bit as it introduced more syntax to do the same things.
>So I wonder, was it easier to write C/C++ or fortran compiled extensions in python than it was in ruby?
Don't know about technical aspects of "easier" but it may have simply been an accident of history.
E.g. in 1995 (before Ruby 1.0 December 1996[0]), David Beazley was already wrapping C Language code to Python. Deep link to presentation:
https://youtu.be/riuyDEHxeEo?t=52m27s
So DB's Python code for scientific code was released in Feb 1996 and presented in July 1996. Python being released in 1991 was already talked about in magazines in 1995. David's presentation also references Jim Hugunin[1] and he authored the 1995 Numeric package which was the ancestor to NumPy. Once an ecosystem gets started, it can attract more mindshare and snowball into an insurmountable lead that neither Ruby nor Julia will ever catch up to.
In other words... If the opposite timeline happened and Ruby was released earlier in 1991 and Python later in 1996, things may have played out differently.
So folks like David Beazley and Jim Hugunin chose Python as the scripting host language for their C Language code probably because Ruby wasn't mature and well-known back in 1995. Apparently, Ruby didn't widely spread outside of Japan until 1998 when the first documentation in English appeared.[2]
In 2009 I began writing code for a new company doing natural language processing. I was the engineer at the time and got to pick my tools. I started with Ruby because I was sick of C++ and Perl and Ruby looked like the future. But I soon discovered the NLTK and then Numpy and so I started playing around in Python. I never again wrote line of Ruby… until the later hired front end devs threw a fit of not being able to use Rails.
It was clear at the time that there basically was no non-web Ruby community. Ruby was Rails and Rails was Ruby. Ruby had a nice little niche in 2009 but the Python had Numpy and there were a lot of ML people doing lots of math and Ruby wasn’t going to cut it unless they wrote their own libraries, which wasn’t worth the effort since Python and Nunpy already existed and already had a growing community behind it.
I honestly think it all boils down to numpy being developed long before matrix libraries became a standard part of software development.
Ruby's early "killer app" (remember that term?) was Rails. Even to this day there is almost no major code out there built in Ruby that isn't ultimately related to building CRUD web apps. While Ruby may be losing popularity now, it moved the web-development ecosystem ahead in the same way that Python has moved the scientific computing world ahead.
20 years ago if you wanted to use open source tools to performant vector code there was Python and a hand full of oss clones of commercial products. Given the Python was also useful for other programming tasks in a way that say Matlab/Octave is not, it was the choice for more sophisticated programmers who wanted an OSS solution and need to do scientific computing. This creates a positive feed back that persists to this day.
Given that Python remains a decent language relative to it's contemporary peers and it has a massive and still growing library of numerical computing software it is extremely unlikely to be dethroned, even by promising new languages like Julia.
Even to this day there is nothing even close to numpy in Ruby. I do DS work in an org that is almost entirely Ruby, but we still use python without question because we know re-implementing all of our numeric code into Ruby would be a fools errand.
Had ruby had early support of matrix math, it wouldn't have surprised me if it would have replaced Python.
But that begs the question -- why did numpy develop in python and not ruby?
The rest of the thread offers some suggestions though. One is simply that python was born first, and got the numpy precursor before ruby 1.0 even happened. Which seems like a thing.
Ruby had a numpy style library since the early 00's, I forget exactly when. But it never got the kind of momentum numpy and the Python ecosystem surrounding it did.
Lots of comments in this thread from people who's Ruby experience is only from the post Rails era after ~2008, and don't understand that the post Rails culture wasn't really a thing when Python was first gaining momentum for scientific computing.
Perl was my horse in the race. I attribute it's, lisp's, ruby's, etc loss to
1. "There should be one-- and preferably only one --obvious way to do it" being part of python's ethos.
2. ipython repl
1. pairs with jaimebuelta's artistic vs engineering dichotomy, but also plays into the scientist wearing many more hats than just programmer. Code can be two or more degrees removed from the published paper -- code isn't the passion. There isn't reason, time, or motivation to think deeply about syntax.
2. For a lot of academic work, the programming language is primarily an interface to an advanced plotting calculator. Or at least that's how I think about the popularity of SPSS and Stata. Ipython and then jupyter made this easy for python.
For what it's worth, the lab I work for is mostly using shell, R, matlab, and tiny bit of python. For numerical analysis, I like R the best. It has a leg up on the interactive interface and feels more flexible than the other two. R also has better stats libraries. But when we need to interact with external services or file formats, python is the place to look (why PyPI beat out CPAN is similar question).
Total aside: Perl's built in regexp syntax is amazing and a thing I reach for often, but regular expressions as a DSL are supported almost everywhere (like using languages other than shell to launch programs and pipes -- totally fine but misses all the ergonomics of using the right tool for the job). It'd love to explore APL as an analogous numerical DSL across scripting languages. APL.jl [0] and, less practically april[1], are exciting.
From what I remember, people were actively promoting Python as the first programming language already in the 90s. Many universities started teaching Python, creating a steady supply of non-CS majors who were familiar with Python but no other language. And because the community was there, people started building the ecosystem.
In contrast, I've never really encountered anyone advocating for Ruby outside web development.
van Rossum was one of the implementers of ABC, which indeed was created to experiment with how to develop a programming language for beginning programmers. (Note: he was not one of the designers of ABC.)
While van Rossum drew from that experience when making Python, the initial driving goal was as a scripting language for system admin tasks in the Amoeba distributed operating system.
David Beazley talks about this in a YouTube video somewhere. (Can't find it right now, maybe someone will in the comments.)
It was a lot of serendipity. Python was up and running when the US national labs wanted to collaborate and their tools all sucked. Since they wanted visualization this left only Tcl/Tk or Python/Tk. And Beazley was hanging around as a grad student in a national lab with a connection machine, no real boss, no real oversight, and very little budget. He built stuff out of Python, and it snowballed to other labs.
Timing could be a factor. Python was released in 1991. Numeric, the ancestor of NumPy, followed in 1995, the same year Ruby was released. So Python already had its hooks into scientific computing before Ruby even started.
Fortran interop (f2py in particular) was a significant factor, and as soon as you get one thing (in this case LAPACK and BLAS bindings) it snowballs. Also, Python is significantly more initially familiar for informal programmers and that’s critical; the hard part of learning a language is often believing that you can -and Ruby looks weirder than Python, so it makes people doubt themselves.
I don't know how easy it is in Ruby so I cannot give you a comparison.
However it is very very easy to write Python bindings for a C/C++ library with minimal work. Solutions range from "just works" like ctypes to "actually integrates with the language" like Cython. You also have automated tools for wrapping like pybind11 which does a lot of the heavy lifting for you.
It was multiple things, really. I would attribute ute some of it to Swig, Perl attrition, SCons/Software Carpentry, integration with GUI libraries, good documentation, and various other efforts in the mid 2000s. A lot of those things were solving research problems simply, and Python’s use just kept expanding.
Python was already taking over in many use cases by late 2000s.
Ruby was known, but it didn’t have the following at multiple levels in academia like Python did
You describe what happened, which I saw happen too. The question I have is why though. Right, why did python's use in scientific computing keep expanding, and not ruby's? Why was python already taing over many use cases by the 2000s, but not ruby? Why did python develop the following at multiple levels in academia, and not ruby? (Why is Perl attrition relevant, when ruby was in fact explicitly based on Perl?)
That's the question, not the answer!
It seems like a lot of the answer is NumPy, which makes the question -- why did NumPy happen on python, not ruby?
Certainly one answer could be "nothing having to do with the features of the language, it's just a coincidence, they chose to write it in Python, if those working on numpy had chosen to use ruby instead, history would be different."
But one hypothesis is that maybe NumPy wouldn't have been as easy in ruby as python.
Someone else suggested the first numpy release happened before the first ruby release, so that could also be an answer.
I think the difference is in the community. I've used both Python (extensively) and Ruby (a little bit). While the capacities of the languages are relatively similar, the people around the languages, at least the ones creating packages and driving the discussion in conferences are actually quite different, for some reason.
People attracted to Ruby are mostly of an "artistic mindset", they want to be expressive, write code that doesn't look like programming code and using "magic" like dynamically created methods, monkey-patching, etc is accepted or even encouraged.
On the other hand, Python attracts more people with "engineering mindset", they like straight forward code that's readable, clear and understandable, even if it's not as expressive. "Magic" elements are frowned upon: for example, imports are explicit and always included in each file.
Obviously, I'm exaggerating it, but I think is a clear differentiation between the communities.
My guess is that the "Python mindset" got into creating better integrations for "engineering applications", like NumPy or SciPy, and that created some positive feedback in certain environments. The main strength of Python is its rich ecosystem of third party packages. There's a compounding effect, making it grow faster and faster.
I think that’s exactly it, and that there’s much less understanding required to start reading and writing Python code. Ruby has some beautiful features, but they make it much less clear to newbies who are trying to figure out what on earth’s going on.
Ruby makes it easy to do "magic". Which is fun to write, but painful to read for others.
I've encountered real cases of ruby code where a simple code snippet behaves differently with and without a `require` (IIRC, some utility function added to a class with monkey patching). In another case I've also had to modify (and to some extent maintain) a codebase that relied on overriding `method_missing` in the happy case / normal flow. I was trying to find out where some method was being defined by grepping the whole codebase. It probably cost me half a day of unabridged profanity.
In theory you can do the same thing with python -- thing is it usually doesn't happen for some reason (likely the ones you mentioned). Something about the language features and the culture in the community lead to devs doing different things with the different languages. But the effect is real, and I know which language to avoid if I had the choice.
I was using Perl and Python in the 1990s for scientific work.
Around 1993 I got hooked on Perl. I read the Perl book and it was great. But 1) I couldn't figure out how to handle complex data structures (this was Perl 4), and 2) I couldn't embed it into other projects.
More specifically, worked on a molecular visualization program called VMD. It had its own scripting language. I wanted a language to embed in VMD that was usable by my grad student users. This is when I first learned about Python, but I chose Tcl because it fit the existing command language almost perfectly.
At around the same time, UCSF started embedding Python for their molecular visualization package, Chimera, so it was already making in-roads in structural biology.
I later (1997) went into more bioinformatics-oriented work, where I did a lot of Perl. I tried out one implementation (a Prosite pattern matcher) in Perl - which took me reading an advanced Perl book to learn how Perl 5 objects worked. I then tried the same in Python, a language I wasn't as familiar with. And it was just so much easier!
At this time Perl was THE language for bioinformatics, but I thought it was a difficult language for complex data structures. (Bioinformatics at that time was mostly string related, plus CGI and databases - Perl was a great fit.)
I then moved over (1998) to cheminformatics, working more directly on molecular graphs. Python was a much better fit for those data structures than Perl. I started using Python full-time, and it's been that way since.
We used a third-party commercial package for the underlying cheminformatics called the Daylight toolkit. It had C and Fortran bindings. Someone else had already written the SWIG configuration to generate Perl, Python, and Tcl bindings, but these still meant manual garbage collection.
I was able to use __getattr__, __setattr__, and __del__ to turn these into a natural-feeling high-level API, hooked into (C)Python's reference-counted garbage collector.
I presented a couple of talks about this work, got an article in Dr. Dobb's (!) and got consulting work helping companies which either had existing Python work, or were moving to Python.
By contrast, I don't think I heard about Ruby until 2000 or so, years after Python started entering structural biology/cheminformatics. [1]
I wasn't particularly cutting edge - others had already developed tool like SWIG, which was because Beazley and others were using Python at LANL. Numeric Python started in part because of work at LLNL and other research organizations. The concept already firmly established was that Python would be used to "steer" a high-performance kernel.
And Python in turn changed, to better reflect the needs of numeric computing, in particular, the "..." notation in array slices was added to make matrix operations easier. (This was 20 years before '@@' was added to simplify matrix multiplication.) I believe the needs of numeric computing also influenced the changed to "rich" comparisons.
This all took place around the time Matz started developing Ruby. Python had a clear head-start. And except for bioinformatics, Perl never had much presence in the fields I worked in.
So:
> why did python's use in scientific computing keep expanding, and not ruby's?
Because Python was in-use several years before Ruby, and already rather visible as one of the three main languages to consider in that space (Tcl and Perl being the other two).
> Why was python already taing over many use cases by the 2000s, but not ruby?
Because people didn't really know about Ruby, while Python already had a pretty large user community. Probably also because Python's work was all in English, while a lot of the Ruby community was using Japanese.
> Why is Perl attrition relevant, when ruby was in fact explicitly based on Perl?
Perl attrition started before Ruby was much known. The complexity of the language, and the cumbersome need to roll-your-own OO, made it difficult for me to recommend to the typical software developers I work with - grad students and researchers in the physical sciences with little formal training in CS. Python by comparison which easier to pick.
So a language which explicitly based on Perl also picks up that negative impression.
(FWIW, I think Tcl is an easier language to start with than Python.)
"""In 1995 the special interest group (SIG) matrix-sig was founded with the aim of defining an array computing package; among its members was Python designer and maintainer Guido van Rossum, who extended Python's syntax (in particular the indexing syntax[8]) to make array computing easier."""
"""The first public release of Ruby 0.95 was announced on Japanese domestic newsgroups on December 21, 1995. ... In 1997, the first article about Ruby was published on the Web. ... In 1999, the first English language mailing list ruby-talk began, which signaled a growing interest in the language outside Japan."""
""" I think my criteria for
selecting Python over Perl is still true for Python over Ruby,
in that it has too many special characters (like @ and the
built-in regexpes), features (like continuations and code
blocks) which are hard to explain well (I didn't understand
continuations until the Houston IPC), and 'best practices'
(like modifying base classes like strings and numbers)
which aren't appropriate for large-scale software
development."""
Nitpick: Numpy is the newest, revised and reconciled vector library for Python; The first one was called “Numeric”; then there was “Numarray” which was not fully compatible, which caused a bifurcated ecosystem; and then IIRC it was Travis Oliphant who decided enough is enough, created Numpy which was somehow magically backward compatible with both, and reunited the community.
> It seems like a lot of the answer is NumPy, which makes the question -- why did NumPy happen on python, not ruby?
One of Python's original use cases was as a macro/script language you could import into your C application. Adding python to your C app took a day or so, and a side effect often was you'd make a Python library out of your app's library and suddenly, you could write standalone Python that called your app's code. Because it was so easy to write a python wrapper for an existing C library, by 1995/96 when Ruby hit the scene, Python already had quite a bit of importable functionality. The first serious web framework for Python was Zope, and it came out in 1998. I think Rails was around 2005/2006, and it was really cool, but one of the rubs was Ruby didn't have the libraries that Python did. In reality, it's amazing how good Ruby and Python have been.
It’s all about the community. As soon as a language gets attached to a profession it’s hard to break. Ruby has primarily been a web dev language, also the syntax is bad =P
> As a rubyist, it makes me sad that python ended up here rather than ruby. And I sometimes wonder why.
Work on numerical packages and scientific computing started almost as soon as the language did, for instance the origins of Numpy lie in the Numeric package which was introduced in 1995.
And the core team introduced several niceties at the behest of the scientific community (advanced slicing for instance, more recently the matmul operator).
not sure. there are many factors that contributed to python's success.
i discovered the language in 98 or 99. it came with some obscure linux distribution and the tkinter module stood out for me. it showed pretty scientific graphs and charts. but the language has to reinvent its community many times since then.
my intuition is that it was popular in europe in the scientific community. not sure i can say the same for ruby.
> As the name suggests, numeric data is manipulated through this package, not in plain Python, and behind the scenes all the heavy lifting is done by C/C++ or Fortran compiled routines.
So I wonder, was it easier to write C/C++ or fortran compiled extensions in python than it was in ruby?