The gist is that basic Python at its very core is -

a) simple b) limited

The language really took off when developers took this simple limited language and pushed it to its very limits using C extensions. The data science explosion opened up the language to a very wide user base.

So to answer your 3 questions: a) Python is not a fast language by any means. There is a lot of overhead in every function call that makes it almost impossible for low latency/real-time use cases. b) I don't think Python is particularly the best language for this. This is just a demonstration of someone building their own custom toolchain to show what is possible with just pure Python. The author has highlighted why they think this is interesting on the website. c) I keep thinking Python will go away soon, and we will see a much better alternative. But the reality is Python is entrenched deeply just like JavaScript. Lot of smart people are putting in a lot of effort to make it better. Personally the ecosystem and packaging story does not annoy me much, but the lack of proper threading (GIL) has hurt my projects more than once.

For your particular pain point, the current community recommended solution is to use uv (https://github.com/astral-sh/uv). There were several detours (pip, pyenv, pipenv, poetry etc.) the community took before they got behind this.

Before data science Python was already heavily used in web backend e.g. Instagram, others.

Yeah true and I think it was heading on a Ruby-like trajectory. It was the data science/ML trend that really cemented it's status.

My impression was that if you had a problem with Python and then added Docker now you have two problems. I worked at one place where the data sci's had an amazing ability to find defective Pythons.

Python is going in the right directions in terms of all the deployability and big issues but it should have been where it is now 7 years ago. Specifically, I sketched out a system that worked like uv but was written in pure Python, I didn't start on it for two reasons: (a) the bootstrapping problem that I couldn't ever stop devs from trashing the Python that it runs in, and (b) from lots of trying it didn't seem possible to convince most Pythoners that pip was broken or that it mattered... uv solved (a) by removing Python from the bootstrap and (b) by being crazy fast.

> Python 2 vs 3

This should not be an issue this day and age. Python 2 shouldn't be used for anything. If you're trying to do something that only works in Python 2, then you're likely doing something very wrong, likely reading for very out-dated material.

> virtual environments

Also shouldn't be an issue today. Ideally, every project has its own virtual environment. Packages should not be installed globally unless managed by your operating system, not pip or other Python package manager.

> libraries for each version

Rarely an issue, but I've certainly run into it, but only with Torch and other AI/ML libraries, all of which are very cutting-edge. The solution usually is to make sure everything is up to date, especially your operating system. If you're on Ubuntu 18.04, you're gonna have a bad time with something that requires Python 3.11 to work.

Python has its warts, sure, but I like it because it's so easy to get stuff done in. It's slow, yes, but that's rarely an issue. I find the syntax the most easy to read of any language I've ever worked with.

Python is just brutally slow. Anything performance-sensitive has to be done with a native module and now that requires all the same compilation and build tooling that everything else does.

The ecosystem is massive and the core team just keeps adding more and more dubious language features and syntax.

Realistically, Python should have been "done" after async/await and fixing str vs bytes.

There are parts of python that chafe, but if I switch to a language which has solved those problems, the set of people I can help falls to... very small. These are people we fought tooth and nail to drag away from excel, we're not going to get them all the way to haskell.

b: while Python is not a high-performance language, python coding is easier than high-performance languages. And programmer time is valuable. But if after coding a project in python, the developer may then find that they need higher performance than what interpreted python offers, and thus might be tempted to redo their program in a high-performance language. But a non-interpreted python processor provides a more appealing alternative to just spend money on an FPGA (or in the future maybe even an ASIC) python co-processor which may be fast enough, rather than wasting programmer time porting their python code to a high-performance language.

Old-timer here, used Python for about ten years professionally (Go now).

c) It’s a monstrous dumpster fire and getting worse over time, but so is everything else (in the same space). I like Go, but I can see how it’s not for everyone.

I've used Python a lot over the last ~10 years. It's probably my favorite language, although I'm not immune to its weak points.

To answer your questions in order,

a) I haven't done much work with embedded Python, but like any dynamically-typed language that runs in a VM there's a lot of runtime infrastructure that adds latency, complexity, energy consumption, bundle size, etc. It sounds like this project aims to remove the vast majority of that. So take startup time, for instance: Normal Python takes ~50ms to fire up the interpreter and get into actual user code. If I'm understanding it correctly, with PyXL that would be vastly lower. Although I guess the ARM chip still has to load the code onto the FPGA, so maybe not, idk.

b) and c) are kind of the same question, to me - at least, "why use Python for embedded" is a subset of "why use Python at all."

For me, Python more than any other language is great at getting out of its own way, so that you can spend your precious brain energy on whatever problem you're solving and less on the tool you're using to solve it. This is maybe less true in recent years, as later Pythons have added a lot more complex features (like async/await, for instance, which I actually really like in Python but definitely adds complexity to the language).

Finally, I think a lot of it comes down to personal style/taste/chance (i.e. if Python is the first language you encounter, you're probably more likely to end up liking Python.) The Zen of Python[0], which you may have seen, does a good job of explaining the Python way of approaching problems, although like I said a few of those principles have been less-rigidly adhered to in recent years (like "there should be only one way to do it.")

If you hang out in Python circles, you'll probably come across the phrase "Python fits your brain." I'm not sure where it was originally coined but it very definitely describes my experience with Python: it (mostly) just works like I expect it to, whether that's with regard to syntax, semantics, stdlib, etc.

Not that it doesn't have its bad points, of course. Dependency management, as you mentioned, can be a bit hellish at times. A lot of it comes down to the fact that dependencies in Python were originally conceived as systemwide state, much like dynamically-loaded C libs on Linux. This works fine until you need to use two different, mutually-incompatible versions of the same lib, at which point all hell breaks loose. There have been various attempts to improve on this more recently, so far uv[1] looks pretty promising, but time will tell.

The one saving grace of Python dependencies is that it has a very rich standard library, so the average Python project tends to have way fewer total dependencies than the average project in, say, JS or Rust.

The typing story for Python is also a bit lacking. Yes, there are now optional type hints and things like MyPy to make use of them, but even if your own code is all completely typed, in my experience it's usually not long before you need to call out to something that isn't well-typed and then your whole house of cards starts to fall apart.

Anyway, just my rambling $0.02.

[0] https://peps.python.org/pep-0020/

Not all rambling, but the exact kind of input I was hoping for. Thank you!

That wasn't my intention at all, but I appreciate that it came across that way to you. Please know that I was/am sincere in my desire to hear the thoughts of others while this is a current topic.

Yeah python has become more and more version and deps hell. Honestly 3 was all cost and no benefit and we'd all be fine if we'd stuck with 2. There were also some early missteps in api design like async and pandas and matplotlib that we all now have to live with. I even ran into problems with PIL changing API for textsize recently. Just a thousand cuts.

And yet for simple little standalone programs and notebooks, particularly for science, it is super simple and natural to turn to it.

Factors I personally think led to Python's popularity:

1) Perl kind of shooting itself in the foot 20 years ago and Python becoming the de facto scripting language for Linux distributions that needed to do anything more complicated than was suitable for shell scripts but didn't require entirely new compiled software projects.

2) The above meant Python is almost always available and a good tool to have handy if you need to do something one-off and simple but more complicated than what you can do with a built-in calculator app. For instance, ever curious if you can pull the exponents off of x509 certificates and manually verify signatures by hand? Pretty easy to do in Python.

3) The C API and compiled modules made it possible to link against pre-existing BLAS implementations, and the extensible syntax and user-defined operators made it possible to mimic the style of MATLAB and R. Thus, Python became a popular choice as a lingua franca for engineers, scientists, and stats geeks who just wanted to do some data exploration or modeling and weren't trying to create shippable software.

4) MIT decided to make Python its primary teaching language in the early 2000s or so and a lot of CS programs in the US followed suit.

5) It became possible at some point to write Microsoft Office macros in Python, giving marginally technical business types a nice option to learn that was more broadly useful than VB script to automate their own workflows.

Why it ever became so popular among actual software developers I have a harder time answering, but for research, exploratory work, prototyping, scripting, workflow automation, it's as good as anything else you can come up with, usually already available, and it has an extremely "batteries included" standard library that means you probably don't need to worry about the kind of ecosystem dependency hell you're envisioning here.

Possibly some factors include the rise of LeetCode, as Python's "executable pseudocode" style means it is very easy to find or translate examples of algorithm implementations into Python solutions for learning, and the fact that a large trend of the post big data era is trying to turn exploratory data analysis pipelining tasks into real software, along with people who used to brand themselves as "data scientists" deciding to become software developers instead, and already knowing Python.

Python also gives you a pretty good first order approximation of a solution when you want to turn some researcher's data model into a service, provided your app is also written in Python. This has become far less important these days with data APIs, ML APIs, standardized formats for model serialization, but previously, a very popular solution to the so-called "two language problem" was just making Python fast enough to let it be both languages itself rather than trying to add web app frameworks to Julia.