Threads are certainly important, but I have to say that I found the multiprocessing package to work very well. I think a lot of the things people think they need threads for would actually be better with multiprocessing instead. Memory protection is good! Shared memory is still available and explicit sharing of just what you need is better in a lot of ways than implicit sharing of everything.
I will be glad if the GIL is fixed but I think people reach for threads too quickly and too often.
If the GIL is fixed, why would you want multiprocessing versus threads? Threads are cheaper to create, easier to communicate between (even if you need to be careful), and simply do different things than what multiprocessing intends (eg easier for blocking I/O on many threads, versus multiprocessing which is really more of a task queue)
Why don't we just run all code in different threads of the same process? Multiple processes are more robust to failure and easier to reason about because they are less tightly coupled and more explicit about sharing. You can do blocking I/O with multiprocessing, you just have to explicitly share buffers.
> Multiple processes are more robust to failure and easier to reason about
Yeah of course, except when you use multiprocessing to do some CPU-intensive work and one of the subprocess get OOMKilled. Now your main process just hang forever. This was reported on bpo roughly 10 years ago and the response is "we can't fix this".
Is this enough to convince you multiple processes sometimes lead to unnecessary complexity?
I will be glad if the GIL is fixed but I think people reach for threads too quickly and too often.