· crossbeam is a building block used in Rayon, which is Rust's main data parallelism library, roughly similar to OpenMP.
· crossbeam channels are very fast. They're in the same league as golang, sometimes faster (depending how you measure of course).
· crossbeam is an order of magnitude faster than channels in the Rust's standard library, while being more powerful and simpler to use. This is an example where experimentation in a 3rd party crate paid off compared to the standard library where a basic implementation went to die.
· it integrates with Rust's borrow checker of course, so you can use complex multi-threaded constructs (e.g. share stack-allocated objects between threads) without fear of data races or use-after-free.
Note that there are plans to integrate crossbeam's channel implementation into the standard library. This won't change the public API but would at least vastly improve the implementation. A similar thing was done with changing the implementation of `HashMap` to be a wrapper around the `hashbrown` library.
Async/await channels need a completely different implementation. It doesn't really make a ton of sense for a single crate to support both sync/threaded channels and async channels.
Async/await are not inherently bad. They are actually very simple and quickly allow for higher level concurrent programming. I implemented async/await in a Scheme and from there was able to easily build actors in the language.
It’s just that certain languages, like Python, seem confused or unable to maintain the simplicity.
the fact that some languages can't do it might point to the fact that it is not as simple as it seems.
network programming is hard. really hard. Saying that async magically removes all that complexity is false. The complexity is moved into the executor, if the executor is out of your control you are up shit creek.
Finally If languages like python can't do it well that says something...
Python struggles with it because so much networking code deep in networking libraries such as gunicorn, database libraries like psycopg, depend (or did depend) upon mutable global state and non-reentrant blocks of code that, upon introducing concurrency result in all a manner of poor behavior.
Do a search for "psycopg2 decryption failed or bad record mac" for example. Or replace psycopg2 with celery, or gunicorn. Python has a difficult history of half-way attempts to parallelize, a half dozen eventlet libraries that never made it to the standard, and only recently - in terms of library support - is an official coroutine implementation available, albeit not as widely used as it needs to be. Library support is still widely lacking.
It's not that Python can't do it well, it's that the ecosystem has dug themselves a pit by doing things that aren't legal Rust and would be frowned upon in other languages. Those hacks to support parallelism and greenlets, gevent, etc. rely upon juggling file descriptors and carefully tweaking global state in each library. It's gnarly stuff and results in errors that are clearly the result of failing to catch all the balls correctly.
Those SSL errors, by the way, are incredibly concerning. It means that wires are getting crossed and if it weren't for SSL authenticating the packets, the wrong data would happily get sent or received to the wrong party. In a network server, yikes.
The way those languages implement something that is uncolored is by making everything async. Rust has some other use-cases where that's not acceptable.
Well that isn't entirely true. Java is doing it in a similar way to Go in that every call to I/O or other function that would block can suspend the current virtual thread. But if it's called in a standard blocking thread then it blocks as expected (in Java w/Loom that is, ofc every goroutine is a green thread). This is of course a ton easier in Java and Go because of the managed memory model.
However I would be interested in hearing what the specific Rust features (or lack thereof) that prevent this being possible. If it's possible to swap the async runtime out it seems like it should also be possible to add enough runtime to a Rust program to allow the equivalent of virtual threads as long as stuff uses stdlib sockets, file I/O etc.
It's not due to the memory model; it's due to the lack of a runtime. In Go/Java, virtual threads are provided by a runtime. Rust tries to stay close to the metal, and so has no runtime. It does not have virtual threads that could be suspended.
Rust used to have virtual threads pre-1.0, but they were removed since the overhead was deemed unacceptable.
Great! I would love that. However, having read the comments of the rust async working group, they think that async is the best thing since sliced bread ... They have no intention of changing IMO. Sigh. Hopefully one day we will get inclusive features rather than exclusive... Cheers
No I think I should put my money where by mouth is: I will take another route: asyncless rust should be simple enough to remove that part and make async/await compiler errors. Add this as a new target to rustup and we are good.
· crossbeam is a building block used in Rayon, which is Rust's main data parallelism library, roughly similar to OpenMP.
· crossbeam channels are very fast. They're in the same league as golang, sometimes faster (depending how you measure of course).
· crossbeam is an order of magnitude faster than channels in the Rust's standard library, while being more powerful and simpler to use. This is an example where experimentation in a 3rd party crate paid off compared to the standard library where a basic implementation went to die.
· it integrates with Rust's borrow checker of course, so you can use complex multi-threaded constructs (e.g. share stack-allocated objects between threads) without fear of data races or use-after-free.