Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Ractor – a Rust-based actor framework with clusters and supervisors (github.com/slawlor)
168 points by snowboarder63 on Feb 16, 2023 | hide | past | favorite | 74 comments
SOME HIGHLIGHTS.

* We have a full supervision tree so actors can monitor other actors for exits and unhandled panic!s (at least the ones that can be caught)

* The actor lifecycle is handled for you in a simple single-threaded, message handler primitive

* You have a mutable state with each message handling call, so you have an easy way to create stateful actors and update that state as messages are processed

* Actors talk to other actors by message passing, but there are remote-procedure-calls (RPCs) so actors can "ask a question" to another actor and wait on the reply.

* A lot of the concurrency primitives are handled by the framework, such as cancellation/termination of actors (both graceful and forceful)

* A Factory primitive in order to formulate distributed processing pools with multiple job routing options

* Early but stable support for a distributed epmd-like cluster environment, where you can talk to actors over a network link. It's an additional crate (ractor_cluster) that builds on ractor to facilitate the inter-connection between nodes and support remote casts and calls to actors on a remote node.

We're openly seeking feedback, so please feel free to utilize the library and let us know if there's anything you find missing or doesn't work as expected!




Looks very nice!

How does it compare to lunatic https://github.com/lunatic-solutions/lunatic ? (besides wasm vs. native)

Also, what's the deployment story?


I mean lunatic runs within Wasm, which isn't the end of the world, but we compile down to native. Lunatic generally seems to want to be the end-all of concurrency but we fit more into an existing Tokio-based environment.

As far as deployment, do you mean like how we came to build this? Or like how you'd actually deploy it...? Because that's really up to the program being built imo


> As far as deployment, do you mean like how we came to build this? Or like how you'd actually deploy it...? Because that's really up to the program being built imo

I mean that deployment of Erlang code on a BEAM cluster is pretty simple, iirc. I wonder if you have a solution that handles that part as conveniently as Erlang/Elixir/BEAM.


Ah yeah we don't have hot-updates like Erlang does (we may never, it depends on how Rust would let us do it). That being said we aren't running on a runtime, so Rust code is native-compiled. You could take down and upgrade a node in a cluster as long as the network protocol doesn't change (or is at least backwards compat), but you wouldn't be able to like upgrade a single actor on a node.


I suspect that it would be possible to somehow implement hot-updates with native, if you're willing to use some container as your "runtime". I haven't investigated this seriously, though, so I may be wrong.


Any connection to Ruby's Ractor class? https://ruby-doc.org/core/Ractor.html


Actually no, I came up with the name independently and only later discovered their lib. I think it's safe to say we diverge a lot in practice though lol


My mind went to the Artificially Insane RACTER: https://en.wikipedia.org/wiki/Racter


Is this in use at WhatsApp (main dev works there). Asking cause I know WhatsApp was built on Erlang, and was wondering if they still are, or have (partially) moved to Rust.


Pretty sure they're still on erlang.


[deleted]


Wait. This package is in use in production at WhatsApp and the only mention of that is many levels deep in a comment thread on HN? That's seriously burying the lede.


At the rewrites happening for performance reasons?

Anything you’ve published/will publish about this?


[deleted]


Neat. How big is the dev team working on this Erlang & Erlang-adjacent stuff?


Not sure what's public here, so let's just say "big", as in 100's+.


Very glad to see this emerge, we have been using actix in production for a year now and it's a bit annoying the state of it. The dominant sentiment in the Rust community right now seems to be to build your own framework, but I really don't agree with this sentiment.

Regarding the implementation, I would have preferred to have different handler implementations for the various message types instead of a big enum. It's not rare to have 4-5 messages for a single actor and the current approach will make it messy very fast.

One big thing of actix is the ability to control the concurrency of message processing in the async case (sequential or parallel). How does Ractor does it?


So to your two parts.

1. If you would want independent message handlers, normally we'd just add functions off the state which would be called strait from the message handler's match statement. I.e. something like self.handle_msg_1(message, state, ...). We looked at having multiple handlers but you kind of either get into generic's hell or having to do some kind of ugly proc macro. We figured this left it up to the implementor on how you want to handle it

2. Sequential processing would really be a factory. That would do parallel work of jobs using a basic actor implementation while handling concurrent work limits, job routing, etc. Otherwise you could always spawn inside your actor and like a wait on multiple handles if you wanted, but that gets messy and kind of diverges from the true intent. There is a factory implementation in the base ractor lib, feel free to take a look!


Is there any resource that can help me properly understand the actor model and make an informed decision on whether it's something everyone should have in their toolbelt or just another technology like blockchain, in search of non-scam usecases with mainstream appeal?


The actor pattern is used quite a bit in the functional world (Akka is pretty popular in the Scala ecosystem). It's just a design pattern that lends itself well to distributed applications with complex communication structure like some microservice systems. Some people use it to represent graph-like data structures with nodes that update themselves based on their neighborhood, though that's kind of gimmicky. There's no connection at all (that I'm aware of) to crypto or fintech scams, and I guess it doesn't hurt to have a look at the actor pattern if you're in a functional language; always good to know what tools exist.


Actors have been used since the 90s for distributed programming, in particular at telcos. To a large extent, microservices are a poor man's reimplementation of actors, with all the drawbacks and not many of the benefits.


Not scam or even fad. It’s legit. However, details vary so I agree with the sibling that “design pattern” is the best classification for it.

I think it’s fair to say that the actor model is analogous to FP, but for distributed (and to a lesser extent multithreaded) computing. Just like FP has friction against imperative programming, there is a similar friction when interoping with traditional systems. Likewise, benefits pile up once you have some critical mass of subsystems that all use the actor model, where certain problems just go away.

I’m personally a big believer, it fits my mental model as I like to think about data flows first. That said, almost all systems today are request-response, so you have to either accept a certain amount of friction or embrace the much smaller ecosystem of message-based systems.

Just like regular programming is going more and more multi-paradigm, incorporating FP and others into mainstream langs, it seems like the distributed world is moving slowly in a multi-paradigm direction as well. For instance, databases and file systems are increasingly using message/event streams and sometimes exposing them directly to users.


Designing Data-Intensive Applications is a good start.


This classic book contains a chapter (chapter 5) on Actors:

https://www.amazon.com/Seven-Concurrency-Models-Weeks-Progra...


Carl Hewitt has been championing it for a while https://arxiv.org/abs/1008.1459 https://youtu.be/7erJ1DV_Tlo


It is definitely in the "everyone should have it in their toolbelt" class. It is one of the fundamental primitives that makes multithreaded programming a sane and sensible thing that almost anyone can learn and do, rather than an insane hellscape of locks. It is not the only such primitive, but it is one of the basic ones.

At the core, this is an actor: Just as an object fundamentally binds some bit of data to some set of methods that operate on it, an "actor" is a binding of a thread to some data, such that no other thread will ever access it. Therefore, there is no locking problem on that data and the actor has full control over it. It embeds an island of single-threaded processing in a sea of multithreading. It's a "server" with all the reductions in complexity you can apply to it for being in process rather than an external process communicated with via some stream.

As with objects, you can elaborate on that theme in a number of ways. You can enforce at the language level that actors have unique access to certain data (Erlang/Elixir, Pony). You need some communication system; you can bless one as part of the language runtime (Erlang/Elixir mailboxes, Go channels). You can build in support for various structures on top of official actors just as you can build support for various object structures once you have an official concept of an object, like supervisor trees and crash notification (Erlang/Elixir). You can provide actor-level polymorphism by defining protocols they have to conform to. You can embed actors into a structured concurrency model (no language yet [1], as far as I know!) to create a "structured actor" model.

However, just like objects, you get hordes of people insisting that their way is the one true way of doing actors and if you can't check off EVERY LAST BULLET POINT FEATURE of their preferred implementation you don't have actors. I imagine what you are picking up about actors are these people, who come away thinking that if you aren't Erlang, you aren't using actors. These people are wrong, just as people who think that if you aren't exactly Java you don't have objects. Actors at their core are very simple; really the only thing you have to have to get them right is to make sure that an actor's data is only accessed from one thread. Just as I tend to take an expansive view of "object orientation" that it is simply about having "methods" of some sort attached to data structures and I don't sit there and argue about how you need to have a "protected" key word and all the structure that implies to have an object system.

While language support for completely rigid isolation of data to a thread is nice in some ways, it is not necessary, and it can also sometimes be a problem if you want to do something else; it is not hard to accomplish the goal with other tools. For instance, in most object oriented systems you can accomplish this with a factory/constructor function that creates an object and spawns its associated thread, using some variant of "private" to make it so only that thread ever receives access to the internal data. I'd even consider that second clause optional in the case of a language like Python, where private-ness is more convention than strict enforcement. But push comes to shove, nothing stops you from writing actors in C, just as you are not stopped from writing objects in C. You just get zero language support.

In fact everything in the paragraph about what you can build on top of actors can be built as libraries. It's just that the tradeoffs vary from system to system. I implemented supervisor trees in Go, for instance. They aren't exactly what is in Erlang, but they largely solve the problems I have. A language may not have support for an "actor protocol" but using an object oriented language's concept of polymorphism allows a rough equivalent through implementing methods that encapsulate communication with the actor does roughly the same thing and solves the same problems. It isn't always perfect, but there's never a perfect solution to every problem in a given language. Having perfect actors implies some other weaknesses somewhere else.

So, yes, the concept of binding data to threads is a critical one to building modern systems. While I generally phrase the errors of the 1990s threading hell world in terms of their locks, a complementary view is to say that the screw up of our first pass at threading, with the attendant fear still echoing through our community to this day, was the attempt to allow all threads access to all data all the time. When you don't do that, you don't get the threading hell of the 90s. Actors are not the only solution to the problem, but binding data to specific threads is one of the basic tools in solving it. Extremely important to sane multithreaded programming.

[1]: Yes, this includes Erlang; in Erlang you can still "spawn" whenever you feel like it. It is true that usage of the OTP library encourages things that look like "structured concurrency" but it does not enforce it, nor does it take advantage of some of the things that structured concurrency would allow. And I'm talking about languages and their strict enforcement here, and as basic as OTP may be, it is still a library.


Here's the chapter "Message Passing and Actors" from the Programming Models for Distributed Programming book by Nathaniel Dempkowski, comparing different types of actor models:

http://dist-prog-book.com/chapter/3/message-passing.html


That is a good, comprehensive link.

I would also say that matches the academic viewpoint fairly well. It shows how my reductionistic view of actors as the important thing being the binding of data to a specific thread of execution is a fairly heterodox view. I don't want to hide that. Note how that link does discuss data, but it isn't anywhere near as front-and-center as I would put it. If I were writing a similar survey, it would probably be my organizing metaphor, rather than message passing. I'm not criticizing that organization; it's a completely valid handle on the problem as well. It's just not what I would use.

At the end, they disclaim Communicating Sequential Processes as not being an actor model. By their terminology, this is correct. By my heterodox view, though, I would still put them under the "actor" category, at least in general (I don't know if it's strictly speaking necessary for the processes to have isolated data, but even if not specified one does wonder what the point of all the communication is on a practical level if the processes can just reach in to each other's data sets), and to me they would be another example of the various elaborations you can apply to them.

Ultimately what kept me out of academia is I had too much of a hybrid engineering/academic view to fit into either one comfortably, and I could tell there wasn't much room for an "in between" all protestations to the theoretical desirability of cross-discipline cooperation notwithstanding. There's a sense in which all the details in that link matter. But there's another sense in which 75% of the problem is solved just by ensuring that you've got some data that can only be accessed by a given thread of execution, another 20% that you can't hardly help but solve beyond that (e.g., you will come up with some sort of messaging solution, and the details of that solution may affect your architecture but broadly speaking any half-sane solution will generally be able in some way to solve any problems you have somehow), and the remaining 5% is academically interesting but from an engineering perspective will generally only start to bite you at very large scales, so are only worth worrying about if you expect to scale that far. The latter point of view is... not the common approach academics take.

But you don't need to worry about a lot of that stuff to just get started. The guidelines I've given in these couple of posts will get you the vast bulk of real value with a fraction of the concepts, and while you will need to choose some sort of system to use your actors, the complexity is less than it appears because most of the time any of these solutions will work for you, or you'll be choosing them on completely other criteria like implementation language or network protocol compatibility.


Does this have soft real-time preemption capabilities like Erlang? IE: Do I get the BEAM's latency ~"guarantees"?


Ah so presently we're using the Tokio scheduler, which does cooperative scheduling not preemptive. We've abstracted pretty much all of our core concurrency primitives to a single module however so the plan is to support future schedulers (perhaps custom) in the future, but that's not the main focus at the moment. However if you have a blocking task, that will utilize a lot of sequential CPU time, the general guidance is to tokio::spawn_blocking(..) in your actor then await the result, so a new dedicated i/o thread will be used and your actor will yield the main scheduler for other workers.


The trouble with spawn_blocking is that it runs on the tokio thread pool... By default the size of that thread pool is gigantic because it is meant to run blocking IO. For computation, you would want to have a number close to the number of threads of the CPU.

... And you can't have both.


How would you compare this to Orleans? (beyond the obvious Rust vs C# bit)


Actually I was not familiar with Orleans until now. I think there's probably similar primitives you could build but from my quick 2s read they're solving a higher order problem. Additionally having written an actor lib in the past in .net, I can say they can't match rust for speed but they do have some nice syntax stuff with reflection we can't do.

So it's a tradeoff on stuff like syntactic sugar and speed imo


Orleans was born to support Halo's server backend and went from there, they know their stuff regarding performance.

> Orleans is used by Microsoft in Azure, Xbox, Skype, Halo, PlayFab, Gears of War, and many other internal services. Orleans has many features that make it easy to use for a variety of applications.

https://learn.microsoft.com/en-us/dotnet/orleans/overview

Until Ractor proves itself in similar workloads, it is a tiny bit more than syntax sugar difference.


[deleted]


Fair enough, then it is proven that it can handle the load.

Then there is the question about the features themselves, besides what any actor framework should provide as baseline.

Orleans is already on version 7.


I still think the design in the beam world is as close as we can get to perfect actors. Its very very elegant in how you can easily resume crashed actors, upgrade them, migrate them.

`handle_call(pattern, from, state)`

Return your reply and your new state and boom. It just works.


Yeah hot upgrades is something we'd love to get to, but probably won't be able to swing in a compiled language. We had worked out something ugly back in the day in a .net style environment (different library at a different job), but it still didn't compare to Erlang's hot upgrade flow.


So a problem I noticed, is you pass mutable state to the actor.

What happens if the actor mutates the state then fails?

Do you just dirty the state and throw it away or do you retry at all?


If the actor fails, it could have failed mid-mutation so we can't trust the "state" object. This would be the same if we passed in an owned state (rather than a mutable reference) and the handler didn't reply with a new state object. If a failure happens, the state is dropped there too (in Rust). I know it's not exactly how Erlang handles it, but it's a tradeoff we've accepted in this case. We're open to suggestions however!


It looks like lunatic has a solution? Not very familiar with the project, but maybe worth checking out.


They don't, at least not yet. https://github.com/lunatic-solutions/lunatic/issues/150

It is WebAssembly based though, so it should be easier than trying to deal with dynamic libraries and Rust's ABI.


Are you going to be able to pattern match as nicely as one can on the BEAM?

In practice, is the handle function going to match on each message and then call a handler for that specific message which may then match on the state?


That's really up to you, Rust has great pattern matching imho, rivaling Erlang's for sure, but with some nice syntax upgrades. Additionally in Ractor, RPC's are strong-typed to the reply type, so a `call` in Erlang might result in the wrong type coming back which then you have to handle in your reply pattern matching or crash. This isn't possible in Rust w/ Ractor, since you _know_ the strong type in advance and can't send an incorrect value.


The no VM thing I think is a downside. Sure raw performance is nice but not having a process eat all system cou resources is the true beauty of Erlang and Elixir to allow you to self heal. Are you handling that with this library?


Can you give an example of what you're referring to? I don't know of anything limiting memory / cpu / etc in Erlang at least of any individual gen_server. We have the Factory processes which can gracefully loadshed, but that doesn't stop you from having a memory leak.

At least Rust doesn't have a garbage collector, so when the actor is stopped + dropped, it'll cleanup not only it's state but also it's message queue's flushing them so that all memory is released at the time of drop.


Docker and kubernetes are here for that


They’ll give you ways to limit the CPU use of the OS process, but not the individual actor “processes” (Erlang overloads the term), which are opaque to k8s/linux/docker.


Because it's related and a powerful alternative for those stuck in C++ land: The C++ Actor Framework (CAF) implements a combination of Akka and Erlang semantics.

Especially the type-safe actor abstractions saved us many bugs, e.g., sending an uint32 but trying to receive a uint64.

https://actor-framework.readthedocs.io/


What would you use this thing for?


Any task that suits the Actor model. FoundationDB famously employs an actor model as it's a nice abstraction for creating distributed systems.


Something I don’t get about FondationDB is how fast their actor/future lib is. It’s insanely fast.


Here is another one in Rust: https://docs.rs/bastion/latest/bastion/ How does Ractor compare to Bastion?


My first observation is that Bastion looks abandoned and last commit was more than a year ago. Maybe it is feature-complete and stable, but..


This looks interesting. I don't think the single-threaded design fits my use cases, I would prefer a thread per actor. Both are valid tradeoffs though, and it's always good to see more projects in the area


Awesome to see a project like!

One question: what happens for panics that won’t be able to be catched by std::panic::catch_unwind?


That's essentially a process-abort level panic, which then your whole environment would die. Unfortunately there's not much we can do about that. This will catch all unwindable panics however, so that's quite a lot but yeah there's an edge-case.

https://doc.rust-lang.org/std/panic/fn.catch_unwind.html


Ok thank you for your answer.


Is the "let it crash" philosophy a part of this or will a panic bring down the whole process?


Panic's (which can be captured) are captured and will be propagated to the supervisors. That's one of the biggest goals of building this, panics in tasks which have been spawned is a nightmare to deal with


Looks interesting. I'm sure I'll try it one day.


When I was looking for a name for my now-archived distributed actors system based on Redis ( https://github.com/buybackoff/Ractor), the choice for Redis+Actor was between Reactor, Redactor and Ractor. Then I found a reference to The Diamond Age book (https://en.m.wikipedia.org/wiki/The_Diamond_Age) where a ractor is a virtual actor in interactive games and movies working from a computer-provided script. I found it somewhat fitting to the actors abstraction. And the most cyberpunky.


Never a bad time to plug Pony lang[1] - a safety-oriented actor-model language. In addition to the numerous safety guarantees, you also get a beautiful syntax and automatic memory management. Really a great language that often gets overshadowed by Rust's hype-turfing.

[1]: https://www.ponylang.io/


Never a bad time to plug

Somebody else's Show HN is not really a great time or venue to tangent off into some language war.


I don't think it's unreasonable to link something relevant in the comments of a Show HN.


Pony is nice but the ecosystem can't compare to Rust or others.


A bit of a chicken and the egg problem, I'd say, but you're right. You can, however, extend your applications using the C API which opens you up to the entire C ecosystem. Someone has to go and write a bunch of bindings though, and that largely hasn't happened yet, probably because the language is still in flux and writing bindings can be time consuming, and having to maintain them against constant breaking changes would be awful.


Language models solve that. AI translation from Rust into Pony.


"[W]ith "NO unsafe code...."*

* except all the unsafe code in the dependencies (e.g. https://github.com/tokio-rs/tokio/search?q=unsafe)


Clearly nothing can be truly safe then if you ever use the standard library ;-)


You know, people outside the Rust community keep bringing this up as some kind of argument against it. But to me, it's one of the biggest points for it; because the important thing about `unsafe` encapsulation isn't what's inside of it, but what's possible on the outside.


I bring it up because Rust evangelists like to pretend that Rust code with no explicit “unsafe” in it is safe. It’s not safe, though, as long as the dependencies use it.

I enjoy coding in Rust. I don’t enjoy the shit name its community gives it.


All safe code that exists in any language is built on unsafe code. Unsafe code is fundamental to computers. Your frame is unhelpful, not illuminating.


All safe language runtimes use unsafe code by this logic.


Does that also apply to GC-languages?


Yes, if you dare to look into a GC implementation, you might see some truly unsafe code indeed


You can't have 100% safety all the way down. Garbage collection doesn't help you if the VM itself has a memory leak or some driver crashes.




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: