The solution seems to be solving for the simplest use case (internal stateless functions) rather than the most complex use case (external state-impactful functions).
Furthermore, the words used aren't really what they should be talking about.
>> Because workflows are just Python functions, the thread can restart a workflow by simply calling the workflow function with its original inputs and ID, retrieved from Postgres.
>> For this model to work, we have to make one assumption: workflow functions must be deterministic.
Yes, "deterministic"... because all state modification is aligned to ensure that.
If instead of a single print() the function/step had 2 print()'s, the state leaks and the abstraction explodes.
The right abstraction here is probably something more functional / rust-like, where external state modifications sections are explicitly decorated (either automatically or by a developer).
That's exactly what this model is! The @Step decorator is for external state modifications. Then @Workflows orchestrate steps. The example shows the simplest possible external state modification--a print to the terminal.
Steps can be tried multiple times (if a failure happens mid-step) but never re-execute once complete. Since idempotency can't be added externally, that's the strongest possible guarantee any orchestration system can give you (and if your step is performing an idempotent operation, which is the safest thing, you can use the workflow ID as an idempotency key). More details in the docs: https://docs.dbos.dev/python/tutorials/workflow-tutorial#rel...
You're forcing adopters to divide any state-impactful activity into its own function (because only functions can be decorated with step, no?). That's seriously inelegant when scaled to larger codebases.
Regional tagging (e.g. safe/unsafe) would be a better approach, as it would allow developers to more naturally protect code, without redefining its structure to suit your library.
You start to grok the problem here, but primarily think about it in terms of databases, which are just one (admittedly common) type of external state:
>> If you need to perform a non-deterministic operation like accessing the database, calling a third-party API, generating a random number, or getting the local time, you shouldn't do it directly in a workflow function. Instead, you should do all database operations in transactions and all other non-deterministic operations in steps.
Note: Think you should really change "all" into "each in a separate transaction/step" there, to communicate what you're recommending?
As a thought exercise: imagine a Python program that automates a third party application via the GUI. Some UI actions cannot be undone (e.g. submit). Some are repeatable without consequence (e.g. navigating between screens).
How would your framework support that?
Because if you can efficiently support the pathological leaky-state case, you can trivially support all simpler cases.
Yeah, this definitely requires splitting state-impactful activity into its own function. That's good practice anyways, though I understand it might be a pain in large codebases. Regional tags are definitely an interesting alternative!
For the UI example, I don't think you'd use durable execution for most of the UI--it's just not needed. But maybe there's one button that launches a complex asynchronous background task, and you'd use durable execution for that (with careful workflow ID management to ensure idempotency and allow you to retrieve the status of the background task).
> splitting state-impactful activity into its own function.
Each separate state-impactful activity into its own function, no?
> For the UI example...
I wasn't talking about building a GUI in a python program: I was talking about using python to automate an external GUI app.
Because that toy example (which also happens to be done in the real world!) encapsulates a lot of the state issues that don't seem well-handled by the current design.
Namely, that external interactions are a mix between stateless and stateful operations.
The solution seems to be solving for the simplest use case (internal stateless functions) rather than the most complex use case (external state-impactful functions).
Furthermore, the words used aren't really what they should be talking about.
>> Because workflows are just Python functions, the thread can restart a workflow by simply calling the workflow function with its original inputs and ID, retrieved from Postgres.
>> For this model to work, we have to make one assumption: workflow functions must be deterministic.
Yes, "deterministic"... because all state modification is aligned to ensure that.
If instead of a single print() the function/step had 2 print()'s, the state leaks and the abstraction explodes.
The right abstraction here is probably something more functional / rust-like, where external state modifications sections are explicitly decorated (either automatically or by a developer).