My own vision is to take a queue that is relatively dumb and make it smarter. I want it to be able to, for example, allow you to rate limit workers without needing to implement this client side. And so on for all of the other bits that one needs to implement in the course of distributed processing.
I’m still figuring out the market. Very large firms spending thousands of queues? Or developers who want a one stop solution built on familiar tech? Or hosting companies who want to offer their own queue as a service?
I've heard this use case come up in hybrid new/legacy products where there is a chasm between legacy scalability (probably a monolith in the compute or storage layer, or both). The "new" side needs to be able to self-regulate and if you can put that into the service-side it would it would be a helpful transition enabler (and, let's face it, given these transitions often never finish your product would be effectively sticky).
I don't think the cost of queues is a problem anywhere (I'm sure it is somewhere, but not a market's worth). The problems created by queues, on the other hand, are myriad and expensive.
My own vision is to take a queue that is relatively dumb and make it smarter. I want it to be able to, for example, allow you to rate limit workers without needing to implement this client side. And so on for all of the other bits that one needs to implement in the course of distributed processing.
I’m still figuring out the market. Very large firms spending thousands of queues? Or developers who want a one stop solution built on familiar tech? Or hosting companies who want to offer their own queue as a service?