This is awesome, I'm incredibly happy to see more alternatives on the Javascript Runtime Space!
In a nutshell: LLRT aims to have incredibly fast startup times compared to Node, Bun or Deno (which is critical in Lambda environments).
LLRT is built using Rust, Tokio and QuickJS under the hood.
Here's their compatibility with Node.js APIs [1]
We (at Wasmer), have been working on a very similar approach: WinterJS, which is using SpiderMonkey instead (but also built on top of Rust and Tokio) [2]. HN announcement was just three months ago [3]
Really excited to see how the ecosystem is evolving!
Deno’s and Bun’s Isolates should be spawning faster than a QuickJS process, they also utilize snapshots for more optimization. QuickJS is going to be faster at startup from 0 though
This seems ideal for places you use Lambda as glue, like for routing requests or authorization policy decisions. But for application that doing a lot of thinking, keep in mind v8 jitless is about 3x faster than QuickJS, and with JIT it’s about 30x faster than quickjs. I’m curious to see where the break-even point is for various workloads comparing the two, and also numbers versus Bun, which starts quicker than Node but has comparable top-end JIT performance.
One good thing Lambda provides is contractual guarantee, if you want 100000 code spun up you can put it in writing and hold AWS against it
but then you realize you can achieve the same thing at 98% discount with traditional "monolith" setups with a load balancer soaking up all the requests
This is only true for certain workloads. Specifically, workloads that you can put behind a load balancer.
One huge advantage of FaaS infrastructure like Lambda is when your workload doesn't need to process >1 rps. Lambda, for example, has tight and unique integration with foundational AWS services like S3, SNS, SQS, and EventBridge.
Lambda is awesome when you need small scale reliability. I shouldn't have to run a VM 24x7 if I don't need to!
Running 1000's of EC2's at scale? Want to handle instance health notifications?
My favorite use of Lambda is S3 object event notifications, it makes it very easy to handle a variety of odd jobs by decoupling storage and compute and managing each independently. S3 Events are exactly the right use case for LLRT.
Yeah, that was what I was alluding to as "aws glue"; for something like S3 object event notifications or IAM policy checks this kind of light weight runtime seems ideal.
Serverless has two things that I really like about it:
1. Scale to zero, which allows me to do things like automatically deploy a full environment for each branch in a project, with a full infra, frontend, backend, database, etc.
2. Good integration with IaC tools, so that I can define my infra and my runners in a single language/tool, be it terraform/cdk or something else. Most "monolithic" setups split configuring the infra and what runs in it into two tools/steps (please let me know of ones that don't!).
But if I actually run a application for a long time with somewhat consistent load there are always cheaper, more performant and flexible solutions than a serverless setup.
That’s a depends situation. If you’re using direct service integration instead of Lambda, it can be cheaper. And, if you use VMs only as the front-end to push batches of requests to serverless compute you get the best of both worlds and the cheapest by a long shot. Not everything can fit in that model, but the things that do are magically inexpensive.
I’m actually working on a product to make that architecture more approachable (and cheaper yet). I’d be happy to hear from folks running network services on VMs and wishing there was a better way.
That’s true, the front-facing VMs always need to be there (hence the product I’m working on — so at least you don’t need to run them). The real work is done in serverless, in batches. Basically I think the world needs something like API Gateway for UDP and custom TCP, but that targets serverless backends rather than VMs.
Sure, if you try to shove everything into that mold it is, but there are absolutely times where it's the best tool for the job. I run my entire, personal, company on top of it and it saves time, money, and energy.
Sure, I pay less than a dollar most months then in my busy months I’ll pay a max of ~$20 for my backend.
It’s event software, like for a food festival. Most of the months there is almost 0 traffic, for 1-3 months there is lower traffic as pre-sales start with random spikes when the event organizers advertise, then for 1-5 days there is high traffic peaking on a single day or two during the event.
Because the load is incredibly spiky it’s nice to let Lambda scale automatically for me. Trying to scale a server, even bare metal, manually would be a pain in the ass. Even as I run this software for multiple events it’s cheaper than $5/mo and a $5 instance wouldn’t be able to handle the load at peak.
If my lambda bill ever got higher than an average of $20/mo I might consider a bare metal or VM server but honestly I prefer not thinking about it. And the profit dwarfs the server costs, like it’s not even close. The lambda server costs, even if they were 50x, would still be minor.
I'm guessing for HTTP the difference in performance is negligible. QuickJS is like 50x smaller than V8 so the gains in startup time and memory usage will have a deeper impact for this use case.
Interesting project: they are trading lower raw throughput in compute-heavy applications for lower latency at startup and (likely) better behaviour w.r.t memory and "random GC jank".
If you are writing an I/O bound JS service that doesn't do a lot of thinking, but does a lot of "moving stuff around", this will probably be a win.
Yes, though don't think of "traditional" JS engines as being all the way on the throughput side. One of the main distinguishing characteristics of JS engines, at least those that are used in web browsers, is that they have to care a lot about startup time and have to juggle a lot of tradeoffs that other managed runtimes typically don't need to bother with.
A surprisingly high percentage of downloaded JS code is never run at all. (I can speculate: event handlers, unshaken trees, shaken trees with paths that happen to not be taken, ...) Of the rest, the vast majority is run once or a handful of times. That means startup time is a high percentage of total time, and therefore engines will JIT very little of that code, and the little that is JITted will use a baseline JIT that pretty much just translates opcodes to unoptimized machine code. The fancy optimization engineering that requires much cleverness and effort is actually applied to a very small percentage of incoming code. It will need to run something like 1000x before it is seen worth bothering with. Latency is more important than throughput for almost all of the JS that the engine sees. (Note that throughput-sensitivity can easily dominate most of the runtime, but that varies widely depending on workload.)
So browser-embedded engines already work hard at optimizing latency. We're not talking Python or Ruby here.
That still leaves a lot of potential for improvement by stripping the engine down and prioritizing latency above all else, though. There's definitely a niche for things like LLRT. And WinterJS, too.
The benchmark shows Node.js taking 1.5s(!) to start vs. LLRT taking under 100ms.
But the benchmark appears to be a very small program that imports the AWS SDK.
Later on in the readme, it's mentioned that LLRT embeds the AWS SDK into the binary.
So... How much of what the benchmark is showing here is just the fact that the embedded SDK loads much faster than application code?
I mean, it's certainly a valid optimization for Lambdas that are primarily making AWS API calls. But it'd also be interesting to know how much faster LLRT is at loading arbitrary code. As well as interesting to know if the embedding mechanism could be made directly available to applications to make their own code load faster.
(Disclosure: I work on Cloudflare Workers, a competitor. But, honestly interested in this.)
I am also a bit surprised they chose a bechmark that includes network latency as well.
Also the 1.5s NodeJS cold-start seems quite high and is not what I would expect at all. Especially when looking at https://maxday.github.io/lambda-perf/
The SDKs are "bundled" in all lambda runtimes but normally not into the binary, what additional performance would that bring?
> The SDKs are "bundled" in all lambda runtimes but normally not into the binary, what additional performance would that bring?
I haven't looked at LLRT's internals, but if I were them, and I were bundling some JavaScript code into a binary and seeking to really optimize startup time, I would probably pre-parse the JavaScript text to produce QuickJS bytecode (or whatever data structures QuickJS actually interprets at runtime; no modern interpreter is actually processing raw text as it goes). In the best case, embedding something like that into the binary could mean that startup processing of the embedded code is O(1) (just like how startup time for a native-code binary is independent of its size, as long as it doesn't have global constructors).
From all that I have read, file-size is a major part of cold-starts as your zip needs to be fetched internally first so there is some app size where increasing the files by pre-parsing might make it slower and only after a certain size would it make sense. Edit: Sorry, you were talking about the runtime, not the app bundle, just realized this now. If it‘s in the runtime it probably is free as that will probably be available in all lambda hosts I guess.
Lambda already has an even more aggressive optimization than just pre-parsing, which they call SnapStart; where it takes a RAM snapshot after init and loads that afterwards but I think it‘s inly available in some regions and some runtimes like Java with very slow cold start times.
Not really. I have built a fetch-like API for hermes on top of libcurl using generated FFI-bindings (they have a bindgen tool in the repo). All I had to do is just write the JS-wrapper that would call the underlying C-lib and return a Promise.
I mean that the AWS SDK is a large API surface, they'd potentially have to re-implement all of it in Rust and then keep up with future changes, otherwise you have to tell people it's not 100% compatible.
I'm told this library is multiple megabytes of JavaScript. (That's why Node takes so long to load it...)
This benchmark is really an esoteric performance comparison, because the AWS SDK isn’t monolithic anymore and you can only include the packages you need (assuming you aren’t using a minifier and tree shaker to reduce the parsed code anyway). It was only a monolithic library back in v1 and they’re on v3 now. For most use cases, you only need to load a small subset of the AWS SDK. I have certainly written node lambdas with only a few hundred ms coldstarts using multiple aws SDK libraries bundled in (secrets manager, dynamo, s3, parameter store…)
AWS generates all their SDKs from something like Smithy. Not to say that that doesn’t have its own problems, but given that they have an existing Rust SDK, it’s not an insoluble problem.
> offers up to over 10x faster startup and up to 2x overall lower cost
i've been burned by Amazon with these type of claims can we get confirmation this is backed up actual real world testing from a non-amazon affiliated source?
I would love to see similarly low startup times with a real JS-to-WASM compiler (rather than today's approach of compiling a JS runtime to WASM).
With WASM GC landed, and WASI preview 3 taking on async / event loops, it seems like this should be possible soon, except for the very large task of actually writing the compiler.
There's a VERY good reason why JS runtimes use JIT's, it's more or less impossible to properly fully type a JS program w/o leaving runtime holes.
If you restrict yourself to a subset or are willing to risk incompatiblities then it will work.
There's some TypeScript to Wasm compilers that should work reasonably well (If you're breaking TS rules you've got yourself to blame a bit more than with JS).
Ref: My thesis was on JS AOT and even if I had the idea of making it commercial I didn't pursue it in the end.
QuickJS and SpiderMonkey in WASM don't use JIT though - they're both just interpreters.
WASM itself is often JIT'ed so a JS-to-WASM compiler that functions like the common JS baseline compilers would still get some benefit of JIT'ing even though the types available to the ICs in WASM may not be as useful as in JS.
Yes, it was written for other students so it's not as clean as most of the referred research papers. Part of the work was implementing a compiler whilst part of was surveying how well real world JS developers would understand language subset limitations.
Most relevant for my above comment re AOT vs JIT is that JS semantics (I think I touched upon it in the thesis) will have the same issues that made JIT's win out with BigInts in the Agesen&Hölzle paper (whilst the paper is on AOT's my goal was to find a solution for game developers so numerical performance was a non-moving goal for me).
If I understand it correctly, it exchanges execution speed (by not using JIT) for faster starts. Instead, they could pick up the old FB project https://prepack.io/ to optimize both start AND execution speed. It would be very interesting to see it together with node's snapshots: https://blog.logrocket.com/snapshot-flags-node-js-v18-8/, which reduces the start significantly.
I am working on similar runtime QuickJS/C/C++ using async stackful coroutines.
Last year tests were about 14 times faster for networking related to node.js
Here are tests results: https://voda.io/test.html
I plan open source it later this year - have improve build and remove some client's code.
pljs (https://github.com/plv8/pljs) is a postgres language plugin that also uses quickjs. given how much faster the startup is compared to v8, and that most postgres functions are very small and do very little compared to a node.js program, it is quite good.
I can definitely see aws' lambda operations gaining quite a bit from this.
I think Jarred Sumner (creator of bun) summarised it well:
Only faster for code that runs under 5 ms. But a lot of code runs under 5 ms.
(I hope the quote is right, I'm not on twitter.)
I think it makes Lambda better, but not more attractable to use cases that are better served by scale-to-zero managed containers runtimes like Fargate.
i wonder why no one is talking about workerd, cloudflares much more mature approach that is behind cloudflare workers. deno is great but has much broader scope than llrt, workerd or winterjs. these are really low latency edge runtimes not general purpose ones.
Since we're paying for it I'd call it more of "depending on the competence of service providers." AWS Lambda experience has been great for us in that sense, so I'd be looking forward to more general availability of this for Lambdas.
My only wish for AWS team is that they measure success based on how complete a product is. There are still 4 year old outstanding critical github issues around Cognito. There are lot of things that are advertised that aren't delivered.
In fact our trust in AWS and the team has plummeted that we ultimately ditched serverless and went str8 to VPS
The days of zero interest rate series B buying up reserved EC2 instances in the millions are gone. As capital gets expensive we see more reluctance to moving to cloud.
AWS claims of "10x faster startup @ half the cost" is confirmation that the Serverless movement is DOA
Amazon's repo here is open source at least so perhaps this could be remedied with an edit to their README and a PR - go for it !
Agree with you there should be credit where credit is due - I have been using QuickJS for some time and its awesome. For the cost of about 1MB you can get entire modern JS in your C/C++ binary.
I'd definitely spend an evening watching a video of you gluing an equivalent solution together. And if you did finished in one evening, you'd be justified in being very proud.
In a nutshell: LLRT aims to have incredibly fast startup times compared to Node, Bun or Deno (which is critical in Lambda environments).
LLRT is built using Rust, Tokio and QuickJS under the hood. Here's their compatibility with Node.js APIs [1]
We (at Wasmer), have been working on a very similar approach: WinterJS, which is using SpiderMonkey instead (but also built on top of Rust and Tokio) [2]. HN announcement was just three months ago [3]
Really excited to see how the ecosystem is evolving!
[1] https://github.com/awslabs/llrt?tab=readme-ov-file#compatibi...
[2] https://github.com/wasmerio/winterjs
[3] https://news.ycombinator.com/item?id=38047872