In posts talking about how someone made something fast, I see the same idea repeat time after time: it has linear time complexity, has great cache locality, and saturates all available processors.
It's annoying to me how much time I had to spend outside of school before I realized that this is the real recipe for fast code. In particular, even code that is nominally O(n^2) by standard measures is still a dis-economy of scale: the bigger the problem the slower it gets. In school, there's a premium given for finding a polynomial-time solution, but in practice every job where I've been tasked with making something fast, only linear time complexity was good enough.
In practice these linear algorithms tend to also be completely data parallel and have great cache locality. Not only do these properties make the code fast the second you harness them, but then tend to be exceedingly simple architectures to maintain over time.
For some other reading I really like on software performance in the real world:
> but in practice every job where I've been tasked with making something fast, only linear time complexity was good enough.
But of course we remember from our CS school that some things simply cannot be done in linear time. I’m not sure what happens when your employer tells you to sort a big list and insists that only linear time complexity is good enough.
To that I find that the answer is usually: find ways to avoid needing to sort the list! This is more viable for many problems than you might otherwise think.
There’s also vastly faster ways of sorting lists of bounded numbers (or tuples), the most linear being: cut straws of the exact right length labelled with an Id, bang them on a table so the ends without the ids all line up, read the ids back in order.
I only said it took linear time, I didn’t say it was fast...
At one job we used to joke "i don't care if it's right, as long as it's fast"
Of course, it was only half a joke. For some pages, we really did want data quickly, so we were willing to accept stale data or somewhat inaccurate data, it that helped. For other pages/sections, accuracy was more important so caching wasn't possible or invalidation was critical.
But product design is always about tradeoffs between what features are desirable, what features are possible, what features are easy to build, and how features interact. Features that are possible but not easy to build can sometimes be replaced with features that are easy to build and close enough.
Radix sort is still O(n logn). More precisely, it's O(w * n) where w is the size in bits of the data being sorted, but then it follows that w scales with log n, since n distinct items requires log n bits and hence you arrive at n logn.
Sure, but it has the classic problems of the memory/storage required for the sort (minimum double all your keys), and a classic math issue relating to it's speed being O(k*N) and since k = number of bits/digits, N can only be as large as the number of bits/digits allow for.
This means quicksort and the like end up faster in practice for any "sort this list" operations. Radix/bucket sorting can be SUPER awesome when what you need is a single level of bucketing at all rather than exact order.
Sure, but it has the classic problems of the memory/storage required for the sort
No problem, just create a file of the necessary size on disk if you don't have enough RAM available.
Today's comment was brought to you by a large number and the letters big-O and c... where c, a large number, is the hidden constant of proportionality waiting to bite you every time you mention big-O. :-)
If we're reading from disk, then we DEFINITELY don't want radix, as then it's almost guaranteed that all time will be dominated by loading pages rather than cpu cache locality being leveraged by linear time algorithms - which is what this entire discussion is about.
If we are talking about linear in number of characters, an easy answer is building a trie and traversing it left to right. (Cache-friendly variation: https://en.wikipedia.org/wiki/Burstsort)
Anything you didn't learn in Christian Science courses at university, you can find and learn more at your local Christian Science Reading Room! Most cities and towns in the USA have one, and even many airports for people on the go. If you're flying through SFO, you can visit the one in terminal 2!
> in practice every job where I've been tasked with making something fast, only linear time complexity was good enough.
Presumably you weren't tasked with making something fast, were linear time complexity was overkill. In other words, quadratic complexity may have been fine in many cases, but you weren't tasked to make those fast.
I'm not saying you claimed otherwise, but somebody might misread you as saying "in practice, only linear time complexity is ever good enough".
It's also worth noting that if your size is bounded a simpler quadratic algorithm might be more maintainable overtime than a linear one (if it's easier to grok). Say max(n) = 25, it doesn't really matter if it's linear (say 4N = 100) or quadratic (N^2 = 625). In practice they're both fast enough (even though one is 6x faster) and not worth optimizing but if it can grown unbounded that's when the BigO runtimes start to matter. For something like ESBuild your input size grows larger and larger as your codebase increases and that's when you start to see slowness creep in.
Esbuild looks amazing. It seems to have been written almost exclusively by one guy (evanw) during 2020. Just the main JS parser file [0] is 12k lines. According to the Github stats he has committed 280k lines, that's almost 1000 lines per day every day since he started. Amazingly productive.
While that is impressive, I think this is also an indication of a problem: the grammar is becoming unwieldy. For example, even for someone as prolific as Evan, he must decide between feature depth (e.g. bundle splitting) and feature breadth (e.g. implementing the equivalent to @babel/preset-flow, which is used by both flow and hegel[0] type systems)[1]
Esbuild supporting Typescript and JSX is undoubtedly a byproduct of these grammar extensions having become popular. But supporting extra grammar extensions does add to complexity, sometimes in non-trivial ways. In Typescript, for example, you can import types using runtime import syntax, making it ambiguous whether running side-effects from a library is intentional or not. This gets problematic once you consider treeshaking and not-really-standard things like conditional resolution via package.json's `browser` field.
It gets even more fun when you realize that module resolution isn't even specified, meaning that as far as a bundler is concerned, something like `import 'lodash'` means completely different things for a browser vs a project installed via npm vs one installed via yarn v2...
> It gets even more fun when you realize that module resolution isn't even specified, meaning that as far as a bundler is concerned, something like `import 'lodash'` means completely different things for a browser vs a project installed via npm vs one installed via yarn v2...
Ugh, why does it have to be different between npm and yarn v2?
I'm an occasional JavaScripter and yetserday wrote an automation with Puppeteer.
The first line I wrote was
import puppeteer from 'puppeteer'
only for Node to fail on that. I thought that syntaxt was supported, but I had to change it to
const puppeteer = require('puppeteer');
I've no idea if it's a problem with my Node version (12) or what, but it was a real "Huh?" moment.
FWIW, that's not an `npm` vs `yarn` thing. That's a "Node has only recently added support for ES module syntax and it's still mostly experimental" thing:
It's an odd thing. The way people talk, you'd think NodeJS and JavaScript are synonymous. Turns out, NodeJS is this generation's JScript, including shoddy support for the language proper, and it includes vendor-specific extensions that you're expected to use instead. It has somehow though managed to escape any of the criticism that gets thrown at other projects pilloried for stuff like this.
Node came out way before any of these features were invented, and the ES committee knowingly broke compatibility when designing for the browser. Modules are supported since v14 but you have to opt in.
They didn't break compatibility. NodeJS never attempted to be compatible with the Web. The design work that TC-39 and the browser makers did is also much better than what the NodeJS community puts out. And it's a design that was in ECMAScript 6. Six years later, and you still have to opt in. The TypeScript team got it right, working in a way that NodeJS should have been taken lessons from.
Yeah, technically better, at the cost of huge complexity.
CommonJS was easy to implement anywhere and dead simple. We only need static analyzers and tree-shaking optimizations because the JS ecosystem went wild on dependencies. I honestly think we were better off 10 years ago.
> NodeJS never attempted to be compatible with the Web
It goes the other way around? Node caved in and support modules now, what has been done in tc-39 for web modules to be compatible with node (considering it predates all of it)?
Modules weren't production-grade until Node version 14. Before that, it used CommonJS modules ("require" instead of "import").
It's because Node was invented before EcmaScript modules existed. CommonJS worked well and supporting both was awkward, so the transition took a while.
> that's almost 1000 lines per day every day since he started. Amazingly productive.
I know what you mean, but I also hate using lines of code per day to measure productiveness, and I hate it when managers in corporations look at number of lines added in PRs to measure how productive a dev is.
I don't know about anybody else but for me a good approximation is the time interval between the first line of code and the critical line of code. The better my solution the shorter it often is. In particular, 'shorter' in the sense that future functionality doesn't have to constantly acknowledge that code, creating a tax on new features that builds until the project dies.
I was going to say "start of coding" but some tasks are just so boring that I may procrastinate on starting it. That also indicates an element of difficulty (in giving a shit), but not what we're trying to measure here.
I discovered the Mikado Method quite on my own as a consequence of trying to tackle increasingly complex cases of 'large-scale' or 'top-down' refactoring before accepting that as the oxyest of morons. I was sold when I realized that when you finally discover the crux of the problem, the 'Rome' that all roads lead to, between half and 80% of the code you just wrote isn't strictly necessary. Throwing it all out wholesale is how you avoid code hoarding. Yes, once in a while you go out to the proverbial trash can to pull something back out that you actually did need, but the tabula rasa aspect is profoundly useful psychologically, especially for that 'tax' bit I mentioned above.
There is always some function in the middle of the call tree, that if you add or change the meaning of an argument then the feature or bug fix becomes a logical conclusion of that new semantic. Either the code above it or below it hardly needs to change, decoupling the feature from all but a handful of functions/concerns.
A bad programmer will happily plow through adding a new argument to fifteen method calls and then propagating that change to a hundred call sites. And that code will be buggy as hell because their coworkers have already written them off. If it's too much code for others to review, you can be sure there are bugs in it. And if you didn't listen the last ten times people told you to stop writing so goddamned much code, you aren't going to listen this time, either. So have at it, sport. We'll just make sure you get the blame for the bugs, until someone in management wakes up. If they don't, then they see that the sections of code people "won't touch" but you will are getting bigger, and mistake your ownership of this code for a sign of prowess instead of a sign that you are inspiring apathy in others.
Just to put the 280k lines into perspective, deletion rate seems close to addition rate[0]; measuring the number of changes made to code is also a poor proxy for productivity, but is at least not quite as bad as the pure "lines added" metric.
Not to be the one who is the one, but 1k lines per day sounds like the opposite of productive. If you still have to add/remove such large swaths of code, you're surely doing something weird.
Still a great achievement to produce a bundler and minifier as a one-person team, but not sure we should use the amount of code line changes as a measure for productivity.
Productivity might be the wrong word. Discipline, grit, perseverance?
I've written parsers/lexers/code generators by hand for much smaller toy languages with no thought about performance at all and those were huge undertakings for me. I was just trying to convey my impressedness with some numbers that we can relate to somewhat.
But yeah, he has a "net" loc contribution of 284-148 = 136k lines of code. Assuming the code base is resonable (which I don't doubt), it's a pretty big project which he has built, which I'm impressed by.
If you read the linked page, it was a very intentional decision:
> Everything in esbuild is written from scratch.
> There are a lot of performance benefits with writing everything yourself instead of using 3rd-party libraries. You can have performance in mind from the beginning, you can make sure everything uses consistent data structures to avoid expensive conversions, and you can make wide architectural changes whenever necessary. The drawback is of course that it's a lot of work.
> For example, many bundlers use the official TypeScript compiler as a parser. But it was built to serve the goals of the TypeScript compiler team and they do not have performance as a top priority. Their code makes pretty heavy use of megamorphic object shapes and unnecessary dynamic property accesses (both well-known JavaScript speed bumps). And the TypeScript parser appears to still run the type checker even when type checking is disabled. None of these are an issue with esbuild's custom TypeScript parser.
1k lines per day sounds pretty reasonable when writing your own TypeScript parser, among other components
> 1k lines per day sounds like the opposite of productive
Having built a compiler before, this seems normal, especially since the parser is written manually (as opposed to generated). Parsers are always a lot of lines of code.
Not all of them are the most complex, and in fact a lot will just be defining all the different nodes of your AST and branching on them, which, as blocks do, tend to inflate code a lot. I wouldn't expect a JavaScript parser to be any more compact though, or the development (especially in a codebase with few contributors) to be any less "thrashy".
I don't think that's even up for debate. As far as I'm concerned they have definitely met their goal.
I find it similar to the fact that, while I am thankful that the operator's manual for my toaster was written at a 4th grade reading level, I find it unlikely that anyone has that task in mind when they say they want to be a writer when they grow up.
I think Go is a genuinely fantastic experiment testing the idea that it's actually easier to write and modify long, boiler-platy, but painfully explicit code.
Yup, that's the one! He's also the author of http://madebyevan.com/. I remember seeing that WebGL water experiment many years ago and being blown away. That FSM Designer came in handy for some classes where I had to draw FSMs too.
SWC, another competitor project written in rust is also handled exclusively by a single person. Half a million lines of code and they have even built a type checker for typescript which is not included.
Evan Wallace (the author) is also the co-founder and CTO of Figma. Absolutely remarkable engineer. His co-founder Dylan is an investor in our company, and I can’t seem to get in touch with Evan at all because he’s always super heads down on something meaningful (such as esbuild). Mad respect for him.
How does he do Figma CTO-ing and at the same time building esbuild? That’s mad hours.
Also one of the things I love about Figma is their obsession on performance. It feels such a joy to use. Runs circles around any Adobe product. It’s written in a browser. Figma and vscode are my inspiration for browser based tools.
>How does he do Figma CTO-ing and at the same time building esbuild? That’s mad hours.
My guess is an alignment between purpose, fun, and skillset. I would say (not for the sake of bragging but for qualifying the legitimacy of my position) - I probably pull similar hours across CTO-ing, investing, and learning biochemistry. Then again, I'm commenting on HN and Evan isn't, so maybe not. :P
I switched from google closure to esbuild and it is amazing how fast and well it works. It even generates smaller code in my use case (200k lines app).
Also the creator is really nice. He implemented two features I requested right away, which surprised me a lot. What a great project!
I've seen other interactions in the issue tracker and the author has been nothing short of responsive and understanding. This is a project worth contributing money to.
Important distinctions when comparing Closure Compiler's performance to other tools:
1. Regarding bundle size: Optimization level used (SIMPLE optimizations yield output that is at least expected from a minifier these days)
2. Regarding compilation speed:
* Whether the JavaScript-only compiler was used over the JVM compiler: besides being slower, a number of optimizations are not available to the former.
* In many conditions a pre-heated JVM (or Nailgun), or using the NPM-distributed native compiler binary yields faster compilation.
* Which compiler flags have been tuned; e.g. such that the parsing of browser externs is bypassed during compilation for Node.js-targeted bundles.
I just serve my ESM modules directly from the filesystem. It works well, builds in literally 0 seconds.
You also don't need a transpiler like Babel if you drop support for IE11. Almost all ES6 features– such as classes, proxies, arrow functions, and async are supported by modern browsers and their older versions. As long as you avoid features like static, private, and ??=, you get instant compile times.
It's not 2015. You don't need a transpiler to write modern JavaScript. As of writing, the latest version of Safari is 14 and the latest version of Chrome is 88. These features fully work in Safari 13 and Chrome 70, which have less than 1% of marketshare.
I would say the primary motivators at this point have shifted from browser compatibility to syntaxes like TypeScript and JSX that will (likely) never be supported natively
> I would say the primary motivators at this point have shifted from browser compatibility to syntaxes like TypeScript and JSX that will (likely) never be supported natively
I suggest using a single-pass transpiler like Surcase[0] that doesn't translate to IE11-compatible syntax, and just loops through the string once, avoiding generating an AST– making it much faster– removing TypeScript annotations and desugaring JSX.
If you additionally need to support IE11, you can use a development build with Surcase and a production build with a bundler like Snowpack. C/C++ developers have been doing things like this for ages: compiling files as objects during development, and compiling them into one big binary for production.
Surcase is often slower than ESBuild since it is single-threaded. The benchmark posted on its README forces ESBuild to run single-threaded, and is thus a bit misleading.
> 1. How do you handle npm dependencies that don't expose ES modules?
See my comment above.
> 2. How do you handle the performance impact of your browser doing multiple sequantial requests because it doesn't know your whole dependecy tree?
Put all your dependencies in the HTML file, so it fetches them all in parallel. So instead of just loading the root application file and forcing the browser to resolve dependencies:
You can see a demonstration of this technique on my production site, a complex JavaScript SPA that loads everything (including dynamic content) in less than one second: http://vnav.io
For a small performance boost, you can load smaller files first via <link rel=preload>.
> Put all your dependencies in the HTML file, so it fetches them all in parallel.
> ...
> You can see a demonstration of this technique on my production site, a complex JavaScript SPA that loads everything (including dynamic content) in less than one second
But it doesn't work like this in practice! There is a limit to the number of concurrent connections a browser will make, Chrome for example is <=10 IIRC. You can see this in the waterfall:
`rendering.js`, the last `.js` in the queue, stalled for 300ms.
Additionally, each round trip is dependent on:
- The user's latency
- Your server's response time
So with each connection there is overhead. You would be much better served concatenating these files.
`util.js` took 510ms, of which 288ms was spent stalled (i.e. waiting for a connection) after which spent 220ms waiting (time-to-first-byte).
Furthermore, Lighthouse gives your page a performance score of 60, which isn't great. Key metrics:
- 1.7 seconds to paint (which is okay)
- 7.7 seconds to interactive (which is terrible)
Finally, why on earth are you redirecting to HTTP after serving an HTTPS response? This makes your page load even slower:
> But it doesn't work like this in practice! There is a limit to the number of concurrent connections a browser will make, Chrome for example is <=10 IIRC. You can see this in the waterfall:
In HTTP/2, which our site will switch to soon, the concurrent limit is 100. The world's changed, change with it, and throw away those bundlers.
Mixed Content: HTTPS sites cannot connect to HTTP resources. The game needs to dynamically scale without expensive load balancers in front. Until I get dynamic DNS and HTTPS working, the site will not have HTTPS.
So your solution to build systems has absolutely none of the benefits that build system do in fact bring, or the ecosystem of a language that you might consider terrible but actually has a decent amount of quality re-useable code. But this works for your very specific use-case?
Alas, many of us still need to support IE11. We alas get a lot of value out of TypeScript, which needs to be transpiled. <1 second with esbuild is pretty good!
In development, use an index.html that loads it directly from the filesystem, using a single-pass annotation remover such as https://github.com/alangpierce/sucrase A single-pass transpiler doesn't build an AST, instead just looping over everything once, which will surely make it faster than esbuild.
You can annotate your code with JSDoc comments and get the full value of TypeScript while just writing JavaScript without any transpilation step[1]. It won’t help you with IE11 support though.
That's a helpful feature but it's by no means a replacement for Typescript. You do not get the full value with just inline comments, in fact that usage is extremely limited.
I generally avoid third-party JavaScript modules for performance and maintainability reasons. When I do use them, I'll frequently vendorize them (put them in my source tree) and make ESM modules for them.
Right, but that is basically saying we should all go back to something similar to before npm/bower/etc.
I love using plain ESM and I prefer to pull in stuff that only relies on that. But when I need graphing on a single page I'm not going to try to vendor Vega and I'm not going to pull it in on all pages. When I need xls parsing I don't want to vendor xls.js and maintain the diff myself.
So I have to use something like snowpack to make it work with my ESM system and dynamic imports.
If you don't have to require heavy libraries for certain pages then that's great and I encourage to use pure ESM without build systems for that, but you should also recognize that not all use-cases are ready for it.
Currently I use snowpack to handle dependencies but still run all my own code unbundled and unminified.
> Right, but that is basically saying we should all go back to something similar to before npm/bower/etc.
That position isn't without good reason. Refer, for example, to the linked post, explaining the engineering decisions behind esbuild. ("It's Golang, not JS" is only part of the answer to "Why is esbuild fast?" The implications seem to be lost on many of esbuild's users, given the nature of esbuild itself and the job it's supposed to do.)
So you’re the one writing everything from scratch even though the wheel has already been invented and perfected and tested over years by many developers.
Also your website is serving dozens of cascading files instead of a single minified one.
Bundlers aren't the "wheels" here. The real "wheels" here are ESM modules, which have been available in every modern browser for a while. Bundlers are just pre-wheel, prehistoric, stopgap technology from the 2010s that we should be moving away from.
And there's no problem serving multiple files in 2020 because we have HTTP2 multiplexing now. In fact, it's probably more efficient than using bundling, because you can cache much better. Minification is also virtually unnecessary with Brotli, and not minifying has the added benefit of making the debugging experience much better.
And bundlers haven't been "perfected". Not even close. Webpack is without a doubt the worst piece of technology in my stack right now, together with Babel. Those two are terrible by themselves, but they also manage to "infect" other things elsewhere in my stack: for example, ESLint and Jest need Webpack/Babel plugins to work.
Basically everything you wrote is wrong. Loading multiple files is never more efficient because compression works on a file at a time in that case, plus subdependencies start loading only after their parents finish downloading. This is CSS’ @import cascades all over again.
The wheels I’m talking about are the actual modules that exist on npm used in production by millions daily. Everyone here can write some code that does something common like, say, left-pad, and then they screw it in some obvious way that was fixed in 2014 by the third user of that library.
Sure I agree that having no build is nice, but user performance is not comparable and you’re limited to your own code.
You seem to be still holding a lot of assumptions from 2010.
In practice compression is not a big issue if you use something with a pre-defined dictionary specialized for HTML/CSS/JS, which is Brotli. The advantages of bundling are not exactly too large. Sure you might not be deduplicating some identifiers, but the lion's share of code in practice is normal javascript/CSS/html.
Also, in the long term, caching and the removal of code added by bundling more than makes up for any losses from compression.
Also, with Multiplexing there's no problem with download multiple files sequentially, because you don't need TCP handshakes and HTTPS negotiation, which were the reason for bundling in 2010. In fact, multiplexing multiple files might be faster in some cases, because you're able to parallelise the downloads and the parsing, rather than having to download a big bundle before parsing it.
There are of course caveats to this, but neither bundling or separate assets are silver bullets.
If you're using straight ESM then you don't need any build tool a lot of the time. If I need any 3rd party module it's either include it in global using an ES5 library, or use https://www.skypack.dev/ which has a ESM ready built module of most npm packages that you can import directly in your code.
This works IF you write 100% vanilla javascript. In larger projects a vanilla javascript approach usually falls short. I cant see a large project being written without a types in 2021. So you still need a build step, no matter if you are using Typescript, flow or some other typed version.
> You also don't need a transpiler like Babel if you drop support for IE11
I really dislike this line of thought: Babel isn't just a polyfill tool. It's an incredible way to enhance your code. Further, the future of javascript changes isn't static and shouldn't be.
There's also a substantial writeup on the architecture of esbuild in the repo [1]. It is definitely worth a read if you are interested in some of the inner workings of esbuild or are curious how various bundler features are implemented. High-quality, in-depth architectural docs for complex projects like this are exceptionally rare.
Slightly tangential, but I really love the minimal but effective site design here. It is minimal, responsive and easy to read. Is this a theme or just the author being very effective and throwing together a documentation site?
It is also all custom: https://github.com/esbuild/esbuild.github.io. YAML files that get converted to plain static HTML pages. It's designed to work well without any JavaScript. Even the animated graphs on the home page are done without JavaScript.
I can confirm that esbuild is very fast - easily bundling my code a hundred times faster than webpack or rollup. And the minified esbuild bundles are only around 10% larger than what is produced by the JS-based bundlers. Considering it saves 30 seconds per build, this is more than an acceptable tradeoff.
It's good that this tool is shaking up the JS development ecosystem. It was getting stagnant (babel) and overly complicated (webpack). We can probably expect more tooling to migrate to golang and rust. I think the esbuild author accomplished his goal.
Esbuild is awesome! I tried it on my latest project and the speed has blown me away, even though it doesn't replace the TypeScript compiler for my usecase.
Its two biggest shortcomings are:
1. It can't compile const enums (TypeScript feature)
2. It can't compile down to ES5 (with some exceptions)
So I still need to run TypeScript + esbuild side-by-side. TypeScript compiles to ES5 and esbuild bundles the ES5 files.
Anyway, it is a massive improvement over TypeScript + webpack.
Curious to how you set this up without going crazy (and CI working)? Just a line in package.json that runs tsc, then another one that runs esbuild on the output?
We really need it to support top level await, and the author doesn't seem to be that interested in supporting it at the moment. Hopefully he will come around soon.
I'm the author of esbuild. Rest assured that I am very interested in supporting it and am actively working on it. It's just unfortunately very complicated to correctly, especially without affecting any of the other existing features. There are some recent updates at the bottom of this thread: https://github.com/evanw/esbuild/issues/253.
The use case is not needing to wrap your code in an IIFE [1] when you would have a top-level await statement. It is useful in entry point of apps where you might need anything that uses async. It's also useful for having async modules. [2]
They're also very helpful for debugging. Using await something in the console is way more convenient than something.then().
I mean the feature itself has been vetted, i.e. it wouldn’t be in the language if enough people didn’t find it reasonably useful. TLA is a substantial feature implementation-wise so I expect TC39 to have thought “yeah that’s still worth it”
TLA was accepted in the specification before the module loading scenarios were fully baked. Probably best to use async IIFEs until the TLA spec bugs are sorted out and the fixes are subsequently implemented in the major browsers and released. Maybe a year from now.
I wish the Angular team put some work into integrating. They are instead doubling down on webpack which is frustrating. After they back stepped regarding the bazel build system I thought that all was lost. Then this beauty came out. Unfortunately it's not as interesting as the incremental compiler.
It's still early days for esbuild. If you follow the commit history there are still some weird bugs. I'd give it a few months to get these things ironed out (along with wider usage) and it'll be good to go.
That would make my day. I have a 64GB RAM PC because I work on 2 or 3 angular apps at a time and since my PC is the fastest I also use it for CI builds on occasion. It's a nightmare to wait 20 minutes for an optimized prod build. And having to upgrade everyone's RAM so they can contribute to the front end is horrible. We are past 10GB of RAM required to start the dev server.
We actually just worked on splitting it up into libraries. The shared library might actually make the ending build larger due to it being potentially duplicated (I'm not certain about this until I test it). Unfortunately that has to go on hold temporarily. It is a large Healthcare application in the therapy space. The ending build is actually fine, but the tooling eats RAM like something else.
Little by little I take large third party libraries and rewrite them on my own to drop 5MB includes. I'm sure that is part of it. Devextreme and Syncfusion are libraries I would strongly recommend avoiding.
Incidentally, a new version of the Bazel rules_nodejs, which somewhat confusingly has both browser and no JavaScript support, includes an esbuild rule.
As for Angular CLI which relies heavily on web pack, it has years of grinding tweaking bug fixing work getting all the right behaviors for all the cases. It's going to take a lot to re-create that in a non-breaking way using something different under the hood.
Vite 2.0 (a build tool like Webpack or Rollup, and powered by Esbuild) was just released earlier today. It's seriously awesome.
I first learned of Esbuild early 2020 when I complained on Twitter that JS build tools are awfully slow [1]. Most my (mainly Vue) projects use Webpack and starting the dev server takes typically anywhere between 5-50 seconds.
With Vite, cold devserver start in a mid-sized project on my 2015 MBP is ~2-5 seconds, but vast majority of the starts happen almost instantly as the deps are cached. Hot module reloading and most changes is also very fast and feel instantaneous (where in Webpack projects they would take anywhere between 1-20 seconds). This has HUGE affect in developing experience as you can just keep writing and almost never need to wait for any changes. (The one exception being TailWind/PostCSS builds that take few seconds every time the config or main import changes.)
I absolutely love Esbuild and Vite. Been using them daily for few months now, and I'm currently in the process of converting all my old vue-cli based projects to Vite using a project template I made that has TypeScript, Tailwind, and e2e tests w/ GitLab and GitHub CI configured [2]. If you work with React or Vue and Webpack and haven't yet tested out Vite or any of the other Esbuild based build tools, definitely do check them out!
1. You can support IE11 without Babel (I got rid of it many years ago already): Typescript has transpilation, and you can add polyfills
2. As much as I like JSDoc-augmented vanilla JS, it doesn't catch nearly as much as real TS
3. Webapps are made of more than just scripts, loaders are even the big reason why Webpack got popular initially. Luckily, Webpack 5 can automatically recognize assets referenced in "new URL()" calls, so you don't need types for images or shaders anymore.
Esbuild is certainly fast but I could never figure out how to get it to preserve some symbols during minification (for when I want to expose some class methods or properties publicly to external consumers)
I've really wanted to use Esbuild in my projects but I haven't figured out a great way to get the desired functionality I want:
I'd like to be able to transpile my code but keep the same directory structure in the output. For example, I have an index.js file that exports all my components and I'd like to just create a "build" folder output with the same exact file structure, but with all the files exported from index.js having been transpiled.
Hey, I was trying to do this same thing for a TypeScript project I was bootstrapping. Digging around the issue tracker, I recall the maintainer stating that this is currently not in the scope of esbuild, and that swc[0] might be better suited for this.
Personally, I wrote a crude JS script using the esbuild transform API + node-watch to compile .ts files and copy files/folders during development. This is all very recent, so I can't ensure that it's bulletproof quite yet.
Esbuild does have a JS transform API, so you could write a command line tool that walks the file system (Babel, ESLint, Prettier, etc. all do this so you have examples), transforms each file, and writes the code back out. Snowpack's build command also does this.
I recently switched to using esbuild [0] and certainly is fast for minifying js and even css!
For my site I basically only do minifying for js, and bundling+minifying for css, and it all gets done in 0.2 seconds. Right now the longest time in my deploy process is actually just doing a git fetch (with depth=1).
Go has shared memory between threads while JavaScript has to serialize data between threads. Both Go and JavaScript have parallel garbage collectors, but Go's heap is shared between all threads while JavaScript has a separate heap per JavaScript thread.
Untrue when using sharedArrayBuffer
For the uneducated like me: if I am writing a NodeJS application in Typescript, can I use this instead of the `tsc` Typescript compiler CLI and get faster compile times?
can esbuild used to do quick development builds with some dev server, like serve node pkg? or just for prod builds? if so, how the pipeline looks like?
I like the idea of having type checking moved out of the watch path, or at least be in a second window watching, so building is faster. But you get your type checking errors too.
You also have the editor finding issues for you, which is where I want that caught.
Yes this seems like an excellent performance boost to development. Parallelize type checking and compilation. We all have these many-core machines now, and so many tools that single thread the world unnecessarily.
Nope. This is wrong. It clearly mentions that its only a Javascript and CSS bundler and nothing much. It doesnt try to solve every problem in the JS world.
None of the other bundlers do type checking by themselves. They are always through plugins.
It's annoying to me how much time I had to spend outside of school before I realized that this is the real recipe for fast code. In particular, even code that is nominally O(n^2) by standard measures is still a dis-economy of scale: the bigger the problem the slower it gets. In school, there's a premium given for finding a polynomial-time solution, but in practice every job where I've been tasked with making something fast, only linear time complexity was good enough.
In practice these linear algorithms tend to also be completely data parallel and have great cache locality. Not only do these properties make the code fast the second you harness them, but then tend to be exceedingly simple architectures to maintain over time.
For some other reading I really like on software performance in the real world:
- https://blog.nelhage.com/post/why-sorbet-is-fast/
- https://blog.nelhage.com/post/reflections-on-performance/