I'd like to see something like this used for big JavaScript bundles.
Many webapps now have 2MB+ of JavaScript, and lots of web companies do daily or even hourly software releases, which effectively means every time I visit your site I have to download many megabytes of JavaScript which takes many seconds. It's frustrating. Diffing the JavaScript and sending just the changed lines of code would save hundreds of millions of people a few seconds each day, and is totally worth it.
All of this has already happened, and all of this will happen again.
- The original rsync author wrote rproxy (https://rproxy.samba.org/), an ingenious system wrapping http inside the rsync protocol. On subsequent visits, the proxies see the resemblance and sends a diff. I've toyed with it myself (https://github.com/rakoo/rproxy/) but I don't think website really care about how disastrously huge and cumbersome they are to use
- another idea is to invert rsync: instead of the client calculating a signature, sending it to the server, and the server calculating the delta for every request of every client, it is possible to have the server calculate the signature once every time actually changes, send the signature to the client, let the client calculate the parts they need and download only them. In short, zsync (http://zsync.moria.org.uk/)
There are a million things already done in this sector, but for some reason it has never caugh up... I think there's just not enough interest
I really like this idea, but I'm not sure if it would work in practice. It's basically working against webpack's code splitting because those are tagged with hashes. Additionally, the first load will still be quite heavy, whereas with code splitting at least there's some semblance of a smaller initial load. Since it's not native to the browser, it would have a nontrivial initialization overhead as well.
I think it could be a cool idea if it's offered as a static file hosting as a service kind of deal that requires zero config, but then again if you're the kind of person that wants a zero config static file hosting, you're probably not pushing daily/hourly software releases?
It would be neat to have something like this more transparently implemented though. Seeing a 2MB+ module rebuild for a minor typo fix is aggravating.
* For each bundle, webpack produces the compiled bundle and a seperate diff file between the latest release and every previously published release. Filenames could be [oldhash]-[newhash].js
* The webpack loader can check if the browser has that specific module cached (I believe service workers are able to inspect the http cache like this), and then based on the version that is cached, request the correct filename for a diff to the latest version.
* Then apply the diff and load the javascript as normal.
I just did a quick test with a webapp of mine, and a small diff file made with the rsync algorithm (via rdiff-backup) of a webpack bundle was just 150 bytes.
> Then apply the diff and load the javascript as normal.
Can service workers mutate the HTTP cache? I haven't seen it yet, but if not, it seems like the client might be forever pinned to the old version. I guess most likely we'd have to avoid the HTTP cache all together and run our own application module caching. That could work, but wonder if the performance overhead would be worth it...
I'll play around this a little more though, it's a really cool idea. Getting a clean API seems tricky but doable.
Many webapps now have 2MB+ of JavaScript, and lots of web companies do daily or even hourly software releases, which effectively means every time I visit your site I have to download many megabytes of JavaScript which takes many seconds. It's frustrating. Diffing the JavaScript and sending just the changed lines of code would save hundreds of millions of people a few seconds each day, and is totally worth it.