The goal was to show the high level change, in a glanceable way, not to get in to individual millisecond comparisons. However, in the future I would pick a different visualization I think :)

The benchmarking has also come under fire. My goal was to just to put the same site/assets on three different continents and retrieve them a bunch of times. No more, no less. I think the results are still interesting, personally. Clean room benchmarks are cool, but so are real world tests, imo.

Finally, there was no agenda with this post to push HTTP/3 over HTTP/2. I was actually skeptical that HTTP/3 made any kind of difference based on my experience with 1.1 to 2. I expected to write a post about "HTTP/3 is not any better than HTTP/2" and was frankly surprised that it was so much faster in my tests.

I think the box plots were a good choice here. I quickly understood what I was looking at, which is a high compliment for any visualization. When it's done right it seems easy and obvious.

But the y-axis really needs to start at 0. It's the only way the reader will perceive the correct relative difference between the various measurements.

As an extreme example, if I have measurements [A: 100, B: 101, C: 105], and then scale the axes to "fit around" the data (maybe from 100 to 106 on thy y axis), it will seem like C is 5x larger than B. In reality, it's only 1.05x larger.

Leave the whitespace at the bottom of the graph if the relative size of the measurements matters (it usually does).

Every day, the stock market either goes from the bottom of the graph to the top, or from the top all the way to the bottom. Sometimes it takes a wild excursion covering the whole graph and then retreats a bit toward the middle. Every day. Because the media likes graphs that dramatize even a 0.1 percent change.

It will be visible if you work in Celsius, a unit that is essentially a cut-off Y axis to better fit the origin within the domains we use it for.

We have an intuitive sense of what 30 degrees is, assuming it is in our preferred system of measurement.

A stock market graph really should be showing the percentage change, not some small absolute change that it’s not immediately understood by the typical layperson.

Meanwhile, there are plenty of practitioners who aren’t obviously to the argument, but rather long past it: they know there are situations where it’s totally legitimate to cut the axis. Other times, they might resort to a logarithmic axis, which is yet another method of making the presentation more sensitive to small changes.

In this case, when we're measuring the latency of requests, without any other context, it's safe to say that relative differences are the important metric and the graph should start at zero.

So while it's true that this isn't universally the correct decision, and it's probably true that people regurgitate the "start at zero" criticism regardless of whether it's appropriate, it does apply to this case.

> if I have measurements [A: 100, B: 101, C: 105], and then scale the axes to "fit around" the data (maybe from 100 to 106 on thy y axis), it will seem like C is 5x larger than B. In reality, it's only 1.05x larger.

If you were interested in the absolute difference between the values then starting your axis at 0 is going to make it hard to read.

[A: 27.0, B: 29.0, C: 28.0]

versus:

[A: 27.0, B: 27.2, C: 26.9]

If scale is fit to the min and max values, the charts will look the same.

Still, as a rule of thumb, when Y axis doesn't start at 0, the chart is probably misleading. It is very rare that the absolute size of the measured quantity doesn't matter.

edit: like this: https://imgur.com/a/7Gvq59j

Personal preference: for large offsets that makes sense. For small ones (~10% of max here) it seems unnecessary, or, to a suspicious mind, meant to hide something ;)

And the numbers are too small to read, and reading numbers on a graph is mixing System 1 and System 2 thinking, anyway.

I agree that the graphs would be better and still impressive even anchored to 0

In fact, you can probably figure it out by yourself just by looking at what goes across the wire.

And yet, it runs the whole modern world. That’s beautiful. I think simplicity is underrated and it’s something I really value when choosing the tools I use.

https://portswigger.net/research/http-desync-attacks-request...

The best way to exploit an HTTP/2 server is to exploit the HTTP/1.1 server behind it [1].

[1]: https://portswigger.net/research/http2

The exploit mentioned is the http/2 front end getting confused by the amount of responses it gets because it never checked how many http/1.1 messages it was forwarding, it just assumed that the http/1.1 headers where matching the http/2 headers. Now in defense of http/2 this was completely expected, called out in the spec. as requiring validation in case of tunneling and ignored by every implementation.

Quoting your link it describes this politely, "By itself, this is harmless. However, modern websites are composed of chains of systems, all talking over HTTP. This multi-tiered architecture takes HTTP requests from multiple different users and routes them over a single TCP/TLS connection [...]"

Lax whitespace rules, lax repeating header rules, no advanced integrity checks like checksums, etc. I see these as signs of simplicity.

The following is an excerpt from Performance of Checksums and CRCs over Real Data[0].

The TCP checksum is a 16-bit ones-complement sum of the data. This sum will catch any burst error of 15bits or less[8], and all 16-bit burst errors except for those which replace one 1’s complement zero with another (i.e., 16 adjacent 1 bits replaced by 16 zero bits, or vice-versa). Over uniformly distributed data, it is expected to detect other types of errors at a rate proportional to 1 in 2^16.

If you're deeply interested in this topic then I would recommend Jonathan Stone's Phd thesis.

[0] https://www.researchgate.net/publication/3334567_Performance...

If you have a shared secret to support doing an hmac then you can certainly set up tls correctly.

TLS does not obviate application layer checks it can only compliment them.

You have to ask yourself "How would this get corrupted"? If the concern is that the data ends up somehow mangled from an external source like a bad router or a lose cable, then the data simply won't decrypt correctly and you'll end up with garbage.

So that leaves you with an application bug. However, an app is just as likely to add a checksum for an invalid body as they are to add one for a valid body.

So, the only fault I could see this fixing is an application which, through corrupt memory or something else, manages to produce a garbage body which fails it's checksum but... somehow... is encrypted correctly. I've never seen that sort of problem.

Not sure what that means precisely, but if you're stretching definitions that far, the same is probably also true of your application checksums/hmacs

> Additionally the TLS connection may not even be between the server and client but a proxy between the server and client, client and server, or both server and client.

So its the same as your hmac check. That can also be processed at any layer.

> While TCP and TLS can get a message over the Internet correctly and securely they can't make any guarantees about the contextual validity of that message.

Can anything at the http protocol layer provide that?

Now, you can also build a lot of complexity upon it. And while the simplicity is nice for inspecting and debugging sometimes, in practice it is rarely used, and it's a very complex beast with TLS, SSL, gzip compression and more.

The arguments are similar to binary logs vs text logs. Except that in HTTP's case, it is already littered with binary.

We don't complain that we can't read our browser's binary but for some reason everyone is convinced that our transport protocols are different and it is imperative to be able to read it with a tool to translate it.

The fact that HTTP 1.1 speeds are even comparable, let alone faster in some situations, should at least tip the scale in its favor.

How much do you understand about gzip or brotili? How much do you know about TLS? Do you understand all of the functionality of TCP? Network congestion and checksuming?

QUIC includes compression, encryption, and transport into the standard which is why it's more complicated. Just because Http 1.1 doesn't include those parts explicitly into the standard, doesn't mean they are unused.

Http 1.1 is only "easy" because it sits atop a sea of complexity that rarely faces the same critiques as Quic does.

Do you understand how to build TCP packets by hand? A lot of the confusion between QUIC and HTTP/3 is intentional because it is somewhat merging the layers that used to be entirely separate.

All the examples elsewhere of "I can telnet/netcat to an HTTP/1.1 server and mostly do the thing" avoid the obvious fact that telnet/netcat is still doing all the work of splitting the messages into TCP packets and following TCP rules.

Presumably QUIC and HTTP/3 are separate enough to still warrant the idea of a telnet/netcat for QUIC taking care of the low level details and then you could try to write HTTP/3 on top of those apps a little easier, in a similar fashion. Though maybe not, without the rich history of telnet-related protocols on top of TCP and HTTP/3 still currently the only protocol targeting QUIC today.

Are these realistic web browsing scenarios? Because that seems false. For most websites http/2 is a pretty significant performance win.

HTTP/3 is actually not super complex. The main trickiness comes from header compression. But unlike HTTP/2 the stateful QPACK compression part in HTTP/3 can actually be deactivated, and just the stateless static dictionary and huffmann compression being used.

Perhaps I'm old fashioned, but I like being able to debug things with telnet/netcat. (Or openssl s_client.)

This is an underappreciated feature of the old protocols. You can telnet into an HTTP, SMTP, POP, or even IMAP server and talk to it with your keyboard (using stunnel for the encrypted versions), or dump the traffic to learn the protocol or debug a problem. Good luck doing that with QUIC.

If you're talking about HTTP 1.1 and not thinking about the complexity of TLS, you're ignoring more than half the LoC in the stack.

I like QUIC because I was never going to roll my own TLS anyway, so I might as well have TLS and streams in the same package.

The post clearly explains that the big advantage of HTTP/3 is that it deals much better with IP packet loss. But then the tests are done without inducing (or at least measuring) packet loss?

I guess the measured performance improvements here are mostly for the zero round-trip stuff then. But unless you understand how to analyze the security vs performance trade-off (I for one don't), that probably shouldn't be enabled.

For desktops HTTP2 is mostly ok, possibly an improvement. For Mobile it wasn't. I raised this when we were trailing in at $financial_media_company. Alas the problems were ignored because HTTP2 was new and shiny, and fastly at the time was pushing it. I remember being told by a number of engineers that I wasn't qualified to make assertions about latency, TCP and multiplexing, which was fun.

I still am not convinced by QUIC. I really think that we should have gone for a file exchange protocol, with separate control, data and metadata channels. Rather than this complicated mush of half remembered HTTP snippets transmuted into binary.

We know that despite best efforts website size is going to grow bigger, in both file size and number. Lets just embrace that and design HTTP to be a low latency file transfer protocol, with extra channels for real time general purpose comms.

QUIC itself doesn't have the request/response style of HTTP, it doesn't know anything about HTTP, it's just datagrams and streams inside the tunnel.

So you could use QUIC to build a competitor to HTTP/3, a custom protocol with bi-directional control, data, and metadata streams.

In fact, I'm looking forward to when Someone Else writes an SFTP / FTP replacement in QUIC. HTTP is already a better file transfer protocol than FTP. (because HTTP has byte-range headers, which AIUI are not well-supported by FTP servers) Think how much we could do if multiple streams and encryption were as simple as importing one library.

Yup, my mistake, I meant to say HTTP3 over QUIC.

At a previous company(many years ago), I designed a protocol that was a replacement for aspera. The idea being that it could allow high speed transfer over long distance, with high packet loss (think 130-150ms ping). We could max out a 1gig link without much effort, even with >0.5% packet loss.

In its present form its optimised for throughput rather than latency. However its perfectly possible to tune it on fly to optimise for latency.

>TLS 1.2 was used for HTTP/1.1 and HTTP/2

>TLS 1.3 was used for HTTP/3.

So the fairer comparison might be TLS 1.3 under all three, but if you need to upgrade why not upgrade the whole HTTP stack?

Because it's a benchmark of HTTP3 and not a comparison of "as it might be stacks".

It would a bit like bench-marking HTTP/1 with RHEL7 and Apache2, HTTP/2 with RHEL8 and NGINX and HTTP/3 with...let's say Alpine and Caddy...it's just not a clean benchmark if you mix more then one component and try to proof that this single one component is faster.

HTTP/3 could perhaps be described as HTTP/2 over QUIC. It's still a very different protocol from HTTP/1.1, even if you were to ignore the transport being used - the way connections are managed is entirely different.

HTTP has a bunch of semantics independent of how it's spelled and HTTP/3 preserves those with a new spelling and better performance

Upgrading just TLS to 1.3 for most people likely just means upgrading to a newer openssl which you probably want to do anyway. In many web server deployment scenarios, deploying HTTP/3 is highly likely to be more involved. The Apache httpd doesn't support H3 at all, I don't know if nginx has it enabled by default these days?

NGINX's QUIC implementation seems to also lack support for QUIC and or HTTP/3 features (such as Adaptive Reordering Thresholds and marking large frames instead of closing the connection).

[1] : https://hg.nginx.org/nginx-quic

[2] : https://github.com/cloudflare/quiche

[3] : https://github.com/ZestProjects/Zestginx

EDIT: A friend of mine who works at VK ("the Russian Facebook") informed me that they're helping out with the NGINX QUIC implementation which is nice to hear, as having a company backing such work does solidify the route a little.

Waiting for Superhub 5 to be officially rolled out before upgrading here!

Yeah, I remember Google engineers steamrolled HTTP/2 through W3C with equally flawed "real world data."

In the end it came out that HTTP/2 is terrible in real world, especially on lossy wireless links, but it made CDNs happy, because it offloads them more than the client terminals.

Now Google engineers again want to steamroll a new standard with their "real world data." It's easy to imagine what people think of that.

You can join in if you want - the IETF is an open organisation. There’s no magical authority. Standards are just written by whoever shows up and convinces other people to listen. And then they’re implemented by any person or organisation who thinks they’re good enough to implement. That’s all.

If you think you have better judgement than the working groups, don’t whinge on hacker news. Turn up and contribute. We need good judgement and good engineering to make the internet keep working well. Contributing to standards is a great way to help out.

Now, nobody of httpbis raised a red flag, and challenged performance figures of HTTP/2 before it became a standard.

Ok, I will take that grumbling on internet forums is a non-solution. How would you suggest contributing to HTTPbis process without flying engineers around the world all year long to attend IETF meetings?

On the matter of QUIC — my biggest discontent with it is that these guys basically recreated SCTP (and botched it at that,) but did it in UDP, without taking advantage of most exiting OS level, and hardware level performance optimisation. There is no chance at all hardware makers will put any effort to support offloading somebody's proprietary weekend project into hardware, and without that it has no chance at adoption, and everybody will be stuck at HTTP/2 now because CDNs are very happy with it, and browsers can't roll back its support.

HTTP/4 is needed now, it needs to be built over SCTP to have any chance at getting hardware offloading.

As for flying people around the world, most of the actual work of the IETF happens on the mailing lists and (in the case of httpbis) on the http GitHub issue tracker. You can attend the meetings virtually, and they go to great length to include virtual attendees - though it’s never quite the same as talking to people in person over drinks or in the corridors. If you think the http working group isn’t taking performance metrics seriously enough, it sounds like you have something really important to contribute to the standards group. That voice and perspective is important.

I agree with you about SCTP being a missed opportunity - though I suspect quic will get plenty of adoption anyway. And I’m sure there’s a reason for not using sctp - I think I asked Roberto Peon a couple of years ago at IETF but I can’t remember what he said. He’s certainly aware of sctp. (For those who don’t know, he’s one of the original authors of quic/spdy from when he was at Google.)

I agree that quic will probably never hit 100% of global web traffic, but I wouldn’t be surprised if it surpassed 50% within a decade or so. And there’s some positives from that - it’s nice to put pressure on internet vendors to allow opaque udp packets to float around the net. Hardware offload aside, that increases the opportunity for more sctp-like protocols on top of udp in the future. It’s just a shame any such attempts will need to layer on top of udp.

Not having it, means CDNs must have 4-5 times more CPU power, on top of natural internet traffic growth. Saying "buy 5 times more servers" will not fly

HTTP/2 is such a hit with CDNs exactly because it let them do more traffic with less servers, though with worse end user experience unless for kind of people who get gigabit at home.

Though I guess I can compare the confidence intervals visually :-P

Tbf, I think for this blog the narrower range does help the first chart as you can 1) easily compare bounds 2) on the full scale they would be nearly at the same spot.

Lots of room for improvement next time I think.

Accessing a page where all useful information I am interested in is text, should not require a new protocol developed by genius engineers to work without delay. If I want to read an article that is 50kB of text, it's not unreasonable to expect that information to be here before my finger leaves the ENTER key, regardless of how its transmitted.

Why isn't that the case?

Because said article is not transmitted with a dollop of html to structure it, and a sprinkle of CSS and JS to make it look nice. It's delivered to me buried in a mountain of extraneous garbage, pulled in from god-knows-where, mostly to spy on me or trying to sell me crap I don't need.

I am not saying "don't invent new protocols". But maybe think about why it was perfectly possible to have functional, fast, and reliable webpages and applications in the 90s and early 00s, despite the fact that our networks and computers were little more than painted bricks and paper-mache by todays standards.

https://idlewords.com/talks/website_obesity.htm

If we don't think about this, then neither QUIC, nor QUIC2 or REALLY_QUIC will save us from being wading through a pool of molasses slow crap. Because inevitably, following each techical improvement that could makes our stack faster, is an even bigger pile of bloat that drags it down again.

[1] Although satire is dead, killed by people actually espousing extreme views on the internet, I still indulge but am forced to explicitly tell the reader: this is ridiculous satire. Of course we should solve the root of the problem and not invent/adopt technology that enables our bad habits.

No, it's not, multi-dimensional optimization and improvement-in-depth is good, actually.

Look back at the small site hosted in Bangalore - That's an Indian version of me. A hacker whose projects can't run on self-hosted Wordpress, who only bought a VPS because their home ISP is not reliable.

With HTTP/1, most of America can't load their site, it times out. With HTTP/2, it takes 2,500 ms. With HTTP/3, it takes 1,000.

A _petty_ software upgrade allows this imagined Indian blogger to gain an audience on _another continent_ without buying more servers, without using a CDN, and without taking advertising deals to afford more stuff.

You know 3 things I hate about the web? Advertisements, CDNs, and having to buy servers.

I promise this is purely geographical, not political - How often do you connect to servers outside the USA and Europe? I mean trading packets with a computer. YouTube uploads don't count because they have a CDN. For me, the answer is "almost never".

The Internet is supposed to be global, but computers are made of matter and occupy space, so connections to nearer servers are still better, and they always will be. But QUIC makes the far-away connections a little less bad. That is a good thing.

That shouldn't happen. Do you have a real site in mind there?

> No, it's not, multi-dimensional optimization and improvement-in-depth is good, actually.

That can be true at the same time as "it's sad that we need this"

Other protocols may work much better in such conditions, for example popular messengers can send and receive text, metadata and blurry image previews just fine. In some cases even voice calls are possible, but not a single website would load ever.

I think HN crowd won't notice the changes. I hope that the protocol would improve experience for smartphone users without reliable 4G connection.

Your messenger isn't sending tens of megabytes for every message.

> metadata and blurry image previews just fine.

One might wonder why they don't just send the 4 MB JPEGs instead of those down scaled to hell previews if they work so well.

> . In some cases even voice calls are possible

Not only kilobytes of data, but kilobytes of data where transmission errors can be completely ignored. I have been in enough VoIP calls to notice how "well" that works in bad conditions.

Everything points to putting websites on a diet being the correct solution.

> And if the connection drops, then it almost certainly won't recover and full page reload is necessary.

I would consider that a browser bug. No idea why the connection wont just timeout and retry by itself.

It would help a lot but it's not a full solution. You also need to stop TCP from assuming that every lost packet is because of congestion, or it will take a slow connection and then underload it by a huge factor.

I mean no, but it can be unreasonable to expect that physical limits get beaten if you have in mind that some people just have a few hundred to a thousand KM between their computer and the server they access.

E.g., in my hometown I may get latencies of 200 ms and more to a few sites, especially simpler ones that have no CDN but are a single server on the other end of the world. Don't get me started if I'm travelling between Vienna and South Tyrol using the train, in some parts (cough Germany) the internet is really spotty and latency spikes up to 10 to 20s are (sadly) rather normal for a few minutes here and there.

Now, with HTTP 1.1 over TCP and TLS involved the setup time already gets me to almost a second of wait time (more than the few tens of ms my finger needs to leave the Enter key) in the former setup and in the latter I may need 30 to 60s, or it even just times out when travelling by train and being in a bad spot, connectivity wise.

QUIC improves there, TLS handshake starts immediately and UDP setup needs less round-times (none) compared to TCP (even with TCP fast-open).

So simple websites can profit too from QUIC, initial load time can get reduced a lot, browsing them is finally doable also on remote, spotty connections. Also, I happen to develop applications that are delivered as web app, they just tend to acquire a certain complexity even if one tries to stay simple, so loading that faster even if nothing is already cached is a welcome thing to me.

Bloated page still will be slow, sure faster than with HTTP 1.1 but still slow, and I definitively would like to see that getting improved, but that's not really related to the issues that QUIC improves on, as simple websites win too when using it; it's just less noticeable there if you already have a somewhat OK connection.

In summary: Why not invent a new protocol if you can significantly reduce overhead for everyone, especially if it can coexist with the simple and established one.

Pretty sure I said nowhere that we shouldn't invent new and better protocols. QUIte the contrary (pardon the pun).

What I am saying is: We should not need to rely on new protocols to make up for the fact that we send more and more garbage, we should send less garbage.

If we can get less garbage, and new and improved protocols, all the better!

But if all we do is invent better protocols, the result will be that what buries the internet in garbage now, will use that better protocol to bury us in even more garbage.

You're saying it's good to make sites smaller, and I agree. You're saying QUIC is good, and I agree.

What do you want? To just complain that bad sites exist? Is this "what-aboutism" for programmers?

I'm sorry, usrbinbash. I'm sorry that bad things happen, and I wish I could make you happy.

How about for people to recognize that this performance gift is easily squandered, and talk about how we're going to prevent bloat so that we can preserve the good performance.

HTTP over QUIC will probably do a similar number of round-trips compared to HTTP over TCP, but it will do much fewer than HTTP over TLS over TCP. There's no getting away from SYN/SYN-ACK/ACK for a reliable protocol, but QUIC can put certificate negotiation information directly here, instead of the TLSoTCP approach of SYN/SYN-ACK/ACK/ClientHello/ServerHello/ClientKeyExchange/ServerKeyExchange.

Additionally, QUIC supports multiple streams over a single physical connection, and correctly implements packet ordering constraints and retries for them. TCP supports a single stream over a connection: any delayed packet will delay the entire stream. In QUIC, a delayed packet will only delay packets from the same logical stream, packets from other streams can still be received successfully on the same connection.

This feature is heavily used by HTTP/3: HTTP/2 introduced a concept of HTTP streams, but all HTTP streams were run over the same TCP connection, so over a single TCP stream: a slow packet on HTTP/2 stream 1 will delay all packets from HTTP/2 streams 2, 3 etc. With QUIC, an HTTP/3 stream is a QUIC stream, so a single slow request will not blkc other packets from other requests from being received.

so we are using a slightly older version of QUIC than the RFC and using go-quic on server and client. and this is what i see when creating a connection:

    Client: Initial (1284 bytes) includes client hello
    Server: Retry (166 bytes) includes retry token
    Client: Initial (1284 bytes) including retry token and client hello
    Server: Server Hello/Encrypted Extensions/Cert 
    Request/Certificate/Certificate Verify/Finished (1284 bytes) 
    Client: Certificate/Certificate Verify/Finished (1284 bytes)

https://datatracker.ietf.org/doc/html/rfc9000#section-8.1

probably what go-quic does is not optimal because you don't always have to validate the address.

   Prior to validating the client address, servers MUST NOT send more
   than three times as many bytes as the number of bytes they have
   received.  This limits the magnitude of any amplification attack that
   can be mounted using spoofed source addresses.  For the purposes of
   avoiding amplification prior to address validation, servers MUST
   count all of the payload bytes received in datagrams that are
   uniquely attributed to a single connection.  This includes datagrams
   that contain packets that are successfully processed and datagrams
   that contain packets that are all discarded.

    A server might wish to validate the client address before starting
   the cryptographic handshake.  QUIC uses a token in the Initial packet
   to provide address validation prior to completing the handshake.
   This token is delivered to the client during connection establishment
   with a Retry packet (see Section 8.1.2) or in a previous connection
   using the NEW_TOKEN frame (see Section 8.1.3).

The infrastructure inertia part isn't even so much a question of technical infeasibility, but greed. So much spending was slotted to carriers to improve their networks, but instead of investing in capacity and protocol upgrades, it all went to lobbying/exec bonuses.

If SCTP really ran on UDP. I'd have no reason to be salty, because we'd already be using it.

> I'd have no reason to be salty, because we'd already be using it.

Direct OS support is a big deal, and UDP gets messed with too. If someone made SCTP-over-UDP the default mode, while changing nothing else, I don't think it would affect adoption at all.

I think you have an overly rosy view of the time. Yes, some things were fast, especially if you had an unusually good connection, but the average person's experience of the internet was much slower and less reliable.

This site, which we're on, delivers pages of comments (this very comment thread is around 50K of comment text) almost instantly.

The Washington Post homepage loads in 600ms for me. 20K of text. Images load within about another 500ms. Their lead article right now is one of those visual feature ones with animations that trigger on scroll, so after the text first appears after 600ms, it takes a second or so longer for the layout to finish. But a routine text article page, I have a scrollable, readable text view within 500ms.

CNN.com, a fraction slower, for a little less text, but a lot more pictures. When I click into one of their articles, within a few seconds, it starts streaming me 1080p video of their news coverage of the article. Imagine that on your 'fast functional 90s and early 00s' web.

Or let's pick a technical resource. go.dev's landing page? No noticeable delay in loading for me, and that page has an embedded form for trying out Go code.

Or reactjs.org, say? Loads up in 200ms for me, and then subsequent navigations on site among documentation pages take about 60ms.

How about a government resource? irs.gov? The homepage maybe loads a little slower than I'd like, but FAQ pages are pretty responsive for me, with text appearing almost as soon as I've clicked the link.

I'm not cherrypicking, these were the first few websites that came to mind to test to see if things are really as bad as you're portraying. And my impression is... you know? It's not that bad?

I am not arguing that there aren't bad sites out there, but we need to stop pretending that the entire world has gone to hell in a handcart and the kids building websites today don't care about optimization. Substantive websites deliver substantive content efficiently and effectively, with a level of functionality and visual flair that the 90s/00s web could only have dreamed of.

Every website which loads way more content than its use case justifies. If the high quality of my device and ubiquitous broadband is the only reason that something appears to be loading fast, that's not good.

>Or let's pick a technical resource. go.dev's landing page? No noticeable delay in loading for me

Among other things, that's because this is a website loading what it has to, instead of everything and the kitchen sink. 885kB in total, and most of that are the images of "companies using go".

But I just picked a few obvious high profile information-oriented sites and none of them had that problem. So… which websites are the bad ones?

Are you looking at the first site that comes up when you search for ‘chocolate chip cookie recipes’ and holding that up as an example that the web is a disaster area?

That’s like picking up a gossip magazine from the rack in the supermarket checkout line and complaining that American literature has really gone to the dogs.

Do you get substantively different experiences on those sites if you don't?

> The tech lead for Google's AMP project was nice enough to engage us on Twitter. He acknowledged the bloat, but explained that Google was "resource constrained" and had had to outsource this project

> This admission moved me deeply, because I had no idea Google was in a tight spot. So I spent a couple of hours of my own time making a static version of the AMP website. . .

> By cutting out cruft, I was able to get the page weight down to half a megabyte in one afternoon of work. This is eight times smaller than the original page.

> I offered my changes to Google free of charge, but they are evidently too resource constrained to even find the time to copy it over.

The bloat and slowness is generally due to incompetence. On one hand it’s merchandising stalking users with poorly written spyware and on the other hand the technology is dictated by the lowest common denominator of developers who cannot perform without monumental hand holding.

Does the end user really want or prefer the megs of framework abstraction? No, that’s immature trash from developers insecure about their jobs. This is the standard of practice and it isn’t going away. In hiring it’s seen as a preference as increased tool proliferation can qualify higher wages.

The only way that’s going to get better is by moving to an alternate platform with more challenging technical concerns than populating content.

With IPv6 and gigabit internet to the house becoming more common in the US there is less and less reason to require web servers, data centers, and other third party concerns. These are incredibly expensive and so long as the platform becomes progressively more hostile to their users emerging alternatives with superior on-demand capabilities will become more appealing.

That way it should be possible to turn off a lot of that garbage that is generic JS. Of course, then the problems becomes website built to not work if you don't have that stuff. I suppose the solution is a protocol where it's not possible to know, i.e any server-side state is disallowed so that you cannot track id users have seen your ads OR the such a response cannot be proven.

What utter arrogance. Developers are more often than not driven by time constraints and demands from higher up the food chain. What you call "bloat" is often not just the easier solution, but the only one.

Of course, "bloat" can come from neglecting the non-bloatiness. But to call this mainly a function of competence is in my opinion misguided.

Stepping back from the nonsense I believe this is a training disparity.

> The bloat and slowness is generally due to incompetence.

I'm sure this is true.

I'm also reminded of "office automation". Conventional wisdom was that new technologies would reduce the use of paper. However, for decades, paper usage went up, and it took a long time for the reduction to happen.

Curious, no?

Was increased paper usage an instance of Jevons Paradox? https://en.wikipedia.org/wiki/Jevons_paradox

I have questions.

So then why did paper usage eventually plummet?

Given long enough time frame, is Jevons Paradox a phase? 150 years after Jevons book The Coal Question, coal consumption is finally tanking. Decades after the start of data processing and office automation, paper usage finally tanked.

Is this generalizable?

Clearly software bloat (and by extension web page bloat) are catalyzed by better tools. Like the democratization of word processing (et al) begat more paper usage, IDEs (et al) begat more code production.

If there is a downside slope to Jevons "rebound effect" (efficiency -> lower cost -> higher consumption), what could it look like for software bloat? What are some possible causes?

For coal and paper, it was displacement by even cheaper alternatives.

What's cheaper than large, slow web pages? Or conversely, how do we raise the cost of that bloat?

Making a huge leap of reasoning:

My optimistic self hopes that the key resource is attention (people's time). Social media values each person's eyeballs at $100/yr (or whatever). So all the nominal costs of all those bloated web pages is pretty cheap.

My hope is the displacement of software bloat will be somehow related to maximizing people's cognitive abilities, making better use of attention. So replace web surfing, doom scrolling, and the misc opiates of the masses with whatever's next.

This hope is strongly rooted in Clay Shirky's works Cognitive Surplus and Here Comes Everyone.

Thanks for reading this far. To wrap this up:

I regard software bloat as a phase, not inevitable.

I imagine a futureperfect media ecosystem not dependent on ad supported biz models, the primary driver for web page bloat.

I have no idea if people smarter than me have studied the tail end of Jevon's Paradox. Or even how much predictive power the Paradox has at all.

I have no idea what the displacement may look like. Maybe patronage and subscriptions. Or maybe universal basic income, so amateurs can self produce and publish; versus "platforms" exploiting user generated content and collaborative editing.

A bit of a scrambled thesis, I know. I'm writing to understand, sound out a new idea. Thanks for your patience.

My browsers shouldn't need to make 100 network connections and download megabytes to display a thousand words of text and some images. It should be about 1 network connection and maybe dozens of kilobytes (for images).

We are not complaining about the web version of productivity apps like email, spreadsheets, project management, etc. loading slowly. But a newspaper article should load faster than a social media feed, and often they don't.

...I mean it's to build a web application, but that's different.

I've been building web stuff since the 90s. Your memory of what it was like is flawed. Sites were slow to load. People complained about images because they took a while to load, and as soon as they completed the page jumped down and you lost your place. Moving from one page to another page on the same site required throwing all the HTML away and starting over, even in a web app like an email client (MSFT literally invented XMLHttpRequest to solve that). The HTML content itself was bloated with styles and font tags and table layouts. Often a tiny article would weigh in at 50KB just because it had been created in DreamWeaver or Hotmetal or something and the code was horrible.

Thr web didn't feel fast back then. It was hellishly slow. I had perf budgets to get sites loaded in under 8s (and that was just measuring the time to a DOMContentLoaded event, not the same as today's Core Web Vitals idea of loaded).

There's no doubt that the web is, in many places, bloated to shit. It's not worse though. It's significantly better. It's just that there's more work to be done.

Its not better if the only thing that keeps it afloat is the fact that broadband is ubiquitous by now (and that isn't even true for most of the world), and hardware got alot better.

The main difference is that many people started using the web after the gamification and bloatation took over, so they are used to adbanners flying in, random videos which start playing, and phone batteries going flat just by looking at a news article.

> It's just that there's more work to be done.

But is that extra work necessary? Look at this threads page on HN. It loads 402 kb worth of content: the html, a tiny amount of JS that isn't even minified (and doesn't have to because its so slim), a small css file, 3 small gifs and the favicon.

That's it. That's all the network-load required to display a usable, information-dense, not hard to look at and performant experience.

if something doesn't start loading in 5 seconds and you are still giving it time without jumping up and down, tcp gave you stockholm syndrome.

I'm not familiar with quic so I don't know how to feel about it, but we are in dire need of an alternative that doesn't require reimplementing the good parts of tcp over udp like every game and communication apps in the world do.

It has unreliable, unordered datagrams, and a huge number of TCP-like streams, all wrapped in the same TLS'd connection.

When I look at QUIC, I see old TCP protocols like IRC and telnet coming back. The difficulty of encryption pushed many applications into HTTPS so they could stay safe, even though HTTPS isn't a perfect fit for every app.

With QUIC saying "You're as safe as possible and you have basically a UDP and TCP portal to the server, have fun", I think we'll see some custom protocols built / rebuilt on QUIC that were lying dormant since the age when encryption was optional.

For instance, multiplayer games could probably just use QUIC as-is. Send the real-time data over the datagrams, and send the chat messages and important game state over the streams. Instead of some custom game communications library, and instead of connecting TCP and UDP to the same server, one QUIC library and one QUIC connection. Now it's within the reach of a solo indie developer who wants to focus on their game-specific netcode, and not on re-inventing networking ideas.

https://quicwg.org/datagram/draft-ietf-quic-datagram.html

They're working on it, I guess.

At least this (compared to unskippable commercials of past) I can bypass looking at.

But that is how 75% of the people reading this make their living...

Actually, in this case "we" means Google and a few other big companies whose aims are not the same as ours - they are the ones in charge. Sure, at times it happens that our interests are somewhat aligned (like page load times) but only as far as it serves them. For Google, a page without their code like ads/analytics is pretty much useless; for us, it's more useful because it doesn't track us and loads faster.

So yes, while they continue doing some work in that respect, I expect it will actually go worse with time as they are focused on average consumer bandwidth in the USA. Once G5 is well entrenched and broadband/fiber gets even faster, we can expect even more bloat on websites and there is not much "we" (=users and developers) can actually do about it.

What I really don't get is why bare html looks so awfully today and you have to add tons of js and css to get something acceptable.

No we don't.

As I have written before in this topic, HN is the perfect example. It pulls in a small css and js file, both of which are so slim, they don't even require minifying. The resulting page is small, looks good, is performant and most importantly does its jobs.

We don't need megabytes worth of cruft to display a good looking page.

It looks good for technical people. It does not look good for others.

I am also sure she would love these selfsame pages to load instantly when shes accessing them via her old cell phone at my uncles house where reception is bad and the device has to revert to 3G.

No, "other" people don't want bloat, tons of adds and "clever designs" (aka. a huge useless picture-banner that contains zero information) either.

https://www.bbcgoodfood.com/ - 12MB https://www.jamieoliver.com/ - 4.3MB https://www.deliciousmagazine.co.uk/ - 8.4MB

It seems their users didn't wanted them to be so barebone.

I am not saying people want bloat, but they won't accept HN-style simplicity. What I'd like to have is good looking sites with no CSS at all, just make them acceptable by default, and a lot of this bloat will simply go away.

The result: traffic congestion on 6 lanes.

https://en.wikipedia.org/wiki/Braess%27s_paradox

That will be the explanation, I’m not too optimistic.

Even Wikipedia begs for money it doesn't need.

He never said experts shouldn’t design protocols, and nothing like that was anywhere near the point.

A good solution could have been AMP: https://amp.dev/

> mostly to spy on me

If only AMP was not beholden to an adcorp.

So it's already possible to achieve quite restricted subsets of what a website can do. The issue is agreeing on what subset is useful enough for many sites.

> TLS 1.3 was used for HTTP/3.

> 0-RTT was enabled for all HTTP/3 connections

Ok then. No potential confounding variables there... none at all.

[For reference its expected that tls1.3 with 0-rtt is going to be much faster than tls1.2 especially when fetching small documents from geographically far away places. To be clear, not doubting that http/3 gives performance improvements in some network conditions, this is just a really bad test and tls version is probably a good portion of the difference here]

The methodology section also doesn't say if 0-RTT was actually enabled during testing. I'd argue that any system or website with an admin interface or user account system should not enable 0-RTT without very strict evaluation of their application protection mechanisms, making it useless for many API servers and data sources. It's fine for static content and that can help a lot, but with the benefit of multiplexing I'm not sure how useful it really is.

For a full handshake: TCP + TLS1.2: 3 RTT TCP + TLS1.3: 2 RTT QUIC including TLS1.3: 1 RTT

And for subsequent connections: TCP + TLS1.3: 1 RTT QUIC including TLS1.3: 0 RTT (but no replay attack protection)

It seems to me just like another excuse to add complexity and to create more bloated websites.

The Google homepage is 1.8 MB at initial load for an image, a field and 3 links, all the other major web operators are not better. Seriously, would they do such pages if they cared for being fast?

[EDIT] For those not liking my comment, I should have said that it is in line with the conclusion of the article: "In general, the more resources your site requires, the bigger the performance improvement you’ll see". I am just questioning the benefit to help the inflation of website traffic, in the end the service is not better, just always heavier (the Google example above is just an illustration).

Arguably it's the other way around. Web sites were already getting extremely complex and bloated, so new protocols are attempting to restore performance that we've lost. I.E. one of the problems HTTP/2 tries to solve is sending multiple files in parallel over the same connection, to avoid the pitfalls of opening lots of simultaneous TCP sockets. This only became a major concern as web sites added more and more assets.

It's definitely a vicious cycle though. It's reminiscent of what happens with hardware. Better hardware incentivizes inefficient software development, to the point where a modern messaging app might not even be usable on a PC from 2003, despite not having much more functionality than similar apps from the era.

Additionally, the Internet is not a static system but a dynamic one. To say something is "fast", means that it should be fast for most people in most conditions. Sinle-link benchmarks are not feasible.

I.e. in a traffic jam I will be very fast when I use the turn-out/emergency lane. But only as long as I am the only one doing it.

Am I missing something, or is this yet another way to track clients across visits? If so, I'm sure Chrome will be faster than Firefox (because it will keep the sessions IDs live forever). Well played, Google.

While someone could use it for some tracking purposes it really is a huge performance boon. There are good intentions on why these features were put in even if they can be abused.

1. https://stuartsmall.com/tlsslides.odp https://github.com/stusmall/rustls/commit/e8a88d87a74d563022...

As a mitigation Chrome (and FF?) have started to partition connections so a different connection would be used to a common 3rd party from different top level origins

You don't want your transport acknowledgement packets to get delayed/lost because of app thread scheduling.

https://caniuse.com/loading-lazy-attr

You can manually enable it if you have Developer mode enabled with:

Develop -> Experimental Features -> Lazy Image Loading

If they just used loading=lazy on the images Safari should ignore it and so all images should load

  <img />

Are there downsides to HTTP/3? When HTTP/2 came out there was discussions about why not to enable.

Benefited from Lucas https://clemente.io/ ... full credit to him, and the various people who worked on the IETF test implementations that then became various production implementations.

I don't credit Caddy with this as they were downstream and just received the good work done. Not that Caddy is bad, but singling them out ignores those that did the hard work.

> Are there downsides to HTTP/3? When HTTP/2 came out there was discussions about why not to enable

1. HTTP/1 and HTTP/2 were natural rate limiters to your application... can your app cope when all requests arrive in a much shorter time window? The traffic pattern goes from somewhat smooth, to somewhat bursty and spikey. If your assets come from a dynamic endpoint or you have authorization on your endpoint that results in database lookups... you'll want to load test real world scenarios.

2. There are amplification potentials when you pass HTTP/3 through layers that map to HTTP/2 and HTTP/1 (the optimisations in the small payload to H3 are "undone" and amplify through H2 and H1 protocols)

3. HTTP/3 combined the transport protocol benefits and the HTTP protocol benefits with TLS benefits... all good, but it is harder for DDoS protection as proxies and middle boxes can no longer differentiate as easily the good traffic from the bad, and may not have visibility over what constitues good at all.

Largely though... worth enabling. And for a good while the last of my points is mitigated by disabling H3 as you'll degrade to H2 cleanly.

I'm also hesitant about HTTP/2 and 3 because of their complexity, but if you trust the systems and libraries underlying your server I don't see any problem in activating either. Just watch out with features like 0-RTT that can have a security impact and you should be fine.

The only downside with HTTP 2 & 3 is software support. If you are in Google, you probably already use it just because the load balancer supports it. With self hosted, you are dependent on whatever you are using adding support. I remember playing with SPDY way back on nginx for example. It wasn't that hard to get going and it made a difference for our mobile users (who are on shitty networks by default).

With anything self hosted, security is indeed a key concern; especially if you are using alpha versions of new functionality like this which is what you would be doing effectively.

I would suggest disabling HTTP/3 for the next 10 or so years and give everyone else the chance to burn through the exploits.

Beware reading these graphs.

Of course, these protocol improvements mostly benefit companies the size of Google. Smaller, independent hosts probably won't get much out of improving the underlying transport outside of a few edge cases.

This blog page loaded very quickly for me compared to most websites, though I don't know how much of that is in the network part and how much of it is because of optimized HTML/CSS/JS.

Also pypy is fast, and the speed of php also heavily depends on version. Not that backend speed even makes a difference that much of the time. 3ms vs 8ms won't matter.

Edit: Ok, apparently there are charts that are not being loaded due to the HN Effect. A clear example of how a blind reader would miss quite a bit of information when the data is only shown in images / other non A11Y compliant resources.

No it doesn't work on Safari only. Turn on Lazy Image loading or use other browser.

How about extending TCP to establish multiple sessions at once instead?

QUIC also handles things like transparent IP-address switching (on either side of the connection).

Also nothing prevents the server from implementing TCP in userland, apart from maybe security concerns, but then if you want low latency you have to discard those anyway.

It's easy with streaming video, when you can just time the requests at a fixed rate. But for stuff like streaming game assets, you never know how long each download will take. Doing parallel requests will just slow down the time for the first asset to appear, and doesn't guarantee that you fill the bandwidth anyway if the latency is high enough...

Is TCP really that fundamentally slow? How could high-level repurposes of UDP be faster than presumably hardware optimized and heavily analyzed TCP stacks?

The lies, damn lies, benchmarks seems a bit applicable too here. He's disabling caching? Really it is testing a specific transport situation, but caching IMO would mitigate a lot of the dramatic advantages? And what about cloudflare edge caching outside the browser? I think he was routing all resource requests through his single server that probably isn't cached properly.

So with good caching will HTTP/3 produce the advantage for the everyman over HTTP/1.1 to justify the attention?

And you don't need hardware optimization until you're getting into many gigabits per second.

https://caniuse.com/http3

Typically servers support multiple HTTP versions.

Huh, this is interesting.

So the current timescale is like:

  HTTP/1: 1996 (though HTTP/1.1 came out in 1997)
  HTTP/2: 2015
  HTTP/3: 2021 (current draft)

So yes, I'd think the rate of progress may well increase. Not all will become standards, but I imagine we might see HTTP/3.1, 3.2, etc. long before we see an entirely new version like HTTP/4

In general connections between CDNs/reverse proxies and origin servers don’t get much benefit from HTTP/2 or HTTP/3. CDNs don’t generally care about connection establishment speed or multiplexing to the origin (the main benefits of newer HTTP versions), since they can just create and maintain N long-lived connections if they want to be able to send N concurrent requests. They generally only bother with HTTP/2 to the origin if they need to support gRPC, which has some unusual connection semantics.

Is the packet loss rate being measured? That's the main cause of head-of-line blocking delays.

Are both ends using persistent TCP connections? If you have to re-open and redo the TLS crypto handshake each time, that's a huge overhead. Does Caddy implement that? Is the CONTENT-LENGTH header set? If not, each asset is a fresh TCP connection.

> “ On average, with HTTP/3 we see the first byte appearing after 176ms. With HTTP/2 we see 201ms, meaning HTTP/3 is already performing 12.4% better!”

https://blog.cloudflare.com/http-3-vs-http-2/

It's true that you miss out on "telnet example.com 80" and typing "GET /", but that honestly hasn't been valid since HTTP/1.1 replaced HTTP/1.0. To some extent this all sounds to me like "I hate screws, they render my trusty hammer obsolete". That's true, but there are also great screw driving tools.

Which sites tend to need.

I'm sure someone can make an equivalent to that. Maybe even just a wrapper around curl...