"We need to make an AMP site to get into this carousel" is a much easier sell (to executives) than "we need to rebuild our site according to some not completely specified principles to hopefully get a higher search rank".
Load time in what browser? Over what connection? When is it loaded - all pictures or just the text? How much text? Does all the JavaScript have to have loaded? What about the calls that makes, do they need to complete? Is it better to have one more second before first render or one more megabyte of data?
It's a multibillion dollar company with thousands of the best tech minds in the world in their employ. With the massive pool of talent they have, this stuff isn't exactly rocket surgery.
Fine, if you can't pick a number out of your ass and say "sites must load in less than 1.5 seconds from <simulated average connection> to appear in our special box" just use it as a ranking value.
Fill the special box with relevant results to the search, and use load time for the result as a weighting. So if your content is ridiculously on-topic but a little slower to load, you'll still probably be in the special little box, but a slightly less specific site thats super fast to load might be before your site.
As has been mentioned: google has used page load times as a metric for a long time. The difference is they didn't add a "special little box" to incentivise sites with otherwise shit decision making skills to do the right thing.
But sure. Tell me how forcing clients to download a bunch of javascript, and introducing forced 8 second blank pages for anyone who dares to not load said javascript is all about making pages faster.
Give me a fucking break. If you want to live in the fucking google sphere, that's your choice, but don't pretend that their motives are anything but hostile to the very concept of the open web.