Cloudflare’s misguided reliance on Javascript Paywalls[1] is fundamentally hostile to open web. It’s essentially a form of DRM.
And they don’t even bother to implement it properly—for example if your site tries to follow best practices and uses a separate domain for your static assets, you will just get errors on your static assets, resulting in a page with broken styling and no images. That despite pissing off your users by having them go through the Google hosted captcha (which also breaks all the time btw[2][3]).
One of the websites that was horribly broken by this was Stack Overflow. As anyone trying to stay safe on public WiFi by using VPN can attest.
And if you turn off Cloudflare's protection to fix this, somebody that wants to censor you and has $20 will use one of the hundreds of DDoS booters (most of them are behind Cloudflare) to nuke your site, unless you're Brian Krebs and qualify for Project Shield.
I'm very optimistic about the direction the internet is going in right now.
As a quick primer, it's a hash that points towards a directory of content. Everything's deduplicated. Based on Torrents, GIT, and self-referential filesystems.
an IPFS hash is immutable. The hash points at the hash, no matter what. Indestructible. Publish stuff by
ipfs add -r folder
An IPNS hash points towards an IPFS hash. It's a pointer you publish every 12 hours. It IS mutable. Do this by
ipfs name publish <ipfs key of resource>
The browser plugins with IPFS running allow you to pull the DNS text record of the IPNS hash, and you never touch the website!
CloudFlare is a anti-DDoS and CDN network. IPFS is a CDN protocol that anyone can join or put files into. It doesn't quite hide the endpoints, but anyone can inject data.
It does what CloudFlare does, but better. And as more people/nodes get online, free and ubiquitous.
More likely the Python client is using the stock python user-agent, this should be customized per application. The reason for this, is most stock scrapers and malicious agents are using stock engines with defaults.
CloudFlare's CAPTCHA can be trivially disabled by the site owners if they so choose. It's a trade-off though - many leave it enabled to prevent a layer 7 attack... there's not many other options to do that and there's absolutely none available to anyone but the biggest sites.
And they don’t even bother to implement it properly—for example if your site tries to follow best practices and uses a separate domain for your static assets, you will just get errors on your static assets, resulting in a page with broken styling and no images. That despite pissing off your users by having them go through the Google hosted captcha (which also breaks all the time btw[2][3]).
One of the websites that was horribly broken by this was Stack Overflow. As anyone trying to stay safe on public WiFi by using VPN can attest.
Coincidentally, Cloudflare has lost Stack Overflow as their customer recently: https://meta.stackoverflow.com/questions/323537/cloudflare-i...
They’re now behind fastly.
_
[1] https://ipfs.pics/QmTZo6oPKHwUgWB7p7LfZwZsVQJV1n7k9qNQNZBCEu...
[2] https://ipfs.pics/QmeuJjgV621NV9aNKyNAUoEHdWZYtzCrwkLHoHneg3...
[3] https://ipfs.pics/QmRWcCkBdaG214GKttkGFcadncUJ6YvfMTSE8jiAxA...
bonus picture: https://ipfs.pics/QmPkncvs2R9EkhZQuzPzWYs4z7UUdKqQzg1k8mc7y5...