Those who don't learn their history are doomed to find new and innovative ways to repeat history.
If you're older than 40, you remember that there did exist an aeon, long, long ago, when people did not use data object serialization formats as config files. When config files were written not to be easy to parse, but to make it easier for human beings to configure software. When nobody believed there was one single good way to do everything. When software was written not to aid computers, but to aid humans.
Yeah. I've got no idea what your parent comment is talking about.
> When config files were written not to be easy to parse, but to make it easier for human beings to configure software.
*eyes rolling*. All I can remember is the hundreds of hours I've spent trying to figure out how to configure something in Apache httpd, BIND, iptables, and god forbid, Sendmail!!
Config files were written not to be easy to <anything>. There was no rhyme or reason. Every project had their own bespoke config. All from the whims and fancies of the devs of the project.
Good thing that was all in the past and I had no job and no responsibilities. If software today made configuration like they did 40 years ago, I'd just give up!
It's embarrassing when people buy a truck and don't use it for work, towing or payload. So you bought a fuel-inefficient non-aerodynamic vehicle whose storage area is open to the air and unusable for passengers, and it's not big enough to carry or tow large items? What a smart choice.
However, with the SUV package and lift kit, this is actually useful. It's basically the same size (and payload and towing capacity) of the 2nd gen Scion xB. A boxy, roomy, small, cheap car. Absolutely useful and great. (Unlike a tiny truck.)
Worth noting that people here are assuming that the author's assumption is correct, that his writing /etc/hosts is causing the 403, and that this is either a consequence of security filtering, or that this combination of characters at all that's causing the failure. The only evidence he has, is he gets back a 403 forbidden to an API request when he writes certain content. There's a thousand different things that could be triggering that 403.
It's not likely to be a WAF or content scanner, because the HTTP request is using PUT (which browser forms don't use) and it's uploading the content as a JSON content-type in a JSON document. The WAF would have to specifically look for PUTs, open up the JSON document, parse it, find the sub-string in a valid string, and reject it. OR it would have to filter raw characters regardless of the HTTP operation.
Neither of those seem likely. WAFs are designed to filter on specific kinds of requests, content, and methods. A valid string in a valid JSON document uploaded by JavaScript using a JSON content-type is not an attack vector. And this problem is definitely not path traversal protection, because that is only triggered when the string is in the URL, not some random part of the content body.
It sure looks like the author did his due diligence; he has a chart of all the different phrases in the payload which triggered the 403 and they all corresponded to paths to common UNIX system configuration files.
Nobody could prove that's exactly what's happening without seeing Cloudflare's internal WAF rules, but can you think of any other reasonable explanation? The endpoint is rejecting a PUT who's payload contains exactly /etc/hosts, /etc/passwd, or /etc/ssh/sshd_config, but NOT /etc/password, /etc/ssh, or /etc/h0sts. What else could it be?
If you change a single string in the HTTP payload and it works, what other explanation makes sense besides a text scanner somewhere along the path to deploying the content?
You're being downvoted because WAFs work exactly like this, and it's intentional and their vendors think this is a good thing. A WAF vendor would say that a WAF parsing JSON makes it weaker.
It's frightening that so many people are convinced the author is correct, when the author never proved they were correct.
The author just collected a bunch of correlations and then decided what the cause was. I've been doing this kind of work for many, many years. Just because it looks like it's caused by one thing, doesn't mean it is.
Correlation is not causation. That's not just a pithy quip, there's a reason why it's important to actually find causation.
Having had three opportunities in my life to diagnose this exact problem and then successfully resolve it by turning off the WAF rule (see my top level comment) - I don't know you or your work history, but trust me, the author is much closer to the truth here than you are.
edit: Also, someone commented here "it was an irrelevant cf WAF rule, we disabled it". Assuming honesty, seems to confirm that the author was indeed right.
Open Source projects used to all be hosted on hundreds of random mirrors. The hosting of which was free and donated, because it was just an HTTP/FTP/RSYNC directory on a file server in a closet in some corporation or university. Didn't even need to be reliable, as there were hundreds of mirrors. Linux distributions, and some very old projects, are still maintained this way.
Nowadays you must have a flashy website. You must host everything on a single managed VCS provider, or a programming package ecosystem hoster. You must depend on corporations to give you free things, in exchange for you giving them everything about you (otherwise you must pay out of pocket for everything). You must do what everyone else does.
Maybe it's impossible to go back to a simpler time. But it's not impossible to change the state of things today.
> Nowadays you must have a flashy website. You must host everything on a single managed VCS provider, or a programming package ecosystem hoster. You must depend on corporations to give you free things, in exchange for you giving them everything about you (otherwise you must pay out of pocket for everything). You must do what everyone else does.
You keep saying _must_. Why _must_ you do what everyone else does? To what end? To get contributors to your project? To get funding? No one is stopping anyone from starting a project on sourceforge.
If it's local-first then it's already subject to rot. Unless they're running it all in containers? In which case local doesn't matter.
If you want to record a runbook, then record a runbook. You can do that a million ways. Text file, confluence doc, screen recording, shell script, etc. People already don't do that; they're not gonna suddenly start doing it more because your UI is fancier.
Personally, I don't want to sit around all day writing code (or docs) to try to get the system to be like X state. I want to manually make it have X state, and then run a tool to dump the state, and later re-run the tool to create (or enforce) that state again. I do not want to write code to try to tell the computer how to get to that state. Nor do I want to write "declarative configuration", which is just more code with a different name. I want to do the thing manually, then snapshot it, then replay it. And I want this to work on any system, anywhere, without dependence on monitoring a Bash shell for commands or something. Just dump state and later reapply state.
Then I should explain why "a blob" is, in a weird way, actually superior to a Dockerfile.
Step 1. You write a Dockerfile. You build it. You test with it. Hey, it works! You push it to production.
Step 2. Years later, you need to patch that container in production. So you change the Dockerfile, rebuild, and re-test.
Step 3. Uh-oh! The tests don't work anymore! What's going on? I changed one line in the Dockerfile but now the app isn't working (or perhaps the build isn't). What's going on?
What's going on is a reproducibility failure. Just having the instructions (or what someone thought was the instructions, or what they were years ago) isn't enough to ensure you get the same results a second time. A million little things you didn't think of may change between builds (and the more time in-between, the more things change), and they may end up breaking things. Without a truly reproducible build, you are likely to have problems trying to rebuild with just the instructions.
That's why with Docker containers, we rely on build artifacts. The container you had two years ago? However that was built, with whatever weird combination of arguments and files-on-the-internet-being-pulled-at-build-time and everything else, it was built successfully. If you want to be really sure you patch it correctly, you pull the old container image (not Dockerfile), modify that, and push that to production as a new container. No rebuilding, just patching. This avoids reproducibility failures.
That same idea is why you'd want to just download a blob and later re-apply it.
The blob was the state of things when it was working. If you tried to just write down the instructions to replicate it, it's likely you'd either 1) get it wrong (it was only working because of some unrelated changes somebody else made and forgot about) or 2) you'd get a reproducibility error.
So "the blob" I'm talking about doesn't have to be a literal binary blob. It could be whatever, we're talking about a theoretical idea here. It could be layers like a container, or metadata in a JSON file, or configuration/code that gets auto-generated, etc. I don't care what it is. It just has to describe the state as it was when X was working. It's then up to the tool to figure out how to get back to that state.
People already write this as "declarative code" for configuration management tools to do the same thing. I'm saying, I don't want to have to write the code. Just dump it out for me.
That’s not what they are saying. They are saying that the system where you have to declare everything manually is annoying (which it is), ideally it would record the changes while you make changes and then deduplicate them, remove unnecessary ones to arrive at the final playbook that can be replayed if needed.
Actually no, that's still just monitoring a series of steps to eventually lead to a thing. I don't want to record a series of steps, I just want to dump existing state of a thing.
Such a process is rarely portable though, and will need to be repeated for each different system, at which point it would be great to already have a declarative description, that can automatically be translated into those steps required to get to state X.
Unless the Dockerfiles are kept secret, any container can be replicated from the given Dockerfile. Barring extreme (distro/system/hardware)-level quirks, a Docker container should be able to run anywhere that Linux can.
You are mixing build time reproduction with run time ones.
Docker images (not files) help with the run time consistency .
Docker (files) barely scratch the surface of build reproducibility. Most applications depend on the distribution package manager ( apt, apk etc) and language package manager (npm, cargo, etc), both sets of them have various challenges in consistent dependency resolution.
In addition build steps might have ordering challenges RPC calls to remote services no longer running and so on.
Anyone trying to to build a docker image from 10 years back experiences this problem
You're right in the absolute form, but I've yet to see a Dockerfile where (with a little thinking and elbow grease) I couldn't "easily" port it or update it, even after years.
It's basically the best and easiest "I am documenting how it works now" thing without any arcane "works on my machine" quirks I have yet found.
So I'm still agreeing here that it's a very good approximation of this idea.
Real reproducability is miles better, but usually cannot be formulated in a ~20 line single file "recipe". (and before anyone mentions Nix.. no, there's so much inherent complexity involved, that doesn't count like "apt-get install docker && docker build ."
A container can very rarely be reproduced by a dockerfile.
I imagine with a lot of discipline (no apt update, no “latest” tag, no internet access) you can make a reproducible docker file…. But it is far from normal.
People rarely mean 100% build reproducibility, but simply within a reasonable limit, Dockerfiles are mostly "run stable" and provide the same OS abstraction and process encapsulation.
Well sure, making a 100% reproducible build is hard - but Docker makes it easier, not harder. If 100% reproducible is the goal, what's easier than docker?
A Dockerfile is essentially a shell script with access to the outside world. It has unconstrained network access. It can access local hardware and filesystem if instructed to. However, it doesn't verify that whatever stuff it took from the outside remains the same across builds. Docker doesn't care if the same Dockerfile builds Apache httpd in one build and Nginx in another. It literally can't get more irreproducible than that.
But mysteriously, people say that Docker is reproducible because, uh, you can download gigabyte-sized binary blobs from the Docker registry. I wonder, what's not reproducible by that metric?
Docker images may be portable compared to binaries targeting traditional FHS distros. But it's not reproducible whatsoever.
Presumably if your goal is a reproducible build you just wouldn't do any unconstrained downloading in the process of designing the dockerfile and building the image. Making a choice to use a tool poorly for you requirements isn't a problem with the tool.
The claim the parent was addressing was that Docker helps with reproducibility. It doesn't. Docker does nothing at all in this regard.
If you want a reproducible Docker image, you're on your own. For example, the most common problem is that many build scripts out in the wild download stuff willy nilly without verifying anything. I've seen NPM package post install scripts do the craziest things. Catching all of that is harder than most would give credit for at first glance, considering that tons of build scripts are written in Turing complete languages. Help from tooling is essential.
When you have to fight the tool to achieve reproducibility, not choosing to do so isn't "using a tool poorly." It's simply using the tool as is. Especially when the vast majority of Dockerfiles out there happily run something along the lines of `apt install foo`, again, without verifying anything.
The Nix package manager. It forces all inputs to a package be specified precisely. Files either have to come from the build recipe or have its hash verified. Builds are properly sandboxed and can't freely access the network. Dependencies are completely static and no "resolution" ever takes place. The tool catches sources of irreproducibility for you.
Full reproducibility isn't easy, there is a cost to it.
However the payoff is rather significant so if you can temper that cost a bit and make it less inconvenient to achieve then you have a winning solution.
Tools that have been designed with reproducibility in mind. Like Guix.
Beware, I am definitely not claiming those are easy to use in general. Just that you can get to reproducibility using them more reliably and maybe easier than with docker.
It kind of sounds like you're describing Ansible. You use modules for common tasks like ensuring a package is installed, a file is present or has certain content, etc. It's declarative and imdempotent.
I've written some fairly complex stuff in Ansible. It is mostly declarative but you should be careful with assumptions about its idempotency, especially if you reach out for community modules.
No, I don't want to write Ansible configuration. I don't want to write any configuration. I just want to dump state and restore it. (To put it another way: I want to auto-generate ansible configuration based on an existing system, and then apply that configuration again later)
what happens when you want to tweak something you did in the middle of this process? do you have to go through the whole flow again manually to make a single change?
I imagine you could either A) just modify the dumped state, B) paramaterize it, C) have the program split up the state into transactions and modify those. The program will probably have to take more than one step, in order, in order to accomplish everything. If it fails, you'd want it to try to undo it, ala transactions. And since it can do all that, it can stop, start, or resume at specific steps.
Like, Terraform has always sucked because there was no way to dump existing resources as new code. So a team at Google made a tool to do it (Terraform-er). If Terraform had already had that feature, and if it didn't rely on having pre-existing state to manage resources, that would be like 90% of the way to what I'd want. Just dump resources as code, then let me re-run the code, and if I want I can modify the code to ask me for inputs or change things. (People think of Terraform as only working on Cloud resources, but you could (for example) make an Ubuntu Linux provider that just configures Ubuntu for you, if you wanted)
Any notion of state that satisfies requirements like
> Just dump state and later reapply state
is necessarily declarative.
> Just dump resources as code,
What is the code for this resource?
VM foo1
Memory 16GiB
Network mynet1
It depends on the current state of the system where the resource is applied. If VM foo1 already exists, with 16GiB of memory, and connected to network mynet1, then the code is a no-op, no code at all. Right? Anything else would be a mistake. For example if the code would delete any matching VM and re-create it, that would be disastrous to continuity and availability, clearly a non-starter. Or, if VM foo1 exists, with 16GiB of memory, but connected to anothernet3, then the code should just change the network for that VM from anothernet3 to mynet1, and should definitely not destroy and re-create the VM entirely. And so on.
It depends what you're talking about; Terraform specifically has a flawed model where it assumes nothing in the world exists that it didn't create itself. Other configuration management tools don't assume that; they assume that you just want an item to exist; if it does exist, great, if it doesn't exist, you create it. But for a moment I'll assume you're talking about the other problem with configuration management tools, which is "which of the existing resources do I actually want to exist or modify?"
That's a solved problem. Anything that you use on a computer that controls a resource, can uniquely identify said resource, through either a key or composite key. This has to be the case, otherwise you could create things that you could never find again :) (Even if you created an array of things with no name, since it exists as an item in a list, the list index is its unique identifier)
Taking Terraform as example again, the provider has code in it that specifies what the unique identifier is, per-resource. It might be a single key (like 'id', 'ASN', 'Name', etc) or a composite key ( {'id' + 'VPC' + 'Region'} ).
If the code you've dumped does not have the unique identifier for some reason, then the provider has to make a decision: either try to look up existing resources that match what you've provided and assume the closest one is the right one, or error out that the unique identifier is missing. Usually the unique identifier is not hard to look up in the first place (yours has a composite identifier: {VM:"foo1", Network:"mynet1"}). But it's also (usually) not fool-proof.
Imagine a filesystem. You actually have two unique identifiers: the fully-qualified file path, and the inode number. The inode number is the actual unique identifier in the filesystem, but we don't tend to reference it, as 1) it's not that easy to remember/recognize an inode number, 2) it can be recycled for another file, 3) it'll change across filesystems. We instead reference the file path. But file paths are subtly complex: we have sym-links, hard-links and bind-mounts, so two different paths can actually lead to the same file, or different files! On top of that, you can remove the file and then create an identically-named file. Even if the file had identical contents, removing it and creating a new one is technically a whole new resource, and has impact on the system (permissions may be different, open filehandles to deleted files are a thing, etc).
So what all of us do, all day, every day, is lie to ourselves. We pretend we can recognize files, that we have a unique identifier for them. But actually we don't. What we do is use a composite index and guess. We say, "well it looks like the right file, because it's in the right file path, with the right size, and right name, and right permissions, and (maybe) has the right inode". But actually there's no way to know for sure it's the same file we expect. We just hope it is. If it looks good enough, we go with it.
So that's how you automate managing resources. For each type of resource, you use whatever you can as a unique (or composite) identifier, guesstimate, and prompt the user if it's impossible to get a good enough guess. Because that's how humans do it anyway.
> Terraform specifically has a flawed model where it assumes nothing in the world exists that it didn't create itself.
I don't think this is accurate. Terraform operates against a state snapshot, which is usually local but can also be remote. But it has several mechanisms to update that state, based on the current status of any/all defined resources, see e.g. `terraform refresh` (https://developer.hashicorp.com/terraform/cli/commands/refre...) -- and there are other, similar, commands.
> But for a moment I'll assume you're talking about the other problem with configuration management tools, which is "which of the existing resources do I actually want to exist or modify?"
I'm not really talking about that specific thing, no. That problem is one of uncountably many other similar sub-problems that configuration management tools are designed to address. And, for what it's worth, it's not a particularly interesting or difficult problem to solve, among all problems in the space.
If you have a desired state X, and an actual state Y, then you just diff X and Y to figure out the operations you need to apply to Y in order to make it end up like X. Terraform does this in `terraform plan` via a 3-way reconciliation merge/diff. Pretty straightforward.
> you just want an item to exist; if it does exist, great, if it doesn't exist, you create it
It's not as simple as whether or not an item should exist. Being able to uniquely identify a resource is step one for sure. But a single resource, with a stable identifier, can have different properties. The entire resource definition -- identifier, properties, and everything else -- is what you type and save and commit and push and ultimately declare as the thing you want to be true (X). That's not code, it's state (definitions). Code is what's executed to diff that declarative state (X) against actual state (Y) to produce a set of delta operations. Or, it's those delta operations themselves.
> If the code you've dumped does not have the unique identifier for some reason, then the provider has to make a decision: either try to look up existing resources that match what you've provided and assume the closest one is the right one...
First, you "dump" state, not code. More importantly, no configuration management system would ever take one identifier and "guesstimate" that it should match a different identifier, because it's "close", whatever that means.
> or error out that the unique identifier is missing. Usually the unique identifier is not hard to look up in the first place (yours has a composite identifier: {VM:"foo1", Network:"mynet1"}). But it's also (usually) not fool-proof.
I really don't understand what you mean, here, nor do I understand your mental model of these systems. It's certainly not the case that my example VM has the composite identifier {vm:foo1 network:mynet1}. The identifier is, intuitively, just foo1. Even if we were to say the identifier were an object, the object you propose is missing the memory size. But more importantly, changing the foo1 VM from network:mynet1 to network:othernet2 probably should not have the effect of destroying the existing VM, and re-provisioning a brand new VM with the new network. Sometimes configuration changes require this kind of full teardown/spinup, but these conditions are generally rare, and all modern configuration management tools avoid this kind of destructive work whenever possible and most of the time.
> So that's how you automate managing resources. For each type of resource, you use whatever you can as a unique (or composite) identifier, guesstimate, and prompt the user if it's impossible to get a good enough guess. Because that's how humans do it anyway.
Just to reiterate, I'm not aware of any configuration management tool that "guesstimates" when making changes in this way. For good reason.
`terraform refresh` (which is now `terraform apply -refresh-only`) is an exception to the rule. Terraform doesn't know what's going on in the outside world. If you write configuration to create a Security Group named "foobar", and do a `terraform plan`, it will say it's about to create "foobar". When you go to apply, it will error out, saying "foobar already exists".
If Terraform wasn't completely idiotic, it could have just checked if it existed in the planning stage. If Terraform was even mildly helpful, it would have suggested to the user at either plan or apply time that the security group already exists, and do you want to manage that with your code? But it doesn't do those things, because it's a completely dumb-ass design.
> I'm not aware of any configuration management tool that "guesstimates" when making changes. Thank God.
Many of them do. Ansible does, Puppet does, Terraform does. They have to, for the same reason as my filesystem example: it's often impossible to know that a resource is unique, because there aren't actually unique identifiers. My definition of "Guesstimation" is specifically "using the identifiers you have available to select an entry from a list of potential options with the closest match". Ansible does this all the time. Puppet and Terraform do this for every provider that doesn't have a totally unique identifier (there basically are no totally unique identifiers, as I pointed out in my filesystem example)
It seems to me that your frustration with Terraform being "completely idiotic" is ultimately frustration with the underlying design model.
> If you write configuration to create a Security Group named "foobar",
That configuration is a declaration: a security group named "foobar" should exist, with the declared properties.
> and do a `terraform plan`, it will say it's about to create "foobar".
That plan would be based on the most recent snapshot of the target "outside world" resources, which, if you haven't synced them recently (or at all) would probably be empty, resulting in `terraform plan` proposing to create foobar afresh.
> When you go to apply, it will error out, saying "foobar already exists"
Sure, which should hopefully make sense. You've declared a resource locally, and asked Terraform to "make it so" basically. But that resource is in conflict with an identical remote resource. You can `terraform refresh` or sync or whatever, to pull down the current relevant remote resource state locally, and then operate from there. Or you can manually blow away the remote foobar and retry. Or etc.
But this kind of situation is not common. Terraform assumes and expects that the declarations (and state) it has access to locally is an authoritative source of truth for what the target remote system(s) should be. The config files define what should be running in AWS, not the other way around.
It's fine if this isn't a fit for your use cases, but I don't think that means the entire tool is stupid or whatever. It just means it's not for you.
When most people say "local" what they mean is "i'm running something on my laptop". That is to say, it's a random operating system, with a randomly set-up environment, with randomly installed tools. Could be different tools that have the same name but incompatible options. Could be any version (version 3 might be incompatible with version 4, but it's the same name for the command). And it will definitely change over time.
This will lead whatever steps you've recorded to 1) stop working on the existing "local" machine, and 2) be incompatible with other people's "local" machines. So the instructions have "rotted".
To avoid this you could write Ansible/Puppet to install and configure all the same tools, but that will break too over time too, and be a maintenance hassle. The only reliable solution is to use containers to run it all in; that guarantees the same version of everything. But at that point it's not really "running locally" anymore, it's running in the container, which is sort of its own bag of issues.
At that point you might as well have a SaaS tool to run your runbook in a cloud environment in containers or something, as that's way easier to set up and manage than either ansible/puppet, or Docker on everyone's machine (there's still a million tech "engineers" out there who don't understand containers).
Pre Cloud at eBay we managed 30k windows servers via a tool called site controller that referenced ALL servers as objects with properties of configuration state, build version and operational state.
ALL databases had their startup configuration parameters defined per instance across datacenters.
Furthermore SRE had tools to rate limit connections or restart of a database so that it was not overwhelmed by incoming connection requests. We also built tools to do fine grained definition of load balancers and what services were behind them to be able to redirect traffic on the fly and then reset to the original mappings once connectivity of the LBs was resolved.
These tools weee centralized and available to both SRE and senior system administrators.
These things evolved to new tools which accommodated a private cloud, then docker then Kubernetes. I left prior to kubernetes implementation.
Hard to word that language to prevent a corporation from forking it, as you have to "fork" the project locally to make modifications and send patches back. I'm sure nobody here wants to stop a random engineer at a corporation from contributing to a community project?
If you want a corporation to avoid it like the plague, just make it GPLv3. If you really want to screw them, go with AGPLv3. This way you keep a true open source license, but don't have to worry about corporate control.
And remember kids: there's hundreds of CAs, they all implement validation independently, and you just need one to do one of the three validation methods wrong to make any cert you want. And there's two dozen different attacks that work aside from bugs in validation. Cert validation is swiss cheese.
But there's a fix: have the registrars confirm authority to issue certs using a public key uploaded by the domain owner. The CSR contains a request signed by the same domain owner key. The CA sends the CSR to the registrar, the registrar verifies it was signed by the same key they have, then they send back a newly signed object (that the eventual-https-end-user can later confirm was signed by the registrar, so everybody knows that every step of the way was confirmed cryptographically). This ensures a single secure authorization by the actual domain owner, verified by the registrar of the domain. All you have to change is how the CA validates, and the registrar needs to handle an extra step or two. Solves 95% of the vulnerabilities.
....but nobody's going to do that, because the fact that Web PKI is swiss cheese hasn't threatened a big enough business yet. Once money or politics is threatened, they'll fix it.
CAA account binding is basically this. Not cryptographically verified but similar idea that the CA confirms you possess a secret (account) before issuing. https://www.rfc-editor.org/rfc/rfc8657
Nope, it's not the same. The point of having the Registrar involved is to side-step the problem of validating a cert request is allowed to request the cert. All the CA validation methods are supposed to be verifying your authorization to request a cert, but they don't do that.
They instead verify your authorization to control DNS records, or IP space, or an e-mail address. And there's dozens of exploits to compromise each of those. And they can be chained. And they can be CA-specific.
That's not domain authorization, and each of those verification methods lacks cryptographic authentication. Only the Registrar controls the domain, so that is the only way to know that the request is genuinely authorized. We're playing a game of telephone, it's not secure, and it's unnecessary. Just get the registrar involved.
With your solution, we end up with the same problem just one layer down. Browsers have to contain a list of 'trusted' registars, and an attacker only needs to find one buggy registrar that will incorrectly sign for a domain the attacker doesn't own.
That's a much simpler problem to solve than the current one. One attack vector to cover, one set of organizations, one trust list. It's definitely no worse than our current predicament.
Basic math shows how much safer the new model would be:
- Assume there are 350 CAs, 3 validation methods, and 12 kinds of exploit per validation method (there are more in some combinations but for simplicity I'll say 12).
(350 x 3 x 12) leaves *12,600* possible attack vectors.
- Now assume there's 2,650 domain registrars, 1 validation method, and 1 kind of exploit.
(2650 * 1 * 1) leaves *2,650* possible attack vectors.
So 4.75x fewer possible attack vectors.
Add to this that with only 1 validation method and 1 feature to support, that's way less code, which means much fewer bugs.
Add to that a cryptographic chain of proof that the key of the domain owner was used to sign the request all the way, which we don't have today.
This math only looks good because you're adding an arbitrary unsubstantiated 12x multiplier to the CA numbers.
Of course, neither of us have actual numbers but my gut instinct is that registars are probably about as secure if not less secure than CAs, and there are nearly 10x as many of them.
Registrars are less secure because CAs have to follow a strict guideline for security. We could just apply that same standard to the registrars. It would be easier for them to follow since it would be far simpler to use 1 validation method, and using cryptographic verification it's easier to automate validating that it was done successfully.
The 12x multiplier comes from the number of attack vectors to validation methods. The whole system is public knowledge; if you know how each part works, and you know about all the possible security exploits out there, you can just count them.
Here's a brief list off the top of my head, it's not exhaustive:
- DNS validation. The CA looks up TXT records of a given subdomain.
Attack 1. DNS cache poisoning
Attack 2. Compromise the credentials of a DNS admin
Attack 3. Zero-day exploit in the DNS server
Attack 4. Zero-day exploit in the DNS web management interface
Attack 5. Incorrectly configured DNS zone transfer settings.
Attack 6. BGP spoofing attack on the target DNS nameserver.
Attack 7. BGP spoofing attack on the CA's DNS resolver.
- HTTP validation. The CA requests a specific URL over HTTP and verifies the contents.
Attack 1. MITM the HTTP request/response. (Can be done anywhere across the network, from the CA internal network to the target internal/external network)
Attack 2-8. Every single DNS attack. You just replace the A/AAAA record when looking up the target HTTP host, with an attacker-controlled http host.
Attack 9. BGP spoofing attack on the IP of the target HTTP host.
Attack 10. Zero-day exploit in the target HTTP server.
Attack 11. Stealing credentials to remotely login to the target HTTP server.
- Email validation. The CA sends an e-mail to the domain and confirms the reply.
Attack 1-7. Every DNS attack.
Attack 8. BGP spoofing attack on the IP of the target MX host.
Attack 9. MITM the e-mail. This one is extra easy as intermediate unencrypted relays are common/expected.
Attack 10. Steal the credentials of the mail server admin to remotely log in and intercept/fake emails.
Attack 11. If it's a site that allows users to register an email address, there are six different email addresses they can try to register; if one works, they can use that to validate certs for that domain.
Attack 12. Zero-day exploit in the mail server software.
Attack 13. Zero-day exploit in the mail server's web management interface.
And this is just the run-of-the-mill exploits most experienced hackers can pull off remotely. Haven't gotten into more advanced things like supply-chain attacks, timing attacks, protocol/algorithm flaws, espionage, social engineering, etc.
The big problem is that since validation uses no cryptography whatsoever, all the attacks are fairly trivial. A BGP attack is so easy that it happens by accident on a monthly basis. MITM is easy. Stealing credentials is easy (ask any botnet admin). Attacking DNS is easy (most people don't uses DNSSEC and even if they do their clients/resolvers aren't enforcing validation).
Do an end-run around all this bullshit by asking the people who actually know who owns the domain (the Registrar) to validate it cryptographically.
I came to the comments hoping for lots of electric apologists not reading the article and I was not disappointed
As a recap (for those who don't like to read)
- 70% of EV cost comes before the vehicle ever moves and must be recouped over the life of the vehicle (and takes much longer than traditional fuels)
- the energy density & cost of certain fuels is the only reason certain vehicles are able to be profitably operated in the first place
- the only way to create enough energy to match said fuels/demand with electrics (at present) would be to hook up coal or nuclear plants to airports, and even then it'd be expensive as shit
- we basically need a 5x improvement in battery energy density at minimum to even think about profitability, and that's only one of the things that would need to be addressed before it's practically feasible
What do you mean? Virtually no one is arguing that marine or air is viable given current tech.
The article author though demonstrates some clear lack of understanding about more viable tech though, given his absurd assertion about the profitability of EV scooter companies. Ground-based EVs like cars and ebikes are clearly here to stay and are going to replace almost all fossil-fueld based equivalents.
We should make Slack a new internet protocol and application standard, and use that going forward to replace e-mail, texting, and the various isolated islands of "secure chat" solutions (WhatsApp, Signal, Telegram, etc). Allow us to retain and control our own data, while also enabling all of the features and functionality we've come to want from modern tools, and be compatible with other solutions.
IRC and e-mail are both old and busted. 99% of the world wants to communicate and share information with more interactive tooling than ASCII text in a console or static HTML in a mail reader. There are alternatives to Slack, but like every networked application created in the last 10 years, none of them define an interoperable standard. They are all their own vendor-lock-in islands.
Even Mattermost, the most polished "open-source" alternative, is not a standard, it's an application. Applications change all the time. Standards don't. Applications lose backwards compatibility, change their licenses, have closed ecosystems of servers. Standards don't. There's a reason that actual standard network protocols continue to work for 40 years, while applications made just a few years ago are dead and buried. Standards last. They enable interoperability in an ecosystem of supported technology. They give us flexibility, choice, competition, portability. The world is better when we have solid standards to build on.
Replace it all with a standard. Let anyone implement the standard, implement a client, a server, etc. And let people choose the tooling they want - but while being interoperable with everyone else's.
(Note that I'm not talking about federated social networks. E-mail and IRC are not social networks, they are communication tools, private by default, and have to be directed at specific individuals or groups)
Comparing XMPP/Matrix to Slack is like comparing Telnet to Chrome. Yes, they both display text, they're both interactive. But the latter has about 5,000x more features, which is why we all use it.
Yet with Slack, it doesn't use a standard. Could you build an app like Slack that includes XMPP/Matrix along with a whole lot of other stuff? Sure. But without the whole kitchen sink, you still don't have a standard other apps will follow. You have a proprietary app plus XMPP. Other apps won't be compatible with it. Which is the case with Slack's competitors.
Think of a web browser. It's larger than a kernel. It's probably the biggest, fattest, meatiest, most feature-rich application in the world. (And it should be, because it's a freakin' application platform at this point.) But it all runs on.... standards! Every part of it. I'm saying, do that, but for the massively feature-rich, complex, large, almost unwieldy, but insanely productive, communications platform that is Slack.
I get that a lot of people don't really understand what the big deal about Slack is. A lot of people thought the same thing about web browsers back in the day. But once they started using them a lot, they got it. It's not just a document viewer, just like Slack isn't just chat.
XMPP exists as a standard, and Google Chat was built on it. Then Google+ came along and needed more features, and instead of adding these features to a standard that supports federation (like XMPP or now Matrix), Vic Gundotra (I assume) did the expedient and stupid thing of building Hangouts Chat like Facebook Messenger, commencing the parade of throwaway Google communications products to come.
Email and chat are identical in the modern age. The only difference is the coat of paint. Once chat could send attachments, do replies, persist history, have group chats, it supplanted email and has continued to do so for more than a decade. Chat even ate email's lunch in the one area it had a cultural advantage, conducting business.
> Replace it all with a standard. Let anyone implement the standard, implement a client, a server, etc. And let people choose the tooling they want - but while being interoperable with everyone else's.
If you're older than 40, you remember that there did exist an aeon, long, long ago, when people did not use data object serialization formats as config files. When config files were written not to be easy to parse, but to make it easier for human beings to configure software. When nobody believed there was one single good way to do everything. When software was written not to aid computers, but to aid humans.
reply