Email is extremely non-brittle, and everyone following the RFC is one main way it is able to be non-brittle. What is naive here, is that you think things are the way there are, for no reason.
Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability.
A 5xx error means 'perm failure'. There are a variety of 5xx class responses, from 'user account deleted' to 'no such domain'. It is, in fact, a working mail system, which responds with 5xx, or 4xx (temp fail) or 2xx (received OK) messages.
A severely broken system is incapable of even responding.
For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it.
By default, MTAs are designed to 4xx(tempfail) on those errors. Loads, and loads, and loads of work to ensure that. Code meticulously crafted. This is how good engineering works. This is how the RFC works.
You either get no ACK for your SYN, because it is so borked, or you get a 4xx if it can run, but has a failure condition.. OR someone did something very, very wrong.
I understand your angst, but the real problem isn't the RFC, or the sender not retrying. The problem is:
- the MTA was up and running
- it had an error condition on its back end
- whomever set everything up, didn't taking into account internal failure conditions, and to respond with a 4xx if that was the case
THAT is where the "good engineering" failed.
Mail accounts are deactivated all the time. Domains are deactivated all the time. 5xx tells us this. Someone replying with 5xx, when they did not mean to, has a configuration issue on their end.
That's all there is to it.
I'll put this another way.
What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again".
I just realised that you may not get the other side of the scenario.
On a 5xx series response, you bounce. Where does the bounce go? Back to the original sender.
This is the 'closing of the loop'. Original sender sees the message was not sent successfully. They can now retry, resend, make a phone call, whatever.
"Hey Bob, what was your email address again? I got a bounce. Ohhh, damn, I typed bod@ instead of bob@. I'll fix and resend.
That's how it works.
That's how it works for:
- a typo in the email address
- when an account is deleted (and maybe your friend has a new email address)
- the remote mail admin did something horribly, horribly wrong, and it bounced as a result
(Horribly, horribly wrong being -- they told their own mail servers that accounts did not exist, who helpfully passed that info on.)
> Email is extremely non-brittle, and everyone following the RFC is one main way it is able to be non-brittle. What is naive here, is that you think things are the way there are, for no reason.
> Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability.
Same people argue that email is a hot mess exactly because it's decades old and comprised of a patchwork of standards. It's not exactly the poster child of how to do this sort of thing well.
> For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it.
Configuration issues can be, and often are, temporary. Also, bugs are a thing.
> THAT is where the "good engineering" failed.
I agree with this.
> What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again".
Of course there's a semantic difference between "come back later" and "go away forever", I'm just arguing that a client that doesn't blindly trust what every connected system tells it is going to more often successfully achieve its goals than one which does.
You're lumping together "email content" with "SMTP". SMTP isn't a "patchwork of standards", it has a very specific RFC for it. SMTP works very, very, very well.
Configuration issues can be, and often are, temporary. Also, bugs are a thing.
This isn't just "a bug" or "a config issue", this is an edge case bug or config issue. I've handled literally hundreds of thousands of mail servers in high avail production, with some of those being extremely high volume.
What you're wanting to change the normal flow of operation, for an extremely rare edge case.
Of course there's a semantic difference between "come back later" and "go away forever", I'm just arguing that a client that doesn't blindly trust what every connected system tells it is going to more often successfully achieve its goals than one which does.
Not true here. In 99.999999% of cases when you get a 5xx response code, the correct thing to do is immediate bounce. This sort of mess, 5xx 'by accident', is insanely rare.
And it's not 'go away forever', it's "this specific mail cannot be delivered, please return it to sender, so they can examine the issue and deal with it".
It's "bring a human into the equation".
How is this an issue? For something which is very, very rare.
> And it's not 'go away forever', it's "this specific mail cannot be delivered, please return it to sender, so they can examine the issue and deal with it".
>
> It's "bring a human into the equation".
>
> How is this an issue? For something which is very, very rare.
You apparently missed the whole point of newsletters automatically unsubscribing users on a 5xx error from the previous discussion that the topmost comment was referring to. Not that you are to blame for missing that because it wasn't restated here, but that's the context of the whole discussion. In this context I also heavily disagree with your statements and agree that among other examples automatically unsubscribing to a single "permanent" error is a brittle system.
If I have sent emails successfully to a certain email address before it is not wise to assume a permanent error on a single response that the RFC specifies as permanent. Email/SMTP is in it's core a stateless protocol, but as a mailing list service I can keep state and provide additional context to errors that can improve my service.
Finally, if you read the RFC 5321 4.2.1. Reply Code Severities and Theory it specifically uses "SHOULD NOT" instead of "MUST NOT" implying that there might be valid scenarios to act differently.
RFC 2119:
> SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.
You apparently missed the whole point of newsletters automatically unsubscribing users on a 5xx error from the previous discussion
There was no previous discussion / topmost thread, as I replied directly to the topmost comment, and this thread forks from that. No mention was made of newsletter unsubscribes thread upwards. Further, I made it quite clear I was discussing SMTP response codes, and not the article directly.
Ergo, your stated context, isn't the context of this whole thread/discussion. Further, from what I see, everyone in this thread is discussing MTAs, SMTP, SMTP return codes.
that the topmost comment was referring to. Not that you are to blame for missing that because it wasn't restated here, but that's the context of the whole discussion.
No, it isn't the context of this thread/discussion, as per above.
If I have sent emails successfully to a certain email address before it is not wise to assume a permanent error on a single response that the RFC specifies as permanent. Email/SMTP is in it's core a stateless protocol, but as a mailing list service I can keep state and provide additional context to errors that can improve my service.
In the context of the RFC, mailing list software should be viewed the same as a 'human being'. Of course it's fine to it to re-send, if the software wishes. Just like it is fine for you to send a mail with your mail client, get a 5xx, and re-try by clicking 'send' once again.
(5xx errors can happen during auth/etc stages too)
However, during this entire thread I've have been peppering the words 'MTA', "smtp server', 'bounce' and more.
Note that the mailing list software you're describing, is receiving a bounce. Bounces only happen from the MTA side. EG, mailing list software doesn't "bounce" anything, ever. Only an MTA does. Without that bounce, most mailing list software won't even know there is an issue.
Others upthread were advocating that MTAs don't bounce on an initial 5xx failure, regardless of my assertions that the client/end user should receive an immediate 5xx perm failure message.
Whether a human, or some automated software, as per my upstream statements, the proper thing to do is bounce back to the sender. 5xx, bounce, return. In 99.9999999% of cases, this is what is needed. This is the majority case.
Then, the mailing list software, the end user, can do as they wish. Including trying a resend. I don't see the conflict here, except it's apparent that a lot of people don't have much in the way of MTA experience. That's not even a knock on them, but it is a bit disheartening to see people suggesting massive MTA behaviour/RFC alteration for no reason.
To speak to mailing list behaviour, as a separate issue from above, repeated hits to 5xx targets will get you blacklisted faster than you can imagine. It's the equivalent of knocking on someone's door, them answering and saying "Sorry, Bob moved out", but you come back all day and night, banging on the door "HEY IS BOB THERE?!"
Yeah. That'll work out well. Just because you can do something, doesn't mean you even remotely should.
I don't really intend to continue the discussion as most things have been said.
Just wanna point you to the context. The topmost comment was referring to his own comment and discussion about the original issue 3 days ago. You can find this here https://news.ycombinator.com/item?id=25438169
So while you didn't have the context, he and probably many others (like myself) did have that context. Again, this is not to blame you, because as I said it wasn't repeated in this thread and you couldn't have known.
> In the context of the RFC, mailing list software should be viewed the same as a 'human being'. Of course it's fine to it to re-send, if the software wishes. [...]
> Then, the mailing list software, the end user, can do as they wish. Including trying a resend. I don't see the conflict here,[...]
A lot of people were arguing 3 days ago that the end user (e.g. the mailing list software) should never try a resend and removing the email immediately (after the first 550 response) is correct behavior and mandaotry by the RFC.
The topmost comment here restated that this is in fact wrong. Based on what I quoted from you here, you're actually agreeing with the topmost comment you were initially disagreeing with.
Here is the relevant part from the topmost comment:
> When this event happened, I said senders need to avoid taking "invalid address" at face value when they've recently succeeded delivering to the same addresses. But despite the RFC saying senders "should not" repeat requests (rather than "must not"), many people had a lot of resistance to this idea, and instead just blamed Google for messing up implementing the RFC.
If you hosted a mail server, would you really want an ever increasing number of mail servers hitting your mail server for invalid email addressess for an indeterminate period of time? The 5xx error code you are returning can't be trusted, after all.
Mail is decades old, built upon millions of hours of work crafting software, RFC standards, and works the way it does, including bounces, to ensure stability.
A 5xx error means 'perm failure'. There are a variety of 5xx class responses, from 'user account deleted' to 'no such domain'. It is, in fact, a working mail system, which responds with 5xx, or 4xx (temp fail) or 2xx (received OK) messages.
A severely broken system is incapable of even responding.
For a mail system to respond with 5xx, when it is internally broken, is 100% a configuration issue. Every MTA on the planet, is designed (eg, postfix, sendmail, etc) to respond with a 4xx TEMP fail message if something is borked. A milter gone bad. A library missing. A full disk. A config issue. An issue forking. Memory limits. All of it.
By default, MTAs are designed to 4xx(tempfail) on those errors. Loads, and loads, and loads of work to ensure that. Code meticulously crafted. This is how good engineering works. This is how the RFC works.
You either get no ACK for your SYN, because it is so borked, or you get a 4xx if it can run, but has a failure condition.. OR someone did something very, very wrong.
I understand your angst, but the real problem isn't the RFC, or the sender not retrying. The problem is:
- the MTA was up and running
- it had an error condition on its back end
- whomever set everything up, didn't taking into account internal failure conditions, and to respond with a 4xx if that was the case
THAT is where the "good engineering" failed.
Mail accounts are deactivated all the time. Domains are deactivated all the time. 5xx tells us this. Someone replying with 5xx, when they did not mean to, has a configuration issue on their end.
That's all there is to it.
I'll put this another way.
What you want to do is make 5xx like 4xx, because you feel there should never be any way for a SMTP server to say "No, really, this email address doesn't exist.. don't bother trying again".