Set up a server with postfix. Don't to this on EC2. Many just blacklist those IPs. Implement DKIM. It isn't too difficult and if you just google for a how to there are plenty. That way, the messages are signed and nice. I would avoid SPF. A lot more people have forwarding addresses now and most forwarding servers don't play nice with SPF and that causes mail to get rejected.

Most importantly, make it very obvious how to stop getting the emails with one click. Users hate messages they can't easily stop. Don't hide it thinking that you'll convince some to stay receiving. They'll send you to SPAM and that's the end of your email server. Make it one click, make it all messages, don't ask for passwords, don't ask for email addresses, nothing. One click opt-out. Anything else and, well, you deserve getting dumped in the SPAM folder.

EDIT: almost forgot, make the address go to a real mailbox if someone replies. Most of the time, the reply will be "take me off". Just click the link from their message and be done with it. Don't tell them how, just do it. It takes less time and makes them happier.

None of the things you've offered as solutions actually guarantee delivery. In fact, I'll wager that emails sent from an email server that's been setup like you propose will not even show up in Hotmail's Spam folder, let alone Hotmail inbox! Having DKIM doesn't do anything for Hotmail or Y! Mail, for example. Spammers use DKIM too. I wouldn't avoid SPF since some European ISPs require it and Hotmail actually does care about it once you actually get past bigger issues that stop Hotmail delivery. And even if one of your test emails does get through Hotmail, a first Spam report (i.e. someone clicks on that dreaded "Spam" button) by one of the recipients will blacklist your whole server and domain, INSTANTLY! And who knows how long it will take you to get off that dreaded spam list. SPF is a must.

In short, setting up your own mail server and thinking that your emails will show up in people's inboxes is naïve. You wouldn't believe what it takes to have a 95% delivery rate. Those email delivery companies have dozens of people who talk daily to postmasters of major email companies and resolve issues.

We went through this process, maybe foolishly, and it took a lot of work and more importantly, a lot of time. And we still have issues with some email services. It will take you about a month to jump through all the hoops to actually have about 80% delivery rate. If I were doing it again, I'd go with a hosted email service instead. Unless you're sending over 10K emails a day, hosted is the way to go. I might make a writeup on the process once I get some free time... let me know if anyone's interested.

We're considering rolling our own email server setup. We send about 30K emails a day in the form of a daily newsletter, and are getting ready to build a custom community update users, that will go to about 10K per day.

A long way of saying I'm VERY interested in your experiences with your setup.

In this world of spam paranoia, I've found it to be way too much trouble to do mass mailings yourself and expect to actually get around all the spam filters. Services like Constant Contact are paid to do just that, so your chances are much better.

I use AuthSMTP for all of my personal and business outgoing email and it's great (I've been using them for almost two years now). I've never had any downtime and they have a simple interface to view sent emails, so I can easily confirm an email was actually sent if I'm in doubt. The one or two times I've interfaced with their support, they seemed technically adept and were very helpful.

For my clients who insist on writing scripts to do their own mass mailings, I always recommend using AuthSMTP and simply writing the script to stay within AuthSMTP's stated volume guidelines (maximum of 50 recipients per single message, though they recommend keeping it to 10). Oh, and they do have a "Show Errors Logged" section that shows you all the emails that didn't go through.

But what I can tell you is that they're on a mailing list that a lot of postmasters are on, and whenever someone has a problem with one of their clients[1], they're very responsive, usually within a matter of hours (sometimes minutes). And that has, without a doubt, kept them in good graces.

So my point here is twofold: A. Constant Contact is, at least, a healthy player in the field, and you're likely to get good throughput with them. B. Don't try to pull any fast ones over these companies, if they don't notice themselves, they'll hear about it very quickly. Make sure you keep your lists clean and you're following every single guideline they give you (and they're usually happy to help you).

[1]. Which happens to any service at some point; they can't exactly hand-vet every mailing that goes through their service.

We're obviously in beta now so the site's marketing copy has a bit of work needed yet, but I'd be happy to get you set up today for a free trial. We've been having superb deliverability with our clients so far and we have a bunch of other nice features you may want to use.

Just email me at matt (at) hellocreature.com

Make sure your SPF works.

Most of the suggestions here are for campaign software, and that might help-- just use a mail merge feature. Such software often has metrics like bounces, opens, spam-responses built in.

Edit: It looks like they do have a programmatic API. Next question: can you do single-opt email? And the cost page makes it look like it's free to send out a single email at a time.

If all this is accurate, it seems like this would be a perfect service. Free, API, single-opt -- so what's the catch?

I do enjoy the API. I actually just spent all day working on implementing part of it in a Wordpress plugin. It's SOAP as opposed to REST, but I guess you can't have everything in life.

I did not double check to make sure that it's exactly what we're talking about, simply trying to help. :)

I realize this first step only works for sending below gmail's per day send.

My plan was to do the following:

1 - use google apps with SPF to ensure my general personal emails gets through

2 - use my web server's local SMTP to send my registration and other user demanded server sent messages. My plan was to add my server's SMTP info in an SPF compatible way (still don't know how to do this part.)

3 - use a service (mailchimp etc.) for newsletters.

Can someone expand on this thinking? Particularly step 2?

In fact, they have an API for transactional e-mails (i.e. ad-hoc ones to welcome a user for example). So you're not just restricted to using them for bulk emails.

It's far superior to Constant Contact's API, which consists of two methods: subscribe and unsubscribe (last time I checked anyway).

Once done, nobody at that ISP is going to get your mail.

That's the advantage big players like aweber have, they can pick up the phone and work out issues like this with an ISP.

BTW, its worth subscribing to campaignmonitor to actually see which ISPs are nuking you globally.