Sending receiving email

Created: 2017-10-29 17:09:40 -0700 Modified: 2020-04-17 14:20:25 -0700

Basics

CAN-SPAM act

https://www.ftc.gov/tips-advice/business-center/guidance/can-spam-act-compliance-guide-business

Email types

Commercial content – which advertises or promotes a commercial product or service, including content on a website operated for a commercial purpose;

Transactional or relationship content – which facilitates an already agreed-upon transaction or updates a customer about an ongoing transaction; and

If you have transactional emails only, then you don’t need to worry about the CAN-SPAM act.

MX records

MX records in DNS are for receiving email, not for sending them. An MX record points either to an A record or an AAAA record, so for example, I made my MX record point to “bot.land” originally, then I had an A record for “bot.land” to point to the IP address of the mail server. At some point, I had a problem with this because I have DNS pointing to AWS servers and another site, so I had to change my MX record to point at “share.bot.land” since that IP address had my mail server. This change happened almost immediately.

Apparently an MX record isn’t absolutely necessary (since it seemed to work for me), but it’s not a bad idea to have one.

You can check to see what your MX records are set to using this page.

DKIM / SPF records

For sending emails, DKIM and SPF are used to verify that the sender is actually authorized to send for a particular domain.

An SPF record basically says “if you receive an email from bot.land, the sender’s IP address needs to match whatever address was listed in the “include:” record below (amazonses.com)“.

To set up DKIM:

Go to SES in the AWS console
Click your domain
Click “View details”
DKIM —> “Generate DKIM Settings”
If you’re using Route 53, you can click a button below that one that will automatically create the records in DNS.
- You’ll see them as CNAME records in Route 53.

To set up SPF records (reference):

Just add this as a TXT record on your domain
- ”v=spf1 include:amazonses .com -all”
- (I copied the quotation marks and everything)
- Also note: Leaf says that if you want to update this to be able to send from other domains that you should edit the existing one rather than replace it with a new one (“you can add as many “include:” for the domains you want to include).

You can validate SPF records using something like mxtoolbox (← warning: don’t stream this since it shows your IP at the bottom)

You can also use a site like SPFWizard to build the SPF string.

Note: once you’ve set up SPF, then you won’t be able to send email from cPanel any longer.

Checking emails / regex

2:42 tsjost: the proper way to do it is to check if it contains a @, then check the domain name for MX records, and then connect to the server to see if the mailbox exists

That’s pretty much it; there’s no catch-all regex (nor should you try to have one) and there’s no other way than checking for MX records to see if you should try to deliver the email. I’m assuming that you’d want to cache whether a particular domain can receive emails so that you don’t need to keep looking up their MX records. I also imagine that using email as a service from some other company would be the way to go about doing this.

Forwarding email

As of 10/29/2017, it doesn’t look like Amazon handles email forwards automatically, so I did these steps:

Set up the email address on my shared host (through cPanel)
- Add a noreply@bot.land account
- Go to “Forwarders” in cPanel and “add forwarder”
- Forward email to “<personal address>+botlandnoreply@gmail.com”, that way I can filter them there.
- Test that I set it up correctly by emailing noreply@bot.land from my personal address. Note: this did seem to take way longer than it should have just for the email to arrive in my webmail through cPanel, and I don’t think it ever forwarded the first email (but the second one worked).
In AWS Route 53, I added an MX record
- Name: blank (so that it’s just bot.land)
- Value: “10 bot.land.”

Reverse DNS

I didn’t end up needing to do anything here, but I wanted to write it down in case I ever needed it in the future.

3:03 UseWorkNL: Also dont forget to set Reverse dns

3:04 UseWorkNL: Reverse DNS is IP address to domain name mapping - the opposite of forward (normal) DNS which maps domain names to IP addresses.

3:05 UseWorkNL: Read more: http://manuals.gfi.com/en/kerio/connect/content/server-configuration/mail-delivery-and-dns-records/essential-dns-records-for-mail-delivery-and-spam-protection-1223.html

3:06 nperry0: if using SES, you wont need to worry about reverse dns

3:07 nperry0: though you can get a static ip from ses, and do the reversing on it - but its only worth on enterprise level.

3:07 UseWorkNL: @nperry0 Yeah, SES will setup reverse-dns for you: Amazon SES users do not need to set up reverse DNS records. Amazon Web Services manages the IP addresses used by Amazon SES, and provides reverse DNS records for these addresses.

Handling bounces properly

If you get a bounce, you need a blacklist set up to make sure that you don’t contact that email address again. There are different kinds of bounces: hard bounces (message will never be able to be delivered), soft bounces (temporarily can’t deliver, maybe you’re sending too fast), spam complaints. Leaf doesn’t recommend trying to write this stuff from scratch (although maybe it’s not terrible; check this page). Bounces can be caused when an email address doesn’t exist or if you try sending too many to the same mail server; it’s just a broad term to mean that the mail server rejected your email address.

Leaf has never used something like this, but he’s heard good things about it: sendy (paid).

Using an external service

If I’m going to use something like Zoho then I would need to point my MX records at Zoho. They should teach you how to set that up. BTW: Zoho has a free plan if I need to use it (scroll down on the pricing page).

Leaf’s nightmare story

Leaf had a catch-all email so that “madeupaddress@hisdomain.com” would still get delivered to his address. He also had an SPF record that was a little more permissive; he used “+all” instead of “-all” so that emails could still be delivered even if they didn’t match the SPF record, so maybe that painted a target on his back. It led to a backscatter attack that caused him to disable the catch-all email address.

He also linked this just for fun: https://en.wikipedia.org/wiki/Joe_job

Troubleshooting

Deliverability issues

If I have deliverability issues, I may need DKIM and SPF records. You can check everything using http://mxtoolbox.com/

Note: that site can check basically anything, but you first need to do a search for the dropdown to be enabled.

”not allowed to send from <domain> per its SPF Record”

I got this error from Valve’s third-party background-check company:

SMTP error from remote mail server after RCPT
    550 5.1.1 198.252.100.72 is not allowed to send from <bot.land> per its SPF Record. Please inspect your SPF settings, and try again. IB508 <http://x.co/srbounce>

Originally, my SPF record looked like this:

“v=spf1 include:amazonses .com -all”

…however, I was trying to send mails from my cPanel host (via Gmail). To allow a cPanel sender to use its own domain to send emails (instead of amazonses.com), you’ll have to modify your SPF record to add another “include” directive like “include:otherdomain .com”. Even if you use something like Gmail via the cPanel sender, the IP address will be cPanel’s.

After updating my SPF record to “v=spf1 include:amazonses .com include:botland .net -all” and verifying that it “took” via mxtoolbox, I sent another email and it worked.

Note that at the time of writing, nslookup shows botland.net resolving solely to 198.252.205.45, not 198.252.100.72 as mentioned in the bounce email, but everything still worked fine.

Time drift

If you get this error: “SignatureDoesNotMatch: Signature expired”, it probably means that your VM has time drift issues. It’s super easy to fix in ~3 minutes by following the steps here (note: I used the Ubuntu steps on Debian).