The Security of Running Your Own DNS & E-mail
At present, I run my own DNS and e-mail servers in a colocated server on hardware I fully control. The main reason for having this server was to have full control over the stack so I could ensure that everything from the mail and DNS server applications themselves all the way to the hypervisor are fully patched. The 2 main reasons are: 1. Not wanting somebody else to get access to the e-mail in my inbox due to somebody else forgetting to patch or being tricked into giving access to my inbox and seeing all the e-mails I’ve ever received, and 2. because if your e-mail or DNS is compromised then almost everything is compromised.
E-mail and DNS are the keys to the kingdom. If somebody controls your DNS, they control your whole domain and can point your records to attacker-controlled servers and issue legitimate SSL certificates. They also get full control over e-mail since your MX records control where your e-mails are to be delivered, and your SPF, DKIM and DMARC records determine who is allowed to send e-mail on the behalf of your domain.
E-mail is also critical: the vast majority of services you use can be reset via e-mail so getting access to your e-mail address is as good as getting access to everything you can reset with that address, which often includes hosting providers and therefore DNS.
However running your own server still leaves you open to the same attacks on the network side. Controlling the IP addresses of your e-mail servers are often enough to gain full control. To be fair, this can be mitigated in many circumstances:
- Let’s Encrypt will validate DNSSEC if it is enabled on the domain, preventing a malicious (by diverting traffic on the network to an attacker-controlled server) from being able to return valid DNS data as confirmed by Let’s Encrypt staff on their forums: https://community.letsencrypt.org/t/lets-encrypt-and-dnssec-verification/213809/5
- You can use CAA records in your DNS to restrict which CA’s may issue certificates on your behalf and what methods they can use to do so, meaning you can enforce DNS-based authentication which itself is protected with DNSSEC: https://letsencrypt.org/docs/caa/
- SMTP between servers is opportunistic by default so gaining control over the IP of the receiving mail server is enough to start receiving their e-mails. However thid can be mitigated using MTA-STS which caused supporting senders to refuse to send to servers that fail SSL validation: https://learn.microsoft.com/en-us/exchange/security-and-compliance/enhance-mail-flow-using-strict-transport-security
For what it’s worth, of these steps the only one I’ve put into practice is DNSSEC, so I may be missing some information here.
Many of these techniques also require the other side to do validation. If the client DNS server doesn’t validate DNSSEC, the client ignores the CAA records (and/or another CA issues a cert without using DNSSEC-signed DNS challenge), or sending SMTP server doesn’t enforce MTA-STS then taking over your DNS or mail server IPs will still be enough to control traffic to your domain.