The journey of an email - from sender to receiver and everything in between

Background

In my previous role, while working in the Insurance domain, we provided a mobile app to all customers who bought cyber insurance. This had some cool value added services that helped small enterprises check if their employees’ knowledge of cyber security is good enough to protect the organisation from being compromised.

Thus we implemented a one-click phishing campaign solution, which enabled users to click a button and initiate a phishing campaign for the organisation. This was a really fun thing to work on. This was when I invested some time looking into how email really worked. I even wrote most of this post at the time. However, there are many such draft posts that I haven’t shared yet. Because that’s how life happens, constant interruptions with small moments of peace and I didn’t deprive myself of sleep to publish blog posts either.

Anyhow, while looking at my long lost git branches of blog posts, this one caught my attention and I thought I’d finish off what I started and here I am.

There was and still is a lot that I didn’t know about emails. And I am certain I have only scratched the surface of the concepts that matter. But I thought it was worth sharing what I researched here so that others who come by get to see and read it all in one place!

Alright, let’s get started then.

Email Protocols

These are protocols that enable emails to be sent and received via the internet.

There are at least two protocols required to enable emails:

• SMTP - Simple mail transfer protocols
  • can send email messages and attachments
• POP - Post Office Protocol or Internet Message Access Protocol (IMAP)
  • used to retrieve emails from an email server

How does email reach its destination?

SMTP makes email possible. If you want to know all about it, you can read the relevant RFC. The original specification for SMTP was published under RFC 821 and later further specifications were introduced in RFC 5321. I personally haven’t read it fully.

It can be a difficult read if you aren’t used to reading much. So I am going to summarise it for you.

Let’s draw the actors out.

Let’s learn more about these actors.

User Agents

The email client you use to send emails - could be something very GUI oriented or something simple and light weight and accessible via command line based on Text. This is what enables you to read, reply, compose, forward, save email messages.

Mail servers

Stores user mail boxes, communicates with local user agents and other mail servers.

SMTP

The application layer protocol that allows emails to be sent on top of TCP.

The journey of an email

The user agent on the left is the sender’s email client. When the sender writes and email and clicks send, the email client uses SMTP protocol to figure out what the recipients email server is based on the domain in the email address.

The Handshake

The SMTP handshake is to ensure that the client and the server get to know each other - who’s who and understand what services they provide and understand the identities of the sender and the recipient. This part also confirms if the recipient exists on the target email server and responds with relevant acknowledgement to the sender’s email server.

After figuring that out, the sender’s email server establishes a TCP connection to the receiver’s email server. Then they start something called an SMTP handshake.

The message is sent

SMTP, like HTTP has a few key commands and this is where the sender sends DATA command to the server and awaits a specific response code. On receiving the code, the sender sends the email. At the end of the email, the server sends something that tells the destination email server that they’ve got the end of the email.

Then the destination email server, acknowledges the receipt of the email stores it in a place from which the recipient can read their email.

End the connection

The source email server sends a QUIT message and closes the connection to the destination server now that the email was sent.

Email protocols

SMTP again

This is a push protocol that’s used for sending the email. You cannot pull emails from the server using this protocol.

Historically for reading emails, you had to login to your local email servers to read emails. Pretty hardcore.

POP3

This is a basic, lightweight protocol that has limited functionality. It uses the TCP port 110 and it has been around for a very long time.

Two popular modes in which POP can operate:

• download and delete
  • gets messages from the server and stores it locally
  • deletes messages from the server
• download and keep
  • keeps pulling emails from the server, however doesn’t delete them from the server.

POP Limitations

• It was designed for a time when emails were sent from one computer. Not great for a multi-device experience. Why? you ask. Read on.
• Leaving emails on the server means, actions performed on the emails in your local machine, isn’t reflected on the server! Making it hard to get back to what you were doing, if you switched machines!
• Pull emails is all is does - cannot organise emails on the server, no concept of remote folders.

IMAP

Email is not downloaded in this one. But it is always kept on the server. However, you manage it entirely from your local client - INBOX. Operates like a client having an API to organise and do things with emails on the server.

This being a newer kid on the block, allows users to create remote folders. They can also retrieve portions of the email - enabling previews of the email!

Web based emails - HTTP

Web based emails became a thing with Hotmail in 1990s. In this case, you access emails on the server via HTTP - using your browser. HTTP is used to push and pull email to and from the server.

As you know, the advantages are that you can access your email from anywhere, any device, so long as it has internet connectivity.

Email Spoofing

Like with any digital communication medium, emails are also vulnerable to malicious actors. A hacker could send an email from anywhere in the world, pretending to be you. Imagine if a naive recipient thought it was you and the contents of the email were not something you would ever send!

This impersonation of email is called Email Spoofing.

This is relatively simple to do as all it takes is to modify the header information of the email. The email header contains metadata regarding FROM, REPLY-TO and RETURN-PATH. SMTP being a simple protocol, doesn’t make any provision to authenticate an email address. So hackers can use this to their advantage.

Let’s look at a couple of email security controls you can setup to ensure that your email address cannot be spoofed easily by a malicious actor. All this is specifically for cases where you have a custom domain associated to the email address.

If you have ever bought and configured a custom domain with an email address, you might recall having to input little pieces of text into your domain provider’s DNS records. Some of these mechanisms are trying to do what is called Email Authentication.

Sender Policy Framework (SPF)

Not sun protection factor!

Suppose person X sends an email posing as person Y from y@mydomain.com using their personal naughty email server. How could the recipient email server tell that the email originated from mydomain.com?

SPF links sender server to a domain on the sender’s DNS. This helps recipient email servers to ensure that the server where the email originated from was actually authorised to send the email. If you have something like an on-premises email server, then your SPF record would include the IP address of your server. However, if you are using a cloud based service like Microsoft 365, then the SPF senders would be the servers of Office 365.

A server admin configures this by adding a TXT record with a certain configuration value to the DNS. The recipient email server then checks the SPF record of the DNS to ensure that the sender server is authorised for sending emails with that domain.

An SPF record in your DNS would look like:

Type: TXT Record
Host: @
VALUE: v=spf1 include:office365.com ~all
TTL: Auto

SPF in Pictures

Thus if the email claims to have originated from an email server that belonged to mydomain.com, the recipient server verifies if the message really came from outbound email servers belonging to that domain. but the outside address states it came from a different IP address.

In fact this is a limitation of SPF. If you were to configure your email server to forward all emails to another server. Then SPF would fail at the last receiver - because the originating IP to the last receiver was different from the actual origin IP and that doesn’t exist in the DNS of the original sender!

DomainKeys Identified Mail (DKIM)

Where SPF was checking if the sending server is authorized to send an email on behalf of a domain, DKIM checks if the email has been tampered with since it was sent!

This method uses digital signatures to link emails back to the original domain using asymmetric encryption. All outbound emails from a domain will be signed with a specific key - a private key. A corresponding public key is published to the DNS of the sending organisation.

1. Sending server, publishes their public DKIM key to their DNS.
2. When the sender sends an email, the email client signs the email using their DKIM private key. The signature could include components of the from, body, subject and many other fields. These fields must remain unchanged in transit for DKIM validation to pass. DKIM-Signature is the header field in the email that contains all the relevant information required by the receiver to validate and verify the DKIM message like
   • The fields used to cryptographically sign the message at the time of sending
   • The email domain that signed the message
   • A DKIM Selector, which is what helps the receiving server to select the right public DKIM key for verification. Domains may have multiple public DKIM keys and hence it is important that the receiver uses the right one for decrypting the message.
3. The receiving server then sees that the email was signed using DKIM because that information is present in the email header in the field - DKIM-Signature.
   1. It validates the version number of the DKIM specification, the identity of the sender’s domain, ensures that FROM field is included in the signature.
   2. Then the receiving server retrieves the public key for sender’s domain using the selector and the domain information in the signature. It then uses the public key to decrypt the encrypted hash sent in the email.
   3. It then computes its own hash
      • If they match, then the message is accepted or received, else the message is quarantined or rejected based on policy set in the receiving server.

DKIM in pictures

Domain based Message Authentication Reporting Compliance

This is another record published at the DNS registrar. It tells recipients what to do if the email received has failed SPF and DKIM. Thus this is a third layer to the email spoofing prevention layers.

The recipient can do the following with the emails:

• None
• Quarantine
• Reject

You can setup an email address and configure DMARC such that this email address will get a report whenever there is spoof attempt that failed both SPF and DKIM.

That sounds very secure, does it not?

These methods that we discussed work very well in protecting your email address from being spoofed and hence protecting it from being blacklisted by email servers.

However, there is no real way to ensure that the emails you receive are really authentic unless the senders go through the pain to setup one of the security controls that we discussed above. Most large corporations do set these things up as they probably have the resources to do so. But there could be small organisations or even freelancers who might have missed this key step when configuring their custom domain based email.

Summary

I hope that gave you an insight into how email works and also some insight into how to secure your email.

References

• How DKIM works
• How does SMTP work
• How does email work?
• Anti-spoofing protection
• SPF
• Hugo Book theme and shortcodes
• SPF record syntax (http only)
• How DKIM, SPF and DMARC work to prevent email spoofing - with all the details and examples of records