Video: SMTP Overview
We’ve put together a short overview video on SMTP:
This video is part of a larger video series on all things DMARC.
The transcript follows:
This short video goes through the basics of email and how it flows around the Internet via the Simple Mail Transport Protocol, known as SMTP.
Special emphasis is given to how SMTP relates to the topic of DMARC, as this video was created by the makers of dmarcian to continue to help spread DMARC around the world.
SMTP is used to deliver email between servers when a piece of email needs to travel across the Internet. The handoff of email between two different servers is what SMTP accomplishes, and SMTP does this using a very conversational set of commands. That is, servers talk to each other, exchange bits of information about themselves, they exchange bits of information about the email that will be handed over, and then the actual piece of email is communicated to the receiving server. This back and forth is referred to as the SMTP conversation.
Before diving into the guts of the SMTP conversation, it’s important to note that a lot happens before SMTP comes into play. First, a piece of email is composed by a person or a piece of software that is churning out email. The person or the software then hits SEND, which causes the email client to connect to an outbound email server. If you’ve ever used an email client, you’ve likely seen the configuration or preferences screen that asks for the server and username+password for sending email. Those credentials are used when the email client connects to the outbound server, and is how the outbound server knows that it is OK to deliver email on behalf of the client.
The outbound server then does something interesting. It determines where the piece of email is going, and it does this by looking at the recipient’s email domain. Once the recipient’s domain is known, the outbound email server asks the global Domain Name System (almost always referred to as “the DNS”) for a list of servers that accept email on behalf of the recipient’s domain. When the list is returned, the outbound email server can then connect to the server that will accept the recipient domain’s email. When that connection is made, the SMTP conversation begins. Keep in mind that the “S” in SMTP stands for Simple!
The SMTP conversation is a series of commands that the outbound email server sends to the receiving server. Each command is replied to by the receiving server, and the replies consist of numeric codes followed by regular text messages that are intended to make people’s lives easier should they ever have to dive into the special hell known as an email log to figure out what happened to a CEO’s email that probably had something to do with dinner parties or scandals involving electric cars.
We’ll walk through a sample conversation. This sample is using “smtp.sample.net” as the outbound server, in blue at the top, and “mail.example.org” as the receiving server, in green. The conversation flows from top to bottom, as indicated by the giant arrow going down the middle of the screen. The very first command that the outbound server issues after connecting to the receiving server is the HELO command, which includes the name of the outbound server. This is sort of like introducing one’s self at a party… Hello My Name Is Outbound Email Server.
As a side note, this is just a sample conversation to show how it works. In the real world, the “HELO” command has largely been replaced with the “EHLO” command (E-H-L-O), which allows the receiving server to pass information about the receiver’s abilities back to the outbound server as part of the reply to the EHLO command. Extensions to SMTP are added every once in awhile, which keeps email fresh, but also turns the “S” part of SMTP into a bit of a joke.
Alright, back from the side-note. The numeric code that the receiving server uses to say “OK, go ahead” is 250. In response to the HELO command, the receiving server here returns 250 and includes it’s own server name. This is like replying to “Hello My Name Is Outbound Email Server” with “Hi There, my name is Receiving Server, let’s talk“.
The Outbound email server then expresses its desire to deliver email by telling the receiving server where email is coming from. This is the MAIL FROM command, and we’re using “[email protected]” in this example. In response to this command, the receiving server sends a reply that says “250 – this sender is OK in my book, go ahead”. Happy with this reply, the outbound server continues and tells the receiver about the recipient of the email. This is the “RCPT TO” command, and the example address is “[email protected]”. Because of the pre-connection dance that was done prior to the SMTP conversation, the outbound server can expect the receiving server to accept email on behalf of the recipient domain—otherwise the outbound server wouldn’t have connected to this receiving server!
So the receiving server replies with 250, and that the recipient is in fact OK.
To finish off this conversation, the outbound server sends the DATA command to tell the receiver that the actual content of the email is coming up, which the receiver acknowledges with a numeric code of 354, the email content is transmitted, and then the outbound server disconnects with the QUIT command.
Note that after the email content is delivered, the receiving server can add stuff after the 250 response. This little bit of extra information is very useful when humans need to get involved to figure out what happened to a piece of email. The operator of the outbound server can contact the operator of the receiving server and ask “what happened to the piece of email that you called 17,763,873?”. The operator on the other end can then track down the fate of the email using the supplied number.
In stripped down form, this is how SMTP works. After the receiving server accepts the piece of email, the email is likely subject to things like anti-spam scanning, DMARC checks, and then maybe on to delivery to an inbox.
What about the actual piece of email? What does that look like?
A single piece of email consists of two sections – headers and the body.
The section containing headers is always at the top of the email. Headers themselves are tag/value pairs where the tag part is the name of the header followed by a colon, and the value part is everything after that. Values can span many lines as long as the continued line starts with white space. In the example here, the first two headers—Return-Path and Delivered-To—are single-line headers, whereas the third one—Authentication-Results—spans 3 lines in total.
A small set of headers are added to an email by the original sender, but as email travels across the internet, the servers that touch the email can add headers. Usually headers are added to make it easier to figure out where the email has passed through, but headers can also include extra information that a receiving server deems useful.
The Return-Path header is a bit special, and is worth noting. It contains a copy of the email address that showed up in the SMTP MAIL FROM command. The Return-Path header is populated by the receiving server, which means that if a piece of email goes through multiple servers on its way to being delivered, the Return-Path header can be modified multiple times. No one said the content of email was simple—it’s not S-MAIL, or SMAIL—so we’ll let it pass.
The last header marks the beginning of the body of the email. In most email clients, except for headers like Subject, From, and Date, most headers are suppressed. The body of the email is what people read. This example is very simple, but most modern pieces of email use MIME (or Multipurpose Internet Mail Extensions) to extend the formatting and content-recognition capabilities of email. More information about MIME is left as an exercise for the listener!
Knowing about SMTP and the structure of email is important when wrapping one’s head around how DMARC works from a technical perspective. DMARC ends up creating a link between the content of an email and a domain using either SPF or DKIM, something that is expanded upon in the DMARC technical overview video.
To get started with DMARC, visit dmarcian.com.
Questions? contact [email protected]
Thank you for watching!
