How E-mail Works

There are several other pieces of information that the server might save into the file, like the time and date of receipt and a subject line, but overall you can see that this is an extremely simple process!

As other people send mail to mbrain, the server would simply append those messages to the bottom of the file in the order that they arrive. The text file would accumulate a series of five or 10 messages, and eventually I would log in to read them. When I want to look at my e-mail, my e-mail client would connect to the server machine. In the simplest possible system it would:

• Ask the server to send a copy of the MBRAIN.TXT file.
• Ask the server to erase and reset the MBRAIN.TXT file.
• Save the MBRAIN.TXT file on my local machine.
• Parse the file into the separate messages (using the word "From:" as the separator).
• Show me all of the message headers in a list.

  When I double-click on a message header, it would find that message in the text file and show me its body.

  You have to admit that this is a VERY simple system. Surprisingly, the real e-mail system that you use every day is not much more complicated than this!

  Understanding the Real E-mail System
  For the vast majority of people right now, the real e-mail system consists of two different servers running on a server machine. One is called the SMTP Server, where SMTP stands for Simple Mail Transfer Protocol. The SMTP server handles outgoing mail. The other is a POP3 Server, where POP stands for Post Office Protocol. The POP3 server handles incoming mail. The SMTP server listens on well-known port number 25, while POP3 listens on port 110 . A typical e-mail server looks like this:

   

  
  
  

  Understanding SMTP
  Whenever you send a piece of e-mail, your e-mail client interacts with the SMTP server to do the sending. The SMTP server on your host may have conversations with other SMTP servers to actually deliver the e-mail.

   

  
  

  Let's assume that I want to send a piece of e-mail. My e-mail ID is brain and I have my account on howstuffworks.com. I want to send e-mail to jsmith@mindspring.com. I am using a stand-alone e-mail client like Outlook Express.

  When I set up my account at howstuffworks, I told Outlook Express the name of the mail server -- mail.howstuffworks.com. When I compose a message and press the Send button, here is what happens:

• Outlook Express connects to the SMTP server at mail.howstuffworks.com using port 25.
• Outlook Express has a conversation with the SMTP server. The conversation is an extremely simple set of text commands and responses (see below). Outlook express tells the SMTP server the address of the sender and the address of the recipient, as well as the body of the message.
• The SMTP server takes the "TO" address (for example, jsmith@mindspring.com) and breaks it into two parts: 1) the recipient name (jsmith) and 2) the domain name (mindspring.com). If the TO address had been another user at howstuffworks.com, the SMTP server would simply hand the message to the POP3 server for howstuffworks.com (using a little program called the delivery agent). Since the recipient is at another domain, SMTP needs to communicate with that domain.
• The SMTP server has a conversation with a Domain Name Server and says, "Can you give me the IP address of the SMTP server for mindspring.com?" The DNS replies with the one or more IP addresses for the SMTP server(s) that Mindspring operates.
• The SMTP server at howstuffworks.com connects with the SMTP server at Mindspring using port 25. It has the same simple text conversation that my e-mail client had with the SMTP server for How Stuff Works, and gives the message to the Mindspring server. The Mindspring server recognizes that the domain name for jsmith is at Mindspring, so it hands the message to Mindspring's POP3 server, which puts the message in jsmith's mailbox.

  If, for some reason, the SMTP server at How Stuff Works cannot connect with the SMTP server at Mindspring, then the message goes into a queue. The SMTP server on most machines uses a program called sendmail to do the actual sending, so this queue is called the sendmail queue. Sendmail will periodically try to resend the messages in its queue. For example, it might retry every 15 minutes. After four hours it will usually send you a piece of mail so that you know there is some sort of problem. After five days, most sendmail configurations give up and return the mail to you undelivered.

  The actual conversation that an e-mail client has with an SMTP server is incredibly simple and human readable. It is specified in public documents called Requests For Comments (RFC) and a typical conversation might look something like this:

  • helo test 250 mx1.mindspring.com Hello abc.sample.com [220.57.69.37], pleased to meet you mail from: test@sample.com 250 2.1.0 test@sample.com... Sender ok rcpt to: jsmith@mindspring.com 250 2.1.5 jsmith... Recipient ok data 354 Enter mail, end with "." on a line by itself from: test@sample.com to:jsmith@mindspring.com subject: testing John, I am testing... . 250 2.0.0 e1NMajH24604 Message accepted for delivery quit 221 2.0.0 mx1.mindspring.com closing connection Connection closed by foreign host.

  What the e-mail client says is in red, and what the SMTP server replies with is in green. The e-mail client introduces itself, indicates the from and to addresses, delivers the body of the message and then quits. You can, in fact, telnet to a mail server machine at port 25 and have one of these dialogs yourself -- this is how people "spoof" e-mail.

You can see that the SMTP server understands very simple text commands like HELO, MAIL, RCPT and DATA. The list of most common commands is:

• HELO - introduce yourself
• EHLO - introduce yourself and request extended mode
• MAIL FROM: - specify the sender
• RCPT TO: - specify the recipient
• DATA - specify the body of the message. To:, From: and Subject: should be the first three lines.
• RSET - reset
• QUIT - quit the session
• HELP - get help on commands
• VRFY - verify an address
• EXPN - expand an address
• VERB - verbose

  Understanding the POP3 Server
  In the simplest implementations of POP3, the server really does maintain a collection of text files -- one for each e-mail account. When a message arrives, the POP3 server simply appends it to the bottom of the recipient's file!

  When you check your e-mail, your e-mail client connects to the POP3 server using port 110. The POP3 server requires an account name and a password. Once you have logged in, the POP3 server opens your text file and allows you to access it. Like the SMTP server, the POP3 server understands a very simple set of text commands. Here are the most common commands:

• USER - enter your user ID
• PASS - enter your password
• QUIT - quit the POP3 server
• LIST - list the messages and their size
• RETR - retrieve a message number, pass it a message number
• DELE - delete a message, pass it a message number
• TOP - shows the top x lines of a message, pass it a message number and the number of lines

  Your e-mail client connects to the POP3 server and issues a series of commands to bring copies of your e-mail messages to your local machine. Generally it will then delete the messages from the server (unless you've told the e-mail client not to).

  You can see that the POP3 server simply acts as an interface between the e-mail client and the text file containing your messages. And again you can see that the POP3 server is extremely simple! You can connect to it through telnet at port 110 and issue the commands yourself if you would like .

  Understanding Attachments
  Your e-mail client allows you to add attachments to e-mail messages you send, and also lets you save attachments from messages that you receive. Attachments might include word processing documents, spreadsheets, sound files, snapshots, pieces of software, etc. Usually an attachment is not text (if it was, you would simply include it in the body of the message). Since e-mail messages can contain only text information and since attachments are not text, there is a problem that needs to be solved.

  In the early days of e-mail, you solved this problem by hand using a program called uuencode. The uuencode program assumes that the file contains binary information. It extracts three bytes from the binary file and converts them to four text characters (that is, it takes 6 bits at a time, adds 32 to the value of the 6 bits and creates a text character). What uuencode produces, therefore, is an encoded version of the original binary file that contains only text characters. In the early days of e-mail, you would run uuencode yourself and paste the uuencoded file into your e-mail message.

  Here is typical output from the uuencode program:

  • begin 644 reports
    M9W)E<" B<&P_(B O=F%R+VQO9R]H='1P9"]W96(V-C1F- BYA8V-<W,N;&]GM('P@8W5T("UF(#(@+60@(C/B('P@8W5T ("UF(#$@+60@(B8B(#X@<V5A<F-HM+61A=&$M)#$*?B]C; W5N="UP86=E<R!/('-O<G0@/B!S=&%T<RTD,0IC< " @M?B]W96)S:71E+V-G:2UB:6XO<W5G9V5S="UD871A+V1A= &$@<W5G9V5S="TDM,0IC<"!^+W=E8G-I=&4O8V=I+6)I;B ]W:&5R92UD871A+V1A=&$@=VAE<F4MM)#$*8W @?B]W96)S:7 1E+V-G:2UB:6XO96UA:6QE<BUD871A+V1A=&$@96UAL:6PM)# $*?B]G971L;V<@/B!L;V=S+20Q
    End
  The recipient would then save the uuencoded portion of the message to a file and run uudecode on it to translate it back to binary. The word "reports" in the first line tells uudecode what to name the output file.

  Modern e-mail clients are doing exactly the same thing, but they run uuencode and uudecode for you automatically. If you were to look at a raw e-mail file that contains attachments, you would find that the attachment is represented in the same uuencoded text format shown above!

  The Simplicity of E-mail
  From this description, you can see that today's e-mail system is one of the simplest things ever devised! There truly is nothing to it. There are parts of the system -- like the routing rules in sendmail -- that get complicated, but overall the system is as simple as it can possibly be.

  The next time you send an e-mail, you will now have the comfort of knowing exactly what is going on behind the scenes!