[ Main / Projects / Docs / Files / FAQ / Links ]
When my ISP (cox.net -- I don't reccomend them) callously decided to block incoming SMTP (TCP port 25) for all of their customers, I was no longer able to receive emails. I'm not sure why they made this decision, as it does not reduce spam; outgoing SMTP would be the correct thing to block were they wishing to block spammers. To understand why this port block made me unable to receive my email, a bit of history may be necessary.
In the UNIX world, the standard way of forwarding email from a server machine to a client machine is simply by using .forward files. It's an elegant solution; the server simply opens a SMTP connect to the client and passes along the mail. With this arrangement, email is quickly delivered, as it is "pushed" to the client. Many people use "Instant Messenger" programs, but email, when correctly set up, was the original instant messenger. Sadly, since most people are forced to passively grab their email over insecure POP3 connections, they're unable to appreciate email as it was meant to be used.
Being something of a perfectionist, I set out to create a setup that would at least partially emulate the results of the traditional UNIX .forward.
After I bit of thought, I came up with a setup that I thought would meet my requirements:
The main goal is to actually pull the data from the remote host and deliver it to the local mail structure (which in my case is a Maildir; you might be using mbox or MH). There are two programs that I know of for this task, the venerable fetchmail and the newcomer, getmail.
Fetchmail works by rewriting the mail headers of retrieved mail and passing it to the local MTA for delivery. It supports a wide range of mail retrieval protocols and is very flexible. It's written in C and is very fast. It can natively run as a daemon and pull mail automatically without needing a crond.
Getmail is newer and more simple. It directly delivers mail to the local filesystem, bypassing the MTA. Getmail does not rewrite headers and can optionally pass mail to a local MDA of your choosing for filtering and delivery. Getmail doesn't support as many protocols as fetchmail, and it's a bit slower since it's written in Python. As an advantage, since it's written in Python, it isn't susceptible to buffer overruns and memory allocation problems (although format and input validation problems are still possible flaws).
I used fetchmail in my setup for a while, but I soon decided to switch to getmail. Getmail doesn't mangle mail headers and isn't so susceptible to causing mail loops. Fetchmail is probably the better choice if you would like a solution that is easier to set up or you have to retrieve your mail using a protocol that getmail does not support.
Before you use either approach, you'll want to make sure that you can log in to your mailserver using RSA authentication. Otherwise, you'll have to type in your account password whenever you retrieve your email, which will probably preclude automated retrieval.
Fortunately, RSA logins are easy to set up; you may already use it as your ssh authentication method. If you don't have to type a password to log in to your mail server via ssh, then you should just go on to the next section.
If you need to set up RSA authentication, first it'll be necessary to generate a RSA key on your local machine. There is a choice involved here. If you log in to your mail server using ssh1, then you'll have to use ssh1-style RSA. If you're using ssh2, you'll have a choice between DSA and RSA. I would suggest using ssh2 DSA authentication if you can. In any case, it's not much different to set up individual methods.
If you're using ssh1, then you should make sure you're logged in on your local machine with your normal user account and invoke:
ssh-keygen -t rsa1
Save the private and public keys to the default locations. Do not set a password on the private key. ssh-keygen will create a file named "identity.pub". You'll want to copy it to your mail machine using scp:
scp identity.pub REMOTEUSER@REMOTEHOST:~/
Now you should log in to your remote host and append identity.pub to your "~/.ssh/authorized_keys" file:
cat identity.pub >> ~/.ssh/authorized_keys
Now try to log in to your remote host again. You should be able to log in without typing a password. If so, then you've successfully set up RSA authentication.
If you're able to use ssh2, then the procedure is almost the same. Instead of using "ssh-keygen -t rsa1", you'll want to use "ssh-keygen -t dsa". ssh-keygen will create an id-dsa.pub file. Append it to your authorized_keys file on the remote host as above and everything should work fine.
If you get stuck, I would suggest man ssh-keygen. The OpenSSH manpages are quite informative and easy to read.
Now that your login is working properly, we can proceed.
If you choose fetchmail, I would suggest a setup similar to this one:
.fetchmailrc
defaults
user REMOTEUSER is LOCALUSER
no rewrite
set daemon 600
poll localhost with protocol pop3 and port 15551:
preconnect "ssh -C -f REMOTEUSER@REMOTEHOST -L 15551:REMOTEHOST:110 sleep 5"
password REMOTEPASSWORD;
With this setup, when you invoke fetchmail with your normal user account, it will fork into the background and retrieve your mail once every ten minutes (600 seconds) with no other intervention required. Make sure that you change the items in all caps to suit your individual configuration.
If you need more help with a fetchmail-based configuration, I would suggest that you take a look at the Secure POP via SSH mini-HOWTO as it goes into far more detail than I do.
I use getmail in my own setup because I like not having to deal with the risks and lossage of mail header rewriting. I also feel a bit more confident in its security. If you do decide to use getmail, be aware that it takes a bit more work to achieve the same result as the fetchmail approach I presented before.
First, it's necessary to write a script that will use getmail to pull mail from the remote server. In my case, I use POP3 over a ssh tunnel, so I use this script:
grabmail
#!/bin/sh PREFIX=/usr/bin LOGIN=YOURLOGIN HOST=YOURHOST $PREFIX/ssh -C -f $LOGIN@$HOST -L 15551:$HOST:110 sleep 5 $PREFIX/getmail
Obviously, this script will need to be modified to suit your own setup. The ssh invocation creates an encrypted tunnel from port 110 on the remote machine to port 15551 on the local machine. The script then runs getmail to actually grab the mail from the remote server.
Before this script will do anything, you'll have to set up your "~/.getmail/getmailrc" file:
getmailrc
[default] verbose = 0 [HOST] server = localhost port = 15551 delete = 1 username = USER password = "PASSWORD" # if you want to directly deliver to a Maildir without a MDA, use this: #postmaster = ~/Maildir/ # if you want to use maildrop as your MDA: postmaster = "|/usr/bin/maildrop 2>/dev/null"
Once again, you'll want to modify this file to suit your own configuration. Make sure that it's not world and group readable, otherwise someone else with an account on your local machine could steal your password simply by reading the file.
At this point, you'll want to test the grabmail script. If it doesn't work, you should fix it before you proceed.
Getmail isn't able to daemonize itself into the background and grab mail at regular intervals on its own, so it's necessary to write a script that will be run at regular intervals by a cron daemon. We've just written the script that you'll be using. Now all that remains is to set up your cron daemon. Your system may already have a traditional cron daemon, and if it does, you can use it if you know how. I don't trust traditional cron daemons very much, so I wrote ncron. The setup I present here uses ncron.
I'll assume that you already have ncron installed and working. If not, I suggest checking out its documentation. It should be fairly informative. Automating the grabmail script should only require you to add a job entry to your "/var/lib/ncron/crontab" file. An example crontab that does nothing else would be:
crontab
!1 command=/home/LOCALUSER/bin/grabmail interval=900 user=LOCALUSER group=LOCALGROUP !0
Of course, this crontab is only an example and will have to be modified to suit your configuration. This job runs grabmail every 15 minutes to retrieve your mail. If ncron is already running, you should send it a HUP signal to reload the configuration file:
kill -HUP `pidof ncron`
At this point, you should be done.