Twitter
Facebook
LinkedIn
Google
Flickr
YouTube
RSSWhy is backing up my e-mail so difficult?
Jun 29
Personal 3 Comments
backup, e-mail, imap, imapsize, mbox, restore, thunderbird, ubuntu, windows, wine
Since I began using Ubuntu I have been taking great joy in automating the download of back up copies of the data I have stored on-line, e.g. the database and files for this very website. However, there is one back up job that has refused to be automated – backing up my e-mail. So this post isn’t going to be my usual how to type of post, it’s going to be outlining what I want to do, the problems that have stopped me getting there.
To start with, all of my e-mail is up in the cloud, because I use IMAP. I don’t want to change this, because with POP I would have to have all my e-mail only on my home computer. That’s no good for me trying to find something in my e-mail on my phone or the web when I’m away from home. Although, I trust no server, so I want to know that if my IMAP server goes boom for whatever reason, it’s okay because I have a copy of all my data at home ready to be uploaded to another server.
My favourite IMAP supporting E-Mail client is Thunderbird (because it’s open source and cross platform). Thunderbird has off-line storage for IMAP, which is great. All I have to do to get an up-to-date copy of my e-mail is load up Thunderbird and tell it to go into off-line mode. It will then download any messages it doesn’t have, and also delete any messages I’ve deleted on the server.
This is great, although there’s two problems. Firstly, there’s no way to automate this. Secondly, if for whatever reason you loose all the e-mail on your server, you can’t use your local copies to synchronise back to the server, using Thunderbird. Instead, Thunderbird only synchronises server to client. So if there’s nothing on the server, it’ll wipe your off-line copies.
Thunderbird does support upload of messages to IMAP servers, so I had thought that when needed to do an emergency restore of my e-mail, I could simply use my file manager, navigate to my Thunderbird profile, and copy my IMAP mail files, to Thunderbird’s “local folder” store. Although, for some reason Thunderbird can’t read these properly, and you see that after doing this, the message count is wrong (and some strange malformed messages appear), so this method will cause the loss of some messages.
I had to think of something else.
IMAPSize
If you Google around for “IMAP backup”, you will invariably come across the Windows application “IMAPSize”. This is a great program that I have used before. I even discovered that IMAPSize supports command line options, and if you can run something by command line, you can set up a schedule for automation! I should add here that I’m in Linux so I had to use IMAPSize through WINE, and I found that IMAPSize was very slow to download messages, and this might have been because of using it through WINE, I don’t know.
Performance aside, I hit a short coming of IMAPSize. While it does indeed perform incremental back up of your e-mail, it doesn’t actually remove local messages that have been removed on the server. I verified this by creating test messages, running IMAPSize backup, then going back to the server to delete and expunge them (just to be sure), then re-running the IMAPSize backup. After checking, I could see that the deleted messages were still in my local backup.
So, this is where I am, I am stuck manually backing up my e-mail every few days, by means of having Thunderbird perform the local synchronisation I require (i.e. matching all changes on the server). Then the only progress I made was that I now know that I have IMAPSize to upload/restore the messages that Thunderbird downloads.
Although, this is a two app solution that I have to do through the graphical user interface. If only IMAPSize would remove messages that are not present on server, and if I could get it to play nicely with Linux’s cron scheduler, I would be happy.
It’s amazing to me that there isn’t a command line program that does what I’m looking for. I’ve asked a lot of people, who know far more about such things than I do, but everyone seems to say the same thing to me, “Don’t know of one but it’d sure be useful“.
So, if you know of something that’ll do what I’m looking for, please let me know in the comments.
It all boiled down to running out of storage quota on my server. My server account had crept over its quota limit without me knowing. Things were still working like this, but since WordPress 2.8 (the software that runs this website) came out on Thursday I ran the upgrade, and then we hit problems.