OfflineIMAP in action

Some time ago I bemoaned the lack of a way to have my computer run an automated back up of my IMAP email. To summarise, I had a couple of point and click methods to back up my email. Although, these were both manual methods, which means I needed to remember. No matter how good my intentions, sometimes I’ll forget, so I wanted something I could automate. This naturally lead to me wanting something that would run on the command line, which I could hence run from a script.

I’m glad to say I finally discovered a Python program called OfflineIMAP that would do just this. It won’t run on Windows, but it will run on any POSIX type operating system, i.e. Linux, FreeBSD, and Max OS. My only complaint with the thing is that if you run it manually in a terminal, you can no longer type anything into that terminal – you have to shut down and re-open the terminal window. Also, if you want to run this without having to input your log-in details (i.e. unattended executions) then you have to have them stored in a clear-text configuration file.

Other than these niggles, it is perfect for my needs. I can run it from my cron scheduler, and it synchronises with my IMAP server. This is actually better than I wanted, as synchronisation means I can use this to upload any messages I’ve archived away from the server, but want to restore.

If you’re interested in trying this, I’ll just mention that I don’t directly call it from a cronojb, but I call it from a script, because of the above mentioned terminal bug. (And no, running it like this from the cron scheduler didn’t work, it had to be ran from a script).

sleep 2
/usr/local/bin/offlineimap&
echo ""

References:

• OfflineIMAP – WikiStart – Software.Complete.Org

As the winter draws in, it’s time for wrap up warm with some open source goodness, as the October 29^th release date of Ubuntu’s latest version, Karmic Koala, draws near. So, let’s take a look at what’s new!

Running the installer from the Live CD is the same user-friendly set up that Ubuntu always has. Although in addition to drive encryption that came in with the last version, there is now an added option to format your hard drive with the new Ext4 file system, a new version of Linux’s native file format.

Once the installer gets running, you get to watch a presentation of screens telling you all the key features of the operating system. This something Windows has had for years, and is a great idea to educate new users on just what they can do with Ubuntu.

Ubuntu's new informative installer

Once you’re into the Ubuntu desktop, everything will be familiar if you’ve used previous versions. Although, you can see straight away a fresh set of icons have been used, and the default wallpaper is much brighter than previous versions.

Browsing the applications and system menus gives you a view of the key new features to Karmic.

The default IM application has been switched from the long-standing Pidgin, to a new but very similar application called Empathy. I expect few users will really notice the difference here.

Ubuntu's new IM client, Empathy

In the applications menu, you will find the “Ubuntu Software Center”. Desktop Linux distributions have long had software to deliver applications to you. Ubuntu’s has been the “Synaptic Package Manager”. Although Software Centre gives you a far more user friendly interface. The distinction of having a single section named “Free software” hints at Cannonical’s long term goals of being able to deliver commercial software to the Ubuntu desktop. This is a boon for developers. Until now, Linux’s repository method of delivering software has been a mixed blessing for them. The aspiration of new developers is to get their work into as many repositories as possible. However, commercial developers have shied away from this because Linux repositories have offered little opportunity for generating an income.

The Ubuntu Software Center

Confusingly, there is another existing application called “Add/Remove Applications” alongside the Synaptic Package Manager in the System/Administration menu. This appears to offer the same software set as Synatpic does, although again with a better user interface, including popularity ratings.

Ubuntu's Add/Remove Applications window

Finally, we have the addition of “Ubuntu One”, Cannonical’s other embryonic software as a service. Ubuntu One is a Dropbox like service (utilising Amazon S3), available to all Ubuntu users. For free, everyone gets 2GB of storage, but this can be expanded to 50GB for $10 (US) a month, although I couldn’t find where you are meant to authenticate your account. You can also access your files via the web and share specific files and folders with specific individuals. The eventual aim is that you will be able to synchronise all of your application settings across multiple Ubuntu computers. This is a promising service, but from my own research I have yet to find any information about privacy and encryption, which are essential to me for any on-line storage service. On the other hand, the Cannonical deserve kudos for finding innovative ways to generate more income from their desktop operating system.

Ubuntu's "Ubuntu One" cloud storage running.

You can find more screen shots at my Ubuntu 9.10 Flickr gallery.

If all of this has caught your interest, then go Get Ubuntu!

S60 & Linux

I have had another article published on AllAboutSymbian.com. This time I wrote about how to maintain connectivity between your phone and computer after switching from Windows to Linux.

This is based on my own experience after switching exclusively to Ubuntu Linux, and having spent time working out how to still support my mobile phone.

Topics covered include: how to back up files, transfer photos, synchronise your music and connect to the internet via your phone.

If you’d like to read it, here’s the link:

- A survival guide to Linux and S60 -

1. Reliability.
   Of course Ubuntu can go wrong but only when I mess with it. Meanwhile, Windows seems to go wrong all on its own with no help from me!
2. Settings are easier to find.
   Everything you could need to alter is either available through the “System” menu, or right clicking a particular object. Windows requires you to drill through multiple layers of menus.
3. Boot speed.
   Compared to Windows, the boot time of Ubuntu is stunning. See for yourself.
4. Disk space.
   Ubuntu takes at most a quarter of the disk space that Vista and Windows 7 takes. That means you have more drive space to use for your own files.
5. Operation speed.
   Windows soon slows down, Ubuntu doesn’t seem to ever slow down. Everything works as quickly as it did yesterday.
6. Applications are easier to install.
   In Ubuntu, you have the “Synaptic Package Manager” which gives you a point and click interface to choose applications to install – think of it like an App Store. Unlike Windows, where you have to hunt the internet for a .exe to download, then run it yourself. In Ubuntu, it’s an all in one operation.
7. Desktop effects.
   Thanks to a system called Compiz, Ubuntu has more snazzy desktop effects than Windows or Mac OSX have ever had. You can set up the combination that is perfectly useful to you.
8. Regular releases.
   With Ubuntu, you have the equivalent of a service pack every six months. Between those releases, on average, you’ll get bug and security fixes every week.
9. The command line.
   This is a controversial one as I wouldn’t suggest novice users jump right into using the command line (but I would encourage you to learn!). Although, I like how I can achieve – with a single line of code – something that would take a few minutes of point and click work. Windows has a command line, but it’s just not as useful.
10. No DRM, i.e. no Windows Genuine Advantage.
    Because Ubuntu is free there’s no nasty DRM to get in your way when you’re installing the system, either on your own computer, or taking your Ubuntu CD to your friend’s computer.

Get Ubuntu!

Tux, the Linux mascot

I’ve been living in Linux for just over three months, so I thought it would be interesting to report on how I’m finding it.

The first thing I need to stress is that I really do think that the GNOME desktop environment is ideal for non-techie users. You can have “GNOME Launchers” which are just the same as Windows “Quick Launch” icons. The Windows “Start” menu idea is extended more logically as you have three menus “Applications”, “Places” and “System” all self-explanatory and logical for a new user to find what they’re looking for. One little criticism here for catering to those migrating from Windows; to open these menus from the keyboard you have to hit Alt+F1, rather than simply hitting the Windows key (know as the “Super key” to Linux/Unix operating systems). I think having the Windows key launch these menus, by default, would be more useful to novice users who are used to the Windows desktop.

The Super (Windows) key

That covers novice computer users, but for slightly more advanced users, Linux of course has lots of freedom to configure your environment. So much so that the limit to which you can customise is only practically limited by your own skill and willingness to learn. Also, there are the 3D desktop effects. These can mimic those found in Windows and OSX, but there are lots more effects, more than I’ve had time or cause to explore. Some are simply for show and I have chosen to them switch off, most notably the rotating desktop cube and the wobbly Windows. Although the ones I do find genuinely useful are the “Shift Switcher” (particularly for flipping through windows on all desktops), and the  enhanced desktop zoom. Other subtle effects for minimizing and maximising windows just add another layer of class to your overall experience.

A BASH terminal

Now, you can’t talk about Linux without mentioning the command line. I accept that this is not a tool for novice users, and that there is a command line interface to Windows too. Comparing the CLI’s of both platforms,  have to say, subjectively, that the Linux/Unix CLI just makes more sense to me then Window’s pseudo-DOS CLI. If I were trying to do tech support for a novice computer user of Linux, it would be a last resort for me to tell them to open the command line. Having said that, I think that if you are doing your own research into a technical problem, the ability to copy and paste some code from a trusted website into your terminal is much more efficient than following some Windows guide with a long set of “point and click here” type instructions. I am neither a computer science grade user, or a novice user, I’m probably somewhere on the advanced side of a medium level user. As such, I have really enjoyed having the command line and writing bash scripts and cron jobs to automate tasks for me. During these three months, it has really made me feel like I was using a computer again rather than just a glorified internet appliance, which is what computing of recent years seems to have turned into.

It is all these things that I do through scripts that I now find Linux indispensable. If I were to go back to using Windows full time, I’d probably need to run a Linux virtual machine all the time, because equivalents all of my bash scripts would be a lot more complicated to set up in Windows.

WINE IS NOT AN EMULATOR!

Now for some negative points. I don’t game that much these days, but occaisionally enough to make it a consideration. I’m particularly partial to a game of Team Fortress 2. Now, this game even taxed my laptop in XP, but I did manage to make it run in WINE, but it wasn’t the best gaming experience I’ve ever had. A 800×600 window which slowed to a halt whenever I got into any point-blank range action. Even Cedega was a let down as it wouldn’t even let me install Steam, which is a MASSIVE over sight for a program focused on enabling Windows games in Linux.

Having said that, there are some very nice open source games supplied with Ubuntu, out of the box, which are fun to play if you just want a 15 minute distraction.  My current favourite is a Lemmings clone called Pingus (yes I know, not exactly fun for all you FPS fans!)

Also, compatibility with new hardware is something that worries me. I don’t know what the current situation is, but I know my laptop is only so well supported because it is old. I am currently looking at replacing my laptop in the next 18 months. Since I want to use Ubuntu as my main operating system, I’m going to have to do more homework than I otherwise would, just to make sure the hardware is supported without me having to do a load of command line hacking that would be beyond my current ability.

So there we have it, my experience so far of living with Linux. To sum it up, I’m happy, I don’t want to switch back to Windows. The only reason I’m keeping Windows around is to run software for my smartphone, for using Chkdsk on some of my removable drives, and in the future I’ll use it again for gaming.

Since I began using Ubuntu I have been taking great joy in automating the download of back up copies of the data I have stored on-line, e.g. the database and files for this very website. However, there is one back up job that has refused to be automated – backing up my e-mail. So this post isn’t going to be my usual how to type of post, it’s going to be outlining what I want to do, the problems that have stopped me getting there.

To start with, all of my e-mail is up in the cloud, because I use IMAP. I don’t want to change this, because with POP I would have to have all my e-mail only on my home computer. That’s no good for me trying to find something in my e-mail on my phone or the web when I’m away from home. Although, I trust no server, so I want to know that if my IMAP server goes boom for whatever reason, it’s okay because I have a copy of all my data at home ready to be uploaded to another server.

My favourite IMAP supporting E-Mail client is Thunderbird (because it’s open source and cross platform). Thunderbird has off-line storage for IMAP, which is great. All I have to do to get an up-to-date copy of my e-mail is load up Thunderbird and tell it to go into off-line mode. It will then download any messages it doesn’t have, and also delete any messages I’ve deleted on the server.

This is great, although there’s two problems. Firstly, there’s no way to automate this. Secondly, if for whatever reason you loose all the e-mail on your server, you can’t use your local copies to synchronise back to the server, using Thunderbird. Instead, Thunderbird only synchronises server to client. So if there’s nothing on the server, it’ll wipe your off-line copies.

Thunderbird does support upload of messages to IMAP servers, so I had thought that when needed to do an emergency restore of my e-mail, I could simply use my file manager, navigate to my Thunderbird profile, and copy my IMAP mail files, to Thunderbird’s “local folder” store. Although, for some reason Thunderbird can’t read these properly, and you see that after doing this, the message count is wrong (and some strange malformed messages appear), so this method will cause the loss of some messages.

I had to think of something else.

IMAPSize

If you Google around for “IMAP backup”, you will invariably come across the Windows application “IMAPSize”. This is a great program that I have used before. I even discovered that IMAPSize supports command line options, and if you can run something by command line, you can set up a schedule for automation! I should add here that I’m in Linux so I had to use IMAPSize through WINE, and I found that IMAPSize was very slow to download messages, and this might have been because of using it through WINE, I don’t know.

Performance aside, I hit a short coming of IMAPSize. While it does indeed perform incremental back up of your e-mail, it doesn’t actually remove local messages that have been removed on the server. I verified this by creating test messages, running IMAPSize backup, then going back to the server to delete and expunge them (just to be sure), then re-running the IMAPSize backup. After checking, I could see that the deleted messages were still in my local backup.

So, this is where I am, I am stuck manually backing up my e-mail every few days, by means of having Thunderbird perform the local synchronisation I require (i.e. matching all changes on the server). Then the only progress I made was that I now know that I have IMAPSize to upload/restore the messages that Thunderbird downloads.

Although, this is a two app solution that I have to do through the graphical user interface. If only IMAPSize would remove messages that are not present on server, and if I could get it to play nicely with Linux’s cron scheduler, I would be happy.

It’s amazing to me that there isn’t a command line program that does what I’m looking for. I’ve asked a lot of people, who know far more about such things than I do, but everyone seems to say the same thing to me, “Don’t know of one but it’d sure be useful“.

So, if you know of something that’ll do what I’m looking for, please let me know in the comments.

My reason for using Linux is that it is “open source” software. What does this mean? Well, it basically means anyone, subject to a peer review type system, is free to contribute and to distribute this software. Unlike proprietary software like Windows or OSX, where if you try to modify that software, the company that makes it could take you to court!

I’ve been using it for a whole month now and I haven’t looked back, yet. Due to having a custom Windows rescue partition boot loader I didn’t dare risk it getting over written with Ubuntu’s boot loader (“GRUB”), I decided to install Ubuntu via “Wubi”. This was a straight forward and painless process. The difference with Wubi, is that instead of partitioning your drive, a large file is created in your Windows drive that the computer sees as a drive in its own right. When you boot up your computer you get an option of booting Windows or Ubuntu.

The environment you get in Ubuntu is called “GNOME”. If you’re used to Windows, it won’t be too confusing to you. Instead of one bar at the bottom, by default you get two, one at the top and one at the bottom. You get menu’s and equivalents of “quick launch” icons and the system tray at the top, and then application tabs on the bottom bar. The good thing about open source based software is freedom, and it’s the same here. The desktop environment I’ve outlined here is totally customisable, you could move things around so that it’s more Windows like, or something completely different.

Unlike Windows, Ubuntu comes pre-installed with applications that you are actually going to be familiar with and will want to use on a daily basis. Although adding and removing software I personally find a lot easier in Ubuntu than in Windows. In Ubuntu there is an applications called the “Synaptic Package Manager” where you can simply point and click at software in an on-line catalogue. There’s no more of this Windows style searching the web for an installation .exe file and running the install. Everything is automated for you in Ubuntu. If you’re a little more adventurous you can use this same automated system via the “Terminal” which is a way of controlling your system by typing in text commands (not everyone’s cup of tea).

I’ve tried Ubuntu several times in the past, but not all of the hardware in my laptop was supported, most notably my webcam. This time my webcam works! Getting to grips with Ubuntu 9.04 wasn’t without little hitches. I found that their default media player somehow broke the function of the volume keys on my laptop, the only fix I had was to change file associations so that application doesn’t get used. Also, the touchpad didn’t work, until I Googled around and found a single text command to type into the terminal that fixed it.

There’s a whole lot of other features new to Ubuntu 9.04 which are well covered elsewhere. I haven’t mentioned them here because I simply haven’t had to use them yet.

If you’re in the Windows or Mac world and like the 3D desktop effects that Vista, Windows 7 and OSX have, then you won’t be disappointed with Ubuntu. When you initially turn on desktop effects you are asked to give your permission for Ubuntu to install drivers for your video card, once this is done it sets up your desktop effects. The desktop effects software is called “Compiz” and you can optionally install the Compiz Settings Manager to get greater control of how your effects work. I’m still fine tuning my settings to get more productivity, but less pointless distraction (and you can easily go from one extreme to the other!). Once you do have your desktop effects set up, I promise you, what you get is far more impressive than anything Microsoft or Apple can give you, and remember, this doesn’t cost you a penny.

I’ve even got a few indispensable Windows applications running in Linux thanks to the WINE application (think of it as an adapter for running Windows software in Linux). One exception is my current favourite computer game, Team Fortress 2. I can play it, but it’s not completely stable and performance isn’t really satisfactory, even with all settings turned down. I’m working on this, but gaming isn’t my top priority.

If you’re not a techie, but you’re still reading this, then well done, and I’d like to encourage you further to try out Ubuntu by pointing out that I’ve had my mother using this too, and she didn’t even notice the difference from Windows XP – she’s just doing her web browsing and e-mail as normal. So if that’s all you do on your computer – take the plunge, go open source and enjoy the freedom.