Tinymail: evolution and intelligent design
Many people today use mobile devices to stay connected. Dirk-Jan Binnema introduces Tinymail, a memory-efficient framework particularly useful to develop e-mail applications for mobile devices.
For several years now, Evolution has been the default e-mail program for the GNOME Desktop. It uses a flexible architecture and provides mature implementations of many e-mail protocols (POP3, IMAPv4, SMTP, and even Exchange, Groupwise, and Hula). Also, it has an advanced user interface (UI), which is getting even more friendly in the next release – as seen at GUADEC.
Great. However, all this requires quite a bit of memory. Tens of megabytes for moderate amounts of e-mail. Now, there’s a lot of memory in your desktop PC, but what about an ‘embedded’ device? For example, the Nokia 770 Internet Tablet. Or the One-Laptop-per-Child (OLPC) machine? In such devices, we need to be very careful with the memory usage. Also, they usually require user interfaces which are very different from the one Evolution offers – if only for the much smaller screen sizes. So, what if we took the mature, non-UI parts of Evolution, cut down on the memory usage, and made it possible to have customized user interfaces? Wouldn’t that be great for embedded devices, and even for low-end PCs?
That is exactly what Philip van Hoof did.
Philip van Hoof (1981) is an energetic software developer from the village of Oud-Turnhout in Belgium. He has strong opinions about software, and no qualms about voicing those (which sometimes causes some friction…). Philip works as a (surprise) software developer. After work, he likes to do even more (Free) software development, and he worked on many projects inside and outside GNOME.
Recently, Philip has been looking into the area of e-mail libraries. He wanted to explore ways of using the e-mail protocol parts of Evolution in a more memory-efficient way. He named the resulting framework after his girlfriend, Tinne. Thus, Tinymail was born.
Tinymail (pronounced Tinniemail) is not an e-mail program, but with tinymail, you can make an e-mail program.
Tinymail is an abstraction layer on top of libcamel (the library on which Evolution is built). It adds some functionality for account management, as well as some widgets for viewing folders, headers and messages. All these things are tightly integrated, yet loosely coupled. That means that you can easily replace parts with your own, and the system will still work.
If you use the pre-fab building blocks that tinymail delivers, you can make a simple e-mail client in only some hundreds of lines of C code, and even less in Python (using the Python binding).
Tinymail offers very substantial memory savings, compared to Evolution – 80% or more, in some tests. It achieves this by being very aggressive with its memory management, and only keeping in memory what is really needed at that time. For example, Evolution keeps the header data for all mailfolders in memory. This is required to support virtual folders (saved searches that look like normal folders). As Tinymail does not support virtual folders, it only needs to keep the currently selected folder in memory. And that saves a lot of memory.
Of course, one should be careful to compare the memory use of a full application like Evolution with that of Tinymail. However, by comparing the memory usage for, say, 500, 1000, 2000 e-mail messages, you can still draw some conclusions.
But Tinymail goes one step further. If you are browsing through your mail headers, you don’t need to have all of them in memory. You only need the ones that you can currently see. Tinymail does some clever tricks with the way it lists messages (using the so-called Proxy design pattern), so that it only allocates memory for the ones that you can actually see. The memory needed for all the others is reduced to the very minimum. While this sounds a bit slower than keeping everything in memory, in practice it does not seem a problem. Even on devices like the Nokia 770, you can smoothly scroll through thousands of messages. A reason for this might be that the less memory use in itself gives performance benefits, due to better cache usage.
Tinymail uses concepts from interface-based programming to present the client programmer with a relatively clean way to program e-mail applications. In fact, evenlibcamel itself is abstracted away in these interfaces, and Tinymail could use an alternative library instead.
Tinymail makes heavy use of the object-orientation (OO) features of the GObject system. The APIs are just interfaces, with many possible implementations. Tinymail typically supplies one particular implementation, but you could simply plug in your own.
For example, Tinymail has an interface for displaying e-mails (TnyMsgIface). It also supplies different implementations for this interface – GtkTextView and Mozilla. You can have your e-mail program display messages in either of those, whatever is the best for the specific situation. But you might still want something else. For example, others have written a GtkHTML implementation. Tinymail does not even have to know it exists. It will use whatever the client programmer specifies, as long as it conforms to the TnyMsgIface interface.
As another example, Tinymail provides the TnyDeviceIface, which is in abstraction of the network connection of a system. If your e-mail program needs to get online, you can ‘ask’ through this interface. And it does not matter what the actual implementation looks like. Tinymail comes with a TnyDeviceIface for the NetworkManager in GNOME. Other people are working on implementations for other devices.
Such software Lego sounds like some long-foregone dream from the mid-nineties. But in Tinymail it actually works quite nicely. Of course, there are limits to the syntactic sugaring that the C programming language can do for you. As with all GObject-based code, there is a lot of explicit up-and-down-casting of pointers involved. And a lot of boilerplate code. Furthermore, writing software with a framework requires a bit more of the programmer than using a simple library. Instead of just calling a bunch of library functions, the software enters into some symbiotic relationship with Tinymail, which requires you to do things in a certain way.
Back to the Future
There has been a lot of interest in the Tinymail project already. For example from Nokia, who did some experimental work with Tinymail already.
Tinymail also runs smoothly on the Nokia 770, as shown in picture 2). And there are the OLPC team members, who are evaluating various solutions. They will probably be interested in the Tinymail Python bindings, which have recently become available.
So, if you are working on a new e-mail client (aren’t we all), you should take a look at Tinymail. It does take some time to get familiar with it, especially if you are not fluent with GObject. Also, Tinymail has not seen an official release yet, and API changes are to be expected. Still, the framework is well designed, fairly complete, and is released under the LGPL license. Furthermore, it comes with a lot of documentation and unit tests.
Even if you are not writing your own e-mail client, or are not a programmer at all, Tinymail can benefit you. The project has sparked a lot of new interest and improvements, which will benefit all Evolution users. Philip has made additional impressive memory reductions to Evolution (not depending on Tinymail), using mmap. At the same time, Ross Burton saved more megabytes by killing off unnecessary copies of libraries. This will all result in a meaner and leaner Evolution which translates to a much nicer experience for all users. It also shows how the work on GNOME for embedded devices ultimately benefits all GNOME users.
This article only gave a short overview of Tinymail. If you want to find out more, you can visit the Tinymail website or join the mailing list. Tinymail is still under heavy development, so this is the time when you can make a change!
About the author
Dirk-Jan Binnema (djcb) has been a computer programmer and GNOME user since 1998 or so. Nowadays, this handsome Dutchman works for Nokia in lovely Helsinki – not programming so much anymore, but still working on all kinds of cool software for the Nokia 770.