The Itch I Can’t Stop Scratching

My first official contribution to an Open Source software project was way back in 2002.  I was solving a problem for my employer, and ended up becoming a developer for the venerable Squirrelmail project. It was an exciting time. The community was vibrant, active, and surprisingly welcoming to a near-complete novice willing to get their hands dirty. Looking back at the code I wrote lo those many years ago makes me want to gouge my eyes out with red-hot sporks, but I can’t deny the impact contributing to that project had on both my mindset and career path. Since then my involvement in Open Source has waxed and waned, but has always remained. That seemingly innocent interaction sparked a lifelong interest in webmail applications, and I have been tinkering with them ever since.

After a brief 5-year stint writing mostly Python and C++ , I started working with PHP full-time again last May when I joined Automattic. I realized pretty soon after starting that my skills were rusty. Like PHP4 rusty. I needed to experiment with the latest-greatest the language had to offer, but in a safe way, and on my own terms. For the third time in my life, I decided to unleash yet another Open Source webmail client on the world. That surge of excitement you are not feeling at this point is totally understandable. Especially considering the code I wrote the first two times would best be stowed away in the “how not to write complex software” file.

I set out with a newly provisioned github repo, the enthusiasm of someone half my age, and some lofty goals:

  1. Build a client with combined views from multiple E-mail accounts, able to speak both IMAP and POP3, and flexible enough to merge other data sources
  2. Turn security up to 11. Perhaps 12
  3. Make it fast, compact, and compliant
  4. Utilize a modular system that all components outside the bare bones framework use. Like an uber-plugin system the whole app runs on
  5. Do all this while pushing myself to learn what great features new versions of PHP have to offer

To get started, I ferreted out and cleaned up the core IMAP, POP3, and SMTP routines from my last webmail project. While I was at it, I modernized the IMAP library to support some useful protocol extensions, and even built some unit tests *gasp*. These libs have been battle-tested against real world server idiosyncrasies for over a decade, so while they may not be ideal from a code design standpoint, they have an established record of compatibility. This is important when dealing with complex protocols that have a myriad of server implementations. I’m looking at you IMAP.

Next I set out to create a simple request and response processing framework – one that uses “modules” to do the real work of building the resulting page. The framework is lightweight (request processing uses on average 2MB of server memory), and leverages some nifty code features. With a framework in place, the next step was to start cranking out module sets for specific functionality. I started with core requirements like laying out the page content and logging in and out. Next I dove into IMAP, since it would be the primary protocol for E-mail access, and easily the most complicated data source to implement.

9 months later I am happy to say I have a pleasant to use E-mail and RSS reader including preliminary SMTP support for outbound mail (very preliminary). It’s easy on the server and the browser, and has some interesting features for combined content views. It is still very much a work in progress, but here are some highlights:

  • Super small pages with minimal server requests. A single page load only requires 3 HTTP requests with a combined response size of about 30KB (gzipped). Email and Feed data are populated via one parallel AJAX call per source, with response sizes of ~1KB. All interface icons are served inline with data urls to keep request count low.
  • Oodles of security features: TLS/STARTTLS support for all protocols; forced HTTPS for browser requests; secure HTTP-only session level cookies; AES compatible encryption for session and persistent data using unique keys; white-listed and typed user input; built-in HTTP POST nonce enforcement; HTTP header fingerprinting; easy output escaping; a two factor authentication module; probably more I’m forgetting.
  • Modules for IMAP, POP3, SMTP, RSS, and several other app components with more on the way. Modules can be enabled or disabled independently. The module system is super flexible and lends itself to some interesting customization options. It might even turn out to be too flexible.
  • Easy-to-extend session management including stock PHP session support and custom DB sessions. The DB session support is not a registered PHP session handler – it is a completely independent implementation.
  • Authentication is also easy to extend and already supports authenticating via IMAP, POP3, or an included PBKDF2 compliant database schema.
  • Database access is not required (unless used for authentication), but can be leveraged for session and persistent data storage with any PDO supported DB. Table definitions are included for Mysql and Postgresql.
  • Validated HTML5 output, including responsive views for mobile devices and HTLM5 local session storage for caching.
  • Lots of other boring technical details really neat stuff!

I could ramble on about this forever, better stop now before I get carried away. No post about half-done probably soon-to-be obsolete software is complete without at least one screenshot. Here is a look at the interface with a combined view of 9 different RSS feeds.

hm3_feedsIt’s not only been a great learning experience to work on this code, it’s been a lot of fun too. The repository is at http://github.com/jasonmunro/hm3/ for anyone who wants to take a look. Documentation is scarce and things are changing quickly, so if you do check it out, use caution :).

2 thoughts on “The Itch I Can’t Stop Scratching

  1. As I was reading this, I kept thinking of Hastymail. Then I saw your Github link and it made sense! HM was great for dealing with some huge customer mailbox problems back in my ISP sysadmin days, *years* ago. Glad to hear you’re still at it.

    I had a lot of fun with Squirrelmail, too, and wrote some plugins for it back in its heyday. Nice to read a post from a kindred soul. Thanks for sharing.

  2. Pingback: Cypht: New Open Source Webmail | Unencumbered by Facts

Leave a comment