Cypht 1.0.0 Released

After more than 3 years of work, over 3,300 commits, 8 release candidates, 126 resolved issues, and 35,000+ lines of code, I’m pleased to announce the first official stable release of the Cypht webmail program is now available! As anyone who has worked in creating releases for software knows, it’s hard to draw a line in the sand. There is nothing worse than creating a release only to find out the next day you forgot something critical or missed an important bug fix. At the same time, creating releases is a crucial part of getting your software into the hands of users.

I created the release branch 2 months ago with the hope that it would only take a week or two to work out the kinks. After eight release candidates, we finally hit the “it’s good enough, let’s do this thing” point. The way I’m structuring releases in git is to create a release branch from the master branch, then porting applicable bug fixes from the master branch to the release branch while putting out pre-release candidates. Point releases will come from the same branch, but primary development continues on the master, until the next major release, which starts the process over again. I first learned this style of releasing from the Squirrelmail project lo these many years ago. In those days we used diff and patch to port fixes from trunk. With “git cherry-pick” this process is a LOT easier.

The downside to this approach is that over time the master branch diverges from the release branch, and it can get harder and harder to port fixes. The solution is to release often, effectively “dead-ending” the prior release branches as new ones are created. This is a good thing since it encourages frequent releasing. Enough has changed in the master branch in the last 2 months, I’m already eyeballing a 1.1 feature release.

I want to thank everyone who contributed code, filled out a bug report, sent me an E-mail inquiry, requested a feature, donated a translation, or told me they love/hate it. The primary force behind Cypht development is what I want a webmail client to do, but feedback is super important to broaden our user base. I greatly appreciate everyone’s feedback and support for the project.

If you are looking for a secure, lightweight self-hosted webmail that provides access to all your E-mail accounts from one place, give Cypht a try and let me know what you think!

https://github.com/jasonmunro/cypht/releases/tag/v1.0.0

Continuous testing for Cypht with Travis CI and BrowserStack

I randomly happened upon Travis CI a few weeks ago. Travis is a “continuous integration” platform that can be tied to a Github account. Every time a change is pushed to the Github repository, Travis can run all your unit tests, and it can connect to a Selenium grid provider like BrowserStack or Sauce Labs to run Selenium tests. All 3 (Travis, BrowserStack and Sauce Labs) provide free versions of their services for Open Source projects. “This sounds really cool!” I thought. And it is. But it took a wee bit of work to get it all running. By wee bit I mean a veritable shit-ton. Hopefully this post will save someone out there the hours of Travis config tweaking it took me to get everything ship-shape.

Travis has a lot of good online documentation, definitely a useful resource to get you started. Basically what Tavis does is spin up virtual machines, that you can control using its setup script. Then it will run whatever commands you want to execute your tests (in my case PHPUnit tests and Selenium tests written in python). By using it’s ability to create a “build matrix”, you can generate different server configurations to run your tests on.

As I write this I am running a build with 75 combinations (!). 5 versions of PHP x 3 different databases x 5 different web browsers. This build is not very practical since it will take about 6 hours to complete, but I had to try it once because of course I did. My standard build is 15 different server configurations (PHP versions x database types) with 5 different browsers (3 server-side combinations per browser).

In no particular order here are some tips and tricks for various parts of the configuration that took some time to figure out.

PHP versions
Setting up multiple PHP versions is a snap. You just list the ones you want in your .travis.yml file (the main Travis configuration file), and BOOM – it creates new instances for each version. The only problem I ran into with this is that PHP 7 versions do not have the LDAP extension enabled, while PHP 5 versions do. You can work around this by adding the extensions during your setup process to the travis.ini file that will eventually become the php.ini file.

PHP version setup

Fix for missing LDAP extentions in PHP 7

PHPUnit versions
PHPUnit has 2 major versions that cover PHP 5.4 through 7.1, so you will need to make sure you have the right version installed in each instance. The easiest way to do this is to wget, chmod, and mv the correct phar file based on the PHP version. Travis makes the version available as an environment variable during the setup process, so by looking at that you can grab the correct PHPUnit file.

Set up different versions of PHPUnit

Services
If you want a PHP enabled web server for your UI tests, and I assume you do since you read this far into the post, you need to install and configure that yourself. I cobbled together a couple of examples from the Travis docs and some various other blog posts to make this work. The example from the Travis docs works fine for PHP 5, however you have to work around an issue with a missing default pool configuration for FPM using PHP 7. I also wanted an IMAP account Cypht could connect to for a more real world experience, so my setup creates a system user to test with, installs Dovecot IMAP, tweaks the configuration slightly, and starts the IMAP service.

Set up Apache with PHP FPM (and the Python Selenium driver)

Default config file for Apache used in the setup

Default FPM pool file used in the setup that fixes PHP 7

Set up a system user with a known password

Set up Dovecot IMAP

Databases
Databases are as easy as PHP version, just list the ones you want in the main Travis configuration file. However they are not configured in a way applications normally use them. The current database for an instance is in an environment variable, so you can use that to determine which database to bootstrap with whatever tables or users you need. Cypht runs tests across Mysql, Postgresql, and Sqlite3.

Database setup (note this is for the massive 75 instance build. You only need one row without the BROWSER part for each database you want to test)

Bootstrap databases for the Cypht unit tests

Browsers
To run selenium tests you need to connect your Travis instance to a selenium grid, like Sauce Labs or BrowserStack. I prefer BrowserStack, but both are great. The online docs for this are pretty comprehensive, and it took a lot less time than I thought to get this working. Then I tried to use different browsers and ran into a serious problem. Chrome, Firefox, and Safari all worked fine, but Edge and Internet Explorer always failed.

By using the replay feature in BrowserStack, I could see that logins to Cypht failed with those 2 browsers. After much head scratching and keyboard bashing, I realized the issue was that these two browsers will not set cookies when accessing a site with the host name of “localhost”. Thankfully there is a work-around for this. You can force the tests to run locally, but also give them a real host name instead of localhost.

Config entry to use a different host name (the important bits are “hosts” and “forcelocal”)

Limitations for Open Source accounts
Travis will allow Open Source accounts to run 5 parallel instances, however both BrowserStack and Sauce Labs only allow 2 parallel connections to their service. In the Travis dashboard you will want to limit your parallel instances to 2 to match the Selenium provider maximum, otherwise those builds will break.

Return value
After the setup completes, Travis runs your “script command”. The return value of this command or commands will tell Travis if your tests were successful or not. You must be sure to return a non-zero value on failure, otherwise Travis won’t know something went wrong with your tests. You can string commands together with “&&” to build a set of commands to run, which will exit immediately if a non-zero value is returned by any command in the list.

Script command Cypht uses

In conclusion, Travis CI rocks for Open Source integration testing and I highly recommend it. Now I have no excuse to not write more tests!

Cypht Webmail Screen Shots – Mobile Version

I had a request on the “Cypht Webmail Screen Shots” post to share some mobile/responsive views of the program. Since I always aim to please, and since the percentage of people with even a shred of interest in this project is so very very small, here it is!

Overall, my design skills fall somewhere between terrible and total shit. My design premise for the Cypht UI has been: “Don’t pretend you know what colors go well together. Don’t fool yourself into thinking you can make something slick. JUST KEEP IT SIMPLE DUMMY“.

I’m pleasantly surprised with the result. It’s pretty standard webmail fare, super simple, and reasonably consistent across browsers.

cypht_mobile10

The main menu. Available by swiping right on any page, or with the awkward looking triangle in the upper left corner of a page.

cypht_mobile9

I pretty much never write E-mail on my phone. But I could!

cypht_mobile8

Yay, only 1 unread message! Also, if you have not checked out Humble Bundle, you should.

cypht_mobile6

The site settings page

cypht_mobile5

General site settings. Kind of squeezed in there, but not too bad!

cypht_mobile3

Search all the things again!

cypht_mobile4

Read a search result. The message view page is tricky on mobile. This one looks pretty good, but others not so much. It’s a work in progress.

cypht_mobile2

Just a teeny tiny calendar. Nothing to see here.

cypht_mobile1

RSS news feed for my favorite Linux news site.

cypht_mobile15

Server management page

cypht_mobile13

“Quick add” dialog on the server page. So cool.

cypht_mobile14

Keyboard shortcut management.

cypht_mobile16

IMAP folder management.

cypht_mobile11

Save settings page.

There are some noticeable omissions to this list, specifically the contacts management interface and the profiles page. I left profiles out because it looks like dog poop, and I left contacts out because I realized the screenshot was of my actual contacts and that would be dumb to post. It looked pretty good though! All in all, Cypht is pretty responsive. Writing this post helped me identify some areas we need to work on, so thanks to the person who requested it!

Cypht Webmail Screen Shots

This is YET ANOTHER post to shill for Cypht, my Open Source webmail project. But instead of droning on endlessly about technical mumbo-jumbo like I usually do, this one has pictures! They are pretty! Let’s look at them!

cypht14

Browsing IMAP folders like it’s the noughts yo!

cypht12

Fancy compose form with an HTML editor. Cypht also supports text only for outbound E-mail (and FTW).

cypht18

The Servers page is where you add data sources to your account. Without data sources there is not much to look at.

cypht8

Site settings page with expandable categories.

cypht13

Search all the things at once!

cypht4

Read one of the things you searched for!

cypht9

This webmail program has a calendar. It’s truly amazing.

cypht5

Home page on mobile (S5 emulation with Chromium)

cypht15

Main menu on mobile

cypht7

RSS feed list view on mobile

cypht16

I almost forgot we support themes. I suck at it, but it’s pretty easy. Maybe somebody who doesn’t suck at it will make some more

cypht17

This theme is called “VT100”. See what I mean about sucking at themes?

There is more, but I’m too lazy to set up decent looking test data and you get the idea. All this uploading of images has made me tired, so I think I will go take a nap.

5 Cool Cypht Webmail Features

Cypht is the Open Source webmail program I have been toiling away on for the last few years. It stands out from the competition because of a few unique options, not that it doesn’t have its own warts. But let’s focus on the positive, and not talk about things like the painful installation process, outstanding bugs, or unfinished features. I wouldn’t be doing a very good job converting this blog into a propaganda platform for the project with a title like “5 Shitty Cypht Webmail Features“. Also, I’m a dork. What I think qualifies as cool is known to be subjective.

1. Stand Alone Authentication
Pretty boring opener, but bear with me. Typically webmail programs are designed to point to an E-mail source, like an IMAP or POP3 server, for authentication. They pass the username and password you give them on to the E-mail server, which then tells the webmail program if you are legit or bogus. Cypht boldly (not really) breaks this paradigm by splitting authentication from your E-mail sources.

We support using LDAP or a database to authenticate users, as well as the old-school method of using a pre-configured IMAP or POP3 server. Adding new authentication mechanisms is designed to be easy (relatively), so any source that can verify your username and password can be coded up. We even support a dynamic login process that lets you pick from common E-mail service providers, and can auto-discover E-mail services for a domain (sometimes).

2. Combined Views
Cypht provides combined lists of E-mail messages from all your accounts. One of the reasons I started working on yet another webmail program, was because I wanted this feature. I spend 95% of my time using these views for my E-mail needs, and very little time browsing folders and pages like it’s 1999 (though Cypht supports this as well)

Show me the latest 20 unread messages from each of my accounts over the last 2 weeks in one list? Done. Search for the boarding pass I misplaced even though I forgot which E-mail account I used to make the reservation, and the plane takes off in 5 minutes? Done (this actually happened to me). If you have more than one E-mail account, combined views quickly become the bee’s knees. If you only have one E-mail address, you probably should have skipped this section.

3. Module Sets
Plugins are cool. Module sets are cooler. First of all, they sound cooler. Secondly, module sets are not just a way to bolt on features. Cypht is entirely built of module sets, and a framework to run them. Only one is required, the “core” set. It does things like basic page layout and login/logout. Everything else (IMAP, SMTP, POP3, RSS, contacts, profiles, the list goes on) is its own module set, and can be enabled or disabled independently.

As if that wasn’t the ultimate in coolness, there’s more! Module sets can override each other. Don’t like the default behavior of a core module? You can change it without hacking a single line of Cypht code by creating your own module set that overrides it. I need a sweater it’s getting so cool in here! There is even some poorly written documentation about module sets for aspiring developers.

4. Focus on Security
Security is serious business for a webmail program. So many attack vectors! From filtering out nasties, to TLS everywhere, to encrypting data at rest – Cypht goes the extra mile to try to cover all the bases. Cypht was built with security and privacy as core design principles.

Securing a complex web application is a process, and we welcome feedback and suggestions to continue to improve. For the gory details, check out our security page with a list of impressive sounding technical stuff.

5. Production and Debug Mode
Cypht has two modes of operation. “Debug” mode is what you use when troubleshooting issues or doing development. “Production” mode is what you use when … in production. Debug mode fire-hoses your PHP log with information about each request, enables all errors and warnings, and activates new modules as you create them for a quick write-then-test cycle. Production mode uses combined and minfied assets, silences warnings, and pre-calculates module dependencies.

If you are looking for a different kind of webmail, one that is lightweight, secure, and has a complicated install process -check out Cypht. Or don’t. It’s cool.