Even More Multiple Monitor Gaming On Linux

It’s been a while since I rehashed this idea as a sad attempt to drive eyeballs to my blog, which these days mainly serves to shill for my Open Source webmail project Cypht (pronounced “sift”). Not, sure, you, have, heard, of, it. You should go check it out, download it, install it, send me questions because the install is hard, really dig it once you get it working, then give me money for a security audit. Or whatever it’s no big.

I haven’t played a lot of these games extensively, many just for a few minutes to test compatibility. Mainly because one of the games in this group is consuming all my “free” time. I’m going to make you read the whole list to find out which. Just kidding, it’s called Rocket League. You should stop reading this post now and play it. Come back later and thank me.

Back already? You’re welcome! Let’s get on with the list

Wasteland 2 Directors cut

https://wasteland2.inxile-entertainment.com/
Steam tells me I have played 4 hours of this game. Having just dusted it off after a year I’m a bit concerned about my memory, because I have no idea what’s going on. It is cool though. Pretty standard point and click RPG to move your group of weirdos, examine stuff, and pick up strange I-don’t-know-how-I-will-use-that items. Fights are turned based on a hex grid with the ability to move/attack within some limitations I don’t remember.

Universe Sandbox

http://universesandbox.com/
Not a game really, but works on my rig and is included for the following reasons:

  1. It’s called Universe Sandbox, which is rad.
  2. Science is awesome, astronomy especially so.
  3. You can do really neat stuff most of which I have only seen on the menus but I did zoom into Saturn and check out the ring rotation.

Robot Roller-Derby Disco Dodgeball

http://www.82apps.com/DiscoDodgeball/
Retro first person “shooter” except it’s dodgeball. The roller derby part basically means you can’t stop moving when you want. I think there are power-ups and leveling up and you can even catch a ball thrown at you. The court has ramps and platforms, and one time I made it to the upper level before a bot nailed me instantly. I have not played it much but it’s pretty fun as far as retro disco dodgeball games go.

Niche

http://niche-game.com/
Everyone knows you have to get up to get down. This game is all about getting down. Seriously. It’s a sim where you have to furiously procreate to survive, hopefully passing on some worthwhile chromosomes. My crude description not withstanding, this is actually a really interesting (and educational) turn based game. It’s also safe for work so to speak, but you really should not be playing games at work.

Martial Law

http://store.steampowered.com/app/346150/Martial_Law/
You are alone in a seemingly abandoned and desolate photo-realistic landscape. Minutes after venturing out into a field, you are near starving. When you approach any potential resources, you are gunned down from unseen locations. This basically sums up my experience.

Madout

http://store.steampowered.com/app/352170/MadOut/
Sort of like Twisted Metal meets Mario Cart. Flashy graphics, simple physics, and easy controls. Like many others on the list, I downloaded, I configured, I played – but then moved on. I’m not sure if it’s because the game itself was lackluster or if I just did not give it a chance because I rushed through it to play Rocket League.

Distance

http://survivethedistance.com/
Another racer with nice graphics and a futuristic beat-the-track/clock type of game play. Like Madout the driving physics are pretty easy to get the hang of, on early levels anyway. Probably has lots of other features I could not be bothered to dig into during my short revisit. I did notice giant pumpkins strewn about the level that slow you down when you plow through them, so there is that.

Divinity Original Sin Advanced Edition

http://www.divinityoriginalsin.com/
Another point-click-to-move RPG with nice graphics and sort of a Diablo feel to it. I have a vague recollection that I enjoyed the 30 minutes I played months ago. I’m putting this one on the get back to it later list for sure.

Besiege

http://www.besiege.spiderlinggames.co.uk/
This is a unique game. Each level requires you to build a war machine that you let loose to destroy enemies or fortresses. There is a huge variety of components to build your machine with. Watching my creation fall apart in seconds because of my comically poor structural engineering skills is hilarious. 100% recommend this one, it’s a gem.

Rocket League

https://www.rocketleague.com/
Many of the games I like tend to be complicated, and increase in complexity as you level up and gain new abilities. Rocket League is one of those rare games with a simple concept, simple controls, playable right away, with endless room to improve. The premise is soccer (as well as other game modes) with a giant ball that you smash into with a car to move down the field. Calling it demolition derby soccer doesn’t really do it justice, but it’s accurate.

The real fun (and frustration), is combining the simple controls to blast your car into the air for “arial” hits. Trying to accurately hit the ball in the air is like trying to get 10 meters in QWOP – surprisingly difficult. You can play with bots offline, or against human opponents online (as well couch co-op with split screen). Be warned – the online community can be iffy (but you can mute the chat thank god). Another online warning – a surprising number of players are seriously good, like stupid good. Of course It’s also possible I’m just stupid bad.

Testing PHP Network Code

I work on this Open Source webmail client. I don’t think I have ever written about it here before. It’s called Cypht. It connects to services, like an IMAP, SMTP, or POP3 server. It uses the PHP function stream_socket_client to create a connection to these services, then it sends commands and reads responses with standard read/write functions like fgets and fwrite.

Recently I decided I hate myself, so I tried to build a way to unit-test this. Turns out it’s possible, and not nearly as hard as I deserve. I did bang my head around the desk area for a few days figuring it out, so not a total loss. Here is how I did it.

Step 1: Abstract low-down-no-good functions

No matter how amazingly awesome your PHP code base is, if your code actually does anything and you want comprehensive unit test coverage, you have no choice but to abstract a few built-in PHP functions that simply don’t play nice (sessions, cookies, header, curl, streams, you get the picture). I use the following pattern for this:

  • Create a class of all static methods that “wrap” the naughty functions
  • Only define that class at run time if it does not already exist
  • Change your code to call the naughty_class::function version
  • Create the same class in your unit test bootstrap, that has friendly versions of these functions (like doing nothing, or returning true or whatever)
  • Include your unit test version before the run time version when running tests.
  • Realize your wildest dreams of success and good fortune.

An example:

class NaughtyFunctions {
    /**
     * @param string $server host to connect to
     * @param integer $port port to connect to
     * @param integer $errno error number
     * @param string $errstr error string
     * @param integer $mode connection mode
     * @param object $ctx context
    */
    public static function stream_socket_client($server, $port,
        &$errno, &$errstr, $timeout, $mode, $ctx) {
        return stream_socket_client($server.':'.$port, $errno,
            $errstr, $timeout, $mode, $ctx);
    }
}

Instead of calling stream_socket_client in code, we call NaughtyFunctions::stream_socket_client with the same (similar) arguments. This pattern (or something like it) is required to make this work, so no skipping step 1. It’s also a great way to deal with PHP functions that disagree with PHPUnit, and as a way to fool tests into taking a different code path they would not normally take, like by overriding function_exists for example. Here is what Cypht uses at runtime:

https://github.com/jasonmunro/cypht/blob/master/lib/framework.php#L59-L202

Step 2: Build a stream wrapper to fake out your code

In PHP you can fake a “stream” AKA a file handle or network connection, by creating and registering a “stream wrapper“. For file operations and stateless protocols like HTTP, this is pretty simple – read until the “file” ends. But for persistent network protocols, this takes a bit of cleverness.

You need the ability to read from the stream until you reach “End Of File” (EOF). But then you need to reset the EOF status the next time you issue a command, so you can read from the stream again. There is no way (I know of) to do this from within the stream wrapper prototype, and we don’t want to alter the network code we are testing.

Thus the cleverness. Using the abstract in step 1, we can save a reference to the stream resource, and rewind it every time we send a new command, effectively resetting the EOF. Seems less clever now that I write this, but it was the most difficult part.

Here is an example of of both the NaughtyFunctions class and a stream wrapper in action:


/**
 * Generic stream wrapper. This will be extended for protocol
 * specific commands and responses.
 */
class Fake_Server {

    /* position within the response string */
    protected $position;

    /* current response string */
    protected $response = '';

    /* list of commands to responses, varies per protocol */
    public $command_responses = array();

    /* open */
    function stream_open($path, $mode, $options, &$opened) {
        $this->position = 0;
        return true;
    }

    /* read */
    function stream_read($count) {
        $this->position += strlen($this->response);
        return $this->response;
    }

    /* write */
    function stream_write($data) {
        $data = trim($data);

        /* look for and set the correct response */
        if (array_key_exists($data, $this->command_responses)) {
            $this->response =  $this->command_responses[$data];
        }

        /* request not found, so set an error value */
        else {
            $this->response = $this->error_resp($data);
        }
        /* CLEVERNESS: here we rewind the stream so we
           can read from it again */
        rewind(NaughtyFunctions::$resource);
        return (strlen($data)+2);
    }

    /* tell */
    function stream_tell() {
        return $this->position;
    }

    /* seek */
    function stream_seek($pos, $whence) {
        $this->position = 0;
        return true;
    }

    /* end of file */
    function stream_eof() {
        return $this->position >= strlen($this->response);
    }

    /* generic error */
    function error_resp($data) {
        return "ERROR\r\n";
    }
}

/**
 * IMAP specific fake server that extends the generic one
 */
class Fake_IMAP_Server extends Fake_Server {

    /* array of commands and their corresponding responses */
    public $command_responses = array(
        'A1 CAPABILITY' => "* CAPABILITY IMAP4rev1 LITERAL+ ".
            "LOGIN-REFERRALS ID ENABLE AUTH=PLAIN AUTH=CRAM-MD5\r\n",
        /* other commands and responses go here */
    );

    /* IMAP friendly error */
    function error_resp($data) {
        $bits = explode(' ', $data);
        $pre = $bits[0];
        return $pre." BAD Error in IMAP command\r\n";
    }
}

/**
 * Naughty functions wrapper to be used in unit tests. Unlike the
 * run time version, this one returns a "connection" to our fake
 * server.
 */
class NaughtyFunctions {

    /* this will hold a reference to our fake network connection */
    public static $resource = false;

    /* we can toggle this to simulate a bad connection */
    public static $no_stream = false;

    /* fake out stream_socket_client and start the wrapper */
    public static function stream_socket_client($server, $port,
        &$errno, &$errstr, $timeout, $mode, $ctx) {

        /* bad connection */
        if (self::$no_stream) {
            return false;
        }
        /* don't call twice from the same test */
        if (!in_array('foo', stream_get_wrappers(), true)) {
            stream_wrapper_register('foo', 'Fake_IMAP_Server');
        }

        /* open, save a reference to, and return the connection
           to our fake server */
        $res = fopen('foo://', 'w+');
        self::$resource = $res;
        return $res;
    }
}

Step 3: Correlate requests and responses for your protocol

Now all you have to do is map requests to the server with appropriate (or inappropriate) responses to exercise your network code from a unit test. In this case that would be adding to the $command_responses array in Fake_IMAP_Server. This is where we cross over from “cool problem solving” to “incredibly tedious unit test production”. looks like I will be receiving extra punishment after all.

Step 4. See a doctor about your wrist pain from writing all the unit tests

Cypht has about 14,000 lines of code I need to test this way. I’m about 1% through the process. I love that it can be done without standing up an IMAP/POP3/SMTP server, but my fingers hurt just thinking about it.

Cypht Development Update

There have been 350+ commits since Cypht 1.0.0 was released, and in this post I’m going to talk about every single one. Kidding of course, but I do want to share some of the super-cool things that have happened since. As always, I want to thank everyone who has written me an E-mail, filed a bug report, submitted a pull request, joined our IRC channel, looked us up on Google, accidentally stumbled across our website, turned me down for grant money, or even thought about Open Source webmail. You guys and gals are the best!

Libsodium
Just before cutting the release, I merged libsodium support. It was a bigger-ish change than I wanted, but I felt adding the ability to leverage well written crypto was worth it. And of course users without libsodium still use our OpenSSL based encryption. Since that time, the PHP maintainers smartly decided to add libsodium as a core extension instead of a PECL package. They have a different calling convention, but I’m happy to say Cypht already supports both.

Travis CI
Travis CI is a freaking awesome service to run unit tests across different system configurations. And like all services this awesome, it’s free for Open Source projects. I have written about it in the past, and improved how we use it since. Now we have 18 different build combinations, from PHP 5.4 to PHP nightly, with 3 DBs, running 5 up, finishing in under 15 minutes. If that didn’t make any sense to you, it’s OK. Know it’s cool, because it is.

Unit tests
Did somebody mention unit test? Oh yeah, I did! Since early on we have had 100% unit test coverage of the Cypht framework. This is good, because it is the environment Cypht modules run in. But it’s not great, because modules are where the action is. Over the last week I have expanded our unit tests to be able to include modules, and have covered 100% of the only required modules, the “core” set. Since then I have come up with what I think is a novel way to use PHP stream wrappers to fake an external network service (like IMAP) to help expand unit tests to other module sets. I’m looking forward to many hours and sore fingers writing tests for all the module sets. Really I am!

Forward compatible
Thanks to Travis CI, Cypht is working flawlessly with PHP 7, 7.1, and nightly builds (eventually PHP 7.2). Man I love that service!

Integration options
Cypht does things differently than most apps. By design. This can make using it to “add webmail to my dynamic site” a bit tricky. Thanks to some great feedback and testing from supporters, we have really advanced this aspect of the program. With our API login module set, you can integrate SSO (Single-Sign-On) for Cypht with any programming language that can make an HTTP API request and build a dynamic form. We also have some PHP integration options, as well as the ability to code your own session and authentication classes without hacking any Cypht internals.

New profiles
In Cypht 1.0.0, profiles are tied to IMAP accounts, and only 1 per account is supported. Since then they have been rewritten, and now support as many profiles as you want. Profiles allow you to correlate an IMAP/POP3 account with an SMTP account, a signature, reply-to, display name and from address. The code is backwards compatible so existing profiles will be converted into the new format the first time you edit them.

Scrutinizer
Last but not least, I want to give a shout-out to Scrutinizer CI, a very cool static analyzer with a free for Open Source service. Static analysis is imperfect, but a great addition to our development process. Aside from just code quality inspection, Scrutinizer runs 16 security related checks. Cypht only fails 15! Another joke, it passes all of them.

It’s not all unicorn farting rainbows since the release. I really wanted to knock out the PGP module set by this time. The proof of concept is there, I just need to CRUD it up. While we have not added a lot of new features over the last 4 months, we have squashed a TON of wiggly little bugs. I smell another official version coming, and for the most part, it does smell like unicorn rainbow farts.

Cypht 1.0.0 Released

After more than 3 years of work, over 3,300 commits, 8 release candidates, 126 resolved issues, and 35,000+ lines of code, I’m pleased to announce the first official stable release of the Cypht webmail program is now available! As anyone who has worked in creating releases for software knows, it’s hard to draw a line in the sand. There is nothing worse than creating a release only to find out the next day you forgot something critical or missed an important bug fix. At the same time, creating releases is a crucial part of getting your software into the hands of users.

I created the release branch 2 months ago with the hope that it would only take a week or two to work out the kinks. After eight release candidates, we finally hit the “it’s good enough, let’s do this thing” point. The way I’m structuring releases in git is to create a release branch from the master branch, then porting applicable bug fixes from the master branch to the release branch while putting out pre-release candidates. Point releases will come from the same branch, but primary development continues on the master, until the next major release, which starts the process over again. I first learned this style of releasing from the Squirrelmail project lo these many years ago. In those days we used diff and patch to port fixes from trunk. With “git cherry-pick” this process is a LOT easier.

The downside to this approach is that over time the master branch diverges from the release branch, and it can get harder and harder to port fixes. The solution is to release often, effectively “dead-ending” the prior release branches as new ones are created. This is a good thing since it encourages frequent releasing. Enough has changed in the master branch in the last 2 months, I’m already eyeballing a 1.1 feature release.

I want to thank everyone who contributed code, filled out a bug report, sent me an E-mail inquiry, requested a feature, donated a translation, or told me they love/hate it. The primary force behind Cypht development is what I want a webmail client to do, but feedback is super important to broaden our user base. I greatly appreciate everyone’s feedback and support for the project.

If you are looking for a secure, lightweight self-hosted webmail that provides access to all your E-mail accounts from one place, give Cypht a try and let me know what you think!

https://github.com/jasonmunro/cypht/releases/tag/v1.0.0

Continuous testing for Cypht with Travis CI and BrowserStack

I randomly happened upon Travis CI a few weeks ago. Travis is a “continuous integration” platform that can be tied to a Github account. Every time a change is pushed to the Github repository, Travis can run all your unit tests, and it can connect to a Selenium grid provider like BrowserStack or Sauce Labs to run Selenium tests. All 3 (Travis, BrowserStack and Sauce Labs) provide free versions of their services for Open Source projects. “This sounds really cool!” I thought. And it is. But it took a wee bit of work to get it all running. By wee bit I mean a veritable shit-ton. Hopefully this post will save someone out there the hours of Travis config tweaking it took me to get everything ship-shape.

Travis has a lot of good online documentation, definitely a useful resource to get you started. Basically what Tavis does is spin up virtual machines, that you can control using its setup script. Then it will run whatever commands you want to execute your tests (in my case PHPUnit tests and Selenium tests written in python). By using it’s ability to create a “build matrix”, you can generate different server configurations to run your tests on.

As I write this I am running a build with 75 combinations (!). 5 versions of PHP x 3 different databases x 5 different web browsers. This build is not very practical since it will take about 6 hours to complete, but I had to try it once because of course I did. My standard build is 15 different server configurations (PHP versions x database types) with 5 different browsers (3 server-side combinations per browser).

In no particular order here are some tips and tricks for various parts of the configuration that took some time to figure out.

PHP versions
Setting up multiple PHP versions is a snap. You just list the ones you want in your .travis.yml file (the main Travis configuration file), and BOOM – it creates new instances for each version. The only problem I ran into with this is that PHP 7 versions do not have the LDAP extension enabled, while PHP 5 versions do. You can work around this by adding the extensions during your setup process to the travis.ini file that will eventually become the php.ini file.

PHP version setup

Fix for missing LDAP extentions in PHP 7

PHPUnit versions
PHPUnit has 2 major versions that cover PHP 5.4 through 7.1, so you will need to make sure you have the right version installed in each instance. The easiest way to do this is to wget, chmod, and mv the correct phar file based on the PHP version. Travis makes the version available as an environment variable during the setup process, so by looking at that you can grab the correct PHPUnit file.

Set up different versions of PHPUnit

Services
If you want a PHP enabled web server for your UI tests, and I assume you do since you read this far into the post, you need to install and configure that yourself. I cobbled together a couple of examples from the Travis docs and some various other blog posts to make this work. The example from the Travis docs works fine for PHP 5, however you have to work around an issue with a missing default pool configuration for FPM using PHP 7. I also wanted an IMAP account Cypht could connect to for a more real world experience, so my setup creates a system user to test with, installs Dovecot IMAP, tweaks the configuration slightly, and starts the IMAP service.

Set up Apache with PHP FPM (and the Python Selenium driver)

Default config file for Apache used in the setup

Default FPM pool file used in the setup that fixes PHP 7

Set up a system user with a known password

Set up Dovecot IMAP

Databases
Databases are as easy as PHP version, just list the ones you want in the main Travis configuration file. However they are not configured in a way applications normally use them. The current database for an instance is in an environment variable, so you can use that to determine which database to bootstrap with whatever tables or users you need. Cypht runs tests across Mysql, Postgresql, and Sqlite3.

Database setup (note this is for the massive 75 instance build. You only need one row without the BROWSER part for each database you want to test)

Bootstrap databases for the Cypht unit tests

Browsers
To run selenium tests you need to connect your Travis instance to a selenium grid, like Sauce Labs or BrowserStack. I prefer BrowserStack, but both are great. The online docs for this are pretty comprehensive, and it took a lot less time than I thought to get this working. Then I tried to use different browsers and ran into a serious problem. Chrome, Firefox, and Safari all worked fine, but Edge and Internet Explorer always failed.

By using the replay feature in BrowserStack, I could see that logins to Cypht failed with those 2 browsers. After much head scratching and keyboard bashing, I realized the issue was that these two browsers will not set cookies when accessing a site with the host name of “localhost”. Thankfully there is a work-around for this. You can force the tests to run locally, but also give them a real host name instead of localhost.

Config entry to use a different host name (the important bits are “hosts” and “forcelocal”)

Limitations for Open Source accounts
Travis will allow Open Source accounts to run 5 parallel instances, however both BrowserStack and Sauce Labs only allow 2 parallel connections to their service. In the Travis dashboard you will want to limit your parallel instances to 2 to match the Selenium provider maximum, otherwise those builds will break.

Return value
After the setup completes, Travis runs your “script command”. The return value of this command or commands will tell Travis if your tests were successful or not. You must be sure to return a non-zero value on failure, otherwise Travis won’t know something went wrong with your tests. You can string commands together with “&&” to build a set of commands to run, which will exit immediately if a non-zero value is returned by any command in the list.

Script command Cypht uses

In conclusion, Travis CI rocks for Open Source integration testing and I highly recommend it. Now I have no excuse to not write more tests!