Introduction

I develop video games since 2002. I worked full-time from 2002 to 2013 in a AAA video game company (mostly C++, PC and Console games). And I run Frozax Games as an independent developer (during the evenings and week-ends) since 2005.

In 2013, I started a new full-time job in a network security company, still as developer. In this company, I discovered Unit Testing and its benefits. In this post, I first write about the lack of tests encountered in my “old” video game developement projects. Then, I explain how and why I decided to write unit test in my latest mobile game projects.

Game development without tests

Some of my previous AAA game projects lasted for 3 years with 20 developers, and I never ever wrote a single unit test. In addition to that, most programmers were highly specialized, working only on a tiny fraction of the whole project (for example, police car AI, terrain rendering, network, UI, gameplay…). Of course, with projects this size, nobody knew all the code. It often lead to scary situations when there is a complex bug to fix (such as a random crash). Sometimes, the developer assigned to the bug changes just one or too lines of legacy code to fix the issue, only to realize a few weeks later, that a corner case is broken in another part of the game.

As the sole developer in Frozax Games, I know every single line of code so there is no “specialization” problem. However, there is still some code that is a few years old that I don’t particularly remember. The fear of breaking something when I update a game a few months after release is still very present. This is a huge risk.

I wanted to lower these risks and be reassured that an update will not break existing code, and that’s why I decided to write unit tests for my mobile games.

Unit testing at Frozax Games

At first, I really wondered how I could write unit tests for a game, because of the strong user-interaction. The result is basically just pixels on a screen :). I just had to accept that not everything is easily testable, such as, for example:
– how to test rendering? it’s dependent on art resources, screen resolution and even hardware
– how to test the appearance of the animation of a 3D character?
– how to automatically test the detection of a multi-touch gesture without actually doing it on your screen with your fingers?

But there’s still a lot of easily (and important) testable code beside these examples. Here is a non exhaustive list of code tested in Thermometers Puzzles, my latest mobile game, released last year:
– Maths functions: interpolation, combination, Min/Max…
– CSV parsing: I use CSV for localization, from a file hosted on Google Docs. therefore there are tricky cases with quotes, commas, international characters, chinese/russian charsets, multi-line translations, utf-8 encoding…
– Some generic helper functions: string management, date/time management, file compression…
– Save game serialization/unserialization: very important code to test, which led me to write specific code to prevent and fix corrupted saves.
– Level generation: my game is a puzzle game with about 1000 levels and this algorithm is the most important one in the game.
– Game hints: the player can ask for hints in the game, I make sure all different hints are working properly.

In my next game (still unannounced and work in progress), I wrote even more tests:
– 2D array class: testing for memory leaks for example
– Network HTTP requests: testing for behavior when receiving 500 or 404 return code from a web server
– Player coins usage: spending, earning and saving the player balance
– Daily level feature: making sure every daily level is different
– Undo feature

As you can see, I also able to test more game-specific features, too, provided the code is properly organized.

Benefits of Unit testing

In my opinion, the main benefits of unit testing (in video games or other domains) are:

– Corner cases: some cases are extremely rare and hard to reproduce. For example, I can test the behavior of the game when it tries to load corrupted save files.
– Better code organisation: you may discover there’s something weird about the architecture of your code when writing tests. It forces you to separate your different features in testable classes. It also simplifies the re-usability of the code in future projects.
– New code does not break existing code: less stress and risk when updating existing feature or adding new ones.
– Non-regression: Writing a test when a bug is found prevent the bug from coming back. Similar bugs coming back every few weeks is really frustrating and boring, I can tell by experience 😉

Drawbacks of Unit testing

Unit testing is not perfect and it comes with a cost: you obviously need to write more code, and it takes time. Just make sure you properly choose the sections of code you want to test, and make sure it’s really worth it.

Another problem you might encounter when writing unit tests is if you happen to update existing test code frequently. This should not happen if you write your tests properly. Make sure you don’t test very low-level methods of your code. Try to test medium/high-level methods and you will be fine. That way, your internal implementation can change, the test code is still valid.

Tools

Depending on the language, game engine or technology you’re using, you need to find the proper tools. I personally develop my mobile games using cocos2d-x, in C++. I’m using gtest, from Google. There is also a mocking module (gmock). I previously made some Unity3D development and wrote tests in C#.

For now, I only run my tests on the Windows build, and I should probably run them also on Android/iOS/Windows Phone. However, as I don’t write much platform specific-code (cocos2d-x does it for me), it’s not that important, in my opinion.

Conclusion

Adding unit testing in my projects really helped me write more reliable code. When I write a new tested feature, I can usually plug it in the game with confidence. My only concern is that there is still a lot of code that I would love to test but I think the time required to do it is still not worth it.

If you want to try it, you can probably first find something small but still useful to test, just to get you started. Progressively you will write tests for your new features. I wouldn’t advice spending to much time writing tests for existing feature, though (or just a non-regression unit test if you happen to find a bug in old code).

Today, I have about 150 tests in my next game. It still needs some manual testing, especially for layouts and rendering. I implemented an automatic tool inside the game that automatically takes screenshots to help me check the localization and layout of the game, but that’s neither automatic nor unit testing.

Hope this helps, feel free to send me your thoughts, remarks, experience about unit testing in games on twitter, facebook or comments.