This post was written by Nicholas Feinberg, one of Quixey’s awesome engineering interns. He is a 4th-year Computer Science student at the top of his class at UC San Diego. He once wrote a Flash game that was played by 300,000 unique users.

One of the best parts of working at Quixey is being able to work on cool projects and being able to take ownership of my work. When I first started at Quixey, I was assigned to build an automated testing system for the jQuery and AJAX code on our site. I learned a lot in the process, and thought it would be valuable to share here.

In our early days, we had a process of manually checking various pages and processes (searching, logging in and out, etc) to make sure that everything worked. Needless to say, the manual testing process was not scalable, and we set out to create an automated system.

We were surprised to find that there weren’t many tools suitable for testing an AJAX site. Of course, there are a great number of unit testing systems for JavaScript out there, such as JsUnit — the problem is that they’re built to test the behavior of JavaScript functions, and not whole DOM-based web GUIs outputted by Javascript code.

At first, we thought a good solution would be to use Selenium, a QA tool that describes itself as a “web application testing system”. Selenium lets you specify actions to test — clicking a button, visiting a URL, entering text, etc — and reports back on whether they work.

The first Selenium product we tried was a Firefox add-on called Selenium IDE. But we quickly discovered we couldn’t write proper unit tests with that – it was only designed for testing rapid prototypes on the fly.

The full Selenium WebDriver product would have let us write proper unit tests – it looked like a solid tool that did what we needed. But since Quixey is written in mostly Python and JavaScript, the lack of support for these languages was a deal breaker.

After a bit more searching, we came across a framework called Windmill, which is basically Selenium for Python and JS — a perfect match for our web stack.

It took me about three weeks to cover a large portion of our site functionality with Windmill tests, automating sequences of events like the clicking and typing required for a user login. It’s still something of a joy to watch the machine automagically type things in. Like your own home-made ghost! Silly, yes, but viscerally satisfying.

The downside of Windmill is that it was designed by a single group of programmers for a single project. As a result, it’s still slightly buggy and could use more documentation.

Today, Windmill tests are an essential piece of Quixey’s QA and deployment processes. It’s a powerful and versatile testing framework, and we look forward to supporting and benefiting from its future development.