Building Quixey’s Automated Testing SystemAugust 25th, 2011 | Posted by in Technology
This post was written by Nicholas Feinberg, one of Quixey’s awesome engineering interns. He is a 4th-year Computer Science student at the top of his class at UC San Diego. He once wrote a Flash game that was played by 300,000 unique users.
One of the best parts of working at Quixey is being able to work on cool projects and being able to take ownership of my work. When I first started at Quixey, I was assigned to build an automated testing system for the jQuery and AJAX code on our site. I learned a lot in the process, and thought it would be valuable to share here.
In our early days, we had a process of manually checking various pages and processes (searching, logging in and out, etc) to make sure that everything worked. Needless to say, the manual testing process was not scalable, and we set out to create an automated system.
At first, we thought a good solution would be to use Selenium, a QA tool that describes itself as a “web application testing system”. Selenium lets you specify actions to test — clicking a button, visiting a URL, entering text, etc — and reports back on whether they work.
The first Selenium product we tried was a Firefox add-on called Selenium IDE. But we quickly discovered we couldn’t write proper unit tests with that – it was only designed for testing rapid prototypes on the fly.
After a bit more searching, we came across a framework called Windmill, which is basically Selenium for Python and JS — a perfect match for our web stack.
It took me about three weeks to cover a large portion of our site functionality with Windmill tests, automating sequences of events like the clicking and typing required for a user login. It’s still something of a joy to watch the machine automagically type things in. Like your own home-made ghost! Silly, yes, but viscerally satisfying.
The downside of Windmill is that it was designed by a single group of programmers for a single project. As a result, it’s still slightly buggy and could use more documentation.
Today, Windmill tests are an essential piece of Quixey’s QA and deployment processes. It’s a powerful and versatile testing framework, and we look forward to supporting and benefiting from its future development.