This page describes utilities and tools I developed to ease the application of open source testing.
qa-visualizer is a set of tools for visualizing software quality. I have been working exclusively on testing software for over five years now. During this time I was constantly reimplementing tools to support software testing like for example logfile parsing. I was using different open source packages for visualization because my taste for software libraries was also evolving.
During the past year I wanted to replace the visualization tools again and I evaluated some tools like Matplotlib, R, Processing, John Resig's Processing.js, and Jit. I read some books about Statistics and Graphics. At one stage I found myself rethinking my approach to visualization altogether. Meanwhile I read through a bunch of books and hundreds of papers, thesis, and websites on information visualization. In January 2010 I registered for a tutorial in Dallas on "Presenting data and information" by Professor Edward Tufte. I hope with his inspiration I will be able to work out a more agreeable approach to visualization of software quality.
I used my XMas break to lay some groundwork for the new qa-visualizer. The new tool-set is now based on YUI3 and CouchDB and so far I am very happy with it. Of cause the logfile parser is still implemented in Python. The prototype of qa-visualizer currently works on about 5.000 quality measurements of the Firefox browser and thanks to CouchDB it is still blazingly faaaaast.
The screenshot shows part of the Firefox browser. The area represents the relative size of the modules and the color its complexity.
django-cp is a collection of continuous performance test for my favourite web framework django. The offspring was made during a sprint at the Birmigham PyConUK in 2008. I sat together with one of the lead developers to discuss what would be helpful in order to identify performance regressions of changes and patches of the django project. The basic idea was to use something like the Mozilla Graph-Server to graphically monitor performance regressions.
YOMU is a Logfile Analyser programmed in Python. I personally use YOMU mainly for visualisation of test results. The visualisation of the test results is presented and distributed as a PDF report. YOMU is designed to be easy configurable in order to analyse all kinds of bulk file contents
The following diagram is the visual representation of the test result contained in over 3 GB of log files. In this specific case the system under test consists of two servers. The diagram shows that response times of both servers are usually < 200 ms. During peak time some responses take over 1 second processing time. In a few extreme cases the response time is over 2 seconds. In summary this means that this particular application is in an healthy state because of acceptable response times.
I have plans to open source YOMU. I need to strip off the application specific parts to put YOMU on for general use. To be useful for others some documentation will be needed, too.
The Test Automation Manual is a collection of best practice approaches to application of open source testing tools. Goal of this manual is to ease the adaption of open source testing tools for all kinds of software development projects.