Welcome to The Hitchhikers Guide to Test Automation!

Tools

	gcview Analysis of JVM garbage collector logs
	yomu Multi purpose logfile analyser yomu is a Logfile Analyser programmed in Python. I personally use yomu mainly for visualisation of performance test results. The visualization of the test results is presented and distributed as a images or a complete HTML report. yomu is basically a Python script template that can be easily adapted to fit your needs when analysing all kinds of bulk file contents. I have a fairly good command of the SQL programming language which I wanted to leverage during my daily work of logfile analysis. This is the fifth complete reimplementation and now I am quite happy with it. The logfile analysis is split into the following processing steps: logfile parsing data clean up (timestamp mangling, removal of duplicate records, etc.) data aggregation visualization
	django-cp django-cp is a tool for implementing continuous performance tests. The tools was build for my favourite web framework django. The offspring was made during a sprint at PyConUK 2008 in Birmingham. I sat together with Jacob (one of the lead developers) to discuss what would be helpful in order to identify performance regressions of changes and patches of the django project. The basic idea was to reuse the Mozilla Graph-Server to graphically monitor performance regressions.
	metrics The metrics tool was implemented to create software metrics for big code bases like Mozilla Firefox, Apache Webserver, or the Python code base itself. One common attribute of big code bases is that they usually contain code from multiple programming languages. Each programming language itself has already a metrics package but unfortunately the metrics are not compatible to each other. For example each metrics implementation has its own philosophy towards what a line of code could be like whether comments are counted or how statements that are spread over multiple lines are counted. From my perspective the only viable approach is to create metrics for the different languages from one package that is able to count the metrics in a consistent way.
	visualizer visualizer is a set of tools for visualizing software quality. I have been working exclusively on testing software for over five years now. During this time I was constantly reimplementing tools for visualizing test results like for example logfile parsing. I was using different open source packages for visualization because my taste for software libraries was also evolving. During the past year I wanted to evolve the visualization tool-set again and I evaluated some tools like Matplotlib, R, Processing, John Resig’s Processing.js, and Jit. I read some books about Statistics and Graphics. At one stage I found myself rethinking my approach to visualization altogether. Meanwhile I read through a bunch of books and hundreds of papers, PhD thesis, and websites on information visualization. In January 2010 I went to Dallas for a tutorial on ‘Presenting data and information’ by Professor Edward Tufte. With his inspiration I was be able to work out a more effective approach to visualization of software quality. I used the 2009 XMas break to lay some groundwork for the new visualizer. The new tool is now based on YUI3 and CouchDB and so far I am very happy with it. Of cause the logfile parser is implemented in Python. Meanwhile the data store has been replaced by a custom webservice and the PostgreSQL database.
	The Hitchhikers Guide to Test Automation If you read this you are already almost save because ‘The Hitchhikers Guide to Test Automation’ has printed ‘DON’T PANIC’ on it’s cover in capital letters. The Hitchhikers Guide to Test Automation is a collection of best practice approaches to application of open source testing tools. Goal of this guide is to ease the adoption of open source testing tools for all kinds of software development projects.

Upcoming events

Come and visit my new talk on software quality visualization at the PNSQC 2010 conference in Portland.

Currently I am preparing the paper and talk on “Visualizing Software Quality”. Besides the conceptual work I will also introduce the prototype of the visualizer tool. I can not wait to get feedback on the work of the past 12 month from experienced testers.

Past Talks

Content

• The Hitchhikers Guide to Test Automation - Mission Statement
  • Introduction
  • Really the mission
• Open source testing tools in practice
  • System under test
  • Initial situation
  • Manual testing
  • Prerequisites & Testability
  • Performance testing
  • Unit testing
  • Continuous integration
  • Acceptance testing/ functional testing
  • Testing tool requirements
• Functional testing of web applications with Selenium
  • Selenium Core
  • Selenium IDE
  • Selenium Remote Control (RC)
  • Selenium Grid
  • Installation
• How to Model and Implement a Domain Specific Language (DSL) for Functional Test Automation
  • Introduction
  • The Problem
  • Domain Specific Languages (DSL) to the rescue
  • DSL Modelling
  • DSL Implementation in Python
  • Related Work
  • Conclusions and further work
• Test Automation Tools and Utilities
  • Working with legacy code
  • Java Garbage Collection analysis and tuning
  • Howto set up a SSL test site with Apache2
  • Create metrics consistently over multiple languages
  • Visualizing Software Quality
  • Creating RESTful webservices with restish
  • timeit - measure the execution of Python code
• Visualization of Continuous Performance Test Results with Mozilla Graph-Server
  • Why Continuous Performance Tests (CPT)
  • Mathematical Foundation of Performance Tests
  • Definition of Continuous Performance Tests (CPT)
  • Implementation of CPT
  • Tools for CPT
• Visualization
  • “Presenting Data and Information” seminar brownbag
  • Visualization Taxonomy
• Past Talks
  • EuroPython 2010 Conference in Birmingham
  • EuroPython 2009 Conference in Birmingham
  • μPy Munich Python User Group May 2009 Meeting
  • μPy Munich Python User Group February 2009 Meeting
  • Open Source Meets Business (OSMB) Congress 2009 Nuremberg
  • μPy Munich Python User Group January 2009 Meeting
  • Mozilla Camp Europe 2008 in Barcelona
  • PyConUK 2008 in Birmingham
• Disclaimer
• Contact/ About
  • Discussion Forum
  • Other means of communication
  • Postal Address