UTAH stands for Ubuntu Test Automation Harness. This is a tool designed to help people with end to end automation. One of the first things I noticed when I started trying to automate tests for Ubuntu is that I lacked the right tools for the job. I was used to have a testing framework, another tool where results were reported in a way that I could have an overall view of them and figure out where we were, instructions on how to add new test cases to the mix to improve the coverage, etc.
In the Ubuntu project things were a bit different 6 months ago. We had several projects solving the same kind of issues in slightly different ways, a fragmented testing landscape, there was a lot of duplicated code and testing projects that slowly died due to no-one having the time to maintain them when they became big enough. This is one of the problems with automated testing. It is easy to get excited and code a lot of stuff just for the fun of it, but it is not that easy to create automated tests in a way that they are cheap to maintain as well as useful.
Ok, so let's say you are convinced that testing is something you need for your project/package and you want to create some tests for it. You do your python test scripts or C programs that report if they pass or fail. Now what? Well, you want to run them in the latest version of Ubuntu, you want to run them on i386, amd64, with this or that kernel, you want to run them in old versions to make sure you are backwards compatible, you want to run them in the current version so that you are compatible with the most modern stuff and quickly you get depressed because you realise that it is not easy to move from having some tests written in python to actually having a good set of results you can work with, provision all those systems and run your tests becomes a huge burden.
UTAH is meant to help people plug their lightweight scripts/test cases and help with the provisioning (this is what we mean by end-to-end: provision, run tests, get results back). If you want to provision a VM or a spare machine at home (HW provisioning not fully implemented yet, working on it), or if you just want to run a bunch of scripts for a particular purpose on your own machine, any of this scenarios should be easily achieved by using the Ubuntu Test Automation Harness. We have made sure that VMs are provisioned and tests can run on them. We are trying to give the community a way to define what the test cases do and a way to run them easily on different configurations.
The tool has been developed and is being Alpha tested. We hope to be able to start substituting current smoke testing jobs in Jenkins for UTAH smoke testing jobs soon. I will tell you all about the migration so that you can decide whether this is the right tool for you! If you want to have a look at the documentation, here it is, but bear in mind this is being reworked whilst we do the migration, to have it completely up to date by the end of it.
We even have a mailing list, in case you want to join this adventure!