Thursday, October 29, 2009

Multiple Tests in a single saru file

Testing with saru is supposed to be easy to do in any language.
So the interface to support multiple tests from a single file has to be easy...

All you need do is print the right stuff to stdout and stderr and return the right value and you're done.

In the single test case all that mattered was the return value, everything else was just informational in the case of failure.

So what should the output look like to make multiple tests work... well here's a sample.

STDOUT
test_00_dummy_pass: OK
test_01_dummy_fail: FAILED
1/2

STDERR
<@saru start test_00_dummy_pass @>
Some info about the dummy_pass test
<@saru end test_00_dummy_pass @>
<@saru start test_01_dummy_fail @>
This test fails 
And so this message will appear in the test output
<@saru end test_01_dummy_fail @>

Whipping up a python script that prints these outputs and running it through
saru-run-test suite .
gives the following results
test.py::test_00_dummy_pass : OK
test.py::test_01_dummy_fail : FAILED???
==MESSAGE==

==STDERR==
This test fails 
And so this message will appear in the test output


1 / 2

Admitadly the output is not that pretty, and the mechanism is not prefectly robust.
But it has done everything I need from running multiple tests from a single file.

Of course you'd probably want to write a helper library to get that outputting correct.
And to help you use "good testing practices" like fixtures.
Some of these helper libraries already exist. The C++ one is already part of saru, as a pure header. There is a python helper library that will be added shortly, and a PHP library that is in development.

I might look at how to use the C++ library in a future post.

Wednesday, October 21, 2009

Testing with saru

So everyone should be running tests on their code. We have hundreds and they're never enough. But how to run and collate results from all these tests. There are plenty of testing frameworks out there, but each one seems married to a particular language. What if parts of your code are in python, parts in C++, parts in php etc. You've been doing the right thing and using "the right tool for the job" but now you have a mish-mash of code. That was the case I was in a while ago, and I decided that I'd be better off having a testing system that could test a bunch of languages. So I wrote saru.

saru is opensource (BSD) and is the simplest testing framework I could come up with.
So how do you use it?

Heres an example test in python
#!/bin/python
# SARU : tag example
import sys
print >> sys.stderr, "Log message"
sys.exit(1)

The same thing again in C++
// SARU : tag example
#include <iostream>
int main()
{
  std::cerr << "Log message" << std::endl ;
  return 1;
}
The convention is that tests are single applications that return 1 for failure and 0 for success. To distinguish test files from other files such as mocks, fixtures or other helper code, tests are tagged with a SARU tag. Now to run these tests
saru-run-tests suite
We get the following output:
example00.py : FAILED???
==MESSAGE==
saru-run-test : execution of test failed with error code 1
==STDERR==
Log message


example01.cpp : FAILED???
==MESSAGE==
saru-run-test : execution of test failed with error code 1
==STDERR==
Log message


0 / 2
Lets change both of those files to return 0 and rerun the tests and we should get
example00.py : OK
example01.cpp : OK
2 / 2
Now this should also catch and report compilation errors in the C++. Theres a bunch of stuff not explained here that I'll detail in following posts including
  1. How to make multiple tests in a single file
  2. How to specify compiler options for C++
  3. What would need to happen to make saru work on windows
  4. How to run subsets of tests
  5. How to extend saru to run other languages
  6. What are these saru logs?
  7. Things that still need to be done to make saru cooler

Tuesday, October 20, 2009

Making cakephp Models and Components user friendly

So cakephp is a handy cake website development platform, but the way models and components work can make refactoring an existing site painful.

Making Models Nice
This is what a basic cakephp controller looks like
class MyController extends AppController
{
  var $uses=array('MyModel');

  do_something()
  {
    $this->MyModel->findById($id);
    ...
    $this->MyModel->save($data);
  } 
}
My problem with this is as your code grows you add more and more models to the uses array. Then when you want to duplicate this functionality in another controller you need to work out which models each function uses. This becomes even more tortuous once parts of these functions have been refactored into components. There has to be a better way, a way that puts the definition of the models used closer to the usage of the models.

So lets dig into how models can be loaded in cakephp. There are a few options...
  1. You can place it in a $uses=array('Foo') variable at the start of your controller
  2. You can do a $this->loadModel('Foo') at the point of usage
  3. you can use the ClassRegistry. which I wont talk about here...
These are all pretty ugly. IMHO the nicest solution is 2. But this leads to a lot of code like this

class MyController extends AppController
{
  function do_something()
  {
    $this->loadModel('Foo');
    $this->Foo->findById( $id );
    ...
    $this->Foo->save( $data );
  }
}
But I think this is better but still kind of ugly. The separation between the initial loadModel and the final save can lead to cases where refactoring is difficult.

So I add the followig function to my AppController class.
function m($modelName )
{
  $this->loadModel($modelName);
  return$this->$modelName;
}
Now the above code becomes
class MyController extends AppController
{
  function do_something()
  {
    $this->m('Foo')->finById( $id );
    ...
    $this->m('Foo')->save( $data );
  }
}
Which I think is much neater. Its now impossible to have the Foo model not instantiated when we need it.

Making Components Nice
So using a component is much like using a model
class MyController extends AppController
{
   var $components=array('MyComponent');
  
   function do_something()
   {
     $this->MyComponent->foo();
   }
}
However there is no loadComponent that can be used to bring the defining of which components we will use and the actual use of the component closer together. So lets write one in app_controller.php
function loadComponent( $name )
{
  if( isset($this->$name) ) return;

  $class_name = $name.'Component';
  if( !class_exists( $class_name ) && !App::import('Component', $name ) )
  {
    throw new Exception('Loading component failed');
  }
  
  $this->$name = new $class_name;
  $this->$name->initialize($this);
}
This laodComponent function is pretty hacky and will not work for all components that are supported by cakephp, nor will it configure all components correctly, but it works for all the components that I have written. You may need to adjust it if you're using funky components.

So now our example would be
class MyController extends AppController
{ 
  function do_something()
  {
     $this->loadComponent('MyComponent');
     $this->MyComponent->foo();
   }
}
Much nicer. However we can apply the same trick we used on models to make it neater again.
Add the followig function to AppController
function c($componentName )
{
  $this->loadComponent($componentName);
  return$this->$componentName;
}
And our example is
class MyController extends AppController
{ 
  function do_something()
  {
     $this->c('MyComponent')->foo();
  }
}

About Me

So this is "The Robots Brain", the blog of Michael Anderson, developer for Run With Robots.

I'll be posting code related thoughts here, mostly C++, python, php and actionscript.

Not sure it will be of much use to anyone but me .. but who knows.