Tricks with nose and python

Nose is a very useful tool for running unittest in python. These are a few tricks you can use. Bellow is my test file - I called it test.py

# dummy case test 

class Test():

    def test_algo(self):
        assert 0 == 0, '0 is not equal to 0'

    def test_failed(self):
        print 'this will fail'
        assert 1 == 0, '0 is not equal to 1'

    def test_fail_inpdb(self):
        # div by 0
        1/0

Now let's see what is this about

1. first I use assert to check if the results match
2. based on assert the first function will pass and second will fail
3. the last function will trigger and Error not a Failure

# running with pdb so any Error not Failure will drop me into python debugger
$ nosetests  --pdb  test.py
.> /home/silviud/PROGS/PYTHON/wal/tests/test.py(14)test_fail_inpdb()
-> 1/0
(Pdb) l
  9             print 'this will fail'
 10             assert 1 == 0, '0 is not equal to 1'
 11     
 12         def test_fail_inpdb(self):
 13             # div by 0
 14  ->         1/0          #### this is the line that triggers the error
[EOF]
(Pdb) c
EF
======================================================================
ERROR: tests.test.Test.test_fail_inpdb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 14, in test_fail_inpdb
    1/0
ZeroDivisionError: integer division or modulo by zero

======================================================================
FAIL: tests.test.Test.test_failed
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 10, in test_failed
    assert 1 == 0, '0 is not equal to 1'
AssertionError: 0 is not equal to 1
-------------------- >> begin captured stdout << ---------------------
this will fail

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 3 tests in 8.453s

FAILED (errors=1, failures=1)

Now let's pretended that I run this regular and is part of my Continuous Integration server which so happen
to be running Jenkins. How can I integrate the python unittests with it ?!
Simple - nose has many plugins and one of them is xunit.

$ nosetests  --with-xunit test.py
....
$ cat nosetests.xml
<?xml version="1.0" encoding="UTF-8"?><testsuite name="nosetests" tests="3" errors="1" failures="1" skip="0"><testcase classname="tests.test.Test" name="test_algo" time="0.000" /><testcase classname="tests.test.Test" name="test_fail_inpdb" time="0.000"><error type="exceptions.ZeroDivisionError" message="integer division or modulo by zero"><![CDATA[Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 321, in run
    testMethod()
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 14, in test_fail_inpdb
    1/0
ZeroDivisionError: integer division or modulo by zero
]]></error></testcase><testcase classname="tests.test.Test" name="test_failed" time="0.001"><failure type="exceptions.AssertionError" message="0 is not equal to 1&#10;-------------------- &gt;&gt; begin captured stdout &lt;&lt; ---------------------&#10;this will fail&#10;&#10;--------------------- &gt;&gt; end captured stdout &lt;&lt; ----------------------"><![CDATA[Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 321, in run
    testMethod()
  File "/home/silviud/Environments/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/silviud/PROGS/PYTHON/wal/tests/test.py", line 10, in test_failed
    assert 1 == 0, '0 is not equal to 1'
AssertionError: 0 is not equal to 1
-------------------- >> begin captured stdout << ---------------------
this will fail

--------------------- >> end captured stdout << ----------------------
]]></failure></testcase></testsuite>

Just by adding --with-xunit made nose to active the xunit plugin and it generated an xml file into nosetests.xml - this file can be used by Jenkins to take decisions if the build failed or not !

Shell parallel processing

This is the description of the tool from the gnu site - Parallel
GNU parallel is a shell tool for executing jobs in parallel using one
or more computers. A job it can be a single command or a small script
that has to be run for each of the lines in the input. The typical
input is a list of files, a list of hosts, a list of users, a list of
URLs, or a list of tables. A job can also be a command that reads from
a pipe. GNU parallel can then split the input into blocks and pipe a
lock into each command in parallel.

The tool can do many things and has some very useful tools that come with it
see sql and niceload.
Bellow you can see some example on how to use it.

#!/bin/sh

# tail log files on different computers
# create a hosts file with all the computers you want to connect

echo '10.100.218.79' >> host.file
echo '107.22.24.219' >> host.file
cat host.file | parallel ssh {} "tail /var/log/php-fpm/error.log | awk '{print \$1,\$2,\$3,\$4,\$5,\$6}'"

[20-Feb-2012 15:05:05.323178] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:06.324028] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:07.324877] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:08.325727] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:09.326568] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:10.327418] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:11.328265] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:12.329118] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:13.329960] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:14.330806] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),

GNU's site has lots more example see Examples