Python unittest AssertionError not being raised

Question

I am trying to write unit tests for my Python package, and I am finding that, when I run the tests, AssertionErrors are not being raised, when they should be. Here is an MWE:

In exampleModule.py I have:

#! /usr/bin/env python
import unittest

class UnitTest(unittest.TestCase):

    def runTest(self):
        print("Starting test...")
        a = 4
        b = 5
        self.assertEqual(a,b)
        print("TEST COMPLETE")
        return

And in testError.py I have:

#! /usr/bin/env python
import unittest

class AllTests(unittest.TestCase):

    def testExample(self):
        from exampleModule import UnitTest
        UT = UnitTest()
        UT.run()
        return

if __name__ == '__main__':
    unittest.main()

When I run testError.py I expect to see the AssertionError reported from the UnitTest in exampleModule.py, however, I simply see the following:

> ./testError.py
Starting test...
.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

Why is the AssertionError not being raised? If I place the UnitTest() class in testError.py (i.e., have everything in the same file) then the AssertionError is raised. So why, when UnitTest is stored in a different file, does the error not get raised?

Answer 1

TestCase.run() creates or updates a TestResult object, on the assumption you intend to do something interesting with those results.

But you immediately throw them away:

    UT.run()

Any failures or errors—including exceptions raised—would have been in that results object.

For example:

def testExample(self):
    from exampleModule import UnitTest
    UT = UnitTest()
    result = UT.run()
    print('Errors:  {!r}'.format(result.errors))
    print('Failures:  {!r}'.format(result.failures))
    return

This prints:

Failures: [(<exampleModule.UnitTest testMethod=runTest>, 'Traceback (most recent call last):\n File "/home/kjc/tmp/exampleModule.py", line 11, in runTest\n self.assertEqual(a, b)\nAssertionError: 4 != 5\n')]

It catches all kinds of exceptions, like this example where I added sys.exit() before the assertion:

Errors: [(<exampleModule.UnitTest testMethod=runTest>, 'Traceback (most recent call last):\n File "/home/kjc/tmp/exampleModule.py", line 11, in runTest\n import sys; sys.exit()\nSystemExit\n')]

For the record, the one passing test your example produced is testExample itself. You can verify this by calling self.fail() early in testExample. (As one commentator said, calling unit tests from a unit test is a very strange thing to do.)

One Solution

You seem to want to run tests normally, but from multiple files. If so, you can make a bunch of typical unittest.main()-style test modules, and then load them manually.

Your test-everything script would look something like this:

#!/usr/bin/env python
import unittest


# Edit this to include new modules or TestCases to run.
_NAMES = ['testmoda', 'testmodb']  # testothermodule.SomeTestCase, etc...


def _main():
    suite = unittest.defaultTestLoader.loadTestsFromNames(_NAMES)
    unittest.TextTestRunner().run(suite)
    return  # WARNING: This always returns a successful exit value.


if __name__ == '__main__':
    _main()

Your test modules would be the usual one-or-more TestCases, with conditional calls to unittest.main(), so each could be run independently:

#!/usr/bin/env python
import unittest


class TestModA(unittest.TestCase):

    def testThatPasses(self):
        self.assertEqual(0, -0)
        return

    def testThatFails(self):
        self.assertEqual(0, float('inf'))
        return

    def testThatBlowsUp(self):
        raise RuntimeError('OMG!')
        return


if '__main__' == __name__:
    unittest.main()

If that's not fiddly enough for you, you can also read up on the discover() method or the module-level load_tests protocol.

Answer 2

The convention to structure unittests is like this:

Hold all tests in tests directory.

test module will be named as test_modulea.py with class TestClassA

nosetests -vw tests to run all tests.