Python unittest: make AssertionError an error instead of a failure

Question

To make migrating existing test suites easier, unittest supports tests raising AssertionError to indicate test failure. However, it is recommended that you use the explicit TestCase.fail*() and TestCase.assert*() methods instead, as future versions of unittest may treat AssertionError differently.

I used quite a few assert statements inside the tested code, but failure of those assertions should really be a test error (i.e. "code did not run properly with those inputs") rather than a failure (i.e. "code gave incorrect output").

I can see the following possible solutions:

Rewrite the tested code to throw better-typed exceptions
Wrap everything inside test methods except the test assertions themselves (self.assertSomething(...)) in try...except AssertionError: raise SomeOtherException blocks
Change unittest's behavior so that it considers those errors rather than failures.

Option 1 would take quite some time, and option 2 feels hacky; option 3 would be the best for me, but is it available? (In case it matters: no, I cannot switch to Python 3.) I do not see anything online, but it is hard to use specific keywords.

MWE:

import unittest


def add_one_to_int(a):
    assert isinstance(a, int)
    return a + 1


class TestAddOne(unittest.TestCase):
    def test_one_plus_one_is_three(self):
        # This tests fails with
        #   AssertionError: 2 != 3
        # which is fine
        self.assertEqual(add_one_to_int(1), 3)  

    def test_add_one_to_str(self):
        # This tests fails with
        #   AssertionError
        # when I would rather have it an error
        add_one_to_int('some string')

if __name__ == '__main__':
    unittest.main(verbosity=2)  # 2 failures instead of 1 failure, 1 error

Inon Peled · Accepted Answer · 2019-03-25 14:40:49Z

I think that option 3 can be achieved through class attribute "failureException", as defined also in unittest docs for Python 2.7:

failureException: This class attribute gives the exception raised by the test method. If a test framework needs to use a specialized exception, possibly to carry additional information, it must subclass this exception in order to “play fair” with the framework. The initial value of this attribute is AssertionError.

So for example:

import unittest

class MyException(Exception): 
    pass

class MyUnitTest(unittest.TestCase):
    failureException = MyException

def add_one_to_int(a):
    assert isinstance(a, int)
    return a + 1


class TestAddOne(MyUnitTest):  # <--------- See above
    def test_one_plus_one_is_three(self):
        # This tests fails with
        #   AssertionError: 2 != 3
        # which is fine
        self.assertEqual(add_one_to_int(1), 3)  

    def test_add_one_to_str(self):
        # This tests fails with
        #   AssertionError
        # when I would rather have it an error
        add_one_to_int('some string')

if __name__ == '__main__':
    unittest.main(verbosity=2)  # ------> FAILED (failures=1, errors=1)

That did the trick. Just for the nitpick: I used MyException(AssertionError) subclassing rather than MyException(Exception), as I feel it is more precise (though it probably does not matter).

snoonan · Accepted Answer · 2023-11-09 22:22:15Z

-1

Better is:

self.assertException(add_one_to_int('some string', AssertionError)

answered Nov 9, 2023 at 22:22

snoonan

1781 silver badge5 bronze badges

Collectives™ on Stack Overflow

Python unittest: make AssertionError an error instead of a failure

2 Answers 2

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Related