Guava has unit test cases automatically generated:
Guava has staggering numbers of unit tests: as of July 2012, the guava-tests package includes over 286,000 individual test cases. Most of these are automatically generated, not written by hand, but Guava's test coverage is extremely thorough, especially for com.google.common.collect.
How they were generated? What techniques and technologies were used to design and generate them?
A large part of this mass of tests is for the Guava collection implementations. They've written generic tests that exhaustively test the collection interfaces, and this generates a suite per implementation. See, for example, classes called CollectionAddAllTester, ListIndexOfTester.
This is all backed by a library called testlib, which ships as part of Guava. This is quite generic. It supports writing generic tests for any interface (not just collections). You can specify Features of possible implementations and test those (e.g. if your set is unmodifiable you expect a different outcome from set.add()), and when you run the tests you specify which features your implementation supports.
It's based on JUnit 3, not 4. Normally, you have a class extending TestCase full of methods named testSomething(), and JUnit runs them reflectively. The testlib library hooks into the running of these tests so that the lifecycle looks like this:
TestCase instanceTestSubjectGenerator – this is the testlib interface that you extend where you actually create the test subjectgetSubjectGenerator() gives access to the test subjectThe key bit is the extra initialisation step that allows them to inject a specific test subject into the generic test case.
I wrote a post on how to write testlib generating suites for your own interfaces.
(Also posted to the same question on the sqa site.)
There are unit test generators. For example, in the .NET world, something like Microsoft Pex could do this.
For example, Microsoft Pex tries based on code analysis all possible values as arguments for a method. Some arguments are expected to let the method throw an exception. Such things can automatically tests created for. Static values like an empty string that is returned in certain cases can also be automatically be tested.
