Granular vs a aggregated tests when dealing with arrays

Question

I write code in a Test-Driven way and I often build my functions starting with tests for the easy edge cases.

For example, given a flat array of Items that have a category property, return an array of Groups, each built with all the items for one of the categories in the input.

One edge case I find helpful to test first is the behavior when the items are all from the same category. This gives me a chance to see how the Group should look like.

In this case there are (at least) two approaches one can take: two granular tests vs. a single one.

Granular

One could write two granular tests for these conditions:

1. Given an array with items all from the same category, the output should be an array with a single group
2. A group should contain all the items for its category and only those

In Swift, these tests might look something like this

func testGroupingArrayOfSameCategoryReturnsOneGroup() {
    let items = [
        Item(name: "a", category: .foo),
        Item(name: "b", category: .foo),
        Item(name: "c", category: .foo),
    ]

    let groups = groupByCategory(items)

    XCTAssertEqual(groups.count, 1)
}

func testGroupingBuildsGroupWithAllItemsForCategory() throws {
    let items = [
        Item(name: "a", category: .foo),
        Item(name: "b", category: .foo),
        Item(name: "c", category: .foo),
    ]

    let groups = groupByCategory(items)

    let group = try XCTUnwrap(groups.first)
    XCTAssertEqual(group.category, .foo)
    XCTAssertEqual(group.items.count, 3)
    XCTAssertEqual(group.items[0].name, "a")
    XCTAssertEqual(group.items[1].name, "b")
    XCTAssertEqual(group.items[2].name, "c")
}

I like how the two behaviors of having a 1:1 match with groups and categories and how the groups are built are separated, but I see a lot of duplication between those tests.

Aggregated

A different approach would be to check both facets of the behavior in the same test.

func testGroupingArrayOfSameCategoryReturnsOneGroupWithAllItemsForCategory() {
    let items = [
        Item(name: "a", category: .foo),
        Item(name: "b", category: .foo),
        Item(name: "c", category: .foo),
    ]

    let groups = groupByCategory(items)

    XCTAssertEqual(groups.count, 1)
    let group = try XCTUnwrap(groups.first)
    XCTAssertEqual(group.category, .foo)
    XCTAssertEqual(group.items.count, 3)
    XCTAssertEqual(group.items[0].name, "a")
    XCTAssertEqual(group.items[1].name, "b")
    XCTAssertEqual(group.items[2].name, "c")
}

Which do you think is clearer and why?

Answer 1

Depends on how you look at it. I am always for the granular.

Granular

• Gives you more information right out of the box, you have higher chances that some tests will fail and some will pass (which is good!).
• All asserts are (should) be executed, that is cool!
• Arguably easier to maintain (you can just easier delete/remove small tests rather than refactor bigger ones).
• It is more verbose, but otherwise I see only benefits.

Agregated

• Gives you one big red "fail" flag.
• You have to dig deeper and find out what is the problem there. You have less tests, but they are all more likely to get red.
• Asserts are together and some of them may not execute because the test already failed at that point (losing useful information).
• That is all fine if making sure you don't miss a bug is your main priority.

Sidenote

All the asserts in your second "granular" tests are just basically checking expected list against another. I would love if you instead constructed complete expected list and then just run single assert comparing structure of those lists. It's more explicit and clear + then it really is just a single assert, which can also be made more powerful.

Answer 2

I prefer the second option: using a test for both facets of the behavior.

While it has a lot of assertions, I feel it covers the behavior well in one go, removing duplication.

From a pure code economy point of view, we achieve the same thoroughness as the two tests scenario with just one extra line of code.

Moreover, in the two tests example, the first test would pass regardless of how the output is built, as long as it contains only one element. Yes, the second test is there to make sure the Group is built properly, but I'm somehow uncomfortable with how close to a false positive the test that only checks the array count is.

If the second tests was to be removed, code coverage tools wouldn't register a change in coverage, but the thoroughness of what is checked would decrease.