C++ Mock Library: Part 3

Question

Parts

Define normal use case patterns:

The next part of this mocking exercise was to simplify the usage.

So here I wanted to define objects that had the standard call pattern for constructors/destructors or normal code. So I came up with the pattern.

Review

The code review is important. But also pointers on the design of the interface would be appreciated.

Expected Use:

TEST(UniTestBloc, MyUnitTestBlocking)
{
    TA_TestNoThrow([](){                    // There is also a TA_TestThrow variant
        ThorsSocket::Socket socket("www.google.com", 80);
    })
    .expectObject(Socket)                   // We have a socket class
                                            // There will be calls in the constructor
                                            // There will be calls in the destructor
                                            // Exceptions in the constructor means destructor not called.
    .run();
}

In the above we have separately defined a Socket thing that defines what is being called in the constructor / destructor of a class object. The TA_TestNoThrow will validate that these calls are all done in the correct order.

The main variation of this is that sometimes the expected calls change (because of input parameters) so we need to allow for simple variations.

TEST(UniTestBloc, MyUnitTestNonBlocking)
{
    TA_TestNoThrow([](){
        ThorsSocket::Socket socket("www.google.com", 80, Blocking::No);
    })
    .expectObjectTA(Socket)
        .expectCallTA(fcntl).toReturn(0)             // Making the socket non blocking adds a call to fcntl
    .run();
}

The third variation is that we inject error conditions into what the mocked functions return thus resulting in different behavior.

TEST(UniTestBloc, MyUnitTestBlockingWithConnectFailure)
{
    TA_TestThrow([](){
        ThorsSocket::Socket socket("www.google.com", 80, Blocking::No);
    })
    .expectObjectTA(Socket)
        .expectCallTA(connect).inject().toReturn(-1)    // Connect Fails
                                                        // This causes an exception so no destructor is called.
        .expectCallTA(close)                            // But we still expect close to be called.
    .run();
}

The expectCallTA() function can be chained with other expectCallTA() functions and with modified calls:

.expectCallTA(<func>)[.inject()]?[.anyOrder()]?[.count(<n>)]?[.toReturn(<value>)]*

// inject()   => Removes any existing calls to <func>
// anyOrder() => By default all expected calls are ordered.
//               If you use `anyOrder()` the order of the call is not looked at.
// count(n)   => The call will return the result `n` times.
// toReturn(V)=> The value "V" will be returned "n" times if not specified the
//               an object constructed with `{}` of the correct type will be returned
//               toReturn() can be used multiple times to specify multiple calls.
//
// TODO: expectedInput(....);  :-)

Also the expectObjectTA() can be chained from other expectObjectTA() calls or after an expectCallTA() set up. A more complicated example:

TEST(UniTestBloc, SecureSSLSocket)
{
    TA_TestThrow([](){
        ThorsSocket::SSLSocket socket("www.google.com", 443, Blocking::No);
    })
    // Note: ThorsSocket::SSLSocket derived from ThorsSocket::Socket
    .expectObjectTA(Socket)
       .expectCallTA(fcntl).toReturn(0)
    .expectObjectTA(SSLSocket)
        .expectCallTA(SSL_connect).inject().toReturn(-1)
        .expectCallTA(SSL_shutdown)
        .expectCallTA(SSL_free)
    .run();
}

Interface Design

The TA_TestNoThrow and TA_TestThrow are specializations of TA_Test. In this review I will show the parts that handle the setting up of the expected function calls (how this is all tied together during the running of the unit tests is for another review).

Simplification with Macros

Any method call that ends in TA is actually a macros. The convention is methodTA(F) will expand to method(MOCK2_BUILD_MOCK_NAME(F)) to simplify the usage. The user of the code simply needs to know the name of the function not the underlying implementaiton:

#define expectInitTA(func)               expectInit(MOCK2_BUILD_MOCK_NAME(func))
#define expectDestTA(func)               expectDest(MOCK2_BUILD_MOCK_NAME(func))
#define expectCallTA(func)               expectCall(MOCK2_BUILD_MOCK_NAME(func))
#define optionalTA(func)                 optional(MOCK2_BUILD_MOCK_NAME(func))

TA_Test

    class TA_Test
    {
        std::vector<TA_Object>          codeBlocks;

        public:
            TA_Test& expectObject(TA_Object const& object);
            TA_Test& expectCode(TA_Object const& object);

            template<typename MockObject>
            TA_Code<MockObject> expectCall(MockObject& action);
            void run();
    };

    // -------------------------
    // TA_Test
    // -------------------------
    
    // The calls `expectObject()` and `expectCode()` are identical.
    // The way the underlying objects are created for code and objects
    // is slightly different but the object they create are the same
    // so the different functions are simply to help self documenting
    // the unit tests.
    TA_Test& TA_Test::expectObject(TA_Object const& object)
    {
        codeBlocks.emplace_back(object);
        return *this;
    }
    
    TA_Test& TA_Test::expectCode(TA_Object const& object)
    {
        codeBlocks.emplace_back(object);
        return *this;
    }
    
    template<typename MockObject>
    TA_Code<MockObject> TA_Test::expectCall(MockObject& action)
    {
        return TA_Code<MockObject>(*this, action, Insert::Append, Order::InOrder);
    }

TA_Code

    // MockObject: This represents a mocked out function object.
    //             This code will be presented in part 4
    //             But because each function has a unique interface this can
    //             be a unique type.

    template<typename MockObject>
    class TA_Code
    {
        protected:
            TA_Test&    parent;
            MockObject& action;
            std::size_t callCount;
            Insert      insert;
            Order       order;
            bool        callSaved;

        public:
            using Ret = typename MockObject::Ret;

            TA_Code(TA_Test& parent, MockObject& action, Insert insert, Order order);

            TA_Code& toReturn(Ret&& result);
            TA_Code& inject();
            TA_Code& anyOrder();
            TA_Code& count(int newCount);

            template<typename NextMockObject>
            TA_Code<NextMockObject> expectCall(NextMockObject& nextAction);

            TA_Test& expectObject(TA_Object const& object);
            TA_Test& expectCode(TA_Object const& object);
            void run();
        private:
            void saveAction(Ret&& result);
    };

    // -------------------------
    // TA_Code
    // -------------------------
    
    template<typename MockObject>
    TA_Code<MockObject>::TA_Code(TA_Test& parent, MockObject& action, Insert insert, Order order)
        : parent(parent)
        , action(action)
        , callCount(1)
        , insert(insert)
        , order(order)
        , callSaved(false)
    {}
    
    template<typename MockObject>
    void TA_Code<MockObject>::run()
    {
        if (!callSaved) {
            saveAction({});
        }
        parent.run();
    }
    
    template<typename MockObject>
    TA_Code<MockObject>& TA_Code<MockObject>::toReturn(Ret&& result)
    {
        saveAction(std::forward<Ret>(result));
        return *this;
    }
    
    template<typename MockObject>
    TA_Code<MockObject>& TA_Code<MockObject>::inject()
    {
        insert = Insert::Inject;
        return *this;
    }
    
    template<typename MockObject>
    TA_Code<MockObject>& TA_Code<MockObject>::anyOrder()
    {
        order = Order::Any;
        return *this;
    }
    
    template<typename MockObject>
    TA_Code<MockObject>& TA_Code<MockObject>::count(int newCount)
    {
        callCount = newCount;
        return *this;
    }
    
    
    template<typename MockObject>
    TA_Test& TA_Code<MockObject>::expectObject(TA_Object const& object)
    {
        if (!callSaved) {
            saveAction({});
        }
        return parent.expectObject(object);
    }
    
    template<typename MockObject>
    TA_Test& TA_Code<MockObject>::expectCode(TA_Object const& object)
    {
        if (!callSaved) {
            saveAction({});
        }
        return parent.expectCode(object);
    }
    
    template<typename MockObject>
    template<typename NextMockObject>
    TA_Code<NextMockObject> TA_Code<MockObject>::expectCall(NextMockObject& nextAction)
    {
        if (!callSaved) {
            saveAction({});
        }
        return TA_Code<NextMockObject>(parent, nextAction, Insert::Append, Order::InOrder);
    }
    
    template<typename MockObject>
    void TA_Code<MockObject>::saveAction(Ret&& result)
    {
        if (parent.codeBlocks.empty()) {
            parent.codeBlocks.emplace_back();
        }
        callSaved = true;
        parent.codeBlocks.back().expectCall(action, callCount, std::forward<Ret>(result), insert, order);
    }

TA_ObjectOption

    template<typename MockObject>
    class TA_ObjectOption
    {
        using ReturnType = typename MockObject::Ret;

        TA_Object&      object;
        ActionStore&    store;
        MockObject*     mockObjectHolder;
        std::size_t     callCount;
        ReturnType      result;
        Required        required;
        Order           order;

        public:
            TA_ObjectOption(TA_Object& object, std::size_t callCount, ActionStore& store, MockObject& mockObjectHolder, Required required, Order order);

            operator TA_Object& ();
            TA_ObjectOption& toReturn(ReturnType&& ret);
            TA_ObjectOption& anyOrder();
            TA_ObjectOption& count(int newCount);

            template<typename NextMockObject>
            TA_ObjectOption<NextMockObject> expectInit(NextMockObject& action);
            template<typename NextMockObject>
            TA_ObjectOption<NextMockObject> expectDest(NextMockObject& action);
            template<typename NextMockObject>
            TA_ObjectOption<NextMockObject> optional(NextMockObject& action);
        private:
            void saveAction();
    };

    // -------------------------
    // TA_ObjectOption
    // -------------------------
    
    template<typename MockObject>
    TA_ObjectOption<MockObject>::TA_ObjectOption(TA_Object& object, std::size_t callCount, ActionStore& store, MockObject& mockObjectHolder, Required required, Order order)
        : object(object)
        , store(store)
        , mockObjectHolder(&mockObjectHolder)
        , callCount(callCount)
        , result()
        , required(required)
        , order(order)
    {}
    
    template<typename MockObject>
    TA_ObjectOption<MockObject>::operator TA_Object& ()
    {
        saveAction();
        return object;
    }
    
    template<typename MockObject>
    TA_ObjectOption<MockObject>& TA_ObjectOption<MockObject>::toReturn(ReturnType&& ret)
    {
        result = std::move(ret);
        return *this;
    }
    
    template<typename MockObject>
    TA_ObjectOption<MockObject>& TA_ObjectOption<MockObject>::anyOrder()
    {
        order = Order::Any;
        return *this;
    }
    
    template<typename MockObject>
    TA_ObjectOption<MockObject>& TA_ObjectOption<MockObject>::count(int newCount)
    {
        callCount = newCount;
        return *this;
    }
    
    template<typename MockObject>
    template<typename NextMockObject>
    inline TA_ObjectOption<NextMockObject> TA_ObjectOption<MockObject>::expectInit(NextMockObject& action)
    {
        saveAction();
        return TA_ObjectOption<NextMockObject>(object, 1, object.init, action, Required::Yes, Order::InOrder);
    }
    template<typename MockObject>
    template<typename NextMockObject>
    inline TA_ObjectOption<NextMockObject> TA_ObjectOption<MockObject>::expectDest(NextMockObject& action)
    {
        saveAction();
        return TA_ObjectOption<NextMockObject>(object, 1, object.dest, action, Required::Yes, Order::InOrder);
    }
    template<typename MockObject>
    template<typename NextMockObject>
    inline TA_ObjectOption<NextMockObject> TA_ObjectOption<MockObject>::optional(NextMockObject& action)
    {
        saveAction();
        return TA_ObjectOption<NextMockObject>(object, -1, object.opt, action, Required::No, Order::Any);
    }
    
    template<typename MockObject>
    inline void TA_ObjectOption<MockObject>::saveAction()
    {
        store.add(order, [mock = mockObjectHolder, r = std::move(result), c = callCount, re = required, o = order](Action action, std::size_t index, bool last) mutable {
            return mock->expectInit(action, index, c, last, std::move(r), re, o);
        });
    }

TA_Object

    class TA_Object
    {
        private:
            template<typename MockObject, typename Result>
            void expectCall(MockObject& action, std::size_t callCount, Result&& result, Insert insert, Order order);

            // Code removed for part 4
    };

    // -------------------------
    // TA_Object
    // -------------------------
    

    template<typename MockObject, typename Result>
    void TA_Object::expectCall(MockObject& action, std::size_t callCount, Result&& result, Insert insert, Order order)
    {
        // Save expected function call into TA_Object (See part 4)
    }

G. Sliepen · Accepted Answer · 2023-09-16 21:14:40Z

I can see how this can simplify some tests. The code looks also quite clean. However, I'm a bit worried about whether you should go this way. Here's why:

About testing how a function is implemented

I see this test as being problematic:

TA_TestNoThrow([](){
    ThorsSocket::Socket socket("www.google.com", 80, Blocking::No);
})
.expectObjectTA(Socket)
.expectCallTA(fcntl).toReturn(0) // Making the socket non blocking adds a call to fcntl
.run();

Why are you testing specifically for how the constructor of Socket is making the socket non-blocking? There are various ways to do this, including:

using fcntl() as you have implemented
using SOCK_NONBLOCK in the call to ::socket()
using MSG_DONTWAIT in calls to sendto() and recvfrom() on the socket

Furthermore, the test you wrote doesn't actually check that fcntl(fd, F_SETFL, O_NONBLOCK) is called, only that fcntl() is called at least once. And even if it sets O_NONBLOCK, it could have a second call to fcntl() that clears O_NONBLOCK. So this test can produce both false positives and false negatives, and if you change the code to use a different method, you break the test.

Much better would be to not test the implementation specifics, but rather the observable effects of the class, although in this case that would be quite a bit more work.

Nesting and ordering ambiguities

In the following code snippet:

.expectObjectTA(SSLSocket)
    .expectCallTA(SSL_connect).inject().toReturn(-1)
    .expectCallTA(SSL_shutdown)
    .expectCallTA(SSL_free)

Do we expect SSLSocket's constructor to make three calls (to SSL_connect(), then SSL_shutdown(), and then SSL_free()), or do we expect SSL_connect() to call SSL_shutdown() which in turn is expected to call SSL_free()? Knowing OpenSSL and seeing the indentation you used I know what to expect here, but what if I did expect the latter?

.expectObjectTA(Socket)
    …
.expectObjectTA(SSLSocket)
    …

Does this mean you expect a Socket object to be constructed first, then a SSLSocket? What if the order is reversed (for example, if SSLSocket had a Socket* member, and new Socket(…) was called inside SSLSocket's constructor's body)? Would the test fail or not?

I see there are anyOrder() and count() member functions, but it's also unclear what they apply to: calls, objects or both? And will anyOrder() only allow previous things to be in any order, or only the following ones, or both?

Much better would be to not test the implementation specifics, but rather the observable effects of the class. This is a different style of tests (I do this as well but separately). This style of tests is to make sure any changes does not change behavior that is not observable. i.e. Did my change stop the close() being called. You need both style of tests to be comprehensive. This also allows me to make sure that things happen as expected when I inject error situations. — Loki Astari
– Loki Astari, Commented Sep 17, 2023 at 20:31
wrote doesn't actually check that fcntl(fd, F_SETFL, O_NONBLOCK) I have now added that functionality. :-) Thanks. But because of the rules I can't change previous review requests. But I will publish this and add links soon. — Loki Astari
– Loki Astari, Commented Sep 17, 2023 at 20:32

Stack Exchange Network

C++ Mock Library: Part 3

Parts

Define normal use case patterns:

Review

Expected Use:

Interface Design

Simplification with Macros

TA_Test

TA_Code

TA_ObjectOption

TA_Object

1 Answer 1

About testing how a function is implemented

Nesting and ordering ambiguities

You must log in to answer this question.

Linked

Hot Network Questions

C++ Mock Library: Part 3

Parts

Define normal use case patterns:

Review

Expected Use:

Interface Design

Simplification with Macros

TA_Test

TA_Code

TA_ObjectOption

TA_Object

1 Answer 1

About testing how a function is implemented

Nesting and ordering ambiguities

You must log in to answer this question.

Linked

Related

Hot Network Questions