Multithreaded local server

Overview

You over-allocate the number of child threads. Remember you should save one thread (the main thread for listening for new messages).

// Remember master thread takes a CPU while listening for connections.
unsigned n_concThreads = std::thread::hardware_concurrency() - 1;

Using multiple threads to handle incoming connection is not the way to go. The problem here is that the parallelization is limited by the number of threads (which is usually in the single digits low double digits if you are lucky).

A single thread can handle thousands of connections simultaneously because most of the time is spent waiting for data on the port. Thus your thread will be idle for most of the time it is handling a request waiting for input. If handled correctly, this idle time could be used to handle other requests.

A good example is the Node.js server. It is single-threaded and will easily handle thousands of incoming connections.

If you want to do this properly you need to use a library like LibEvent (or you can do it manually with select(), pselect() or ppoll()). This allows you to handle multiple sockets with the same thread.

But to be blunt. Doing this is not trivial and not for the beginner. I would opt for a server that already does this part of the work for you. Apache for heavyweight, or nginx for lightweight, or if you want to use another language Node.js. All these servers allow you to write your own request handling code but do the complex stuff for you.

Code Review

Bad form to have global variables.

std::unordered_map<int, std::string> requestQueue;
std::mutex requestQueue_mutex;
std::condition_variable processorsThreadSwitch;
bool gotNewRequests = false;

Would be much nicer to wrap this in a class and pass a reference to each string. That would make the application much more expandable in the long run.

OK. This is speculation. But I find it strange that something would throw a std::string! It would be more normal to throw something derived from std::runtime_error.

    try
    {
        app.createServer(pathToSocket);
    }
    catch (const std::string & err)
    {
        MS::log(SERVICE_NAME, "Failed to start the service. Error: " + err, MS::MessageType::FatalException);

        return -1;
    }

Also I would not return -1. I would rethrow the exception. This gives the calling function more information about the issue.

OK. A bit of a hack. But clever way of finding the number of processors.

    unsigned n_concThreads = std::thread::hardware_concurrency();

    if (!n_concThreads) //if the query failed...
    {
        std::ifstream cpuinfo("/proc/cpuinfo");

        n_concThreads = std::count(std::istream_iterator<std::string>(cpuinfo),
                        std::istream_iterator<std::string>(),
                        std::string("processor"));

        if (!n_concThreads)
            n_concThreads = 6; // ~number of CPU cores. TODO: make the number of worker processes/threads configurable using a config file
    }

But it is OS-specific. So I would wrap it in a function that is OS specific. That way on Linux you can do your clever check. On Windows it defaults to 6 (and the next person can simply replace the Windows version with a Windows specific hack without having to understand your Linux hack).

Not a fan of detaching the thread.

    for (int i = 0; i < n_concThreads; ++i)
    {
        std::thread t (pullRequests);
        t.detach();
    }

You lose all control of the thread. You also lose any warnings when you have a bug in your code and you accidentally shut down while one of the threads is still executing. Using detach() should be a last resource when all neat ways of doing something have been tried and failed.

The locking in pullRequests() seems highly roundabout. Move all the locking into a separate function. That way your scope lock behaves like it should.

I would have written it like this:

std::unordered_map<int, std::string> getNextTask()
{
     std::unique_lock<std::mutex> writeLock(requestQueue_mutex);

     // note this can be written more succinctly.
     // But I am to lazy to look up condition variables at the moment.
     // The following three lines should be a one liner
     // with wait taking a lambda
     while (!gotNewRequests) {
         processorsThreadSwitch.wait(writeLock);
     }

     std::unordered_map<int, std::string> result = std::move(requestQueue);
     return result;
}

void pullRequests()
{
    while(/*test*/)
    {
        // STUFF
        std::unordered_map<int, std::string> queueCopy = getNextTask();
        // STUFF
    }
}

Seems like some repeated code:

            if (!sendResult.isValid())
                    MS::log(SERVICE_NAME, "Could not send the response for request: " + queueCopy.begin()->second, MS::MessageType::Error);
            }

            if (!uds.closeConnection(queueCopy.begin()->first))
                MS::log(SERVICE_NAME, "Could not close the connection.", MS::MessageType::Error);

If you find repeated code then put it in a function.