I have a distributed application backed by a distributed database. (Distributed here means running across multiple hosts with one or more instances on each host). Application has several toolings around it, each does a different operation such as getting some form of update, starting certain application-specific processes, etc.
Both databases and applications have their own set of configurations; all the tools also have configurations around them. Many of these config parameters cut across the tools.
In order to manage this entire system from start to updates to backup to queries to destroy, I have commands for each. Commands here in most cases mean running a binary/executable with certain command-line arguments. Due to the distributed nature, often the commands will have to be executed across all the hosts part of the system.
Update: I would be maintaining several such systems. Hence, in the end, I might have to manage let's say 20 systems; each running the application at version x.x backed by a database running at version y.y
What is the best way to manage such an application?
To give more context, here is how I am thinking of managing it. Design a server application (in java/go) that exposes APIs to execute each of the commands as stated above. Have a superset of different configurations, and store that config in another database. Clients (i.e maintainer of the application) would send some sort of identifier information about the application as part of the request; the server would receive the request, get the config from the database and execute that command. This server would run on all the hosts part of my system. For the above example, if let's say each system requires 5 hosts; I would have this server running on a total of 20*5=100 hosts.
I feel like this is a general problem and I am confident of there being some industry-standard tool or at least a conceptual solution for the same. If so, any guide to it would be really helpful.