bash -c "exec -a myProcessName ./script.sh &" doesn't send that script into backgound

Question

From a bash script, I need to start three processes and put to each of these processes a name, in order to be able to stop them in a later stop command.

# Démarrer les applications eco emploi : backend métier, backend du front (Java), et front (Angular)
demarrer_applications() {
  emit "application" "start" "init" "Démarrage des applications ecoemploi" "Soumission des scripts de démarrage"

  # Partie métier, backend Java
  log "INFO" "Demande de démarrage de l'application métier ecoemploi (Backend Java)"
  START_METIER=$(bash -c "exec -a $ECOEMPLOI_METIER_PROCESS_NAME ./submit_ecoemploi_metier_back.sh &")

  if [ $? -ne 0 ]; then
     emit "application" "start" "fail" "Soumission de ecoemploi backend métier échouée" "$START_METIER"
     log "ERROR" "La demande de soumision de l'application métier ecoemploi (Backend Java) a échoué : $START_METIER"
     exit $?
  fi

  # Partie front, backend Java
  log "INFO" "Demande de démarrage de l'ihm ecoemploi (Backend Java)"
  START_IHM_BACK=$(bash -c "exec -a $ECOEMPLOI_IHM_BACK_PROCESS_NAME ./submit_ecoemploi_ihm_back.sh &")

  if [ $? -ne 0 ]; then
     emit "application" "start" "fail" "Soumission de ecoemploi backend ihm échouée" "$START_IHM_BACK"
     log "ERROR" "La demande de soumision de l'application ihm ecoemploi (Backend Java) a échoué : $START_IHM_BACK"
     exit $?
  fi

  log "INFO" "Demande de démarrage de l'ihm ecoemploi (Front Angular)"
  START_IHM_FRONT=$(bash -c "exec -a $ECOEMPLOI_IHM_FRONT_PROCESS_NAME ./submit_ecoemploi_ihm_front.sh &")

  if [ $? -ne 0 ]; then
     emit "application" "start" "fail" "Soumission de ecoemploi ihm front échouée" "$START_IHM_FRONT"
     log "ERROR" "La demande de soumision de l'application ihm ecoemploi (Front Angular) a échoué : $START_IHM_FRONT"
     exit $?
  fi

  log "INFO" "Les demandes de soumission des applications ont été faites."
}

Summarizing that script, it attempts to run these commands:

bash -c "exec -a $ECOEMPLOI_METIER_PROCESS_NAME ./submit_ecoemploi_metier_back.sh &"
bash -c "exec -a $ECOEMPLOI_IHM_BACK_PROCESS_NAME ./submit_ecoemploi_ihm_back.sh &"
bash -c "exec -a $ECOEMPLOI_IHM_BACK_PROCESS_NAME ./submit_ecoemploi_ihm_back.sh &"

Where each of these submit sub-scripts have this form:

#!/bin/bash
source ecoemploi-start-stop-common.sh

cd "$ECOEMPLOI_METIER_HOME"

if [ $? -ne 0 ]; then
   emit "application" "start" "fail" "Le répertoire d'installation de l'application métier, $ECOEMPLOI_METIER_HOME, est absent" ""
   log "ERROR" "Le répertoire d'installation de l'application métier, $ECOEMPLOI_METIER_HOME, est absent"
   exit $?
fi

./start.sh

here, for example, start.sh will start a Java 17 application:

java --add-exports java.base/sun.nio.ch=ALL-UNNAMED \
   --add-opens java.base/java.util=ALL-UNNAMED \
   --add-opens java.base/java.nio=ALL-UNNAMED \
   --add-opens java.base/java.lang=ALL-UNNAMED \
   --add-opens java.base/java.lang.invoke=ALL-UNNAMED \
   -jar target/application-metier-et-gestion.jar

and ecoemploi-start-stop-common.sh is only here to define few directory locations and two custom functions: emit to send a message to a Kafka topic, and log to log into the console:

#!/bin/bash
#
# Répertoires d'installation de l'application, et noms des processus qui vont être lancés par bash/exec
ECOEMPLOI_HOME="/home/lebihan/dev/Java/comptes-france"
ECOEMPLOI_METIER_HOME="$ECOEMPLOI_HOME/metier-et-gestion/ApplicationMetierEtGestion"
ECOEMPLOI_IHM_FRONT_HOME="$ECOEMPLOI_HOME/web-client/ApplicationEtude/etude"
ECOEMPLOI_IHM_BACK_HOME="$ECOEMPLOI_HOME/web-client/ApplicationEtude"

ECOEMPLOI_METIER_PROCESS_NAME="ecoemploi_backend_metier"
ECOEMPLOI_IHM_BACK_PROCESS_NAME="ecoemploi_ihm_backend"
ECOEMPLOI_IHM_FRONT_PROCESS_NAME="ecoemploi_ihm_front"

# Communication avec Kafka
KAFKA_BOOTSTRAP_SERVER_PORT=9092
BOOTSTRAP_SERVER="localhost:$KAFKA_BOOTSTRAP_SERVER_PORT"
TOPIC_VIE="ecoemploi-statut"

# Logger un message, horodaté
# $1 Sevérité
# $2 Message
log() {
   TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S")

   if [ "$1" = "ERROR" ]; then
      echo "$TIMESTAMP [$1] $2" >&2
   else
      echo "$TIMESTAMP [$1] $2"
   fi
}

# Emettre un évènement dans le $TOPIC_VIE de Kafka
# $1 : Composant
# $2 : Phase : init, start
# $3 : Statut : start, fail
# $4 : description
# $5 : Message
emit() {
# Laisser la ligne ci-dessous non indentée pour que EOF fonctionne.
MESSAGE_EMIT_KAFKA=$(cat << EOF
{
   "composant": "$1",
   "phase": "$2",
   "statut": "$3",
   "description": "$4",
   "message": "$5"
}
EOF
)

  MESSAGE_EMISSION_VERS_KAFKA=$(echo "$MESSAGE_EMIT_KAFKA" | kafka-console-producer.sh --broker-list $BOOTSTRAP_SERVER --topic $TOPIC_VIE)

  if [ $? -ne 0 ]; then
    log "WARN" "Le message vers le topic $TOPIC_VIE n'a pu être émis : $MESSAGE_EMISSION_VERS_KAFKA"
  fi
}

---

My problem is that in my bash function demarrer_applications(), called after few other ones and producing the last [INFO] log,

2023-04-14 10:01:09 [INFO] Démarrage de Zookeeper puis de Kafka...
2023-04-14 10:01:09 [INFO] Zookeeper et Kafka ont démarré.
2023-04-14 10:01:10 [INFO] Le topic ecoemploi-statut existant sera réutilisé sur Kafka localhost:9092.
2023-04-14 10:01:11 [INFO] Postgresql n'est pas démarré. Démarrage en cours...
2023-04-14 10:01:13 [INFO] Postgresql est prêt.
2023-04-14 10:01:14 [INFO] Demande de démarrage de l'application métier ecoemploi (Backend Java)

do run the wished application, with the command:

bash -c "exec -a $ECOEMPLOI_METIER_PROCESS_NAME ./submit_ecoemploi_metier_back.sh &"

but the & after the ./submit_ecoemploi_metier_back.sh doesn't send that application into the background. And my main script cannot to continue its work and launch the following applications next. It hangs here, waiting for a Ctrl-C.

How shall I submit my submit_ecoemploi_*.sh scripts, in the background, with bash and exec commands? Thanks a lot!

---

@StéphaneChazelas

I used your solution to adapt my function (I only put here the changed lines):

demarrer_applications() {
  # Partie métier, backend Java
  NAME=$ECOEMPLOI_METIER_PROCESS_NAME bash -c 'exec -a "$NAME" ./submit_ecoemploi_metier_back.sh &'

  NAME=$ECOEMPLOI_IHM_BACK_PROCESS_NAME bash -c 'exec -a "$NAME" ./submit_ecoemploi_ihm_back.sh &'

  NAME=$ECOEMPLOI_IHM_FRONT_PROCESS_NAME bash -c 'exec -a "$NAME" ./submit_ecoemploi_ihm_front.sh &'
}

it submits perfectly, and I my application see its three components started and is useable.

but a ps -f returns me that:

UID          PID    PPID  C STIME TTY          TIME CMD
lebihan   309213  309207  0 11:18 pts/0    00:00:00 bash
lebihan   322835    2710 21 11:26 pts/0    00:00:04 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:Ma
lebihan   324800    2710  0 11:27 pts/0    00:00:00 /bin/bash /home/lebihan/dev/Java/comptes-france/submit_ecoemploi_metier_back.sh
lebihan   324803    2710  0 11:27 pts/0    00:00:00 /bin/bash /home/lebihan/dev/Java/comptes-france/submit_ecoemploi_ihm_back.sh
lebihan   324804  324800  0 11:27 pts/0    00:00:00 /bin/bash ./start.sh
lebihan   324806  324803  0 11:27 pts/0    00:00:00 /bin/bash ./start.sh
lebihan   324808  324804 99 11:27 pts/0    00:00:47 java --add-exports java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --
lebihan   324809    2710  0 11:27 pts/0    00:00:00 /bin/bash /home/lebihan/dev/Java/comptes-france/submit_ecoemploi_ihm_front.sh
lebihan   324811  324806 52 11:27 pts/0    00:00:07 java --add-exports java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --
lebihan   324812  324809  0 11:27 pts/0    00:00:00 /bin/bash ./start.sh
lebihan   324814  324812 51 11:27 pts/0    00:00:07 ng serve
lebihan   325316  309213  0 11:27 pts/0    00:00:00 ps -f

And I don't see the process names changed.
Then the stop command isn't able to stop the submit_ecoemploi_ihm_front.sh, submit_ecoemploi_ihm_back.sh, submit_ecoemploi_metier_back.sh

Answer 1

output=$(cmd &)

Collects the output of the subshell that runs the cmd & code via a pipe; all the processes started within that subshell inherit that pipe as their stdout whether they're started asynchronously or not.

The shell will read from the other end of that pipe to make up the expansion until it reaches end-of-file on that pipe which will happen only when all the file descriptors (by any process) open on the writing end of the pipe have been closed, so typically when all the processes have died.

In any case, you can't capture the output of those processes if you return before they have output it.

You can close stdout in those processes started asynchronously there to release the fd open on that pipe if you don't need the rest of the output.

For instance:

output=$(sh -c 'echo some output; sleep 1; exec>&-; sleep 120' &)

Will return after 1 second instead of 121, leaving sh and sleep 120 running in background after stdout has been closed.

Better than exec>&- would be to use exec>/dev/null which would also close the fd to the pipe, but avoid leaving stdout closed which is a bad idea in general as applications usually expect it to be opened.

If you want to capture the output of several commands running concurrently, you can't do that with variables and command substitution. In bash which doesn't have support for select/poll like functionality, best would be to use temporary files:

exec -a name1 cmd1 > "$temp1" &
exec -a name2 cmd2 > "$temp2" &
exec -a name3 cmd3 > "$temp3" &
wait
output1=$(<"$temp1")
output2=$(<"$temp2")
output3=$(<"$temp3")

(assuming bash).

No need for those bash -c ... here, and btw those:

bash -c "exec -a $ECOEMPLOI_METIER_PROCESS_NAME ./submit_ecoemploi_metier_back.sh &"

Are wrong as the contents of that ECOEMPLOI_METIER_PROCESS_NAME ends up being interpreted as bash code which is not what you want. You want is to be taken as the argument to -a, so it should be:

NAME=$ECOEMPLOI_METIER_PROCESS_NAME bash -c '
  exec -a "$NAME" ./submit_ecoemploi_metier_back.sh &'

Or maybe:

bash -c '"$@" &' bash exec -a "$ECOEMPLOI_METIER_PROCESS_NAME" ./submit_ecoemploi_metier_back.sh

But those bash -c are completely superfluous if that's a bash script. Just do:

exec -a "$ECOEMPLOI_METIER_PROCESS_NAME" ./submit_ecoemploi_metier_back.sh &

In any case, if you run a script.sh with a #! /bin/bash shebang (so the .sh suffix is misleading, use .bash or no suffix) as exec -a name ./script.sh arg, the shell will do a execve("./script.sh", ["name", "arg"], env), but as that's a script, on Linux at least, that will in turn run execve("/bin/bash", ["/bin/bash", "./script.sh", "arg"], env), and the name will be lost.

You can use:

exec -a name bash ./script.sh

In which case ps -f will show name ./script.sh. Beware that bash can adjust its behaviour based on the argv[0] it receives. For instance, if invoked with a argv[0] of sh or ending in /sh, it will enable its POSIX mode.

Or:

BASH_ENV=./script.sh exec -a name bash -c ''

In which case ps -f will show "name -c ", but beware that BASH_ENV will end up in the environment and mean script.sh will be sourced any time another bash script (including the bash -c inline ones) is started by script.sh likely causing infinite recursion.

Answer 2

What you have described mostly works for me. I had to change a little part though. Here's my worked example:

First, the contents of script.sh

#!/bin/bash
while [ -f /tmp/script.run ]       # Run only while this file exists
do
    date >/tmp/script.date 2>&1    # Update the output file
    sleep 0.5                      # Pause briefly
done
rm -f /tmp/script.date

Now we'll run it in the way you suggest,

touch /tmp/script.run
bash -c "exec -a myProcessName ./script.sh & "

Let's look at the process tree with ps -f

UID        PID  PPID  C STIME TTY          TIME CMD
roaima    9301  9296  0 09:41 pts/0    00:00:00 -bash
roaima    9913     1  0 09:46 pts/0    00:00:00 /bin/bash /home/roaima/script.sh
roaima    9919  9913  0 09:46 pts/0    00:00:00 sleep 0.5
roaima    9920  9301  0 09:46 pts/0    00:00:00 ps -f

You can see that the process is running (PIDs 9913 and 9919) but the process name has not been applied. That's because it's a script, and the script is executed by the process listed on its #! line.

Let's stop the process, wait a moment, and try again with a different approach

rm -f /tmp/script.run    # Remove the flag file
ps -f                    # Confirm the process has ended

touch /tmp/script.run
bash -c "exec -a myProcessName bash ./script.sh & "

This time the explicit invocation of bash has been renamed as expected:

UID        PID  PPID  C STIME TTY          TIME CMD
roaima    9301  9296  0 09:41 pts/0    00:00:00 -bash
roaima   11745     1  2 09:53 pts/0    00:00:00 myProcessName ./script.sh
roaima   11777 11745  0 09:53 pts/0    00:00:00 sleep 0.5
roaima   11778  9301  0 09:53 pts/0    00:00:00 ps -f

But actually you might as well avoid the outer invocation of bash and just call exec directly,

exec -a myProcessName bash ./script.sh &

Finally, I'd suggest that a better way to keep track of your processes is not by process name but by PID:

./script.sh &
scriptPID=$!

kill -0 $scriptPID && echo "The script is still running with PID $scriptPID"

Remove the flag file /tmp/script.run, pause a moment, and then repeat the kill … && echo … command. You'll have no output because the script is no longer running.

Answer 3

Something similar to what roaima wrote about the well-known location:

  # Initialize a file for collecting background process numbers:
    sync_file=/tmp/syncfile.123
    printf ""  > $sync_file
    # Repeat this block for all your parallel commands:
    # This can be a loop or a fixed number of process submissions
       # Run your command(s) in a sub-shell:
       (
        # Name your process using a single word:
        pname=MyProcess.1
        # Save the name and PID of the bg process 
        printf "$pname:$?\n" >> $sync_file
        # run your command(s)
        my_command
       ) &
    # End block
    
    # This part can be run in a separate script:
    # Kill your processes using PID list you have stored in the file
    sync_file=/tmp/syncfile.123
    for a in $( cat $sync_file )
    do
        procname=${a%:*}  # Get the process name 
        pid=${a#*:}       # Get the PID
        echo "Process: $procname PID: $pid"
        kill -9 $pid
    done