Here is what finally worked for me (on Ubuntu 18.04).

The /home/username/start_script.sh (don't forget the chmod +x for that file):

#!/bin/bash
set -x
set -e

byobu list-sessions | grep my-app || byobu new-session -d -s my-app

byobu list-windows -t my-app | grep start-script || byobu new-window -t my-app -n 'start-script'

byobu send-keys -t my-app:start-script "cd /home/username/scripts/ && python userscript1.py" C-m

And the /etc/systemd/system/my-app.start_script.service:

[Unit]
Description=My app start script

[Service]
Type=oneshot
ExecStart=/bin/bash -l -c '/home/username/start_script.sh'
User=username
Group=usergroupname

[Install]
WantedBy=multi-user.target

Then install it thanks to sudo systemctl enable my-app.start_script.service.

To see the logs of the service: sudo journalctl -u my-app.start_script.service.

Thanks to this answer, this one and this one for the hints.

Here is what finally worked for me (on Ubuntu 18.04).

The /home/username/start_script.sh (don't forget the chmod +x for that file):

#!/bin/bash
set -x
set -e

byobu list-sessions | grep my-app || byobu new-session -d -s my-app

byobu list-windows -t my-app | grep start-script || byobu new-window -t my-app -n 'start-script'

byobu send-keys -t my-app:start-script "cd /home/username/scripts/ && python userscript1.py" C-m

And the /etc/systemd/system/my-app.start_script.service:

[Unit]
Description=My app start script

[Service]
Type=forking
ExecStart=/bin/bash -l -c '/home/username/start_script.sh'
User=username
Group=usergroupname

[Install]
WantedBy=multi-user.target

Then install it thanks to sudo systemctl enable my-app.start_script.service.

To see the logs of the service: sudo journalctl -u my-app.start_script.service.

Thanks to this answer, this one and this one for the hints.