realtime pthread created from non-realtime thread with init.d

Question

I have a piece of C++ code that runs just fine when I run it from a Linux terminal, but which throws an EPERM error when run from a SystemV (init.d) script on bootup. The error comes from a pthread_create with the following bit of attributes assigned to the thread attempting to be created:

pthread_t reading_thread;

pthread_attr_t read_attr;
struct sched_param read_param; 
pthread_attr_init(&read_attr);
pthread_attr_setschedpolicy(&read_attr, SCHED_FIFO);
pthread_attr_setinheritsched(&read_attr, PTHREAD_EXPLICIT_SCHED);
read_param.sched_priority = 30;
pthread_attr_setschedparam(&read_attr, &read_param);

k = pthread_create(&reading_thread, &read_attr, Reading_Thread_Function, 
            (void*) &variable_to_pass_to_Reading_Thread_Function); // Will return EPERM

This code works just fine when run from my terminal. It also runs just fine in the init.d script when I call "/etc/init.d/myinitdscript start". It also runs fine as "sudo service myinitdscript start". The init.d script contains the following:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          myinitdscript
# Required-Start:    $local_fs $remote_fs $syslog $network
# Required-Stop:     $local_fs $remote_fs $syslog $network
# Default-Start:     3 4 5
# Default-Stop:      0 1 6
# Short-Description: Starts my daemon
# Description:       Verbose explanation of starting my daemon
### END INIT INFO
PATH=/sbin:/usr/sbin:/bin:/usr/bin
LOG=/home/someusershome/initd.log
NAME=myinitdscript
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
[ -x "$DAEMON" ] || (echo "$DAEMON not found. Exiting $SCRIPTNAME." >> $LOG 2>&1 && exit 0)
USERTORUNAS=a_user_on_my_system
SOURCE_SCRIPT=/home/$USERTORUNAS/source_script
DAEMON_ARGS="some_args_for_script"

. /lib/init/vars.sh

. /lib/lsb/init-functions

# Source this script for environmental variables
[ -f $SOURCE_SCRIPT ] && . $SOURCE_SCRIPT

# This is called when called with 'start', I am skipping that for succintness
do_start()
{
    start-stop-daemon --start --make-pidfile --pidfile $PIDFILE --test --background --chuid $USERTORUNAS --startas /bin/bash -- -c "exec $DAEMON -- $DAEMON_ARGS >> $LOG 2>&1 " || return 1
    start-stop-daemon --start --make-pidfile --pidfile $PIDFILE --background --chuid $USERTORUNAS --startas /bin/bash -- -c "exec $DAEMON -- $DAEMON_ARGS >> $LOG 2>&1" || return 2
}

If I activate this init.d script using:

update-rc.d myinitdscript defaults 99

it will error on boot with an EPERM error thrown at the pthread_create call (k = 1, aka EPERM). I can run this using sudo service myinitdscript start, and it will run just fine. I can also call /etc/init.d/myinitdscript start, and it will run just fine. It is only when I let the system run this script on boot that it fails.

I find that if I add to my start-stop-daemon calls the option "-P fifo:99" I don't get the EPERM error and the code runs okay except at too high a priority so I won't call this a fix. The only part of the code that needs to run real-time is that pthread created from in the code. So I suppose this has to do with my permissions to create a real-time thread with priority 30 from within a normally scheduled thread.

Why does my script need special scheduling policies/priorities when run from boot versus when I manually start the init.d script or through service?

EDIT: Running on Ubuntu 12.04.

EDIT2: I tried adding a call to "ulimit -r" inside my code which the start-stop-daemon call starts, and I get unlimited, so as far as I can see, there shouldn't be any permissions issue going with SCHED_FIFO:30 there

EDIT3: It turns out I am running Upstart, and Upstart has an init script called rc-sysinit.conf which starts all the SystemV style scripts. So perhaps Upstart is screwing up my permissions.

Answer 1

The answer seems to have been to put the following in my init.d script, which I put just before the start-stop-daemon calls in do_start:

ulimit -r ## (where ## is a sufficiently high number; 99 works)

The way I was able to determine this was by making system calls to ulimit -a inside of a bash command inside of my code:

bash -c "ulimit -a"

The bash part is necessary, because ulimit -a is a shell builtin. ulimit -a on /bin/sh returns different information unrelated to real-time priority. For some reason, I found that my real-time priority was limited to 0 (no real-time priority) when my service is started at boot. When I run it with service or by calling the init.d script, it inherits my permissions which allow for real-time priority. But when the system calls it through the Upstart/SystemV backwards compatibility system, it doesn't get that elevated privilege. I suppose this might relate to posts I have seen that say Upstart doesn't read /etc/security/limits.conf which is where you would set system-wide real-time priority permissions for non-privileged users.

If anyone can verify or explain why this solution works, I would love to hear it.