How to list all the files used during Linux boot

Question

I would like to get the list of files that are used during the process of Linux boot. We are developing the protected enterprise system based on the RHEL 6.4. The integrity of specified files will be checked by a special hardware.

So the question is - how to get the list of these files (with resolved dependencies coming from different booting services and daemons).

Answer 1

Set up the audit subsystem to record calls to open.

auditctl -a exit,always -S open,openat,creat,execve

Do that from the initramfs, so that the rule is in place when the main system (/sbin/init on the real root filesystem) starts.

Note that what you're proposing to do won't bring any real security on a typical setup. Anyone who can replace these files by other versions has root access and can also feed bogus data to the logging system.

If the boot media is protected externally so that root cannot modify it (e.g. because it's read-only or under the exclusive control of a secure bootloader), and if loading kernel modules is blocked, then measuring the files can be reliable if it's done right. However, if all you're doing with the measures is comparing them with reference values, this is more difficult and less effective than using an integrity-protected root filesystem (i.e. in practice on dmcrypt, with Trusted Grub for the bootloader).

Answer 2

A good starting place would be /etc/rci.d where i is a number representing the runlevel you're booting into. For example, if your server is headless, i will typically be 3. Looking under /etc/rc3.d will show what services are being started when you boot into runlevel 3.

Answer 3

Make sure atime is enabled for your root and boot filesystems in your kernel (or that noatime is not set), then after booting you can use stat to check the access time for every file and see which ones were accessed during boot.

Answer 4

Thanks to RHEL support, the clear solution has been discovered. It is based on systemtap kernel module usage. Quoted from here to avoid link rot. And thank you again for all of your advice :)

I could not even imagine that systemtap is able to start even before the init script and track the booting process. I very appreciate the Red Hat Support and personally Pushpendra Chavan for help with this perfect tool (unfortunately I don't know developers this method belongs to exactly - otherwise I'd credit them in the first place).

So we need to create two simple scripts:

bootinit.sh:

#!/bin/sh


# Use tmpfs to collect data
/bin/echo "Mounting tmpfs to /tmp/stap/data"
/bin/mount -n -t tmpfs -o size=40M none /tmp/stap/data

# Start systemtap daemon & probe
/bin/echo "Loading bootprobe2.ko in the background. Pid is :"
/usr/bin/staprun \
    /root/bootprobe2.ko \
    -o /root/bootprobe2.log -D

# Give daemon time to start collecting...
/bin/echo "Sleeping a bit.."
sleep 5

# Hand off to real init
/bin/echo "Starting."
exec /sbin/init 3

and bootprobe2.1.stp written in embedded systemtap scripting language:

global ident

function get_usertime:long() {
  return task_utime() + @cast(task_current(), "task_struct", "kernel<linux/sched.h>")->signal->utime;
}

function get_systime:long() {
 return task_stime() + @cast(task_current(), "task_struct", "kernel<linux/sched.h>")->signal->stime;
}

function timestamp() {
  return sprintf("%d %s", gettimeofday_s(), ident[pid()])
}

function proc() {
  return sprintf("%d \(%s\)", pid(), execname())
}  

function push(pid, ppid) {
   ident[ppid] = indent(1)
   ident[pid] = sprintf("%s", ident[ppid])
}

function pop(pid) {
  delete ident[pid]
} 

probe syscall.fork.return {
  ret = $return
  printf("%s %s forks %d  \n", timestamp(), proc(), ret)
  push(ret, pid())
}

probe syscall.execve {
  printf("%s %s execs %s \n", timestamp(), proc(), filename)
}

probe syscall.open {
  if ($flags & 1) {
    printf("%s %s writes %s \n", timestamp(), proc(), filename)
  } else {
    printf("%s %s reads %s \n", timestamp(), proc(), filename)
  }
} 

probe syscall.exit {
  printf("%s %s exit with user %d sys %d \n", timestamp(), proc(), get_usertime(), get_systime())
  pop(pid())
}
<linux sched.h=""><linux sched.h="">
</linux></linux>

In order to receive the list of files accessed during the booting process in systemtap log format we should implement the following:

Download and install the PROPERLY named versions of systemtap and kernel debuginfo packages (I have been given this link, but you'd better use this if you're on CentOS);

Create /tmp/stap and /tmp/stap/data

mkdir -p /tmp/stap/data

Place bootprobe2.1.stp and bootinit.sh into /root and make them executable: chmod +x /root/boot*

Edit bootinit.sh and change 'exec /sbin/init 3' to 'exec /sbin/init 5' if 5 is your default runlevel.

Create the .ko module from bootprobe2.stp

 cd /root
 stap bootprobe2.1.stp -m bootprobe2 -p4

Reboot.

Halt grub (press Esc or Shift) and press 'a' on the default kernel. At the end of the kernel line enter the following and press enter:

init=/root/bootinit.sh,

Normal boot will resume. After logging in, kill the stapio process, copy bootprobe2.log out of the tmpfs /tmp/stap/data directory and unmount it.

killall stapio
cp /tmp/stap/data/bootprobe2.log /tmp/stap/
umount /tmp/stap/data  

Now check the file /tmp/stap/bootprobe2.log file for the list of all files which are read during boot.