Skip to main content
1 of 5
zedmelon
  • 155
  • 10

apparmor: mysql permissions--with no recent changes

Mysql server is being blocked by (I think) apparmor, but I'm at wits' end to determine why/how. I'm not overly familiar with apparmor.

I know I shouldn't uninstall apparmor--for at least two reasons--but I've used enough profanity (and given this issue too much time) to not consider it. Hopefully I'm merely missing something simple and will learn here.

The failure began today and follows no system changes. MySQL's error log laments permissions

Can't open and lock privilege tables: Table 'servers' is read only

I've been unable to find anyone with this issue who isn't currently moving their default database store. I moved mine as well--two years ago. The apparmor config is unchanged since 2014/04/21:

/files/bak/tmp/ rw,
/files/bak/tmp/* rwk,
/files/bak/mysql/ rw,
/files/bak/mysql/** rwk,

I've verified filesystem permissions:

# find mysql/ -type d -exec chmod 700 {} \;
# find mysql/ -type f -exec chmod 660 {} \;
# chown -R mysql: mysql 

I reloaded apparmor, installed apparmor-utils, pushed mysql to complain

# aa-complain mysql
# apparmor_status
apparmor module is loaded.
5 profiles are loaded.
4 profiles are in enforce mode.
   /sbin/dhclient
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/tcpdump
1 profiles are in complain mode.
   /usr/sbin/mysqld
1 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
1 processes are unconfined but have a profile defined.
   /sbin/dhclient (495)

...but viewing syslog still suggests apparmor is blocking mysql after service mysql start:

apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=13899 comm="apparmor_parser"

Before I found the apparmor issue I tried restoring DBs from backups, which also failed with write permissions:

Can't create/write to file '/files/bak/mysql/dbCooper/wp_cb_contact_form.MYI'

I verified the filesystem is 'rw' (even though the above find -exec would have failed anyway):

mount
/dev/xvdb on /home type ext4 (rw,noatime)

I've even tried stopping apparmor, but the syslog still shows can't open and lock privilege tables after this:

# service apparmor stop
  [...redacted teardown msg...]
# /etc/init.d/apparmor teardown
  * Unloading AppArmor profiles
# service apparmor status
apparmor module is loaded.
0 profiles are loaded.
0 profiles are in enforce mode.
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

Is it possible for mysql to lock database files and fail to unlock them when the daemon crashes? If so how would I clear the lock?

I'm currently running my DB with

mysqld --skip-grant-tables

...so I know the executable can run, and the databases are at least somewhat valid (the sites all appear normal). Am I missing something dumb?

I tried trimming this post to relevant info--glad to post the entire (cfg|log) where useful.

Thanks for reading

If this post should be moved to another Stack Exchange (ServerFault?) I apologize--and mods feel free to correct it.

zedmelon
  • 155
  • 10