Problem cause AFAIK
ip netns add ${NSNAME} creates a persistent namespace, by mounting it to /run/netns/${NSNAME}.
This in itself does not involve mount namespaces, yet.
Doing ip netns exec ${NSNAME} cmd spawns a temporary mount namespace that exists until cmd exits, after that the mount namespace gets cleaned up, only network namespace remains.
This is required to achieve the magic mounting of /etc/netns/${NSNAME}/*.
Unfortunately any mounts done by the cmd are cleaned up too.
From man ip-netns:
ip netns exec automates handling of this configuration, file convention
for network namespace unaware applications, by creating a mount name‐
space and bind mounting all of the per network namespace configure
files into their traditional location in /etc.
A solution
Short version
# Init for persistent mount namespaces
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
$ NSNAME=red
# Create persistent network+mount namespaces
$ ip netns add ${NSNAME} \
&& touch "/run/mntns/${NSNAME}" \
&& ip netns exec "${NSNAME}" \
sh -x -c "nsenter -t $$ --mount mount --bind /proc/\$\$/ns/mnt '/run/mntns/${NSNAME}' && :"
# Do NOT use `ip netns exec`
# (Re)enter the namespaces with `nsenter`, mounts are persistent
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
program arg1 arg2
Long version
It is possible to create both network and mount namespaces persistent.
This is largely adapted from examples in the unshare man page: http://man7.org/linux/man-pages/man1/unshare.1.html
The main implication is that instead of ip netns exec the nsenter must now be used to use both the network+mount namespaces, at least where mounts matter anyway.
Step 1. Prepare. Create directory for persistent mount namespaces
This needs to be done once, e.g. after boot, NOT for every namespace.
The path here is arbitrary.
I'm trying to use similar naming to that used by ip netns.
# Try to create a dir for persistent mount namespaces
# and if it got created (e.g. first attempt since boot),
# then bind mount it with --make-private as required by `unshare --mount=/path`
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
Note, the mkdir ... && mount ... are not atomic and where there are multiple concurrently starting services doing this sequence then a race condition may occur.
Step 2. Create network+mount persistent namespaces
2a. method A, simple, no magic bind mounts.
This creates persistent mount+network namespaces.
It is simple but does not create magic bind mounts for /etc/netns/${NSNAME}/*.
$ NSNAME=red # give it a name
# For network namespace use same paths as used by `ip netns` for interoperability
# touch can fail to update timestamp if file is already mounted
# Try to create files for persistent mounts.
# and if touch is successful
# then create persistent mount and network namespaces
# Instead of `true` can run a command right away e.g. /bin/bash
# As it is this is approximate equivalent of `ip netns add ${NANAME}`
$ touch /run/netns/${NSNAME} /run/mntns/${NSNAME} \
&& unshare --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} true
2b. method B, leverage ip netns exec to make mounts but make persistent.
To get those bind mount done by ip netns exec ... but make them persistent, can do setup as follows.
Note this must be done once ONLY when creating the namespace, not every time.
$ ip netns add ${NSNAME} \
&& touch "/run/mntns/${NSNAME}" \
&& ip netns exec "${NSNAME}" \
sh -x -c "nsenter -t $$ --mount mount --bind /proc/\$\$/ns/mnt '/run/mntns/${NSNAME}' && :"
For an attempted explanation see towards the end.
Steps 3. Use it
Now the nsenter must be used to launch commands inside both the namespaces as required to achieve persistent mount results.
This is regardless if setup was done with unshare or ip netns exec NAME sh ... nsneter ... mount ....
# Get a shell inside the namespaces
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
/bin/bash
e.g. with OP's commands:
# Do your additional mounts (can also be done above as command to `unshare`)
# This is approximate equivalent of
# `ip netns exec ${NANAME} mount --bind /etc/red/run/ /run/`
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
mount --bind /etc/${NSNAME}/run/ /run/
# Re-enter the namespaces, check that the mounts are still there
$ nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
sh -c 'ls /run/'
Important
This approach works, but instead of using ip netns exec use nsenter --net=... --mount=...!
If ip netns exec ... is used it will replace (for the duration of its run) the mount namespace, which can be ok for commands that do not care about mount's persistence, e.g.
# This does not touch any files, only operates on network device
ip netns exec ${NSNAME} ip link set up dev lo
Step Z. - Clean up
Regardless which of the two methods above was used to create the namespaces, the clean up can be done by either of the two methods:
# This deletes the /mnt/netns/${NSNAME} for us
ip netns del ${NSNAME}
# Unmount and delete mount namespace holder
umount "/mnt/mntns/${NSNAME}" \
&& rm "/mnt/mntns/${NSNAME}"
or
# Unmount and delete both namespaces' holders
umount "/mnt/mntns/${NSNAME}" "/mnt/netns/${NSNAME}" \
&& rm "/mnt/mntns/${NSNAME}" "/mnt/netns/${NSNAME}"
Explanation
Apologies for long text. This is in part for my own benefit.
This command from step 2b above deserves some explanation.
The command again.
$ ip netns add ${NSNAME} \
&& touch "/run/mntns/${NSNAME}" \
&& ip netns exec "${NSNAME}" \
sh -x -c "nsenter -t $$ --mount mount --bind /proc/\$\$/ns/mnt '/run/mntns/${NSNAME}' && :"
From man unshare
The namespaces can optionally be made persistent by bind mounting
/proc/pid/ns/type files to a filesystem path and entered with
nsenter(1) even after the program terminates (except PID namespaces
where permanently running init process is required). Once a persistent
namespace is no longer needed, it can be unpersisted with umount(8).
Explanation start
Here is what it does, if broken down into simple steps.
Note, two shells are required for this demo.
# SHELL-1
# Init
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
# Prep
$ NSNAME=red
# Create namespace
$ ip netns add ${NSNAME}
# Start a shell inside the namespace
$ ip netns exec "${NSNAME}" /bin/bash
# Now inside a shell inside a namespace.
# Now need to make a bind mount it this shell's mount namespace
$ echo $$ # Use this in SHELL2 AS SHELL1_PID
# Go to SHELL-2 before exiting this one
# SHELL-2
# Set this to PID from SHELL-1
$ SHELL1_PID=????
$ NSNAME=red
$ touch "/run/mntns/${NSNAME}"
$ mount --bind /proc/${SHELL1_PID}/ns/mnt "/run/mntns/${NSNAME}"
Now both network+mount namespaces are persistent and can be re-entered with nsenter as shown way earlier.
The important part is mounting ip netns exec's /proc/pid/ns/net into the outer shell's mount namespace before ip netns exec.. exits.
Explanation cont.
So the answer is: from within ip netns exec ... do nsenter, but not as a direct command ip netns exec ${NSNAME} nsenter ...exec() without fork() therefore leave and destroy the temporary namespace, but rather cause a fork() by running an intermediate shell that keeps the temporary namespace active, and the forked nsenter does mount.
This following is closer to how the one long command works.
# Init
$ mkdir /run/mntns \
&& mount --bind --make-private /run/mntns /run/mntns
# Prep
$ export NSNAME=red
$ touch "/run/mntns/${NSNAME}"
$ export MOUNT_PARENT_PID=$$ # for simplicity
# Create namespace
$ ip netns add ${NSNAME}
# Start a shell inside the namespace, the intermediate shell is important
$ ip netns exec "${NSNAME}" /bin/bash
# Now inside a forked namespaced shell
# Need to mount `ip netns exec` temporary mount namespace to a file
# within MOUNT_PARENT_PID's namespace to make it persistent
# nsenter will reference it later
$$ nsenter -t ${MOUNT_PARENT_PID} --mount \
mount --bind /proc/$$/ns/mnt "/run/mntns/${NSNAME}"
# Yes, here $MOUNT_PARENT_PID can be replaced by $PPID
# But if any intermediate sub-shells and forks get introduced
# then $PPID can point to a different process.
It seems some versions of bash invoked with -c command for a single command optimise out the fork() and that fork() is really needed here, so force it to fork() with && : at the end of the command (See short version at the start).
Manually mounting /etc/netns/${NSNAME}/*
The original answer had this section, so here it is, although likely not very useful.
Re-Creating /mnt/netns/${NSNAME}/ bind mounts*
The downside of using unshare is you don't get the automatic bind mounting of /etc/netns/NETNS_NAME/* that is provided by ip netns exec ${NSNAME} cmd, which is not hard to add manually.
# either during unshare or later as nsenter.
# The important difference is here it is done once and result persists
# unlike with `ip netns exec` which does this every time
NSNAME="${NSNAME}" \
nsenter --net=/run/netns/${NSNAME} --mount=/run/mntns/${NSNAME} \
/bin/sh -e -x -c \
'cd "/etc/netns/${NSNAME}"; for f in *; do mount --bind "/etc/netns/${NSNAME}/${f}" "/etc/${f}"; done;'
