2

Under an Ubuntu 18.04 host, I have set-up an Ubuntu 10.04 guest VM from a cloned HD. The VM fires up fine, I can ssh into it from the host, but it fails to communicate outside of the host.

My question is what is wrong with my configuration below:

Guest VM:

Started with qemu-system-x86_64 G.qcow2 -m 4096 -smp 4 -no-acpi -enable-kvm -name system76 -vga std -device virtio-net,netdev=net0 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no,br=br0vm

Manual network configuration with the GUI / network manager. Static IP 192.168.118.18, mask 255.255.255.0, gw 192.168.118.1, dns 192.168.118.1 (I assumed the tactic I use with my router would work here too, though that may be a mistake).

Somehow ifconfig reports interfaces I did not think were configured. I thought that with the qemu line above we defined net0, but, below, we see eth1 and virbr0!

# ifconfig
eth1      Link encap:Ethernet  HWaddr 52:54:00:12:34:56  
          inet addr:192.168.118.18  Bcast:192.168.118.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe12:3456/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1203 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1606 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:114773 (114.7 KB)  TX bytes:290952 (290.9 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:313 errors:0 dropped:0 overruns:0 frame:0
          TX packets:313 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:45604 (45.6 KB)  TX bytes:45604 (45.6 KB)

virbr0    Link encap:Ethernet  HWaddr 1e:8f:5c:98:b1:25  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::1c8f:5cff:fe98:b125/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:84 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:13004 (13.0 KB)

Host configuration:

apt install uml-utilities
apt install bridge-utils
vi /etc/sysctl.conf
    net.ipv4.ip_forward = 1
sysctl -p
ip link add name br0vm type bridge
ip addr add 192.168.118.1/24 dev br0vm
ip link set br0vm up

tunctl -t tap0 -u asoundmove
ip link set tap0 up

brctl addif br0vm tap0

mkdir /etc/qemu
vi /etc/qemu/bridge.conf
    allow br0vm

# test access to the guest - it works
ssh [email protected]

iptables -t nat -A POSTROUTING -o br0vm -j MASQUERADE
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -o eno1 -i br0vm -j ACCEPT

Trying things out:

I can ping the host from the guest:

ping 192.168.118.1
PING 192.168.118.1 (192.168.118.1) 56(84) bytes of data.
64 bytes from 192.168.118.1: icmp_seq=1 ttl=64 time=0.152 ms
64 bytes from 192.168.118.1: icmp_seq=2 ttl=64 time=0.145 ms

I cannot ping the switch/router to which the host is connected from the guest, the following ping returns nothing:

ping 192.168.117.1

Both ping requests (117.1 & 118.1) show in tcpdump on the guest eth1 and host br0vm, below the tcpdump for the ping 192.168.117.1 requests on the guest.

tcpdump -i br0vm not port ssh
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0vm, link-type EN10MB (Ethernet), capture size 262144 bytes
11:57:34.211291 IP 192.168.118.18 > routerlogin.net: ICMP echo request, id 55320, seq 1, length 64
11:57:35.216163 IP 192.168.118.18 > routerlogin.net: ICMP echo request, id 55320, seq 2, length 64
11:57:36.215940 IP 192.168.118.18 > routerlogin.net: ICMP echo request, id 55320, seq 3, length 64
11:57:37.215919 IP 192.168.118.18 > routerlogin.net: ICMP echo request, id 55320, seq 4, length 64
11:57:39.205837 ARP, Request who-has host.hostname tell 192.168.118.18, length 28
11:57:39.205859 ARP, Reply host.hostname is-at 4e:31:b7:14:1b:a9 (oui Unknown), length 28

watch -d -n 1 iptables -nvL
Chain INPUT (policy ACCEPT 2604K packets, 21G bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
   50  4200 ACCEPT     all  --  br0vm eno1    0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 1974K packets, 43G bytes)
 pkts bytes target     prot opt in     out     source               destination

The 50 4200 counters increment with every ping request

watch -d -n 1 iptables -nvL -t nat
Chain PREROUTING (policy ACCEPT 4283 packets, 510K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 4175 packets, 503K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 9155 packets, 663K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 9227 packets, 666K bytes)
 pkts bytes target     prot opt in     out     source               destination
   25  3789 MASQUERADE  all  --  *      br0vm  0.0.0.0/0            0.0.0.0/0

However the 25 3789 counters do no increase with every ping request.

From the host, this works:

ping 192.168.117.1
PING 192.168.117.1 (192.168.117.1) 56(84) bytes of data.
64 bytes from 192.168.117.1: icmp_seq=1 ttl=64 time=0.609 ms
64 bytes from 192.168.117.1: icmp_seq=2 ttl=64 time=0.585 ms

What am I doing wrong that the IP traffic on the 118 subnet does not get forwarded to the 117 subnet?

EDIT:

Additional information:

ip -br link
lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> 
eno1             UP             30:9c:23:9b:eb:df <BROADCAST,MULTICAST,UP,LOWER_UP> 
br0vm            UP             4e:31:b7:14:1b:a9 <BROADCAST,MULTICAST,UP,LOWER_UP> 
tap0             UP             4e:31:b7:14:1b:a9 <BROADCAST,MULTICAST,UP,LOWER_UP> 

ip -br address
lo               UNKNOWN        127.0.0.1/8 ::1/128 
eno1             UP             192.168.117.110/24 fe80::8fae:b4f2:8b90:7601/64 
br0vm            UP             192.168.118.1/24 fe80::a46d:86ff:fe2a:ddbf/64 
tap0             UP             fe80::4c31:b7ff:fe14:1ba9/64 

ip route
default via 192.168.117.1 dev eno1 proto static metric 100 
169.254.0.0/16 dev eno1 scope link metric 1000 
192.168.117.0/24 dev eno1 proto kernel scope link src 192.168.117.110 metric 100 
192.168.118.0/24 dev br0vm proto kernel scope link src 192.168.118.1
Improve this question
5
  • 1
    probably wrong masquerade rule. what is eno1? you should edit your answer and use modern commands to give your configuration on host: ip -br link; ip -br address; ip route Commented Jun 19, 2019 at 12:10
  • eno1 is the only physical ethernet device on the host, I am using commands from various kvm/qemu related guides, I have no idea what modern command would be, nor how to find the equivalent to the old ones. I updated my question above with the ip commands you suggested. Commented Jun 19, 2019 at 13:11
  • 1
    Sorry modern commands meant ip link and ip address rather than ifconfig and ip route rather than route. That's because some features are better seen with those newer. I wanted to be sure to catch eno1 whcn appears only in one iptables rule. Commented Jun 19, 2019 at 13:12
  • You were right, I had misinterpreted the guide and wrote the MASQUERADE rule with the bridge interface instead of the physical interface - thanks to you I spotted this and it now works. - if you write it in an answer, I'll mark it as accepted. Commented Jun 19, 2019 at 13:27
  • Just did. And if you carefully read the 2nd link about netfilter and bridge, you'll know why I chose to write it as the second rule instead of the first (eg: this could bite if you ever use the specific iptables match physdev because it auto activates bridge-nf) Commented Jun 19, 2019 at 13:36

1 Answer 1

Reset to default
3

The MASQUERADE rule happens at POSTROUTING: that is after a routing decision was already made and a destination interface was already chosen. To communicate with outside, the host will use 192.168.117.1 via eno1. So the MASQUERADE rule criteria should be when using the output interface eno1 rather than br0vm.

You should thus have used:

iptables -t nat -A POSTROUTING -o eno1 -j MASQUERADE

but since this can have unwanted effects (eg: masquerade an other host's IP to its default one etc.) and since there are some side effects of a bridge that might appear later for completely unrelated reasons (described here and there) if you do some specific changes, here's IMHO the best simple rule to use instead:

iptables -t nat -A POSTROUTING -s 192.168.118.0/24 ! -d 192.168.118.0/24 -j MASQUERADE
Improve this answer
4
  • Thank you, great explanation, I can now ping the world (tried a google IP address), so you solved the question above. However I still am unable to use DNS for some reason. With my manual IPv4 configuration on the guest I tried 192.168.118.1 (which is the address for the bridge device) and I tried 192.168.117.1 (which is the address for my WAN router and the setting I use on the system I cloned, so I know it works there) but neither works with my guest VM. Any ideas? Commented Jun 19, 2019 at 13:45
  • No, my host does not run a dns service, so I guess the 118.1 option is moot (out). As to the 117.1 option, when I try a ping www.google.com, nothing turns up on tcpdump or iptables -nvL -t nat, so it does not even leave my guest! I don't understand this. Commented Jun 19, 2019 at 18:58
  • start with tcpdump on the guest then. Commented Jun 19, 2019 at 19:08
  • Sorry, my fault, when you change the name server you point to in network manager, it seems like it only takes effect when the network is stopped and restarted. Problem now solved. Commented Jun 20, 2019 at 8:19

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.