2 
                             192.168.86.0 (1Gb switch)
                             -------------------------
                              /\                    /\
                              |                      |
================   ===========|===========   ========|=======   ================
   10.0.0.80           192.168.86.80           192.168.86.86       10.0.0.86
 enxa0cec8b5b0fa          enp86s0                                      |
       |      Linux Server                              macOS client   |
=======|==================================    =========================|========
       |                                                               |
       +-------------------- Cross-over ethernet cable ----------------+
                                 2.5Gb NFS Network
                    

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp86s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 48:21:0b:59:33:75 brd ff:ff:ff:ff:ff:ff
    inet 192.168.86.80/23 brd 192.168.87.255 scope global noprefixroute enp86s0
       valid_lft forever preferred_lft forever
    inet6 fe80::4a21:bff:fe59:2275/64 scope link 
       valid_lft forever preferred_lft forever
3: enxa0cec8b5b0fa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether ba:3f:0c:1c:33:f7 brd ff:ff:ff:ff:ff:ff permaddr a0:ce:c8:b5:b0:fa
    inet 10.0.0.80/16 brd 10.0.255.255 scope global noprefixroute enxa0cec8b5b0fa
       valid_lft forever preferred_lft forever
    inet6 fe80::ca9a:84e1:ffbe:6196/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: wlo1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b0:dc:ef:33:f7:51 brd ff:ff:ff:ff:ff:ff
    altname wlp0s20f3
5: vmnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 00:50:56:c0:33:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.70.1/24 brd 192.168.70.255 scope global vmnet1
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fec0:1/64 scope link 
       valid_lft forever preferred_lft forever
6: vmnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 00:50:56:c0:33:08 brd ff:ff:ff:ff:ff:ff
    inet 192.168.80.1/24 brd 192.168.80.255 scope global vmnet8
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fec0:8/64 scope link 
       valid_lft forever preferred_lft forever

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    100    0        0 enxa0cec8b5b0fa
0.0.0.0         192.168.86.1    0.0.0.0         UG    101    0        0 enp86s0
10.0.0.0        0.0.0.0         255.255.0.0     U     100    0        0 enxa0cec8b5b0fa
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 enxa0cec8b5b0fa
192.168.70.0    0.0.0.0         255.255.255.0   U     0      0        0 vmnet1
192.168.80.0    0.0.0.0         255.255.255.0   U     0      0        0 vmnet8
192.168.86.0    0.0.0.0         255.255.254.0   U     101    0        0 enp86s0

# ip route show
default via 10.0.0.1 dev enxa0cec8b5b0fa proto static metric 100 
default via 192.168.86.1 dev enp86s0 proto static metric 101 
10.0.0.0/16 dev enxa0cec8b5b0fa proto kernel scope link src 10.0.0.80 metric 100 
169.254.0.0/16 dev enxa0cec8b5b0fa scope link metric 1000 
192.168.70.0/24 dev vmnet1 proto kernel scope link src 192.168.70.1 
192.168.80.0/24 dev vmnet8 proto kernel scope link src 192.168.80.1 
192.168.86.0/23 dev enp86s0 proto kernel scope link src 192.168.86.80 metric 101

The setup has two networks, 192.168.86.0 and 10.0.0.0. The Linux server (Ubuntu 22.04) has two interfaces, enxa0cec8b5b0fa (10.0.0.80) and enp86s0 (192.168.86.80). Only 192.168.86.80 (Linux) and 192.168.86.86 (Mac) have direct access to the Internet through the 1Gb ethernet switch. The ethernet cross-over cable is connected to 2 usb-ethernet 2.5Gb dongles, which are used exclusively for NFS — as the 1Gb network on 192.168.86.0 is too slow for the use.

NFS (on 10.0.0.0) on both sides works fine; the two networks (192.168.86.0, 10.0.0.0) can be accessed simultaneously. The problem is that once the 10.0.0.0 (NFS) network is on, the Internet connection drops on the Linux server – on the Mac it works fine, i.e., the two networks can be accessed simultaneously, with the Internet.

Improve this question
2
  • 1
    In your route -n output you have two default routes, and the active one is out 10.0.0.1. From what you show, that's going to disable access to the internet. Your storage network should not have a default gateway associated with it. Commented Jul 27, 2024 at 12:17
  • Just an aside: 1Gbps+ Ethernet almost always guarantees auto MDI-X, so a crossover cable probably isn't necessary if you ever change this setup. Commented Jul 28, 2024 at 2:35

1 Answer 1

Reset to default
3

The problem is that you have two default routes.
If a connection doesn't have a internet connection, it should not have a default route.

Delete this:

0.0.0.0         10.0.0.1        0.0.0.0         UG    100    0        0 enxa0cec8b5b0fa
Improve this answer
3
  • Thanks! route del -net 0.0.0.0 gw 10.0.0.1 netmask 0.0.0.0 dev enxa0cec8b5b0fa Commented Jul 27, 2024 at 12:40
  • 2
    You may also want to find out what added that default route and remove it there too. Commented Jul 27, 2024 at 12:45
  • Indeed, I edited /etc/NetworkManager/system-connections/eth_10x.nmconnection and removed the default gateway, so under [ipv4], there is: address1=10.0.0.80/16⏎ ignore-auto-dns=true⏎ ignore-auto-routes=true⏎ method=manual. Commented Jul 27, 2024 at 13:50

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.