Why doesn't systemd-resolved use my local DNS server?

Question

I'm using a local BIND9 server to host some local dns records. When trying to dig for a local domain name I can't find it if I don't explicitly tell dig to use my local BIND9 server.

user@heimdal:~$ dig +short heimdal.lan.se
user@heimdal:~$ dig +short @192.168.1.7 heimdal.lan.se
192.168.1.2

Ubuntu 17.04 and systemd-resolved are used. This is the content of my /etc/resolved

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.

nameserver 127.0.0.53

And the output from systemd-resolve --status

Global
         DNS Servers: 192.168.1.7
                      192.168.1.1
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

The DNS Servers section does seem to have rightfully configured 192.168.1.7 as the main DNS server (my local BIND9 instance). I can't understand why it's not used ... ?

Answer 1

So, changing my wired eth0 interface to be managed solved this issue for me.

Changing ifupdown to managed=true in /etc/NetworkManager/NetworkManager.conf

[ifupdown]
managed=true

Then restart NetworkManager

sudo systemctl restart NetworkManager

After this it works flawlessly..

This was not 100%. I also applied theses changes to try and kill resolver

sudo service resolvconf disable-updates
sudo update-rc.d resolvconf disable
sudo service resolvconf stop

Big thanks to this blog post regarding the subject: https://ohthehugemanatee.org/blog/2018/01/25/my-war-on-systemd-resolved/ (if unavailable use https://github.com/ohthehugemanatee/ohthehugemanatee.org/blob/main/content/blog/source/2018-01-25-my-war-on-systemd-resolved.markdown)

Lets pray this works.. This whole systemd-resolve business is just so ugly.

Answer 2

My guess is that your systemd-resolved service is configured correctly, but it never gets to see the request. The .local domain is treated specially by systems running mDNS. avahi-daemon, which provides mDNS/DNS-SD services (aka "Bonjour" on Apple products) can be configured to take precedence over DNS during name resolution; it appears that Ubuntu does this.

There are a few options that you could choose from:

1. Rename your .local domain to something different (possibly .internal or .lan). This may be the easiest to do in practice because you just have to change a couple of things on your DNS server, and it works best with Avahi. I would recommend this method.

2. Alter your /etc/nsswitch.conf file by putting the dns entry in front of the mdns entries.

3. Alter Avahi's configuration to change the mDNS domain from .local to something else by editing /etc/avahi/avahi-daemon.conf and changing (or adding) domain-name=.something (located in the [server] section). You'll need to do this on every computer that uses mDNS so they still work together.

Answer 3

Seems this would be better as a comment, but not enough reputation....

Civing's self-answer was most along the lines of what I wanted.

I also had to add dns=none to the [main] section of /etc/NetworkManager/NetworkManager.conf, so it looks like this:

[main]
plugins=ifupdown,keyfile
dns=none

I have just updated to xubuntu 18.04, from 14.04, and I have a LAN that's older than that, with many little adjustments accrued over the years. So I want my DNS to do what I want (yes, I have purchased many copies of Cricket Lius's book over the years, starting with the second edition).

As an aside, I had previously been adding the DNS resolving information I want to see to the file /etc/resolvconf/resolv.conf.d/head.

In a nutshell, once I had a working /etc/resolv.conf, as root:

cat /etc/resolv.conf >> /etc/resolvconf/resolv.conf.d/head

But now, I just edit /etc/resolv.conf directly, and it stays put. Visitors to my LAN, who are using systemd/resolvconf, are SOOL. They do not exist.

Reading man 8 resolvconf helped. A lot. I did not follow the instructions for putting things where the ifup program could find them. Mostly because there's a whole superstructure in the GUI that was already being ignored by whatever was done during the upgrade. That seems to be a bigger issue (WTF, Ubuntu?).

So this is fugly, and there is still the issue that what I had (long ago) entered into the network control panel GUI was not being obeyed by the newly-upgraded system, but that's a totally different question, once I figure out how to ask it.

Answer 4

For me, running a recently installed 18.04, I made the first change cited by @Civing:

[ifupdown]
managed=true

then, noticing that /etc/resolv.conf was always pointing stub-resolv.conf and that a reasonable resolv.conf with the proper LAN DNS server was being generated, changed the symlink:

/etc/resolv.conf -> /run/systemd/resolve/resolv.conf

and then local all hostnames resolved via ping.

It remains to be seen how long this keeps working.

When I initially installed, wireless network setup failed, and I can't help but wonder if the installation left /etc/resolv.conf in this initial state.

So, one suggestion is to look at what resolved is generating; you may have a working basis already.

Answer 5

I was using Ubuntu 18, and I'm more used to debian and centOS. Cue confusion.

resolve.conf said it's using systemd-resolved ok...

systemd shows global and per-link settings ok...

research shows that may mean it's using netplan

So, open up /etc/netplan/01-network-manager-all.yaml, and it says...renderer: NetworkManager

Turns out NetworkManager is in the gui, and a way to enable per-link settings. But...you can install nmcli and configure this sucker on the command line....

I hope this helped someone.