Have you seen this error when mounting a CIFS share on boot?
mount error: could not resolve address for [cifs share address]: Unknown error
Boy, do I have blog post for you.
Context
I maintain a personal Nextcloud instance over at Hetzner Cloud using their Storage Box for data. I’m also using ZeroTier as a management network, that also connects the VM to my IPA network and DNS.
Having performance issues with WebDav mount I’ve been using so far I tried to switch to CIFS.
Problem
The share would mount without an issue when done by hand or using a mount -a option from the terminal. It would however fail with previously mentioned resolve error on boot.
I didn’t like common ‘solution’ found over the Internet to ‘just use an IP address’. I don’t control the CIFS server, and it could change the IP at any time resulting in unexpected failures.
Solutions
1 - wait for network
See, when system mounts a CIFS share via /etc/fstab or systemd \*.mount files it does not care for your network unless you tell it to. That’s step one.
In general adding a \_netdev option in fstab or an After=network-online.target in the systemd unit file should do the job.
It doesn’t always do though. Mount has very short timeouts (by default at least), and doesn’t retry on failure.
2 - wait for the proper network
In my case, network being up is not enough to reach DNS. The management (and DNS) network starts up with a service called zerotier-one.service which setups a virtual adapter.
With systemd it’s trivial:
[Unit]
After=zerotier-one.service
[...]
With /etc/fstab one can use a x-systemd.after= option:
[...],_netdev,x-systemd.after=zerotier-one.service 0 0
This still didn’t work for me - turns out it takes a short while for zerotier daemon to actually connect to the network after declaring the service started. It was enough for the mount to determine it can’t reach the DNS server and fail.
At this point I decided this can’t be done with fstab - I need to use systemd or scripts in /etc/rc.local (which defeats the purpose )
2.5 - use a timer
ExectStartPre is not an option for [Mount] sections, so this common suggestion wont work here.
[Mount]
ExecStartPre=/bin/sleep 30
The alternative is to use a .timer systemd unit with the same name as our mount, and enable that instead.
[Timer]
OnBootSec=1min
This solution should work, I haven’t tried it however, as I find crude sleep commands inelegant and still error-prone with little debug information provided.
3 - make sure DNS is up
The final solution I came up with also employs an additional systemd unit, but this time of type oneshot to check on boot if DNS is up by trying a DNS lookup in a loop until success. This has an additional benefit of delaying the start of a service in case of an external DNS failure without the need of manual service restart. Create this file as /etc/systemd/system/dns-check.service and enable the unit.
[Unit]
Description=check if DNS is up before proceeding
After=network-online.target
[Service]
Type=oneshot
ExecStartPre=/bin/bash -c 'until host [storage box username].your-storagebox.de; do sleep 1; done'
ExecStart=/bin/bash -c 'echo "DNS up!"'
[Install]
WantedBy=multi-user.target
Next, make the mount unit start after dns-check:
[Unit]
Description=cifs mount Nextcloud data
Requires=dns-check.service
After=dns-check.service
[Mount]
What=//[storage box username].your-storagebox.de/[folder]
Where=[mount point path]
Options=iocharset=utf8,rw,credentials=/etc/[cifs credentials].txt,uid=apache,gid=apache,_netdev,file_mode=0660,dir_mode=0770,vers=3.0
Type=cifs
[Install]
WantedBy=multi-user.target
Optionally overrde the httpd (apache) systemd unit file and make the unit wait until mount is finished so Nextcloud wont try to access it’s data folder until it’s there.
# cp /usr/lib/systemd/system/httpd.service /etc/systemd/system/
# nano /etc/systemd/system/httpd.service
[Unit]
Description=The Apache HTTP Server
After=network.target remote-fs.target nss-lookup.target mnt-data.mount
Depends=mnt-data-nc_mkwlab_eu.mount
[...]
That’s it, no more mount errors on boot.
While my case is specific to using ZeroTier, I think it’d apply to any late DNS coming up late after network-online.target is reached by systemd.