fsckd locks up system on boot after dist-upgrade

Question

Problem

After upgrading from stretch to buster, my laptop gets stuck in boot during fsckd check, with the message:
fsckd-cancel-msg:Press Ctrl+C to cancel all filesystem checks in progress
It does not respond to 'Ctrl+C' or anything else.

Background

Upgrade

I upgraded from stretch to buster following this tutorial. To guard against link rot, here are the commands in consecutive order:

su
sed -i 's/stretch/buster/g' /etc/apt/sources.list
apt update
apt upgrade
apt dist-upgrade
apt autoremove
apt clean
shutdown -r now

There were no error messages.

Boot

During the reboot, there was the obligatory fskd check, which never caused any problems before. Now, however, it ends with the above mentioned error message. Manually powering off and on again lead to the same result.

What I tried

This question has the same problem on Ubuntu. Since Debian and Ubuntu are similar, I thought the accepted answer could fix my problem. My approach follows the answer:

Rescue mode

I did not have have a Live USB at hand, so isntead I booted in Linux 4.19.0-12-amd64 x86_64 (rescue mode) and provided the root password for maintenance.

fdisk -l

Device     Boot  Start       End   Sectors   Size Id Type
/dev/sda1  *      2048    499711    497664   243M 83 Linux
/dev/sda2       501758 625141759 624640002 297.9G  5 Extended
/dev/sda5       501760 625141759 624640000 297.9G 83 Linux

I then ran a manual fsck on all the listed disks, also on /dev/mapper/debian--vg-root and /dev/mapper/debian--vg-home, all of which came back clean. I then rebooted, thinking maybe the problem was solved.

Rescue mode graphical

Alas, rebooting showed the same error. So I again powered off and on manually, again booting in rescue mode. This time however, instead of providing the root password, I pressed Ctrl+D to continue. It finished booting up, showing nothing but a black screen. So I switched tty with Ctrl+Alt+F1, logging in with my non-root standard user. I then started a X11 session with startx (no good reason for this other than me being more comfortable with a graphical session).
This showed me already that the upgrade had worked, since the optics of buster are a little different than in stretch. Just to be sure I checked my distro with lsb_release -a

No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:    10
Codename:   buster

Bad block scan

I ran sudo e2fsck -fccky on all /dev/sdaX and /dev/mapper/debian-vg-X, which took almost two hours. A reboot showed that the error still remained.

Gparted

I ran gparted, but since my disk is LUKS encrypted, I couldn't see anything useful (related question).

Checking /var

I saw in multiple places that a full /var can cause similar problems. Since I couldn't check it with gparted like advised, I checked it like proposed here:

sudo du -ks *|sort -nr

However, the output does not look like it's very full:

698456  lib
100720  cache
53824   log
6688    backups
76  spool
68  tmp
32  snap
8   mail
4   opt
4   local
0   run
0   lock

Also, the output of df looks like this:

Filesystem                  1K-blocks     Used Available Use% Mounted on
udev                          1988192        0   1988192   0% /dev
tmpfs                          402032     6508    395524   2% /run
/dev/mapper/debian--vg-root  28703652 10165160  17057380  38% /
tmpfs                         2010160   133528   1876632   7% /dev/shm
tmpfs                            5120        4      5116   1% /run/lock
tmpfs                         2010160        0   2010160   0% /sys/fs/cgroup
/dev/sda1                      240972    89448    139083  40% /boot
/dev/mapper/debian--vg-home 273421644 24010648 235452240  10% /home
/dev/loop0                     212864   212864         0 100% /snap/firefox/172
/dev/loop2                     100096   100096         0 100% /snap/core/10185
/dev/loop1                     100096   100096         0 100% /snap/core/10126
tmpfs                          402032       24    402008   1% /run/user/1000

So I don't think that's the problem.

Stopping fsck on boot

Booting on battery power

So I thought I could just prevent fsck from running on boot, like here. However, booting on battery power and pressing Ctrl+C had no discernible effect, still the same error.

tune2fs

As proposed here I used tune2fs -c 0 -i 0 /dev/sda1. But even though Maximum mount count is set to -1 and Check interval to 0 (<none>), the error remains.

I also found this solution, but since this is for archlinux, and also I don't know anything about grub, I did not attempt it.

Additional information

• On boot, I was confused to see that there are apparently two kernel versions on my laptop, 4.19 and 4.9. Is this because of the upgrade or was it always there and I just never saw it?
• A common issue seems to be when nvidia drivers are installed, which is why the tutorial linked above recommends to remove it before upgrading. I never had nvidia drivers installed however, so that can't be the reason.
• A similar question was solved by installing nvidia-drivers. However, apt tells me Unable to locate package nvidia-drivers.

Question

How can I prevent fsckd from locking up my system on boot?

Answer 1

I suggest checking that the volume hosting Debian has enough available space. If it is somehow full or almost full, you need to increase its available space then restart. Debian will boot. Voilà.

---

Below is the same suggestion as above. But with details for those not familiar with how to check available space and increase it.

We are facing the same challenge with one of our devices. In our case, the cause of this challenge was that somehow, the device volume which stores Debian was full. Zero available space. In turn, during boot, fsck was automatically started. fsck needs a small amount of space to be able to complete its automation. In turn, fsck was stuck. As it was not able to complete its automation. Unfortunately, fsck does not check the available space before starting its automation. This would be a nice new feature.

We were able to reproduce this challenge on 3 other devices. Simply by filling their storage.

Steps to resolve this challenge:

1. Using Terminal, execute this command to find if the volume(s) are full

   sudo df -h

2. Alternatively, if you have access to GNOME, using Terminal, execute this command to open Baobab

   sudo baobab

3. Wait a few seconds or minutes for Baobab to analyze the volume. It will show which folder(s) use the most space. The interface is self explained.

4. If needed, increase the available storage. To do so, for example, if appropriate, run those commands to clear temporary cached installation packages:

   sudo apt-get clean

   sudo aptitude clean

5. Reboot

6. fsck will automatically complete its automation. Sometime this is very fast. It won't even display. This is normal.

7. Debian will boot. Voilà.