The first things to note is that device files are allowed to not support seek; perhaps surprisingly, seeking on such files succeeds without moving the position in the file.
Before version 9.6, coreutils tail would try to seek from the end, and calculate an end position based on the position after seeking (however many bytes were requested after the position). Thus with /dev/zero, it would decide it needed to read from position 0 to n. It would then look at the amount of data to read to determine how to read it: if a page (in your case 4,096 bytes) or less, it would pipe until the end of the file; if more than a page, it would dump the required number of bytes. Crucially, piping here keeps reading until the end of the file, which never happens! So reading more than a page’s worth of data completes, whereas reading a page or less loops forever.
This was reported as a bug in 2024, with the same expectation as you — that the command should loop. However it was fixed in 9.6 so that reading from such devices never loops.