It's not going to be that easy: dialog --gauge expects the percentages from stdin, as one number per line.
However, unsquashfs doesn't have that kind of output! It uses console control characters to update its progress bar, probably (at least it doesn't link against libncurses).
The important part is line 145 in progressbar.c in the squashfs-tools source code:
printf("\r[");
…
printf(" %3d%%", percentage);
The control character \r jumps to the beginning of the current line, then that line is overwritten with the progress bar, then the percentage is finally printed.
so you'll probably need something like
unsquashfs linux.squash | tr $'\r' $'\n' | sed -ne 's/^.* \([[:digit:]]*\)\%$/\1/p' | dialog --gauge
but I doubt it will work. Most probably, there will be buffering, and all your percentages will appear at once (try without the | dialog --gauge).
Honestly, the progressbar of unsquashfs does work. Simply show that instead of a dialog showing the same?
Other than that, there's things like libraries which can uncompress squashfs, and I guess you could use these to build your own program that outputs percentages instead of anything else. Or, you could add a command line switch to mksquashfs and unsquashfs's source code and make them do that – probably the easiest solution here.
-ndisables the progress bar, so that's not going to help.