Using the perl rename utility (not the rename command from util-linux, that has completely different and incompatible command-line options):
rename -n -v 's/\$+//g' *.mp3*
That will remove all $ characters from anywhere in the filenames.
If you want it to remove $ characters only if they are at the end of the filenames:
rename -n -v 's/\$+$//' *.mp3*
BTW, the above commands use rename's -n option as a dry run to show you what rename would do without actually renaming anything. When you are sure it does exactly what you want without any unintended consequences, remove the -n from the command line. Optionally remove the -v (verbose) option too.
The perl rename utility is in the rename package on Debian and derived distros. This package contains both the File::Rename CPAN module and the rename utility.
Earlier versions of the perl package on Debian used to include an old version of rename, with some notable bugs. If you have that version installed, you should install the rename package ASAP.
This perl-based rename is also sometimes installed as either file-rename or prename. prename is usually the older buggy version.