How to fix UTF-8 / locale problem in openSUSE Linux (in terminal / file names and program dialogues)

Question

I have some file / folder names which are not 7-bit clean and they are not displayed correctly in my openSUSE system.

Example for the folder /music/Gabriel_Fauré:

# ls -1d /music/Gabriel_Faur?
/music/Gabriel_Faur??

Perhaps the locale for LC_CTYPE is not set to some UTF-8 value?

# locale
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_GB.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_PAPER=a4
LC_NAME=en_US.UTF-8
LC_ADDRESS=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_ALL=

Well, that's a nice LC_CTYPE for Unicode, I think! What does the error message say?

Funnily enough, setting LC_ALL to the exact value of LC_CTYPE will work:

# setenv LC_ALL en_US.UTF-8
# ls -1d /music/Gabriel_Faur?
/music/Gabriel_Fauré

However, I do NOT want to set LC_ALL to en_US.UTF-8 (or anything, really) because it messes up some other settings! It would be no fix but only a bad workaround for me.

Also, why is LC_CTYPE ignored by /bin/ls and/or my shell when printing characters to the screen?

In Arch Linux I would check whether locales are generated but I found nothing on the subject in openSUSE. Also, the locale does seem to exist.

EDIT:

# ls -1d /music/Gabriel_Faur? | hexdump -C
00000000  2f 6d 75 73 69 63 2f 47  61 62 72 69 65 6c 5f 46  |/music/Gabriel_F|
00000010  61 75 72 c3 a9 0a                                 |aur...|
00000016

So it's correct UTF-8 (as far as I can tell).

EDIT2:

# locale -a | grep en_US
en_US
en_US.iso885915
en_US.utf8
# locale -a | wc -l
495

EDIT3 (after correct Answer):

# unsetenv LC_PAPER
# unsetenv LC_ALL
# ls -1d /*/Gabriel_Faur?
/music/Gabriel_Fauré

The LC_PAPER=a4 variable prevents UTF-8 encoded Unicode characters from being printed (no pun intended) on screen!

Answer 1

What that locale: Cannot set LC_ALL to default locale: No such file or directory message tells you is that one of the locales you're trying to use doesn't exist. It's not about the $LC_ALL environment variable, locale is just reporting an error when the setlocale(LC_ALL, "") call it does to initialise localisation based on environment variables returns NULL indicating a locale configured via one of the various LC_*/LANG variables cannot be found.

Here, since it works with LC_ALL=en_US.UTF-8 which overrides all the other ones, the problem must be with LC_PAPER=a4. a4 is not the name of a valid locale on your system and is causing setlocal(LC_ALL, "") to fail.

When setlocale() fails, the behaviour defaults to the C locale, where the character encoding is ASCII. In the C locale, every byte is a character but 0xc3 and 0xa9 are unknown ones as they are not in ASCII, so ls -q (and -q is enabled when the output goes to a terminal) renders them as ?.

You can see the list of available locales on your system with:

locale -a

You probably won't find a a4 in there. If you want the paper size to be A4, where locale -k LC_PAPER outputs:

height=297
width=210
paper-codeset="UTF-8"

You'll probably want to use a European locale for $LC_PAPER, something like en_GB.UTF-8.

Answer 2

Missing locale can be fixed in OpenSuse with:

zypper in glibc-locale

I needed this to use a en_GB locale while building a MicroOs image with kiwi