Inconsistent bitwise shifting result in C code

Question

I'm writing a C program and need to create a bitwise mask that could potentially fill a computer word (64 or 32 bits) with all 1s. I found a bug in my code but I can't make sense of it because when I run the operation using literal values it gives the expected value, but when I store the parts in variables and do the same operation I get a different result.

#include <stdio.h>
#include <stdint.h>

#define min(a,b) (((a) < (b)) ? (a) : (b))

typedef uint64_t WORD;

int main() {
    
    WORD one = 1;
    
    printf("mask0=%lu\n", (one<<64) - one);
    
    uint32_t shift_amt = 2;
    uint32_t pos_in_word = 62;
    uint32_t my_min = min((uint32_t)64, pos_in_word + shift_amt);
    WORD mask = (one<<my_min) - one;
    
    printf("one=%lu, my_min=%d pos_in_word=%d, shift_amt=%d, mask1=%lu\n", 
            one, my_min, pos_in_word, shift_amt, mask);

    return 0;
}

Building with command: gcc -Wall -std=c99 main.c -lm -o main under Linux Ubuntu.

I expect the value to be 18446744073709551615 (all 1s) in both cases. In the first case, mask0=18446744073709551615. In the second case, mask1=0. I don't understand how. I'd really appreciate some help please.

Answer 1

You're performing an invalid shift.

Shifting by a value greater than or equal to the bit size of the operand being shifted triggers undefined behavior in your code. This is spelled out in section 6.5.8p3 of the C standard:

The integer promotions are performed on each of the operands. The type of the result is that of the promoted left operand. If the value of the right operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined

If you had done multiple shifts to keep under this limit, you would get the expected result:

printf("mask0=%" PRIu64 "\n", ((one<<32)<<32) - one);

Note that using the wrong conversion specifier in printf also makes your program have undefined behavior, and you are consistently using conversion specifiers that may be invalid in some implementations and valid in other implementations. When printing variables of fixed width integer types, use the macros defined in <inttypes.h> to make sure you get the correct conversion specifiers. Example:

printf("one=%" PRIu64 ", my_min=%" PRIu32 " pos_in_word=%" PRIu32
       ", shift_amt=%" PRIu32 ", mask1=%" PRIu64 "\n",
       one, my_min, pos_in_word, shift_amt, mask);

Answer 2

There are multiple issues in your code:

• the macro min expands to an expression that evaluates the arguments twice, which may pose a problem if you pass arguments with side effects. You do not do this in the posted code, but you might want to check other uses in your code base... Macros with such issues used to be named in uppercase to warn programmers of the different semantics from functions. With modern compilers, defining min as an inline function is highly recommended.

• you shift values by 64, which is the width in bits of the shifted type. This has undefined behavior. The reason for this is most recent CPUs only use the low 6 bits of the shift count for 64-bit values, resulting in no change to the argument value (shifting by 0 is a no-op) while some more ancient or exotic hardware use the full value of the shift argument and evaluate to a different value. In such ambiguous cases, the C Committee decides to make the behavior undefined to keep the translation to code as minimal as possible. Other languages do specify that such shift operations mask the shift count by 63.

• you should use the macros defined in <inttypes.h> to pass uint32_t and uint64_t values to printf instead of hard-coding %u and %lu as some legacy systems define long as 32-bit types instead of 64-bit types.

  You should compile with extra warnings to let the compiler flag these operations as having undefined behavior.

Note that (one << 64) would be 0 if the full shift did occur, so you could just write (WORD)0 instead. In your program, you compute one << my_min, which has undefined behavior if my_min is greater or equal to 64, so you just cannot use this expression, you must add a test:

WORD mask = (my_min >= 64 ? (WORD)0 : one << my_min) - one;

Also note that printing bit patterns in hexadecimal is much more readable.

Here is a modified version:

#include <stdio.h>
#include <stdint.h>

uint32_t min_u32(uint32_t a, uint32_t b) { return a < b ? a : b; }

typedef uint64_t WORD;

int main(void) {
    
    WORD one = 1;
    
    printf("mask0=%#"PRIx64"\n", -one);
    
    uint32_t shift_amt = 2;
    uint32_t pos_in_word = 62;
    uint32_t my_min = min_u32(64, pos_in_word + shift_amt);
    WORD mask = (my_min >= 64 ? (WORD)0 : one << my_min) - one;
    
    printf("one=%#"PRIx64", my_min=%"PRIu32" pos_in_word=%"PRIu32", shift_amt=%"PRIu32", mask1=%#"PRIx64"\n", 
            one, my_min, pos_in_word, shift_amt, mask);

    return 0;
}

For your purpose, filling a word with all one bits, here are 2 simple portable methods:

• use the bit complement operator on a suitably typed 0 value:

  WORD mask = ~(WORD)0;
  

• even simpler: just use -1:

  WORD mask = -1;