Menu ▾ ▴

#20 double to unsigned long long precision error

2.0
pending
nobody
None
1
2013-02-26
2013-02-25
Anonymous
No

Hi, there is a bug with unsigned long long values. At least on my system where sizeof(unsigned long long) = sizeof(double) = 8

The problem is that internally you use double for all intergral types, which might cause
loss of precision for big unsigned long long numbers.

this fails on my system:

unsigned long long ullValue = 63660062001020978;
 double doubleValue = 63660062001020978;

 double unsignedlonglong_to_double = static_cast<double>(ullValue);
 unsigned long long double_to_unsignedlonglong = static_cast<unsigned long             long>(doubleValue);

qDebug() << "unsigned long long: " << ullValue;
qDebug() << "double: " << doubleValue;
qDebug() << "unsigned long long to double: " << unsignedlonglong_to_double;
qDebug() << "double to unsigned long long: " << double_to_unsignedlonglong;

output is:

unsigned long long: 63660062001020978 <-- compare this
double: 6.36601e+16
unsigned long long to double: 6.36601e+16
double to unsigned long long: 63660062001020976 <-- with this

cast from double <--> unsigned long long might cause problems both ways.

Discussion

  • Artyom Beilis

    Artyom Beilis - 2013-02-26
    • status: open --> pending
     

  • Artyom Beilis

    Artyom Beilis - 2013-02-26

    CppCMS represents numeric values as double internally so it wouldn't be possible to fix in without chaning an interface. So at this point I can't fix it.

    You must be aware of this limitations.

     

  • Artyom Beilis

    Artyom Beilis - 2013-02-26

    Ticket moved from /p/cppcms/bugs/94/

     

  • Artyom Beilis

    Artyom Beilis - 2013-02-26
    • milestone: 1.2 --> 2.0
     

Anonymous
Anonymous