1

I'm using SQL Server and in one of the tables I have a nullable float column. When I look at that value in Microsoft SQL Server Management Studio I have a value in one of the records as 1.3. I also have a client app that gets data from the database via Entity Framework (EF) and it has a corresponding nullable double (double?) property for that column. Now when I get a value from the database it has 1.2999999523162842 values in it.

All these records (more than 1 million) will be used in interpolation, extrapolation, calculation and any smallest deviation becomes a significant value. So all values on the client side have to match values in the database.

I can't round on the client side to the first decimal number, because in the database there are values with 2, 3, 4, or more decimal points.

My question is how can I make sure that all values in the database will be the same values on the client after loading them from database via EF?

Improve this question

1 Answer 1

Reset to default
9

Isn't float a binary floating point value in SQL Server? If so, your value isn't really 1.3, as that isn't exactly representable in binary floating point. Options:

  • Perhaps your field is effectively a decimal floating (or fixed) point type, in which case you should use decimal on the .NET side as well.
  • If you're using binary floating point in SQL server, I'd expect the result in .NET to be the same: the nearest binary floating point value of the appropriate type to the number you originally entered. Just because SQL Server is showing you the value as 1.3 doesn't mean it is exactly 1.3.

Using my DoubleConverter, it looks like the closest double value to 1.3 is actually 1.3000000000000000444089209850062616169452667236328125 - so I'm not sure why you'd be getting the value you're actually seeing. The closest float value is 1.2999999523162841796875, which sounds like it's what you're getting. So it's as if your value in the database is being converted as a 32-bit floating point number instead of a 64-bit one.

Is your database field actually something like FLOAT(23)? If so, that corresponds with a .NET float instead of double. That would explain things.

Improve this answer
Sign up to request clarification or add additional context in comments.

18 Comments

Or presumably it could be possible that somewhere between database and where it is being tested in the code it has been forced through a float accidentally somewhere?
@Chris: Possibly - but I'm assuming for the moment that the OP has reported the result as soon as it's come from the database. I think a problematic schema is the more likely culprit, but who knows? :)
Yeah, I just recall many times having problems with string data coming out of the database truncated and it turned out that it was fine in the database, its just that a stored procedure that used it was trying to put it into a varchar(50) when it was a varchar(100). Takes ages to figure out if you aren't really thinking about it because you don't think to check that your variables are the right size until you've been bitten by that a few times. :) The OP can worry about it from now though. :)
Thanks for helping. I checked database and database has float(53). Also I am checking value right after I pulled data from table by EF, so there are no conversions or anything in between done. When I was using EF designer, EF designer automatically created double? property for that column. So as you can see I have Float(53) - 8 bytes in the database and I have double in C# code.
@Vlad: In that case it ought to be okay. What happens if you try to populate the data with the-closest-double-to-1.3? Can you then read it out again? Do you appreciate the point that the actual value will never be exactly 1.3?
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.