ASP.Net conversion on using float numbers

Question

I have a conversion or some kind of type casting problem in my asp.net/C# math app. A contractor wrote some code to calculate a relative standard error in an assembly. But when I run it, apparently doesn't work right.

The data types are as follows:

        public double dSwx = 0.0, dSw = 100.0;
        public double dTx = 0.0, dTwn=0.0;
        public float fEstimateVal = 0.0F;       //  For Estimate Value, it should be same as Swx/Sw

        //i did a system.out on this in my web app and they came out as:
        //zeroDataCell.dSwx = 0.0;
        //zeroDataCell.dSw = 100.0;

        zeroDataCell.fEstimateVal = (float)(zeroDataCell.dSwx / zeroDataCell.dSw) * 100.0f;

       //so now zeroDataCell.fEstimateVal should be 0.0, but my code blows 
       //through the if statement below, is there some conversion problem? 
       //should i use EqualsTo?            


        if (zeroDataCell.fEstimateVal != 0.0f)
            zeroDataCell.fRse = zeroDataCell.fTwx / zeroDataCell.fEstimateVal * 100.0f;0

Why does fEstimateVal that eqauls zero stil run through the if loop?

what is the type of the fEstimateVal field?

platon
– platon

2011-07-07 19:47:18 +00:00
Commented Jul 7, 2011 at 19:47 — platon
– platon, Commented Jul 7, 2011 at 19:47
lemme double check on the fEstimate data type

cdub
– cdub

2011-07-07 19:50:00 +00:00
Commented Jul 7, 2011 at 19:50 — cdub
– cdub, Commented Jul 7, 2011 at 19:50

Tigran · Accepted Answer · 2011-07-07 20:00:07Z

2

Dealing with float/double numbers is always precision balancing. If you want to compare it against clear 0, convert it to integer and compare after.

Or add some precisio stuff like

((int)(floatnumber * 100)) == 0.

edited Jul 7, 2011 at 20:00

answered Jul 7, 2011 at 19:54

Tigran

62.3k8 gold badges90 silver badges124 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Tigran Over a year ago

Sorry, but writing from mobile, so don't have code formatting:)

platon · Accepted Answer · 2011-07-07 19:56:59Z

1

The following code works fine here:

if (zeroDataCell.fEstimateVal != 0)
  zeroDataCell.fRse = zeroDataCell.fTwx / zeroDataCell.fEstimateVal * 100.0f;

answered Jul 7, 2011 at 19:56

platon

5,3801 gold badge24 silver badges25 bronze badges

2 Comments

platon Over a year ago

let me know the results, please

cdub Over a year ago

I used the above to explicitly force it to int and multiplied it by 100.

Collectives™ on Stack Overflow

ASP.Net conversion on using float numbers

2 Answers 2

1 Comment

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Related