This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

round result of floating point calcualtion

Hello,

I have a problem with the accuracy of the result when I divide 2 float values.

This is the line in my source Code:

 dds_word.long_dds = 2.147483648E16 / 7.1312E7 

The result which I get is 3.011392E8

The unrounded result is 3.0113916845E8

There is no difference when I work with float or double variables. Is there any possibility to get more accuracy?

Parents
  • Yes it is an integer value which I try to compute. The problem when I divide the values is the following.
    The value 2.147483648E16 is fix but the second value is variable from 70700000, 70700001, 70700002 .....124999999, 125000000. When I pre-divide this value then the value 124999999 looks like this 124. Then my problem is much bigger because many results are the same.

    Jan

Reply
  • Yes it is an integer value which I try to compute. The problem when I divide the values is the following.
    The value 2.147483648E16 is fix but the second value is variable from 70700000, 70700001, 70700002 .....124999999, 125000000. When I pre-divide this value then the value 124999999 looks like this 124. Then my problem is much bigger because many results are the same.

    Jan

Children