This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

round result of floating point calcualtion

Hello,

I have a problem with the accuracy of the result when I divide 2 float values.

This is the line in my source Code:

 dds_word.long_dds = 2.147483648E16 / 7.1312E7 

The result which I get is 3.011392E8

The unrounded result is 3.0113916845E8

There is no difference when I work with float or double variables. Is there any possibility to get more accuracy?

Parents
  • Hmm... is that value you're trying to compute actually a integer, by any chance? If so, maybe you should just ditch the whole floating point stuff and use unsigned longs (after pre-dividing both constants by 1e7, e.g., they're both in the range of unsigned long). Or hard-code the actual result you wanted instead of dividing through magic constants in the source code.

Reply
  • Hmm... is that value you're trying to compute actually a integer, by any chance? If so, maybe you should just ditch the whole floating point stuff and use unsigned longs (after pre-dividing both constants by 1e7, e.g., they're both in the range of unsigned long). Or hard-code the actual result you wanted instead of dividing through magic constants in the source code.

Children