• State Not Answered
  • Locked Locked
  • Replies 20 replies
  • Subscribers 21 subscribers
  • Views 3511 views
  • Users 0 members are here
2025 survey

We are running a survey to help us improve the experience for all of our members. If you see the survey appear, please take the time to tell us about your experience if you can.

Options
Related
How was your experience today?
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Getting odd floating point results.

Nick S 

((lat2-lat1)*BETA*(lat2-lat1)*BETA + (long2-long1)*delta*(long2-long1)*delta)

I'm writing some C code and I am getting some screwy results. Basically lat1, lat2, long1, long2 are all latitudes and longitudes in radians, delta is another precalculated parameter. BETA is a defined constant.

As far as I can tell this code always results in an answer equal to BETA regardless of changing inputs. Additionally when I brake out the two addition terms: (lat2-lat1)*BETA*(lat2-lat1)*BETA and (long2-long1)*delta*(long2-long1)*delta), and compute them separately I get a result of "0". I have been trying to wrap my head around what is wrong without success for a day now. Any suggestions for things to try would be appreciated.

Thanks in advance.

Parents
  • 0 in reply to ImPer Westermark

    Note that a PC using single-precision arithmetic will not compute in the same way that the Keil code does. When Keil multiply two float values they do it in single precision. A PC will have extended precision for intermediate operations. Most PC compilers then have compilation flags to control if the generated code should follow the IEEE regulations for storing back intermediate results into limited-sized variables or if the compiler should keep the intermediate values in the floating-point registers (which normally are 80-bit large).

    But in your case, you will get very large numbers because you do not divide with 2*pi before you square.

Reply
  • 0 in reply to ImPer Westermark

    Note that a PC using single-precision arithmetic will not compute in the same way that the Keil code does. When Keil multiply two float values they do it in single precision. A PC will have extended precision for intermediate operations. Most PC compilers then have compilation flags to control if the generated code should follow the IEEE regulations for storing back intermediate results into limited-sized variables or if the compiler should keep the intermediate values in the floating-point registers (which normally are 80-bit large).

    But in your case, you will get very large numbers because you do not divide with 2*pi before you square.

Children
  • 0 in reply to ImPer Westermark

    But in your case, you will get very large numbers because you do not divide with 2*pi before you square.

    That's got nothing to do with the problem. Given his input coordinates are in radians, dividing by 2*pi would be correct only if accompanied by multiplying the result by (2*pi)^2 afterwards --- a waste of effort for nothing unless intermediate results were in danger of hitting overflow. Which they aren't.

    If one really wanted to reduce number scales, it's the earth's radius in meters that should be factored out:

    ecc=ALPHA/BETA # a constant

    arc_lat = lat1 - lat2
    arc_long = ecc*cos(lat1)*(long1-long2)

    result_angle = hypot(arc_lat, arc_long)

    result_meters = result_angle*BETA