• State Not Answered
  • Locked Locked
  • Replies 20 replies
  • Subscribers 21 subscribers
  • Views 3512 views
  • Users 0 members are here
2025 survey

We are running a survey to help us improve the experience for all of our members. If you see the survey appear, please take the time to tell us about your experience if you can.

Options
Related
How was your experience today?
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Getting odd floating point results.

Nick S 

((lat2-lat1)*BETA*(lat2-lat1)*BETA + (long2-long1)*delta*(long2-long1)*delta)

I'm writing some C code and I am getting some screwy results. Basically lat1, lat2, long1, long2 are all latitudes and longitudes in radians, delta is another precalculated parameter. BETA is a defined constant.

As far as I can tell this code always results in an answer equal to BETA regardless of changing inputs. Additionally when I brake out the two addition terms: (lat2-lat1)*BETA*(lat2-lat1)*BETA and (long2-long1)*delta*(long2-long1)*delta), and compute them separately I get a result of "0". I have been trying to wrap my head around what is wrong without success for a day now. Any suggestions for things to try would be appreciated.

Thanks in advance.

Parents
  • 0 in reply to ImPer Westermark

    earth is not flat, so pythagoras is no fun

    It's not quite that bad. As I said, the formula uses Pythagoras on arc lengths instead of straight lines (secants). I.e. roads, not tunnels.

    But yes, a less incorrect function would have to go from arcs to secants, use Pythagoras, then convert the resulting secant back into a surface arc.

Reply
  • 0 in reply to ImPer Westermark

    earth is not flat, so pythagoras is no fun

    It's not quite that bad. As I said, the formula uses Pythagoras on arc lengths instead of straight lines (secants). I.e. roads, not tunnels.

    But yes, a less incorrect function would have to go from arcs to secants, use Pythagoras, then convert the resulting secant back into a surface arc.

Children
  • 0 in reply to HansBernhard Broeker

    Thanks all for the help, I believe I have resolved this with your help.

            arc_lat = lat1 - lat2;
        part1 = cos(lat1);
        part2 = long1-long2;
        arc_long = ECC*part1*part2;
        part1 = pow(arc_lat,2);
        part2 = pow(arc_long,2);
        return BETA * sqrt( part1+part2 );

    Without using pow() I could not get valid results for computing the hypotenuse...

  • 0 in reply to Nick S

    Without using pow() I could not get valid results for computing the hypotenuse...

    So are you telling us that not only did you observe a difference between pow(x,2) and x*x here, but pow(x,2) was closer to correct? Now that would be seriously worrying. If there's a difference at all, it's pow() that should be less precise than simple multiplication.

    What were the actual results for both methods?

  • 0 in reply to HansBernhard Broeker

    What I mean by invalid results was the same supposed cancellation problem we were discussing before. x*x resulted in a 0 answer where pow(x,2) gave me an acceptable result.