This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Getting odd floating point results.

((lat2-lat1)*BETA*(lat2-lat1)*BETA + (long2-long1)*delta*(long2-long1)*delta)

I'm writing some C code and I am getting some screwy results. Basically lat1, lat2, long1, long2 are all latitudes and longitudes in radians, delta is another precalculated parameter. BETA is a defined constant.

As far as I can tell this code always results in an answer equal to BETA regardless of changing inputs. Additionally when I brake out the two addition terms: (lat2-lat1)*BETA*(lat2-lat1)*BETA and (long2-long1)*delta*(long2-long1)*delta), and compute them separately I get a result of "0". I have been trying to wrap my head around what is wrong without success for a day now. Any suggestions for things to try would be appreciated.

Thanks in advance.

Parents
  • But in your case, you will get very large numbers because you do not divide with 2*pi before you square.

    That's got nothing to do with the problem. Given his input coordinates are in radians, dividing by 2*pi would be correct only if accompanied by multiplying the result by (2*pi)^2 afterwards --- a waste of effort for nothing unless intermediate results were in danger of hitting overflow. Which they aren't.

    If one really wanted to reduce number scales, it's the earth's radius in meters that should be factored out:

    ecc=ALPHA/BETA # a constant

    arc_lat = lat1 - lat2
    arc_long = ecc*cos(lat1)*(long1-long2)

    result_angle = hypot(arc_lat, arc_long)

    result_meters = result_angle*BETA

Reply
  • But in your case, you will get very large numbers because you do not divide with 2*pi before you square.

    That's got nothing to do with the problem. Given his input coordinates are in radians, dividing by 2*pi would be correct only if accompanied by multiplying the result by (2*pi)^2 afterwards --- a waste of effort for nothing unless intermediate results were in danger of hitting overflow. Which they aren't.

    If one really wanted to reduce number scales, it's the earth's radius in meters that should be factored out:

    ecc=ALPHA/BETA # a constant

    arc_lat = lat1 - lat2
    arc_long = ecc*cos(lat1)*(long1-long2)

    result_angle = hypot(arc_lat, arc_long)

    result_meters = result_angle*BETA

Children
No data