This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Getting odd floating point results.

((lat2-lat1)*BETA*(lat2-lat1)*BETA + (long2-long1)*delta*(long2-long1)*delta)

I'm writing some C code and I am getting some screwy results. Basically lat1, lat2, long1, long2 are all latitudes and longitudes in radians, delta is another precalculated parameter. BETA is a defined constant.

As far as I can tell this code always results in an answer equal to BETA regardless of changing inputs. Additionally when I brake out the two addition terms: (lat2-lat1)*BETA*(lat2-lat1)*BETA and (long2-long1)*delta*(long2-long1)*delta), and compute them separately I get a result of "0". I have been trying to wrap my head around what is wrong without success for a day now. Any suggestions for things to try would be appreciated.

Thanks in advance.

Parents Reply Children
  • #define BETA  6356752.0
    
    float procedure( float lat1, float long1, float lat2, float long2 ) {
            float delta;
            delta = ALPHA * cos(lat1);
            return ((lat2-lat1)*BETA*(lat2-lat1)*BETA + (long2-long1)*delta*(long2-long1)*delta);
    }
    

    lat1 = 0.727414575
    long1 = -1.24152067
    lat2 = 0.727336559
    long2 = -1.24135643

    I expect this to return approximately 737991.335
    and instead it returns 6356752.0 (same as BETA).

  • Sorry, #define ALPHA 6378137.0

  • "... instead it returns 6356752.0 (same as BETA)."

    For me, it does not return that value or the value you expect, but rather 857998.3.

    I suggest you break the long return expression into small intermediate expressions with values assigned to separate float variables so you can see the intermediate results.

    I'd have to look into it further, but I wonder if the large difference in magnitude between intermediate values combined with the limited resolution of a 32-bit float is causing problems.

  • I just double checked my hand calculations... the result you got is correct, please disregard the one I initially supplied.

    It still does not work for me, I can only assume some other condition is impacting the proper execution or compiling of this code.

  • FWIW, here's the code I used to test:

    #include <reg52.h>
    #include <math.h>
    #include <intrins.h>
    
    #define ALPHA 6378137.0
    #define BETA  6356752.0
    
    float procedure( float lat1, float long1, float lat2, float long2 ) {
        float delta;
        delta = ALPHA * cos(lat1);
        return ((lat2-lat1)*BETA*(lat2-lat1)*BETA + (long2-long1)*delta*(long2-long1)*delta);
    }
    
    void main(void)
    {
        float f;
    
        f = procedure(0.727414575, -1.24152067, 0.727336559, -1.24135643);
    
        for (;;) {
            _nop_();
        }
    }