Cortex-A8 NEON strange performance issue

Note: This was originally posted on 10th July 2013 at http://forums.arm.com

Hi all,

As an experienced assembly programmer, I'm currently working on NEON tutorials.

First example done, it beats the original C function by a huge margin :


pld [r1, #64*0]
pld [r1, #64*1]
pld [r1, #64*2]
pld [r1, #64*3]

    ldr  r12, [sp]
    vdup.16 d0, r2 //coeff
    vdup.16 d1, r3 //intercept

1:
        pld  [r1, #64*4]
        vld1.16  {d28-d31}, [r1,:256]!

        vmull.s16   q12, d28, d0[0]
        vmull.s16   q13, d29, d0[0]
        vmull.s16   q14, d30, d0[0]
        vmull.s16   q15, d31, d0[0]

        vaddw.s16   q12, q12, d1
        vaddw.s16   q13, q13, d1
        vaddw.s16   q14, q14, d1
        vaddw.s16   q15, q15, d1

        vqrshrun.s32    d24, q12, #8
        vqrshrun.s32    d25, q13, #8
        vqrshrun.s32    d26, q14, #8
        vqrshrun.s32    d27, q15, #8

        vst1.16  {d24-d27}, [r0,:256]!

        subs    r12, r12, #16
    bgt  1b
    bx      lr


However, there are some pipeline stalls here and there, and the dual-issuing isn't fully in effect. So I decided to optimize it :



pld  [r1, #64*0]
    pld  [r1, #64*1]
    pld  [r1, #64*2]
    pld  [r1, #64*3]
    pld  [r1, #64*4]
    ldr  r12, [sp]
    vdup.16 d0, r2 //coeff
    vdup.16 d1, r3 //intercept
    vld1.16  {d28-d31}, [r1,:256]!
    vmull.s16   q12, d28, d0[0]
    vmull.s16   q13, d29, d0[0]
    vmull.s16   q14, d30, d0[0]
    vmull.s16   q15, d31, d0[0]
    1:
        vaddw.s16   q12, q12, d1
        vld1.16  {d20-d23}, [r1,:256]!
        vaddw.s16   q13, q13, d1
        vaddw.s16   q14, q14, d1
        vaddw.s16   q15, q15, d1
        vqrshrun.s32    d24, q12, #8
        vqrshrun.s32    d25, q13, #8
        vqrshrun.s32    d26, q14, #8
        vqrshrun.s32    d27, q15, #8
        vmull.s16   q8, d20, d0[0]
        vmull.s16   q9, d21, d0[0]
        vmull.s16   q10, d22, d0[0]
        vmull.s16   q11, d23, d0[0]
        vst1.16  {d24-d27}, [r0,:256]!
        vaddw.s16   q8, q8, d1
        vaddw.s16   q9, q9, d1
        vaddw.s16   q10, q10, d1
        vaddw.s16   q11, q11, d1
        subs    r12, r12, #32
        vqrshrun.s32    d16, q8, #8
        vqrshrun.s32    d17, q9, #8
        vqrshrun.s32    d18, q10, #8
        ble  2f
        pld  [r1, #64*4]
        vld1.16  {d28-d31}, [r1,:256]!
        vqrshrun.s32    d19, q11, #8
        vmull.s16   q12, d28, d0[0]
        vmull.s16   q13, d29, d0[0]
        vmull.s16   q14, d30, d0[0]
        vst1.16  {d16-d19}, [r0,:256]!
        vmull.s16   q15, d31, d0[0]
    b 1b
    2:
    vqrshrun.s32    d19, q11, #8
    vst1.16  {d16-d19}, [r0,:256]!
    bx      lr


There is no pipeline stall at all in this optimized version, and all the memory accessing instructions are surrounded by data processing ones, so they get dual issued twice - at top and bottom. - looks good~

Then the bitter surprise comes while benchmarking on my 4th gen iPod touch and iPhone4 though (iOS6.13 if it matters) : The "optimized" version is about 5% slower than the initial one.


(r12 gets the value of a full HD resolution(1920*1080), and both functions are called several thousand times in a row)

I've been trying several variations, repositioning subs and pld around, nothing matters, the optimized version is ALWAYS slower.

And I removed the pld instruction completely, both functions become much slower as expected, but voila, the optimized version is now about 5% faster.


It seems I did the right thing in removing pipeline stalls and unfolding dual issuing capability, but I must have done something the L2 cache doesn't like in the optimized version.

Can someone give me some insight regarding this? I'm really curious, desperate, frustrated or whatever since without knowing the reason of this strange behavior, all the efforts I put in mastering NEON would be in vain.


Thanks in advance.
  • Note: This was originally posted on 17th July 2013 at http://forums.arm.com


    Your problem seems very strange

    - Does r0 and r1 reference the same memory zone ? if this is the case try to reference different memory space.

    - Can you try your code without any NEON instruction (except memoryt acces of course) ! May be you had saturate the memory acces capacity (what is the hardware you are using ?) !

    - Finally is there any chance you benchmark method were wrong ?


    I will try you code on my beagleboard to see !

    Etienne


    - r0 and r1 point on different memory blocks.

    - I'm using iPod touch 4th gen and iPhone4, both same gen :(

    - I've been trying different benchmarking methods, from simple Log to nanosec counter offered by Apple. I'm not saying they can't be wrong, but they all show the initial version to be the fastest.

    Thank you again.


    PS : Just finished a high precision 8x8 LLM iDCT with fixed point math, more accurate than with float. Can do about 1.5Million times iDCT/sec on my iPod touch (800Mhz Coretex A8). Not bad, huh?
  • Note: This was originally posted on 17th July 2013 at http://forums.arm.com


    > In the extended version It's exactly 64Bytes per iteration. No mistakes here.

    What I meant is that you have 2 VLD instructions (loading a total of 2 cachelines) per iteration, but you only have 1 PLD instruction. So you should add an extra PLD instruction in your loop, to preload 2 cachelines not just 1, otherwise you are only preloading half of your data!



    Two vld's loading 32bytes each = 64bytes = 1 cache line. :)
    Putting one additional PLD didn't help.

    Thanks anyway
  • Note: This was originally posted on 13th July 2013 at http://forums.arm.com


    Hi.

    The main NEON optimisation method is to let some times between register loading/saving instructions and compute operations

    Try something like this



    @ preload
            vld1.16   {d28-d31}, [r1,:256]!

    @ first operation to tranfert register
            vmull.s16   q12, d28, d0[0]
            vmull.s16   q13, d29, d0[0]
            vmull.s16   q14, d30, d0[0]
            vmull.s16   q15, d31, d0[0]

    @ you'll have to make 1 less iteration
            subs    r12, r12, #16
    .loop:


    @ load for next iteration
            vld1.16   {d28-d31}, [r1,:256]!

    @ working on preloaded datas

            vaddw.s16   q12, q12, d1
            vaddw.s16   q13, q13, d1
            vaddw.s16   q14, q14, d1
            vaddw.s16   q15, q15, d1

    @ prepare for saving
            vqrshrun.s32    d20, q12, #8
            vqrshrun.s32    d21, q13, #8
            vqrshrun.s32    d22, q14, #8
            vqrshrun.s32    d23, q15, #8



    @ let sone time before saving - transfert register
            vmull.s16   q12, d28, d0[0]
            vmull.s16   q13, d29, d0[0]
            vmull.s16   q14, d30, d0[0]
            vmull.s16   q15, d31, d0[0]

    @ saving
            vst1.16   {d20-d23}, [r0,:256]!


            subs    r12, r12, #16
    bgt   .loop

    @ need to finsh last iteration

            vaddw.s16   q12, q12, d1
            vaddw.s16   q13, q13, d1
            vaddw.s16   q14, q14, d1
            vaddw.s16   q15, q15, d1

            vqrshrun.s32    d20, q12, #8
            vqrshrun.s32    d21, q13, #8
            vqrshrun.s32    d22, q14, #8
            vqrshrun.s32    d23, q15, #8

            vst1.16   {d20-d23}, [r0,:256]!


    That sould works.
    But I do not test the code !


    I wish so much I could give you a success report, but unfortunately, it's not the case :(

    Anyway, thank you very much for your effort. I really appreciate that.

    I had to omit the optimization part in my tutorial. I just submitted the first part.

    http://armneon.blogspot.com

    cya
  • Note: This was originally posted on 13th July 2013 at http://forums.arm.com


    Yes I think Webshaker's idea will give you good results, because it is easy to read ARM & NEON docs and think that counting the CPU clock cycles for your instructions will be enough to estimate the total time of your loop, when in fact most code in mobile spends more time loading & saving memory than performing CPU calculations. So your main goal should be to minimize the amount of time wasted while waiting for data to load & store. This usually means accessing memory in good cache-friendly ways, using main memory the least you can inside your loop, AND trying to increase the number of useful operations that happen between loading a value from memory and actually using it in another instruction.

    Also, having longer loops can sometimes reduce the speed of your code, so this might also be influencing your perf.


    I've been trying about 20 variations, long loops, short loops,,,,, including what webshaker suggested and what you are talking about.

    However, the unoptimized original is ALWAYS the fastest as soon as PLD is enabled.

    There must be still something we aren't aware of....
More questions in this forum