Process ADC data, moved by DMA, using CMSIS DSP: what's the right way?

This is about the code that you posted in NXP Community.

Your code temporarily converts the negative ADC values to positive (by "manual" two's complementing) and eventually you convert back to negative by multiplying by -1. I suggest that you explore alternate, which can be more efficient, ways of converting from 12-bit signed integer to floating-point format.

A possible way of converting the ADC data from 12-bit signed integer to 32-bit floating-point format is by transforming to 32-bit two's complement code then assigning the result to a floating-point variable (simply let the compiler do the rest of the work).

If the ADC data is in (12-bit) offset binary code it can be converted to 32-bit two's complement format by simply subtracting the offset (in this case the offset is 2048)

    adcBuff[i] - 2048

where adcBuff[] is an array of signed 32-bit integer (int32_t). We then let the compiler convert it into floating-point format by assigning to a floating-point variable. The complete statement would be

    float32Buff[i] = adcBuff[i] - 2048;

where float32Buff[] is an array of 32-bit floating-point (float32_t) data.

If the ADC data is in 12-bit two's complement code it can be converted to 32-bit two's complement format by simply sign-filling bits 12 to 31 (replicating bit 11 into bits 12 to 31). Sign-filling can be done by (logical) shifting the 12-bit ADC data 20 bit positions to the left then performing an arithmetic shift right by 20 bit positions

    (adcBuff[i] << 20) >> 20

Note that there is no test for the state of bit 11. I just can't clearly recall if I previously encountered a compiler which cancels (no corresponding code generated) opposing shifts like this. Including the assignment to a floating-point variable, the complete statement would be

    float32Buff[i] = (adcBuff[i] << 20) >> 20;

If the compiler produces no code for the opposite shifts, you should devise a way to circumvent that or you can resort to alternative methods. An alternate method of converting the ADC data is to transform from 12-bit two's complement to 12-bit offset binary format. The subsequent conversion to 32-bit two's complement can be done as described above for the offset binary format. Two's complement code can be converted to offset binary format by simply complementing the leftmost bit (in this case bit 11)

    adcBuff[i] ^ 0X00000800

The complete conversion statement would be

    float32Buff[i] = (adcBuff[i] ^ 0X00000800) - 2048;

This is about the code that you posted in NXP Community.

Your code temporarily converts the negative ADC values to positive (by "manual" two's complementing) and eventually you convert back to negative by multiplying by -1. I suggest that you explore alternate, which can be more efficient, ways of converting from 12-bit signed integer to floating-point format.

A possible way of converting the ADC data from 12-bit signed integer to 32-bit floating-point format is by transforming to 32-bit two's complement code then assigning the result to a floating-point variable (simply let the compiler do the rest of the work).

If the ADC data is in (12-bit) offset binary code it can be converted to 32-bit two's complement format by simply subtracting the offset (in this case the offset is 2048)

    adcBuff[i] - 2048

where adcBuff[] is an array of signed 32-bit integer (int32_t). We then let the compiler convert it into floating-point format by assigning to a floating-point variable. The complete statement would be

    float32Buff[i] = adcBuff[i] - 2048;

where float32Buff[] is an array of 32-bit floating-point (float32_t) data.

If the ADC data is in 12-bit two's complement code it can be converted to 32-bit two's complement format by simply sign-filling bits 12 to 31 (replicating bit 11 into bits 12 to 31). Sign-filling can be done by (logical) shifting the 12-bit ADC data 20 bit positions to the left then performing an arithmetic shift right by 20 bit positions

    (adcBuff[i] << 20) >> 20

Note that there is no test for the state of bit 11. I just can't clearly recall if I previously encountered a compiler which cancels (no corresponding code generated) opposing shifts like this. Including the assignment to a floating-point variable, the complete statement would be

    float32Buff[i] = (adcBuff[i] << 20) >> 20;

If the compiler produces no code for the opposite shifts, you should devise a way to circumvent that or you can resort to alternative methods. An alternate method of converting the ADC data is to transform from 12-bit two's complement to 12-bit offset binary format. The subsequent conversion to 32-bit two's complement can be done as described above for the offset binary format. Two's complement code can be converted to offset binary format by simply complementing the leftmost bit (in this case bit 11)

    adcBuff[i] ^ 0X00000800

The complete conversion statement would be

    float32Buff[i] = (adcBuff[i] ^ 0X00000800) - 2048;

Extremely helpful. What I do need more right now is an insight from a skilled eye to point out what could be possible limitation of my design.
I'm not experienced so I read some chapters on this interesting book: The Definitive Guide to ARM® Cortex®-M3 and Cortex®-M4 Processors, Third Edition: Joseph Yiu: 9780124080829: Amazon.com:…   

Here there are lots of useful information about the cycles needed for each operation and I'll try to do rough calculations about how many of them my code needs.
I hoped I could squeeze things to fit in that 5 cycles boundary (which I was aware of since the beginning), but, as I understand, I might be wrong.
Anyway, I'm planning to follow your advice and, since I'll need to sample impulsive signals, to use the ADCHS in non-continuous mode, using the thresholds I can set and processing data when no info is being sampled. What do you think about it?

Again, thank you very much: as I said I'm developing this alone and for the first time, therefore your help is highly appreciated.

Cheers,

Andrea

Good post goodwin!

goodwin wrote:

    (adcBuff[i] << 20) >> 20

I just can't clearly recall if I previously encountered a compiler which cancels opposing shifts like this.

Such compilers are faulty and would generate incorrect code.

The compiler is free to optimize the code, however, making the code behave incorrectly is not allowed.

An exception: If the resulting value of the shift is unused, then of course the compiler is allowed to remove it completely.

Another exception: If the compiler can 'see' that only the low 12 bits are used anyway, it is free to ignore the line completely (GCC is capable of doing that, as the optimizer is very sophisticated).

I would expect that an optimizing compiler would reduce the above to a single instruction:

... for signed values ...

sbfx rT,rS,0,12

... and for unsigned values ...

ubfx rT,rS,0,12

-I know that GCC is working well, regarding optimizing and code reduction.

Indeed, the optimization that goodwin mentioned will improve performance dramatically!

The problem with floating point and Interrupt Service Routines (ISR) is that if you use a floating point register in the ISR, it will need to be saved before it's used and restored before the interrupt is ended.

-This is because if you're using floating point registers anywhere in the other parts of your program, the values would be messed up randomly if the registers are changed by the interrupt.

However... If you're using floating point only in the interrupt, you won't have any problems.

[Note: sometimes it would be worth it to use hardware floating point in ISR and software floating point at task-time; eg. if you're only using software floating point for printing out values once per second, that would be the optimal solution. Mixing software and hardware floating point is an advanced topic, however and is not recommended the first 3 days of your life as a programmer].

Building upon goodwin's answer, I'd like to suggest a complete loop:

void Twos2Dec_Remapp(const uint32_t *twosBuff, float32_t *decBuff, uint32_t buffLength)
{
     register uint32_t       i;
     register float          *d;
     register const uint32_t *s;

     s = (int32_t *) &twosBuff[buffLength];
     d = &decBuff[buffLength];
     i = -buffLength;  /* convert length to negative index for speed */
     do
     {
          d[i] = (s[i] & 0xfff) - 2048;  /* most likely, this is the correct calculation */
     }
     while(++i);
}

I believe the above would produce the optimal binary code (by 'hinting' the compiler how).

The most expensive part is to convert to a float! In fact, I think it's a very bad idea using floats if you don't have hardware floats.

Imagine that for each float operation, a huge block of code is executed. Each line of code usually takes 1 or 2 clock cycles (unless we're speaking about dividing), so if you can, use fixed point.

Converting to a 16:16 fixed point is real easy; you just need to change the type 'float *' to 'int32_t *' and the shift operations to ...

          d[i] = (s[[i] << 20) >> 4);

-Because fixed point is just an "integer part" and a "fractional part". The integer part is the same as your integer value, the fractional part is 0.

So in hexadecimal, a 16:16 fixed point would lookk like this:

0xiiiipppp

-Also make sure that your destination buffer is not in the SRAM section, which the DMA is using, in order to avoid disturbing the DMA.

The larger the DMA buffer is, the larger the 'propagation delay' will be (eg. time between input and output)

If your data is output in real-time, you will want a small DMA buffer.

But if the DMA buffer is very small, the CPU will spend a lot of time executing non-essential code (such as entering/leaving subroutines, instead of actually working on the data).

-So you'll need to find the right balance and when you've found a size that juuust works, give it a little more room; 40% extra is often a good choice. I would not recommend less than 10% extra.

I think that's the way to do it but as I stated I am not an expert. Instead I would further suggest that you access LabTool's documentation and open-source software. LabTool is also based on LPC4370/LPC-Link 2 so I hope you find it helpful to your thesis.

Thanks for the detailed reply Jens.
Right now I'm doing the sign-aligned stuff inside Thibaut's function (https://www.m4-unleashed.com/parallel-comparison/ ), which is called during the DMA's Transfer Completed ISR.
here's my code:

uint32_t MAXmin;
int16_t sample[NUM_SAMPLE] = {0};
int16_t sample2[NUM_SAMPLE] = {0};
uint16_t shiftBits = 4;
uint16_t wordLenght = 8; /*Figured out looking at the registers address in while debugging*/

uint32_t SearchMinMax16_DSP(int16_t* pSrc, int32_t pSize)
{
    uint32_t data, min, max;
    int16_t data16;

    /* max variable will hold two max : one on each 16-bits half
     * same thing for min
     */

    /*Sign Extension*/
    *pSrc = (*pSrc) << shiftBits;
    *pSrc = (*pSrc) >> shiftBits;
    *(pSrc + wordLenght) = (*(pSrc+wordLenght)) << shiftBits;
    *(pSrc + wordLenght) = (*(pSrc+wordLenght)) >> shiftBits;

    /* Load two first samples in one 32-bit access */
    data = *__SIMD32(pSrc)++;
    /* Initialize Min and Max to these first samples */
    min = data;
    max = data;
    /* decrement sample count */
    pSize-=2;

    /* Loop as long as there remains at least two samples */
    while (pSize > 1)
    {
        /*Sign Extension*/
         *pSrc = (*pSrc) << shiftBits;
         *pSrc = (*pSrc) >> shiftBits;
         *(pSrc + wordLenght) = (*(pSrc+wordLenght)) << shiftBits;
         *(pSrc + wordLenght) = (*(pSrc+wordLenght)) >> shiftBits;


        /* Load next two samples in a single access */
        data = *__SIMD32(pSrc)++;
        /* Parallel comparison of max and new samples */
        (void)__SSUB16(max, data);
        /* Select max on each 16-bits half */
        max = __SEL(max, data);
        /* Parallel comparison of new samples and min */
        (void)__SSUB16(data, min);
        /* Select min on each 16-bits half */
        min = __SEL(min, data);

        pSize-=2;
    }
    /* Now we have maximum on even samples on low halfword of max
     * and maximum on odd samples on high halfword */
    /* look for max between halfwords 1 & 0 by comparing on low halfword */
    (void)__SSUB16(max, max >> 16);
    /* Select max on low 16-bits */
    max = __SEL(max, max >> 16);

    /* look for min between halfwords 1 & 0 by comparing on low halfword */
    (void)__SSUB16(min >> 16, min);
    /* Select min on low 16-bits */
    min = __SEL(min, min >> 16);

    /* Test if odd number of samples */
    if (pSize > 0)
    {
        data16 = *pSrc;
        /* look for max between on low halfwords */
        (void)__SSUB16(max, data16);
        /* Select max on low 16-bits */
        max = __SEL(max, data16);

        /* look for min on low halfword */
        (void)__SSUB16(data16, min);
        /* Select min on low 16-bits */
        min = __SEL(min, data16);
    }

    /* Pack result : Min on Low halfword, Max on High halfword */
    return __PKHBT(min, max, 16); /* PKHBT documentation */
}

At line 33 there's the bit extension.

Great analysis about the clock/sample Jens!

That means if your code runs from SRAM, then per sample, it should cost 7 clock cycles if P is 1.

Sounds great! How can i be sure this is happening?

Speaking of the compilare, this is the out put of my actual configuration in lpcxpresso (S2D.c is the file containing the code we are talking about):

rm-none-eabi-gcc -nostdlib -L"/home/abet/LPCXpresso/link2_2/lpc_board_nxp_lpclink2_4370/Debug" -L"/home/abet/LPCXpresso/link2_2/lpc_chip_43xx/Debug" -L"/home/abet/LPCXpresso/link2_2/CMSIS_DSPLIB_CM4/lib" -Xlinker -Map="S2D.map" -Xlinker --gc-sections -Xlinker -print-memory-usage -mcpu=cortex-m4 -mthumb -T "S2D_Debug.ld" -o "S2D.axf"  ./src/S2D.o ./src/cr_startup_lpc43xx.o ./src/crp.o ./src/sysinit.o   -llpc_board_nxp_lpclink2_4370 -llpc_chip_43xx -lCMSIS_DSPLIB_CM4

Memory region           Used Size  Region Size  %age Used

      RamLoc128:            6688 B      128 KB      5.10%

        RamLoc72:               0 GB        72 KB      0.00%

       RamAHB32:               0 GB        32 KB      0.00%

       RamAHB16:               0 GB        16 KB      0.00%

    RamAHB_ETB16:         0 GB        16 KB      0.00%

   RamM0Sub16:               0 GB        16 KB      0.00%

     RamM0Sub2:               0 GB          2 KB      0.00%

                 SPIFI:          13668 B         4 MB      0.33%

Also,    

arm-none-eabi-gcc --version

gives:

arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.2.1 20151202 (release) [ARM/embedded-5-branch revision 231848]

Looking at the project properties as suggested by Jens I found out that I had no optimization level here:

So I'm going to turn this on and implements Inside Thibaut's function the sign-extension the Jen's way! And see if I get some good news!

goodwin, jensbauer, I'd like to thank you very very much for the effort you're putting into helping me: I'd really offer you a beer (or 2) if we weren't spread around the globe (but, I mean, never say never).
Anyway, I'm working hard on my project in these days and today I'd like to write here a small recap. I hope this isn't off-topic: I know I asked info about floating point conversion so, I would be happy to give the correct answer to Goodwin cause his insight is great, but then I'd need to open up a new post to continue the interesting optimization discussion.
Please, let me know if a way is better than the other and what I should do, I really don't want to take advantage of anyone.

The 'Big' Picture

In this project I need to use the ADCHS to sample gaussian-shaped impulsive signals like these:

These are generated by analog electronics and we're working in the field of ionizing-radiations, so let's say these spikes represent two photons and have a 3us full width.
My goal is to reach a 100kHz count rate: this means 10us between each peak, and therefore to process a MinMax algorithm on the data with this requirement (this would be great, but I'm trying to find a compromise between count rate and hardware costs).

Yesterday there was an important update to the project and a collegue was able to tell me that we need at least 40 samples (or 128 on the entire 10us duration) per spike, so this acquisition can't be slower than roughly 1.25 Msps.

(Note that I still hope to be able to perform a 40Msps acquisition and understand how to get out the most from this micro, since this would open a whole set of possibilities for the applications of my project)

Memory Layout

I'm not an expert on this topic, and since I'm self taught on this I think i could also be somewhat slow, but since you point out that some info about my memory layout should be very helpful, I'll try to describe it.
From my LPCXpresso IDE:

Note that:

• I needed to add the SPIFI Flash in order to use the Link2 as evaluation board (as described here: Introduction to Programming the NXP LPC4370 MCU Using the      LPCxpresso Tools and Using Two LPC-Link2 Boards and here: Using an LPC-Link2 as an LPC4370 evaluation board | NXP Community
• I'm declaring my gloabl arrays like so:

int16_t sample[NUM_SAMPLE] = {0};
int16_t sample2[NUM_SAMPLE] = {0};

Elaboration: is the LPC4370 @204MHz fast enough?

As jensbauer said now I'm trying to figure out which is the best DMA buffer lenght, and, sadly, as  goodwin said @40Msps I'm not able to perform a continuous read + elaboration.

I tried:

• the CMSIS dsp functions (How to prepare ADC data for Q31_t CMSIS DSP functions? , thanks to dwhite85: this is why I was able to get rid of the float type.)

arm_shift_q31(sample, shiftBits, sampleTmp, NUM_SAMPLE);
arm_max_q31(sampleTmp, NUM_SAMPLE, &maxSample, &sampleIndex);

• thibaut's solution:  https://www.m4-unleashed.com/parallel-comparison/   (thanks for this one!). Eventually modified with the sign-extension suggested by goodwin

MAXmin = SearchMinMax16_DSP(sampleTest, NUM_SAMPLE);

Here we have some results from the tests I did:

  So it turns out that:

1. I know that thibaut's implementation is giving me both the Max and the Min, but shouldn't it be faster anyway? Am I missing something about optimization level? I must admit I'm still stucked with the standard LPCXpresso one.
2. My sin extension implementation is dead-slow. Here's my code, and I just notice I have few extra that are not needed (fixing them..).

*pSrc = (*pSrc) << shiftBits;
*pSrc = (*pSrc) >> shiftBits;

Any help on this optimization problem is highly appreciated of course!

Hope this wasn't too long/boring, if so let me know and i'll do everything I can to: give a better explanation, divide the problem in small ones and find the right place in the forum for every piece.
Again, thank to those who helped since today, lovely community!

Regards,
Andrea