uDMA Transfer

I'm using uDMA to transfer content from source to destination. During each transfer, I see the source pointer and destination pointer is moved from the memory to the DMA. Ideally, we need to do that only once. Because of the source/destination pointer movements, each transfer is taking 4 cycles. Why is it required to move the source pointer and destination pointer in each cycle? Is there a mode in which this can be avoided?