Take the 2-minute tour ×

Device GeForce GTX 680

In the program, i have very long array to be processed inside kernel.(Approx 1 GB of integers).As per need,My array is divided into blocks sequentially with some overlap(overlap between blocks is k). I have fixed size of each block(block size is m) .Now, array will be divided in order (0,m) (m-k, (m-k) +m) ,....)

As per above calculation, no of blocks needed in my program will be approximately (1GB / m) Since total number of blocks is limited in GPU, how can i effectively do it?. Should i call kernel in iterative manner from host without any loops inside kernel?? or should i call kernel once and then loop inside kernel for multiple iterations? or should i call kernel only once with total no of blocks = (1 GB /m) ??

What can be put as best value for number of blocks for this program and what methods?

share|improve this question

1 Answer 1

I would suggest the following sequence for the first version of your app:


  • allocate room on the GPU for two non-overlapping blocks of the array on the GPU (slot 1 and 2)
  • copy the first non-overlapping block to slot 1


  • copy the next non-overlapping block to slot 2
  • run a kernel that runs on slot 1 and partially into slot 2
  • copy contents of slot 2 to slot 1 (GPU to GPU memory copy)

In a later version, you can avoid the GPU to GPU copy by copying alternately into slot 1 and slot 2 and wrapping the addressing around in the kernel so that instead of overflowing slot 2, it starts at the beginning of slot 1. Think of it as slot 1 and slot 2 being arranged into a ring buffer. You can also improve performance by adding more slots and asynchronously copying blocks of the array to new slots while the kernel is running on previous slots.

share|improve this answer
Then what should be the ideal kernel invocations and number of blocks for each kernel? –  user1352179 Nov 25 '12 at 20:38
I think a good rule of thumb is to have at least one thread to fill each slot in the pipeline in each core. More threads than that won't hurt performance, but won't help either. At that point, it's just what is most convenient for you. As far as I know, NVIDIA has not disclosed the latency of the pipelines, but it may be around 10 clocks. If so, you should try to schedule at least 1536 cores * 10 clock latency = 15360 threads per kernel. –  Roger Dahl Nov 25 '12 at 22:38
It's hard to say how many threads you should have per block. You use the CUDA Occupancy Calculator (included with CUDA) to get suggestions for this number. However, you should also try to adjust it up and down compared to the number the calculator provides and time the results. 128 and 256 are commonly used. You want the number to be divisible by 32. –  Roger Dahl Nov 25 '12 at 22:43

Your Answer


