title: Python: rewrite a looping numpy math function to run on GPU
body: <p>Can someone help me rewrite this one function <em>(the <code>doTheMath</code> function)</em> to do the calculations on the GPU? I used a few good days now trying to get my head around it but to no result. I wonder maybe somebody can help me rewrite this function in whatever way you may seem fit as log as I gives the same result at the end. I tried to use <code>@jit</code> from <code>numba</code> but for some reason it is actually much slower than running the code as usual. With a huge sample size, the goal is to decrease the execution time considerably so naturally I believe the GPU is the fastest way to do it. </p>
  
  <p>I'll explain a little what is actually happening. The real data, which looks almost identical as the sample data created in the code below is divided into sample sizes of approx 5.000.000 rows each sample or around 150MB per file. In total there are around 600.000.000 rows or 20GB of data. I must loop through this data, sample by sample and then row by row in each sample, take the last 2000 (or another) rows as of each line and run the <code>doTheMath</code> function which returns a result. That result is then saved back to the hardrive where I can do some other things with it with another program. As you can see below, I do not need all of the results of all the rows, only those bigger than a specific amount. If I run my function as it is right now in python I get about 62seconds per 1.000.000 rows. This is a very long time considering all the data and how fast it should be done with.</p>
  
  <p>I must mention that I upload the real data file by file to the RAM with the help of <code>data = joblib.load(file)</code> so uploading the data is not the problem as it takes only about 0.29 seconds per file. Once uploaded I run the entire code below. What takes the longest time is the <code>doTheMath</code> function. I am willing to give all of my 500 reputation points I have on stackoverflow as a reward for somebody willing to help me rewrite this simple code to run on the GPU. My interest is specifically in the GPU, I really want to see how it is done on this problem at hand.</p>
  
  <p><strong>EDIT/UPDATE 1:</strong>
  Here is a link to a small sample of the real data: <a href="https://mab.to/Nx8GvwTjQ" rel="noreferrer">data_csv.zip</a> About 102000 rows of real data1 and 2000 rows for real data2a and data2b. Use <code>minimumLimit = 400</code> on the real sample data </p>
  
  <p><strong>EDIT/UPDATE 2:</strong>
  For those following this post here is a short summary of the answers below. Up until now we have 4 answers to the original solution. The one offered by @Divakar are just tweaks to the original code. Of the two tweaks only the first one is actually applicable to this problem, the second one is a good tweak but does not apply here. Out of the other three answers, two of them are CPU based solutions and one tensorflow-GPU try. The Tensorflow-GPU by Paul Panzer seems to be promising but when i actually run it on the GPU it is slower than the original, so the code still needs improvement.</p>
  
  <p>The other two CPU based solutions are submitted by @PaulPanzer (a pure numpy solution) and @MSeifert (a numba solution). Both solutions give very good results and both process data extremely fast compared to the original code. Of the two the one submitted by Paul Panzer is faster. It processes about 1.000.000 rows in about 3 seconds. The only problem is with smaller batchSizes, this can be overcome by either switching to the numba solution offered by MSeifert, or even the original code after all the tweaks that have been discussed below.</p>
  
  <p>I am very happy and thankful to @PaulPanzer and @MSeifert for the work they did on their answers. Still, since this is a question about a GPU based solution, i am waiting to see if anybody is willing to give it a try on a GPU version and see how much faster the data can be processed on the GPU when compared to the current CPU solutions. If there will be no other answers outperforming @PaulPanzer's pure numpy solution then i'll accept his answer as the right one and gets the bounty :) </p>
  
  <p><strong>EDIT/UPDATE 3:</strong>
  @Divakar has posted a new answer with a solution for the GPU. After my testings on real data, the speed is not even comparable to the CPU counterpart solutions. The GPU processes about 5.000.000 in about 1,5 seconds. This is incredible :) I am very excited about the GPU solution and i thank @Divakar for posting it. As well as i thank @PaulPanzer and @MSeifert for their CPU solutions :) Now my research continues with an incredible speed due to the GPU :) </p>
  
  <pre><code>import pandas as pd
  import numpy as np
  import time
  
  def doTheMath(tmpData1, data2a, data2b):
      A = tmpData1[:, 0]
      B = tmpData1[:,1]
      C = tmpData1[:,2]
      D = tmpData1[:,3]
      Bmax = B.max()
      Cmin  = C.min()
      dif = (Bmax - Cmin)
      abcd = ((((A - Cmin) / dif) + ((B - Cmin) / dif) + ((C - Cmin) / dif) + ((D - Cmin) / dif)) / 4)
      return np.where(((abcd &lt;= data2a) &amp; (abcd &gt;= data2b)), 1, 0).sum()
  
  #Declare variables
  batchSize = 2000
  sampleSize = 5000000
  resultArray = []
  minimumLimit = 490 #use 400 on the real sample data 
  
  #Create Random Sample Data
  data1 = np.matrix(np.random.uniform(1, 100, (sampleSize + batchSize, 4)))
  data2a = np.matrix(np.random.uniform(0, 1, (batchSize, 1))) #upper limit
  data2b = np.matrix(np.random.uniform(0, 1, (batchSize, 1))) #lower limit
  #approx. half of data2a will be smaller than data2b, but that is only in the sample data because it is randomly generated, NOT the real data. The real data2a is always higher than data2b.
  
  
  #Loop through the data
  t0 = time.time()
  for rowNr in  range(data1.shape[0]):
      tmp_df = data1[rowNr:rowNr + batchSize] #rolling window
      if(tmp_df.shape[0] == batchSize):
          result = doTheMath(tmp_df, data2a, data2b)
          if (result &gt;= minimumLimit):
              resultArray.append([rowNr , result])
  print('Runtime:', time.time() - t0)
  
  #Save data results
  resultArray = np.array(resultArray)
  print(resultArray[:,1].sum())
  resultArray = pd.DataFrame({'index':resultArray[:,0], 'result':resultArray[:,1]})
  resultArray.to_csv("Result Array.csv", sep=';')
  </code></pre>
  
  <p>The PC specs I am working on:</p>
  
  <pre><code>GTX970(4gb) video card; 
  i7-4790K CPU 4.00Ghz; 
  16GB RAM;
  a SSD drive 
  running Windows 7; 
  </code></pre>
  
  <p>As a side question, would a second video card in SLI help on this problem?</p>