title: Vectorized way for nested for loop situation with dataframes in Python
body: <p>I'm a little new to Python, so I am struggling to reduce the time complexity of my problem.</p>
  <p>I have created a data frame as follows:</p>
  <pre><code>  volume    SAs  HOEP     Date      Time         Diff_200    Diff_210   Diff_220    Diff_230    Diff_240    ... Diff_340    Diff_350    Diff_360    Diff_370    Diff_380    Diff_390    Diff_400    Difference_410  Difference_420  Difference_430
  451.586788  12  0.00193 2022-06-20  23:00:00    251.586788  241.586788  231.586788  221.586788  211.586788  ... 111.586788  101.586788  91.586788   81.586788   71.586788   61.586788   51.586788   41.586788   31.586788   21.586788
  398.799094  12  0.00481 2022-06-20  22:00:00    198.799094  188.799094  178.799094  168.799094  158.799094  ... 58.799094   48.799094   38.799094   28.799094   18.799094   8.799094    -1.200906   -11.200906  -21.200906  -31.200906
  349.545342  12  0.02966 2022-06-20  21:00:00    149.545342  139.545342  129.545342  119.545342  109.545342  ... 9.545342    -0.454658   -10.454658  -20.454658  -30.454658  -40.454658  -50.454658  -60.454658  -70.454658  -80.454658
  309.973705  12  0.04074 2022-06-20  20:00:00    109.973705  99.973705   89.973705   79.973705   69.973705   ... -30.026295  -40.026295  -50.026295  -60.026295  -70.026295  -80.026295  -90.026295  -100.026295 -110.026295 -120.026295
  288.761856  12  0.06420 2022-06-20  19:00:00    88.761856   78.761856   68.761856   58.761856   761856  ... -51.238144  -61.238144  -71.238144  -81.238144  -91.238144  -101.238144 -111.238144 -121.238144 -131.238144 -141.238144
  ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
  620.289423  12  0.00000 2021-06-21  04:00:00    420.289423  410.289423  400.289423  390.289423  380.289423  ... 280.289423  270.289423  260.289423  250.289423  240.289423  230.289423  220.289423  210.289423  200.289423  190.289423
  614.984896  12  0.00000 2021-06-21  03:00:00    414.984896  404.984896  394.984896  384.984896  374.984896  ... 274.984896  264.984896  254.984896  244.984896  234.984896  224.984896  214.984896  204.984896  194.984896  184.984896
  543.722130  12  0.00442 2021-06-21  02:00:00    343.722130  333.722130  323.722130  313.722130  303.722130  ... 203.722130  193.722130  183.722130  173.722130  163.722130  153.722130  143.722130  133.722130  123.722130  113.722130
  515.074752  12  0.00731 2021-06-21  01:00:00    315.074752  305.074752  295.074752  285.074752  275.074752  ... 175.074752  165.074752  155.074752  145.074752  135.074752  125.074752  115.074752  105.074752  95.074752   85.074752
  470.910849  12  0.02276 2021-06-21  00:00:00    270.910849  260.910849  250.910849  240.910849  230.910849  ... 130.910849  120.910849  110.910849  100.910849  90.910849   80.910849   70.910849   60.910849   50.910849   40.910849
  </code></pre>
  <p>where, the Diff_number column is basically the difference between the volume column and the number for the particular row and the particular column. Ex for the first row: Diff_200 = 451.586 - 200 = 251.586 and so on.</p>
  <p>My aim is to create a new list and insert these columns alongside the diff columns. The new list will run from 0 to (column sum/hours)(where hours = 5824 when I am running the iteration for 7 days a week and hours = 4160 when I am running the iteration for 5 days a week) for every single column. The values will be also be added on the basis of a time constraint, that is they will only be added if df['Time'] is within 7 am to 11 pm.</p>
  <p>First I will generate the columns for 7 days a week, for the mentioned time frame and for hours = 5824</p>
  <p>So far I have written this code.</p>
  <pre><code>#Code for 7 days and 16 hours(7 am to 11 pm)
  allowed_time = ['7:00:00','8:00:00','9:00:00','10:00:00','11:00:00','12:00:00','13:00:00','14:00:00','15:00:00','16:00:00','17:00:00','18:00:00','19:00:00','20:00:00','21:00:00','22:00:00','23:00:00']
  hours_fixed = 5824
  col_names = ['Diff_'+str(y) for y in reqd_list]
  for col in col_names:
      colsum = 0
      max_fixed = 0
      new_list = []
      for index in df_current.index:
          colsum = colsum + df_current[col][index]
      max_fixed = colsum/hours_fixed
      new_list = np.arange(0, round(max_fixed), 10).tolist()
      column_names = [col+&quot;_&quot;+str(i) for i in new_list]
      for j in range(len(new_list)):
          for idx in df_current.index:
              if any(x in df_current['Time'][idx] for x in allowed_time):
                  df_current[column_names[j]] = df_current[col] - new_list[j]
    
  </code></pre>
  <p>I want to repeat this and generate columns for 5 days(only weekdays) of a week from the columns generated from 7 days a week, for the mentioned time frame and for hours = 4160.</p>
  <p>So far I am using this.</p>
  <pre><code>#Code for 5 days and 16 hours(7 am to 11 pm): Basically same as 7 days with only the constraint in weekday()
  df1 = df_current.iloc[:, 29:]
  namesofcolumns = df1.columns.values.tolist()
  hours_peak = 4160
  for colu in namesofcolumns:
      colsum = 0
      max_peak = 0
      new_list = []
      Column_Names = []
      for ix in df_current.index:
          colsum = colsum + df_current[colu][ix]
      max_peak = colsum/hours_peak
      new_list = np.arange(0, round(max_peak), 10).tolist()
      print(new_list)
      Column_Names = [colu+&quot;_&quot;+str(x) for x in new_list]
      for y in range(len(new_list)):
          for idex in df_current.index:
              if ((any(z in df_current['Time'][idex] for z in allowed_time)) &amp; (df_current['Date'][idex].weekday() &lt;=4)):
                  df_current[Column_Names[y]] = df_current[colu] - new_list[y]
  </code></pre>
  <p>The code for 5 days and 16 hours is taking ages to execute, so I was wondering if I can reduce the time complexity severely by using functions for both 7 days and 5 days and vectorizing the code, but since I am relatively new to Python, I don't know how.</p>