title: Re-partitioning dask dataframe to reduce shuffling during rolling
body: <p>I'm working for a company that needs to do some pretty intense aggregation</p>
  
  <p>For every sale they made in the last two years, they want rolling aggregates concerning data for the month prior to that sale. The aggregates themselves are not important. The two years of data are around 5.5 million rows.</p>
  
  <p>I figured out that I should use Dasks rolling function, with a data range indices for the dataframe.</p>
  
  <p>The machine I'm using has has 4 cores so I decided to use 8 partitions on the data and re-partitioned it into 8, 3 month partitions.</p>
  
  <p>However, there's a problem. The first month of each of these 3 partitions is going to have to roll into another partition. This will involve shuffling around 2 million rows!</p>
  
  <p>I was thinking of somehow incorporating a copy of the month that will be rolled over into the beginning of each partition but I can't think of an efficient way of doing this. </p>
  
  <p>Any suggestions?</p>