title: Schedule batch SQL in NiFi
body: <p>I'm using NiFi to connect 2 systems:</p>
  
  <ul>
  <li>Source one generating events in a Kafka topic</li>
  <li>Destination one where I will only consider the Oracle database.</li>
  </ul>
  
  <p>I need to reduce the JSON coming in the Kafka topic and push them in appropriate tables. No major issues in doing this but... The source system is generating too many events and the destination database triggers processes for every modifications. And is not sized to handle that many processes.</p>
  
  <p>So I'm doing bulk update in my DB, using the <code>PutSQL Processor</code> behind a <code>Text Processor</code> + <code>Update Attribute Processor</code> + <code>ReplaceText Processor</code> (as shown here for example: <a href="https://community.hortonworks.com/articles/91849/design-nifi-flow-for-using-putsql-processor-to-per.html" rel="nofollow noreferrer">https://community.hortonworks.com/articles/91849/design-nifi-flow-for-using-putsql-processor-to-per.html</a>).</p>
  
  <p>But this workflow allows me to update my DB based on a number of elements to put in it (my batch size).</p>
  
  <p><strong>I would like to bulk update on a regular, time based, basis.</strong> Reason is that source events are not coming linearly, and destination database cannot accept being more than 5 minutes "away" from the source. So I need to schedule my bullk update at worst every 5 minutes.</p>
  
  <p>I can't see right now how to do this. Please could you tell me which processors/solution you would you?</p>
  
  <p><em>PS: Of course, tons of better solutions exist, like not triggering heavy processes on each commit in my destination database, but changing this "good old system" is not affordable right now.</em></p>
  
  <p>Cheers,
  Olivier</p>