title: Feedback on Python RSYNC script wanted
body: <p>This script appears to work great, but I'm sure this could be optimised by some of you gurus!</p>
  
  <p>Purpose of script:</p>
  
  <ul>
  <li>"watch" a directory for new files with particular file extension</li>
  <li>ensure file is not still being copied</li>
  <li>rsync the file to remote server</li>
  <li>rsync deletes the file</li>
  <li>script loops continuously forever</li>
  <li>survives internet/network outage</li>
  <li><p>leaves no partial file on remote server</p>
  
  <pre><code>import os
  import subprocess
  import time
  import logging
  import datetime
  from sys import argv
  
  if len(argv) &lt; 3:
      exit('Please provide two arguments - Source Destination')
  
  
  LOC_DIR = argv[1]
  REM_DIR = argv[2]
  
  POLL_INT = 10
  RUN_INT = 60
  FILE_EXT = '.mov'
  
  
  # logging setup
  logging.basicConfig(filename='%s' % os.path.join(LOC_DIR, '%s script.log' % datetime.datetime.now()),level=logging.DEBUG)
  
  # make an easy print and logging function
  def printLog(string):
      print '%s %s' % (datetime.datetime.now(), string)
      logging.info('%s %s' % (datetime.datetime.now(), string))
  
  
  # get the files with absolute paths
  def getFiles(path):
      return [os.path.join(path, entry) for entry in os.listdir(path)]
  
  
  # check if file is still being copied (file size has changed within the poll interval)
  def checkSize(path):
      same = False
      while same is False:
          printLog("Processing '%s'" % os.path.basename(path))
          printLog('Waiting %s seconds for any filesize change' % POLL_INT)
          size1 = os.path.getsize(path)
          time.sleep(POLL_INT)
          size2 = os.path.getsize(path)
          if size1 == size2:
              same = True
              printLog('File size stayed the same for %s seconds' % POLL_INT)
              return same
          else:
              printLog('File size change detected. Waiting a further %s seconds' % POLL_INT)
  
  
  # check if correct file extension
  def checkExt(path):
      if path.endswith(FILE_EXT):
          return True
  
  
  # rsync subprocess
  def rsyncFile(path):
      printLog("Syncing file '%s'" % os.path.basename(path))
      try:
          command = ['rsync', '-a', '--remove-source-files', path, REM_DIR]
          p = subprocess.Popen(command, stdout=subprocess.PIPE)
          for line in p.stdout:
              printLog("rsync: '%s'" %line)
          p.wait()
          if p.returncode == 0:
              printLog('&lt;&lt;&lt; File synced successfully :) &gt;&gt;&gt;')
          elif p.returncode == 10:
              printLog('****** Please check your internet connection!! ******  Rsync error code: %s' % p.returncode)
          else:
              printLog('There was a problem. Error code: %s' % p.returncode)
      except Exception as e:
          logging.debug(e)
  
  
  # main logic
  def main():
      all_files = getFiles(LOC_DIR)
      files = []
      for f in all_files:
          if checkExt(f):
              files.append(f)
      if len(files) == 1:
          printLog('&lt;&lt;&lt; Found %s matching file &gt;&gt;&gt;' % len(files))
      elif len(files) &gt; 1:
          printLog('&lt;&lt;&lt; Found %s matching files &gt;&gt;&gt;' % len(files))
      for f in files:
          if checkSize(f):
              rsyncFile(f)
      printLog('No files found.  Checking again in %s seconds' % RUN_INT)
      time.sleep(RUN_INT)
      printLog('Checking for files')
      main()
  
  if __name__ == "__main__":
  
  
      main()
  </code></pre></li>
  </ul>