original_docstring: 
      gets all json files in a specified filepath and add each file as an element to a list, then calls different functions to process and
      populate data from either JSON log files or JSON song files
      
docstring: gets all json files in a specified filepath and add each file as an element to a list, then calls different functions to process and
  populate data from either JSON log files or JSON song files
docstring_tokens: gets all json files in a specified filepath and add each file as an element to a list then calls different functions to process and populate data from either JSON log files or JSON song files
code: def process_data(cur, conn, filepath, func):
      """
      gets all json files in a specified filepath and add each file as an element to a list, then calls different functions to process and
      populate data from either JSON log files or JSON song files
      """
      # get all files matching extension from directory
      all_files = []
      for root, dirs, files in os.walk(filepath):
          files = glob.glob(os.path.join(root,'*.json'))
          for f in files :
              all_files.append(os.path.abspath(f))
  
      # get total number of files found
      num_files = len(all_files)
      print('{} files found in {}'.format(num_files, filepath))
  
      # iterate over files and process
      for i, datafile in enumerate(all_files, 1):
          func(cur, datafile)
          conn.commit()
          print('{}/{} files processed.'.format(i, num_files))
code_tokens: def process_data ( cur , conn , filepath , func ) : all_files = [ ] for root , dirs , files in os . walk ( filepath ) : files = glob . glob ( os . path . join ( root , '*.json' ) ) for f in files : all_files . append ( os . path . abspath ( f ) ) num_files = len ( all_files ) print ( '{} files found in {}' . format ( num_files , filepath ) ) for i , datafile in enumerate ( all_files , 1 ) : func ( cur , datafile ) conn . commit ( ) print ( '{}/{} files processed.' . format ( i , num_files ) )
short_docstring: gets all json files in a specified filepath and add each file as an element to a list, then calls different functions to process and
  populate data from either JSON log files or JSON song files
short_docstring_tokens: gets all json files in a specified filepath and add each file as an element to a list then calls different functions to process and populate data from either JSON log files or JSON song files
comment: """
      gets all json files in a specified filepath and add each file as an element to a list, then calls different functions to process and
      populate data from either JSON log files or JSON song files
      """$SEP$# get all files matching extension from directory$SEP$# get total number of files found$SEP$# iterate over files and process