title: Is ConfigParser slow? ConfigParser vs. an ugly algorithm
body: <p>I was trying to improve a function I created to parse a bunch of INI files into a single JSON file. When I wrote this function I was a newbie and I didn't know <code>configparser</code> module, so now I want to exploit it.</p>
  <p><strong>INPUT</strong>: INI file (parent) containing a long set of references to other INI files (children).</p>
  <p><strong>GOAL</strong>: to convert the INI parent file to JSON and include <strong>some</strong> info taken from <strong>all</strong> the children files into it. In other words: take some info from a set of INI files and export them into a single JSON file.</p>
  <p><strong>Question</strong>: I expected my new code to be at least as fast as the old one, but it isn't: it is 2 times slower. Why is that? Is it ConfigParser or is it me? Performance are really important and my function takes one second to parse around 900 INI files, while the old one takes half a second.</p>
  <h2>Parent Example</h2>
  <p>(it can have from hundreds of lines to <strong>tens of thousands</strong>):</p>
  <pre><code>[General]
  Name = parent
  ...
     
  [Item 000001]
  Name = first item
  path = &quot;path/to/child_1.ini&quot;
  ...
  
  [Item 000002]
  Name = second item
  ...
  
  [...]
  
  [Item 001000]
  Name = thousandth item
  ...   
  </code></pre>
  <h2>Child Example</h2>
  <p>(it can have from less than 100 lines to about 200):</p>
  <pre><code>[General]
  Name = name
  ID = 12345
  ...
  
  [Options]
  ...
  </code></pre>
  <h2>JSON Output Example</h2>
  <pre><code>{
      &quot;Parent&quot;: {
          &quot;Name&quot;: &quot;parent&quot;,
          &quot;Count&quot;: &quot;1000&quot;,
          [...]
          &quot;child1&quot;: {
              &quot;Name&quot;: &quot;name&quot;,
              &quot;ID&quot;: &quot;12345&quot;,
              &quot;Option1&quot;: &quot;...&quot;,
              &quot;Option2&quot;: &quot;...&quot;,
              &quot;Option3&quot;: &quot;...&quot; 
          },
          &quot;child2&quot;: {
              &quot;Name&quot;: &quot;name2&quot;,
              &quot;ID&quot;: &quot;22222&quot;,
              &quot;Option1&quot;: &quot;...&quot;,
              &quot;Option2&quot;: &quot;...&quot;,
              &quot;Option3&quot;: &quot;...&quot; 
          },
          [...]
          &quot;child1000&quot;: {
              &quot;Name&quot;: &quot;name1000&quot;,
              &quot;ID&quot;: &quot;12332&quot;,
              &quot;Option1&quot;: &quot;...&quot;,
              &quot;Option2&quot;: &quot;...&quot;,
              &quot;Option3&quot;: &quot;...&quot; 
          }
      }
  }
  </code></pre>
  <h2>OLD CODE</h2>
  <pre><code>def split_string_by_equal(string):
      str_operands = string.split(' = ')
      first_part = (str_operands[0]).strip()
      second_part = (' '.join(str_operands[1:])).strip()
      return [first_part, second_part]
  
  def parse_ini_to_json(path):
      parent_dict = {}
      child_dict = {}
      num_child = 1
      parent_directory = os.path.dirname(testflow)
      with open(path, 'r') as parent_file:
          for line in tfl_file:
              left_part = split_string_by_equal(line)[0]
              right_part = split_string_by_equal(line)[1]
              if left_part in SOME_WORDS:
                  parent_dict.update({left_part: do_something(parent_directory, right_part)})
              elif left_part == 'Count':
                  parent_dict.update({'Count': right_part})
              elif left_part == 'JohnDoe':
                  parent_dict['JohnDoe'] = right_part
              elif 'Item' in line:
                  if child_dict:
                      parent_dict.update({'test{}'.format(num_child): child_dict})
                      child_dict = {}
                      num_child += 1
              elif left_part in SOME_OTHER_WORDS:
                  child_dict.update({left_part: right_part})
              if left_part == 'path':
                  child_dict.update(extract_data_from_child(right_part))
      if child_dict:
          parent_dict.update({'child{}'.format(num_test): child_dict})
      return parent_dict
  
  def extract_data_from_child(path):
      &quot;&quot;&quot; same methodology used in above function &quot;&quot;&quot;
      [...]
      return child_dict
  </code></pre>
  <h2>NEW CODE</h2>
  <pre><code>def get_config_parser(path):
      config = configparser.ConfigParser()
      config.optionxform = str
      config.read(path)
      return config
  
  def parse_ini_to_json(path):
      config = get_config_parser(path)
      parent_directory = os.path.dirname(testflow)
      parent_dict = {}
      for key in config['Folders'].keys():
          parent_dict[key] = do_something( parent_directory, config['Folders'][key])
      parent_dict['Count'] = config['General']['Count']
      parent_dict['JohnDoe'] = config['General']['JohnDoe']
      counter = 1
      for key in config.keys():
          if 'Item' in key:
              child_dict = {}
              for child_prop in config[key].keys():
                  if child_prop in SOME_WORDS:
                      child_dict[child_prop] = config[key][child_prop]
              child_path = config[key]['path']
              child_dict.update(extract_data_from_child(child_path))
              child_dict[f'child{counter}'] = child_dict
              counter += 1
      return parent_dict
  
  
  def extract_data_from_child(path):
      config = sysfunc.get_config_parser(path)
      child_dict = {}
      for key in config['General'].keys():
          if key in SOME_KEYWORDS:
              child_dict[key] = config['General'][key]
      for key in config['Levels'].keys():
          if key in SOME_OTHER_KEYWORDS:
              child_dict[key] = config['Options'][key]
      try:
          some_value = config['Levels']['SomeKey']
      except KeyError:
          pass
      for key in config['Options'].keys():
          value = config['Options'][key]
          key_enabled = key.strip() + 'enabled'
          try:
              if config['Options'][key_enabled] == '0':
                  continue
          except KeyError:
              continue
          if 'false' in value:
              value = '0'
          elif 'true' in value:
              value = '1'
          child_dict[key] = value
      return child_dict 
  </code></pre>