title: How to write a memory efficient Python program?
body: <p>It's said that Python automatically manages memory. I'm confused because I have a Python program consistently uses more than 2GB of memory.</p>
  
  <p>It's a simple multi-thread binary data downloader and unpacker.</p>
  
  <pre><code>def GetData(url):
      req = urllib2.Request(url)
      response = urllib2.urlopen(req)
      data = response.read() // data size is about 15MB
      response.close()
      count = struct.unpack("!I", data[:4])
      for i in range(0, count):
          UNPACK FIXED LENGTH OF BINARY DATA HERE
          yield (field1, field2, field3)
  
  class MyThread(threading.Thread):
      def __init__(self, total, daterange, tickers):
          threading.Thread.__init__(self)
  
      def stop(self):
          self._Thread__stop()
  
      def run(self):
          GET URL FOR EACH REQUEST
          data = []
          items = GetData(url)
          for item in items:
              data.append(';'.join(item))
          f = open(filename, 'w')
          f.write(os.linesep.join(data))
          f.close()
  </code></pre>
  
  <p>There are 15 threads running. Each request gets 15MB of data and unpack it and saved to local text file. How could this program consume more than 2GB of memory? Do I need to do any memory recycling jobs in this case? How can I see how much memory each objects or functions use?</p>
  
  <p>I would appreciate all your advices or tips on how to keep a python program running in a memory efficient mode.</p>
  
  <p><strong>Edit:</strong> Here is the output of "cat /proc/meminfo"</p>
  
  <pre><code>MemTotal:        7975216 kB
  MemFree:          732368 kB
  Buffers:           38032 kB
  Cached:          4365664 kB
  SwapCached:        14016 kB
  Active:          2182264 kB
  Inactive:        4836612 kB
  </code></pre>