title: Seek into a file full of pickled objects
body: <p>I have this huge file with objects pickled in, lets assume:</p>
  
  <pre><code>for object in objects:
     pickle.dump(myfile,object)
  </code></pre>
  
  <p>The objects are of different size although they are the same type.<br>
  The file gets filled for a long time on different occasions, but from time to time, when the dumping process gets restarted, I would need to read the last objects.<br>
  Something like this:</p>
  
  <pre><code> myfile.seek(-1000,2)
   while myfile.tell() &lt; mysize:
      objects.append(pickle.load(myfile))
  </code></pre>
  
  <p>Now, this obviously doesn't work because -1000 isn't normally at the start of one of the objects and pickle raises an exception etc...<br>
  While I could just try except:pass and let pickle fail till it finds something pickable, I don't really like the idea and I suspect it does advance the file too much on certain read tries and I could be missing few objects.</p>
  
  <p>Reading the file from the beginning is not an option because of its size.</p>
  
  <p>Any ideas for this to work? Is there any way for pickle to check if the current file cursor points to something that looks like an object or not?</p>