title: How to find a good/optimal dictionary for zlib 'setDictionary' when processing a given set of data?
body: <p>I have a (huge) set of similar data files. The set is constantly growing. The size of a single file is about 10K. Each file must be compressed on its own. The compression is done with the zlib library, which is used by the <code>java.util.zip.Deflater</code> class. When passing a dictionary to the Deflate algorithm using <code>setDictionary</code>, I can improve the compression ratio.</p>
  
  <p>Is there a way (algorithm) to find the 'optimal' dictionary, i.e. a dictionary with the overall optimal compression ratio?</p>
  
  <p>See <a href="http://www.gzip.org/zlib/manual.html#deflateSetDictionary" rel="noreferrer">zlib manual</a></p>