title: What is the best compression library for very small amounts of data (3-4 kib?)
body: <p>I am working on a game engine which is loosely descended from Quake 2, adding some things like scripted effects (allowing the server to specify special effects in detail to a client, instead of having only a limited number of hardcoded effects which the client is capable of.) This is a tradeoff of network efficiency for flexibility.</p>
  
  <p>I've hit an interesting barrier. See, the maximum packet size is 2800 bytes, and only one can go out per client per frame. </p>
  
  <p>Here is the script to do a "sparks" effect (could be good for bullet impact sparks, electrical shocks, etc.)
  <a href="http://pastebin.com/m7acdf519" rel="noreferrer">http://pastebin.com/m7acdf519</a> (If you don't understand it, don't sweat it; it's a custom syntax I made and not relevant to the question I am asking.)</p>
  
  <p>I have done everything possible to shrink the size of that script. I've even reduced the variable names to single letters. But the result is exactly 405 bytes. Meaning you can fit at most 6 of these per frame. I also have in mind a few server-side changes which could shave it down another 12, and a protocol change which might save another 6. Although the savings would vary depending on what script you are working with.</p>
  
  <p>However, of those 387 bytes, I estimate that only 41 would be unique between multiple usages of the effect. In other words, this is a prime candidate for compression.</p>
  
  <p>It just so happens that R1Q2 (a backward-compatible Quake 2 engine with an extended network protocol) has Zlib compression code. I could lift this code, or at least follow it closely as a reference.</p>
  
  <p>But is Zlib necessarily the best choice here? I can think of at least one alternative, LZMA, and there could easily be more.</p>
  
  <p>The requirements: </p>
  
  <ol>
  <li>Must be very fast (must have very small performance hit if run over 100 times a second.) </li>
  <li>Must cram as much data as possible into 2800 bytes</li>
  <li>Small metadata footprint</li>
  <li>GPL compatible</li>
  </ol>
  
  <p>Zlib is looking good, but is there anything better? Keep in mind, none of this code is being merged yet, so there's plenty of room for experimentation.</p>
  
  <p>Thanks,
  -Max</p>
  
  <p>EDIT: Thanks to those who have suggested compiling the scripts into bytecode. I should have made this clear-- yes, I am doing this. If you like you can browse the relevant source code on my website, although it's still not "prettied up."<br>
  This is the server-side code:<br>
  Lua component: <a href="http://meliaserlow.dyndns.tv:8000/alienarena/lua_source/lua/scriptedfx.lua" rel="noreferrer">http://meliaserlow.dyndns.tv:8000/alienarena/lua_source/lua/scriptedfx.lua</a><br>
  C component: <a href="http://meliaserlow.dyndns.tv:8000/alienarena/lua_source/game/g_scriptedfx.c" rel="noreferrer">http://meliaserlow.dyndns.tv:8000/alienarena/lua_source/game/g_scriptedfx.c</a><br>
  For the specific example script I posted, this gets a 1172 byte source down to 405 bytes-- still not small enough. (Keep in mind I want to fit as many of these as possible into 2800 bytes!)</p>
  
  <p>EDIT2: There is no guarantee that any given packet will arrive. Each packet is supposed to contain "the state of the world," without relying on info communicated in previous packets. Generally, these scripts will be used to communicate "eye candy." If there's no room for one, it gets dropped from the packet and that's no big deal. But if too many get dropped, things start to look strange visually and this is undesirable. </p>