Take the 2-minute tour ×

Which serialization method/function/BIF etc has best performance in Erlang? My case is:

  1. I have memcache with data i need to retrive every request.
  2. Retrevied data is binary term (when i do binary_to_term a have a List of small data)
  3. I need to do some logic on that data so i need to unserialize them.
  4. On the other system i'm setting that data in that memcache so i can change everything.

Please help me choosing the method of serialization. Only performance is my goal. Thanks!

share|improve this question
Can you clarify your architecture? Are the memcache instances in point 1 & 4 the same? Is the data used by non erlang applications? Is binary_to_term too slow? If so, what are you current benchmark numbers and what level do you need to reach? Did you try using mnesia or ets? Why not keep processes alive on the node that manage the data you now put in memcache. –  Ward Bekker Oct 29 '11 at 21:27
1. Memcache instance is the same one. 2. data is not used by erlang applications but it would be nice if they can in the future. –  Tereska Oct 30 '11 at 9:12
My goal is fast as possible deserialize data for client. serialization could be slow, deserialization must be fast. –  Tereska Oct 30 '11 at 9:15
add comment

2 Answers

up vote 4 down vote accepted

binary_to_term is your friend. This is exactly what you need.

If you want to utilize this data buy other not-erlang systems here is a list of libraries to help you: http://bert-rpc.org/

share|improve this answer
So there is no better thing in terms of performance than binary_to_term? if that is true than i am set ;) –  Tereska Oct 30 '11 at 12:23
now im doing something like this: 1. is data in memcache -> get -> binary_to_term 2. if not generate content -> memcache -> set -> term_to_binary –  Tereska Oct 30 '11 at 12:57
Tereshka we use very similar approach to store data in riak (There are almost no difference between riak and memcache from application point of view). –  Max B Oct 30 '11 at 15:43
and you are using binary_to_term BIF's for data desiged only for Erlang apps and BERT for others? right? –  Tereska Oct 30 '11 at 17:32
Yes. When there is a need to send binary-data over character sensible channels we also use base64:decode/1 and base64:encode/1 functions. –  Max B Oct 30 '11 at 18:46
show 1 more comment

An alternative to the Binary Erlang format might be Google protocol buffers. It's a "a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more". The Erlang implementation that is used by Basho's Riak product is available on Github.

share|improve this answer
Hi WardB, at the time I tried protobuf (about 4 month ago) its implementation didn't properly support required types. –  Max B Oct 31 '11 at 6:25
Hi MinimeDJ, good to know as I want to use protocol buffers on a project very soon. Thx –  Ward Bekker Oct 31 '11 at 7:13
I have tried everything available (protobuf, Thrift and few other custom solutions) and I believe that binary_to_term is a best thing to do this job. All other solutions have their features you really don't need while binary_to_term just works. Of course it is not SO flexible like others, but it has no feature-bugs and it is very fast. –  Max B Oct 31 '11 at 7:19
add comment

Your Answer


