{"action":"created","issue":{"url":"https://api.github.com/repos/sksamuel/avro4s/issues/199","repository_url":"https://api.github.com/repos/sksamuel/avro4s","labels_url":"https://api.github.com/repos/sksamuel/avro4s/issues/199/labels{/name}","comments_url":"https://api.github.com/repos/sksamuel/avro4s/issues/199/comments","events_url":"https://api.github.com/repos/sksamuel/avro4s/issues/199/events","html_url":"https://github.com/sksamuel/avro4s/issues/199","id":324941453,"node_id":"MDU6SXNzdWUzMjQ5NDE0NTM=","number":199,"title":"10x slower than java serialization if avro4s is used as per the Readme.md","user":{"login":"kostaskougios","id":1302046,"node_id":"MDQ6VXNlcjEzMDIwNDY=","avatar_url":"https://avatars3.githubusercontent.com/u/1302046?v=4","gravatar_id":"","url":"https://api.github.com/users/kostaskougios","html_url":"https://github.com/kostaskougios","followers_url":"https://api.github.com/users/kostaskougios/followers","following_url":"https://api.github.com/users/kostaskougios/following{/other_user}","gists_url":"https://api.github.com/users/kostaskougios/gists{/gist_id}","starred_url":"https://api.github.com/users/kostaskougios/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kostaskougios/subscriptions","organizations_url":"https://api.github.com/users/kostaskougios/orgs","repos_url":"https://api.github.com/users/kostaskougios/repos","events_url":"https://api.github.com/users/kostaskougios/events{/privacy}","received_events_url":"https://api.github.com/users/kostaskougios/received_events","type":"User","site_admin":false},"labels":[{"id":261189409,"node_id":"MDU6TGFiZWwyNjExODk0MDk=","url":"https://api.github.com/repos/sksamuel/avro4s/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/sksamuel/avro4s/milestones/2","html_url":"https://github.com/sksamuel/avro4s/milestone/2","labels_url":"https://api.github.com/repos/sksamuel/avro4s/milestones/2/labels","id":3573237,"node_id":"MDk6TWlsZXN0b25lMzU3MzIzNw==","number":2,"title":"2.0","description":null,"creator":{"login":"sksamuel","id":743706,"node_id":"MDQ6VXNlcjc0MzcwNg==","avatar_url":"https://avatars3.githubusercontent.com/u/743706?v=4","gravatar_id":"","url":"https://api.github.com/users/sksamuel","html_url":"https://github.com/sksamuel","followers_url":"https://api.github.com/users/sksamuel/followers","following_url":"https://api.github.com/users/sksamuel/following{/other_user}","gists_url":"https://api.github.com/users/sksamuel/gists{/gist_id}","starred_url":"https://api.github.com/users/sksamuel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sksamuel/subscriptions","organizations_url":"https://api.github.com/users/sksamuel/orgs","repos_url":"https://api.github.com/users/sksamuel/repos","events_url":"https://api.github.com/users/sksamuel/events{/privacy}","received_events_url":"https://api.github.com/users/sksamuel/received_events","type":"User","site_admin":false},"open_issues":10,"closed_issues":0,"state":"open","created_at":"2018-08-15T15:12:40Z","updated_at":"2018-08-16T00:56:11Z","due_on":null,"closed_at":null},"comments":9,"created_at":"2018-05-21T14:40:52Z","updated_at":"2018-08-16T00:56:22Z","closed_at":null,"author_association":"NONE","body":"First of all I want to say that this is a very useful library, thanks!\r\n\r\nI've noticed an easy performance trap if the library is used as per the examples. Say I create this utility function:\r\n\r\nobject AvroSerialization\r\n{\r\n\tdef serializeSingleBinary[T: SchemaFor : ToRecord](t: T): Array[Byte] = {\r\n\t\tval bos = new ByteArrayOutputStream\r\n\t\tval bos = new ByteArrayOutputStream(512)\r\n\t\tval aos = AvroOutputStream.binary[T](bos)\r\n\t\ttry aos.write(t) finally aos.close()\r\n\t\tbos.close()\r\n\t\tbos.toByteArray\r\n\t}\r\n}\r\n\r\nNow if I repeatedly call this 100000 and compare it with the equivalent calls in java serialization, avro4s is 10x slower (900 ms for java serialization, 9 secs for avro4s). My guess is that the implicit instances SchemaFor and ToRecord are created over and over again for each call.\r\n\r\nThis code too suffers from the same slowdown:\r\n\r\n\tfor (i <- 1 to 100000) {\r\n\t\tval bos = new ByteArrayOutputStream(512)\r\n\t\tval aos = AvroOutputStream.binary[Page](bos)\r\n\t\ttry aos.write(pg) finally aos.close()\r\n\t\tbos.close()\r\n\t\tbos.toByteArray\r\n\t}\r\n\r\nIf I change the above code to java serialization, it is 10x faster. My guess again is that AvroOutputStream.binary[Page](bos)(... this creates a new instance of implicits for each call).\r\n\r\nI believe most people will use it just like the code above which means their code will be quite slower than the equivalent java serialization.\r\n\r\nIs there a fix for the above? My work around is to create the class below, which means the implicit instances will be passed on once only by the compiler:\r\n\r\npackage com.aktit.avro\r\n\r\nimport java.io.{ByteArrayInputStream, ByteArrayOutputStream}\r\nimport com.sksamuel.avro4s._\r\n\r\nclass AvroSerializer[T: SchemaFor : ToRecord : FromRecord]\r\n{\r\n\tdef serializeSingleBinary(t: T): Array[Byte] = {\r\n\t\tval bos = new ByteArrayOutputStream(512)\r\n\t\tval aos = AvroOutputStream.binary[T](bos)\r\n\t\ttry aos.write(t) finally aos.close()\r\n\t\tbos.close()\r\n\t\tbos.toByteArray\r\n\t}\r\n\r\n\tdef deserializeSingleBinary(a: Array[Byte]): T = {\r\n\t\tval ais = AvroInputStream.binary(new ByteArrayInputStream(a))\r\n\t\ttry ais.iterator.next finally ais.close()\r\n\t}\r\n}\r\n\r\nThen use the same instance always: \r\nobject DTOSerializers\r\n{\r\n\t// One instance is the more performance efficient due to the reuse of avro4s implicit instances.\r\n\tval pageSerializer = new AvroSerializer[Page]\r\n}\r\n\r\nNow avro4s is 2x faster than the java serializer.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/sksamuel/avro4s/issues/comments/413387228","html_url":"https://github.com/sksamuel/avro4s/issues/199#issuecomment-413387228","issue_url":"https://api.github.com/repos/sksamuel/avro4s/issues/199","id":413387228,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzM4NzIyOA==","user":{"login":"sksamuel","id":743706,"node_id":"MDQ6VXNlcjc0MzcwNg==","avatar_url":"https://avatars3.githubusercontent.com/u/743706?v=4","gravatar_id":"","url":"https://api.github.com/users/sksamuel","html_url":"https://github.com/sksamuel","followers_url":"https://api.github.com/users/sksamuel/followers","following_url":"https://api.github.com/users/sksamuel/following{/other_user}","gists_url":"https://api.github.com/users/sksamuel/gists{/gist_id}","starred_url":"https://api.github.com/users/sksamuel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sksamuel/subscriptions","organizations_url":"https://api.github.com/users/sksamuel/orgs","repos_url":"https://api.github.com/users/sksamuel/repos","events_url":"https://api.github.com/users/sksamuel/events{/privacy}","received_events_url":"https://api.github.com/users/sksamuel/received_events","type":"User","site_admin":false},"created_at":"2018-08-16T00:56:22Z","updated_at":"2018-08-16T00:56:22Z","author_association":"OWNER","body":"For 2.0 lets change the signature to accept a schema."}}
{"actor":{"display_login":"sksamuel"}}