{"action":"created","issue":{"url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/188","repository_url":"https://api.github.com/repos/zendesk/ruby-kafka","labels_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/188/labels{/name}","comments_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/188/comments","events_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/188/events","html_url":"https://github.com/zendesk/ruby-kafka/issues/188","id":155137019,"number":188,"title":"\"nil can't be coerced into Fixnum\" exception after producer.deliver_messages","user":{"login":"nickmarrone","id":1848528,"avatar_url":"https://avatars.githubusercontent.com/u/1848528?v=3","gravatar_id":"","url":"https://api.github.com/users/nickmarrone","html_url":"https://github.com/nickmarrone","followers_url":"https://api.github.com/users/nickmarrone/followers","following_url":"https://api.github.com/users/nickmarrone/following{/other_user}","gists_url":"https://api.github.com/users/nickmarrone/gists{/gist_id}","starred_url":"https://api.github.com/users/nickmarrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nickmarrone/subscriptions","organizations_url":"https://api.github.com/users/nickmarrone/orgs","repos_url":"https://api.github.com/users/nickmarrone/repos","events_url":"https://api.github.com/users/nickmarrone/events{/privacy}","received_events_url":"https://api.github.com/users/nickmarrone/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/zendesk/ruby-kafka/labels/bug","name":"bug","color":"fc2929"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2016-05-16T22:29:40Z","updated_at":"2016-05-17T22:53:38Z","closed_at":"2016-05-17T03:21:13Z","body":"Hello,\r\n\r\nIn my code, I am using the synchronous producer to produce a message, and then deliver it to the kafka queue. \r\n\r\n```\r\nserialized_protobuf = protobuf_message.serialize_to_string\r\nproducer.produce(serialized_protobuf, topic: 'my_topic')\r\nproducer.deliver_messages\r\n```\r\n\r\nAlthough many messages succeed, some of the time `deliver_messages` will raise an error `nil can't be coerced into Fixnum`. The stack trace is\r\n\r\n```\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:30:in `+'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:30:in `concat'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/produce_operation.rb:64:in `block in send_buffered_messages'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:44:in `block (2 levels) in each'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:43:in `each'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:43:in `block in each'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:42:in `each'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/message_buffer.rb:42:in `each'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/produce_operation.rb:57:in `send_buffered_messages'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/produce_operation.rb:45:in `block in execute'\r\n/vendor/bundle/ruby/2.1.0/gems/activesupport-3.2.12/lib/active_support/notifications.rb:125:in `instrument'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/instrumenter.rb:19:in `instrument'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/produce_operation.rb:39:in `execute'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/producer.rb:297:in `block in deliver_messages_with_retries'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/producer.rb:289:in `loop'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/producer.rb:289:in `deliver_messages_with_retries'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/producer.rb:239:in `block in deliver_messages'\r\n/vendor/bundle/ruby/2.1.0/gems/activesupport-3.2.12/lib/active_support/notifications.rb:125:in `instrument'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/instrumenter.rb:19:in `instrument'\r\n/vendor/bundle/ruby/2.1.0/gems/ruby-kafka-0.3.7/lib/kafka/producer.rb:232:in `deliver_messages'\r\n```\r\n\r\nThe error appears to occur in when an empty array of messages is passed to MessageBuffer#concat and it attempts to add nil bytes to @bytesize. Although fixing the symptom appears trivial, I am at a loss to explain why there are no messages to send since my code calls the produce method immediately before deliver_messages. \r\n\r\nThanks\r\n\r\nEDIT: This is using v0.3.7 of the gem. "},"comment":{"url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/comments/219878099","html_url":"https://github.com/zendesk/ruby-kafka/issues/188#issuecomment-219878099","issue_url":"https://api.github.com/repos/zendesk/ruby-kafka/issues/188","id":219878099,"user":{"login":"nickmarrone","id":1848528,"avatar_url":"https://avatars.githubusercontent.com/u/1848528?v=3","gravatar_id":"","url":"https://api.github.com/users/nickmarrone","html_url":"https://github.com/nickmarrone","followers_url":"https://api.github.com/users/nickmarrone/followers","following_url":"https://api.github.com/users/nickmarrone/following{/other_user}","gists_url":"https://api.github.com/users/nickmarrone/gists{/gist_id}","starred_url":"https://api.github.com/users/nickmarrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nickmarrone/subscriptions","organizations_url":"https://api.github.com/users/nickmarrone/orgs","repos_url":"https://api.github.com/users/nickmarrone/repos","events_url":"https://api.github.com/users/nickmarrone/events{/privacy}","received_events_url":"https://api.github.com/users/nickmarrone/received_events","type":"User","site_admin":false},"created_at":"2016-05-17T22:53:37Z","updated_at":"2016-05-17T22:53:37Z","body":"Ah, I see. \r\n\r\nThe async producer will not work for me because I need to be able to ACK or fail each message as they go through my data processing pipeline. FYI, our message size, even compressed, can be several hundred KB. \r\n\r\nLooking in more closely, I see that the async producer uses a producer under the hood. Each producer is creating its own queue and buffer. \r\n\r\nSimilarly, the client is creating a new connection_builder, broker_pool, and cluster. \r\n\r\nSo, I wanted to confirm -- you are saying that the producer is not thread safe. But, is there any reason it would not be thread safe if I create a separate instance of the client and the producer for each thread? "}}