{"action":"created","issue":{"url":"https://api.github.com/repos/openzipkin/zipkin-java/issues/70","repository_url":"https://api.github.com/repos/openzipkin/zipkin-java","labels_url":"https://api.github.com/repos/openzipkin/zipkin-java/issues/70/labels{/name}","comments_url":"https://api.github.com/repos/openzipkin/zipkin-java/issues/70/comments","events_url":"https://api.github.com/repos/openzipkin/zipkin-java/issues/70/events","html_url":"https://github.com/openzipkin/zipkin-java/issues/70","id":127347655,"number":70,"title":"Kafka Span Transport","user":{"login":"adriancole","id":64215,"avatar_url":"https://avatars.githubusercontent.com/u/64215?v=3","gravatar_id":"","url":"https://api.github.com/users/adriancole","html_url":"https://github.com/adriancole","followers_url":"https://api.github.com/users/adriancole/followers","following_url":"https://api.github.com/users/adriancole/following{/other_user}","gists_url":"https://api.github.com/users/adriancole/gists{/gist_id}","starred_url":"https://api.github.com/users/adriancole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adriancole/subscriptions","organizations_url":"https://api.github.com/users/adriancole/orgs","repos_url":"https://api.github.com/users/adriancole/repos","events_url":"https://api.github.com/users/adriancole/events{/privacy}","received_events_url":"https://api.github.com/users/adriancole/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-01-19T02:01:56Z","updated_at":"2016-03-10T10:06:47Z","closed_at":null,"body":"Tracers in python, ruby, and java support a kafka transport. We need to implement this feature such that zipkin-java-servers can accept spans in the same way.\r\n\r\nRequested by @ewhauser\r\n\r\n> I'm looking at what exists in zipkin today because I'm looking to deploy a couple large installations of zipkin. We mostly work in Java, so zipkin-java-server makes sense for us. We also want to use HTTP as the transport mechanism because we will have some cross data center traffic.\r\nI was looking into adding support for writing spans to Kafka, but there's a couple issues I need to figure out. 1) we probably need a way of differentiating between read and write span stores since Kafka would be the writer and another store would provide the reads 2) the interface for SpanStore currently takes an Iterator<Span> and I really want the raw byte[] so I don't have to serialize again before writing to Kafka"},"comment":{"url":"https://api.github.com/repos/openzipkin/zipkin-java/issues/comments/194769260","html_url":"https://github.com/openzipkin/zipkin-java/issues/70#issuecomment-194769260","issue_url":"https://api.github.com/repos/openzipkin/zipkin-java/issues/70","id":194769260,"user":{"login":"adriancole","id":64215,"avatar_url":"https://avatars.githubusercontent.com/u/64215?v=3","gravatar_id":"","url":"https://api.github.com/users/adriancole","html_url":"https://github.com/adriancole","followers_url":"https://api.github.com/users/adriancole/followers","following_url":"https://api.github.com/users/adriancole/following{/other_user}","gists_url":"https://api.github.com/users/adriancole/gists{/gist_id}","starred_url":"https://api.github.com/users/adriancole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adriancole/subscriptions","organizations_url":"https://api.github.com/users/adriancole/orgs","repos_url":"https://api.github.com/users/adriancole/repos","events_url":"https://api.github.com/users/adriancole/events{/privacy}","received_events_url":"https://api.github.com/users/adriancole/received_events","type":"User","site_admin":false},"created_at":"2016-03-10T10:06:47Z","updated_at":"2016-03-10T10:06:47Z","body":"@ewhauser another late thought.. I can see what you mean about the transport needing to be defined. Here's a client-side in htrace https://github.com/apache/incubator-htrace/blob/master/htrace-zipkin/src/main/java/org/apache/htrace/Transport.java\r\n\r\nThe overall concern about re-serialization is an interesting one, particularly as depending on types used, this can be tricky. For example, the only current span store that stores serialized thrifts is cassandra, yet they still need to be unpacked to build indexes.\r\n\r\nI'm struggling with Iterator, too, but for different reasons. It is here to support lazy sampling, but in practice the list counts are small. I'll open another issue on this."}}