{"action":"created","issue":{"url":"https://api.github.com/repos/jaegertracing/jaeger/issues/422","repository_url":"https://api.github.com/repos/jaegertracing/jaeger","labels_url":"https://api.github.com/repos/jaegertracing/jaeger/issues/422/labels{/name}","comments_url":"https://api.github.com/repos/jaegertracing/jaeger/issues/422/comments","events_url":"https://api.github.com/repos/jaegertracing/jaeger/issues/422/events","html_url":"https://github.com/jaegertracing/jaeger/issues/422","id":259947216,"node_id":"MDU6SXNzdWUyNTk5NDcyMTY=","number":422,"title":"How to support plugins","user":{"login":"yurishkuro","id":3523016,"node_id":"MDQ6VXNlcjM1MjMwMTY=","avatar_url":"https://avatars2.githubusercontent.com/u/3523016?v=4","gravatar_id":"","url":"https://api.github.com/users/yurishkuro","html_url":"https://github.com/yurishkuro","followers_url":"https://api.github.com/users/yurishkuro/followers","following_url":"https://api.github.com/users/yurishkuro/following{/other_user}","gists_url":"https://api.github.com/users/yurishkuro/gists{/gist_id}","starred_url":"https://api.github.com/users/yurishkuro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yurishkuro/subscriptions","organizations_url":"https://api.github.com/users/yurishkuro/orgs","repos_url":"https://api.github.com/users/yurishkuro/repos","events_url":"https://api.github.com/users/yurishkuro/events{/privacy}","received_events_url":"https://api.github.com/users/yurishkuro/received_events","type":"User","site_admin":false},"labels":[{"id":358839840,"node_id":"MDU6TGFiZWwzNTg4Mzk4NDA=","url":"https://api.github.com/repos/jaegertracing/jaeger/labels/enhancement","name":"enhancement","color":"84b6eb","default":true},{"id":358839841,"node_id":"MDU6TGFiZWwzNTg4Mzk4NDE=","url":"https://api.github.com/repos/jaegertracing/jaeger/labels/help%20wanted","name":"help wanted","color":"128A0C","default":true},{"id":681654291,"node_id":"MDU6TGFiZWw2ODE2NTQyOTE=","url":"https://api.github.com/repos/jaegertracing/jaeger/labels/roadmap","name":"roadmap","color":"c2e0c6","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":27,"created_at":"2017-09-22T21:51:36Z","updated_at":"2018-11-23T23:06:12Z","closed_at":null,"author_association":"MEMBER","body":"We continue to be asked if we can \"support X as storage backend\" (e.g. #331, #421). Provided that the authors are willing to contribute, maintain, and support such backend implementations, we still have an open question of whether we want to accept those contributions into the main `jaeger` repository. I could be wrong, by my initial reaction is that we should keep them in separate \"contrib\" style repositories, for the following reasons:\r\n  1. having half a dozen implementations is going to bloat the size of the binary, increase compile / testing time\r\n  1. having them in core repo suggests that they are officially supported, same as Cassandra/ES, but we don't have expertise in all those different storage solutions, and cannot be on the hook to support them\r\n\r\nIf we do keep them in the contrib repos, however, we need an approach to allow end users to use those implementations without having to rebuild the backend from source.\r\n\r\nOne such approach is using Go plugins (https://golang.org/pkg/plugin/), for example as done in [Kanali](https://github.com/northwesternmutual/kanali/blob/master/PLUGIN_GUIDE.md). I think it is feasible to package plugins as individual containers and mount them into a shared volume where the main Jaeger binaries can locate and load them.\r\n\r\ncc @pavolloffay @black-adder - any thoughts?\r\n\r\n*Update 12/28/2017*\r\n\r\nAn alternative approach mentioned in the comments below is the *sidecar plugin model* (e.g. https://github.com/hashicorp/go-plugin) where the plugin runs as a separate process and the main binary communicates with it via RPC, e.g. gRPC streams. It's worth noting, however, that this approach is still a special case of in-process plugin model, so we need to start there and answer [the questions below](https://github.com/jaegertracing/jaeger/issues/422#issuecomment-353428289). For each type of plugin we can support a built-in \"sidecar\" variant.\r\n\r\n*Update 8/2/2018*\r\n\r\nPer https://github.com/jaegertracing/jaeger/issues/422#issuecomment-410129850, I think the following is a reasonable, actionable, and realistic plan. If someone **wants to volunteer**, ping me.\r\n\r\n* [ ] define protobuf version of SpanWriter and SpanReader interfaces\r\n* [ ] implement gRPC client and server, where server delegates to the respective storage.SpanReader/Writer interfaces, and client implements them.\r\n* [ ] extend storage factory with two new types, e.g. `h-plugin` (h for harshicorp) and `g-plugin` (plain gRPC). The `h-plugin` should support a cli flag for the name of the plugin executable. The `g-plugin` should support a cli flag for grpc server host:port.\r\n* [ ] implement in-memory storage as `g-plugin` using gRPC client/server defined above\r\n* [ ] implement one of the other storage types (Cassandra or ES) as `h-plugin` as a template.\r\n* [ ] TBD: how to pass configuration to the h-plugins. Because plugins are plain executables, they can use viper just like the main binaries, and the cli flag with the plugin command line might be a long string (or the user can pass params via env vars). We probably should provide a template for main, so that the actual main for a plugin is very short.\r\n* [ ] update documentation with example of building an `h-plugin`.\r\n* [ ] replace Cassandra with in-memory shared service in crossdock integration test.\r\n\r\n*Update 9/4/2018*\r\n\r\nSomeone pointed out that Go's pkg/plugin now supports MacOS and Linux. This removes a significant development hurdle with using the native plugins, and makes it a viable option which is probably simpler to implement than the gRPC-based harshicorp model."},"comment":{"url":"https://api.github.com/repos/jaegertracing/jaeger/issues/comments/441327871","html_url":"https://github.com/jaegertracing/jaeger/issues/422#issuecomment-441327871","issue_url":"https://api.github.com/repos/jaegertracing/jaeger/issues/422","id":441327871,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MTMyNzg3MQ==","user":{"login":"yurishkuro","id":3523016,"node_id":"MDQ6VXNlcjM1MjMwMTY=","avatar_url":"https://avatars2.githubusercontent.com/u/3523016?v=4","gravatar_id":"","url":"https://api.github.com/users/yurishkuro","html_url":"https://github.com/yurishkuro","followers_url":"https://api.github.com/users/yurishkuro/followers","following_url":"https://api.github.com/users/yurishkuro/following{/other_user}","gists_url":"https://api.github.com/users/yurishkuro/gists{/gist_id}","starred_url":"https://api.github.com/users/yurishkuro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yurishkuro/subscriptions","organizations_url":"https://api.github.com/users/yurishkuro/orgs","repos_url":"https://api.github.com/users/yurishkuro/repos","events_url":"https://api.github.com/users/yurishkuro/events{/privacy}","received_events_url":"https://api.github.com/users/yurishkuro/received_events","type":"User","site_admin":false},"created_at":"2018-11-23T23:06:12Z","updated_at":"2018-11-23T23:06:12Z","author_association":"MEMBER","body":"@olivierboucher yes, they are, however, for the proof of concept, I would suggest only focusing on the spanstore reader and writer, as they are the most stable/mature/important. The sampling store is the least important (even master does not implement it for anything but Cassandra, and the adaptive sampling code that uses it is not fully oss yet)."}}
{"actor":{"display_login":"yurishkuro"}}