{"action":"created","issue":{"url":"https://api.github.com/repos/nats-io/go-nats-streaming/issues/129","repository_url":"https://api.github.com/repos/nats-io/go-nats-streaming","labels_url":"https://api.github.com/repos/nats-io/go-nats-streaming/issues/129/labels{/name}","comments_url":"https://api.github.com/repos/nats-io/go-nats-streaming/issues/129/comments","events_url":"https://api.github.com/repos/nats-io/go-nats-streaming/issues/129/events","html_url":"https://github.com/nats-io/go-nats-streaming/issues/129","id":197996365,"number":129,"title":"Data race on publish?","user":{"login":"olivere","id":134848,"avatar_url":"https://avatars.githubusercontent.com/u/134848?v=3","gravatar_id":"","url":"https://api.github.com/users/olivere","html_url":"https://github.com/olivere","followers_url":"https://api.github.com/users/olivere/followers","following_url":"https://api.github.com/users/olivere/following{/other_user}","gists_url":"https://api.github.com/users/olivere/gists{/gist_id}","starred_url":"https://api.github.com/users/olivere/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olivere/subscriptions","organizations_url":"https://api.github.com/users/olivere/orgs","repos_url":"https://api.github.com/users/olivere/repos","events_url":"https://api.github.com/users/olivere/events{/privacy}","received_events_url":"https://api.github.com/users/olivere/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-12-29T11:56:06Z","updated_at":"2016-12-29T17:49:06Z","closed_at":"2016-12-29T17:12:28Z","body":"We're evaluating NATS Streaming Server for the PubSub system in a microservices architecture. We currently use it only for simple \"fire-and-forget\" messaging such as progress notifications.\r\n\r\nWhile using that, we found that the number of goroutines in go-nats-streaming seems to be ever increasing. This is what we get after running the web server part for a while. It seems that the goroutines are stuck in [`stan.go:402`](https://github.com/nats-io/go-nats-streaming/blob/master/stan.go#L402).\r\n\r\n```\r\ngoroutine profile: total 168\r\n69 @ 0x3f20d 0x3f2fe 0x13b08 0x1380d 0x58d855 0x72111\r\n#\t0x58d854\tgithub.com/meplato/cloud/vendor/github.com/nats-io/go-nats-streaming.(*conn).publishAsync.func1+0x144\t/Users/oliver/gocode/src/github.com/meplato/cloud/vendor/github.com/nats-io/go-nats-streaming/stan.go:402\r\n\r\n8 @ 0x3f20d 0x50133 0x4eddc 0x25167e 0x72111\r\n#\t0x25167d\tgithub.com/meplato/cloud/vendor/google.golang.org/grpc.newClientStream.func2+0x54d\t/Users/oliver/gocode/src/github.com/meplato/cloud/vendor/google.golang.org/grpc/stream.go:204\r\n```\r\n\r\nAfter a while, this number exceeds the number of goroutines that the race detector considers to be a problem (`race: limit on 8192 simultaneously alive goroutines is exceeded, dying`).\r\n\r\n```\r\ngoroutine profile: total 1141\r\n1042 @ 0x3f20d 0x3f2fe 0x13b08 0x1380d 0x58d855 0x72111\r\n#\t0x58d854\tgithub.com/meplato/cloud/vendor/github.com/nats-io/go-nats-streaming.(*conn).publishAsync.func1+0x144\t/Users/oliver/gocode/src/github.com/meplato/cloud/vendor/github.com/nats-io/go-nats-streaming/stan.go:402\r\n\r\n8 @ 0x3f20d 0x50133 0x4eddc 0x25167e 0x72111\r\n#\t0x25167d\tgithub.com/meplato/cloud/vendor/google.golang.org/grpc.newClientStream.func2+0x54d\t/Users/oliver/gocode/src/github.com/meplato/cloud/vendor/google.golang.org/grpc/stream.go:204\r\n```\r\n\r\nHere's how we publish messages:\r\n\r\n```\r\n// PublishJSON publishes a message serialized as JSON.\r\n// Strings, pointers to strings, and byte arrays get sent as-is,\r\n// all other objects are serialized as JSON.\r\nfunc (b *natsBroker) PublishJSON(ctx context.Context, topic string, msg interface{}) error {\r\n\tswitch msg := msg.(type) {\r\n\tcase string:\r\n\t\treturn b.conn.Publish(topic, []byte(msg))\r\n\tcase *string:\r\n\t\tif msg == nil {\r\n\t\t\treturn b.conn.Publish(topic, nil)\r\n\t\t}\r\n\t\treturn b.conn.Publish(topic, []byte(*msg))\r\n\tcase []byte:\r\n\t\treturn b.conn.Publish(topic, msg)\r\n\t}\r\n\r\n\t// Fallback to JSON encoding\r\n\tdata, err := json.Marshal(msg)\r\n\tif err != nil {\r\n\t\treturn err\r\n\t}\r\n\treturn b.conn.Publish(topic, data)\r\n}\r\n\r\n...\r\n\r\n// Publish progress\r\nsch.broker.PublishJSON(context.Background(), sync.ProgressTopic, sync.ProgressEvent{\r\n\tCurrent:    int(current),\r\n\tTotal:      int(total),\r\n\tETA:        eta.Unix(),\r\n})\r\n```\r\n\r\nWe're using the current version of NATS streaming server from the GitHub repository:\r\n\r\n```\r\n[52172] 2016/12/29 12:30:02.190442 [INF] Starting nats-streaming-server[cloud-nats] version 0.3.7\r\n[52172] 2016/12/29 12:30:02.190681 [INF] Starting nats-server version 0.9.6\r\n[52172] 2016/12/29 12:30:02.190712 [INF] Listening for client connections on 0.0.0.0:4222\r\n[52172] 2016/12/29 12:30:02.190806 [INF] Server is ready\r\n[52172] 2016/12/29 12:30:02.469346 [INF] STAN: Message store is MEMORY\r\n[52172] 2016/12/29 12:30:02.469358 [INF] STAN: --------- Store Limits ---------\r\n[52172] 2016/12/29 12:30:02.469362 [INF] STAN: Channels:                  100 *\r\n[52172] 2016/12/29 12:30:02.469369 [INF] STAN: -------- channels limits -------\r\n[52172] 2016/12/29 12:30:02.469372 [INF] STAN:   Subscriptions:          1000 *\r\n[52172] 2016/12/29 12:30:02.469376 [INF] STAN:   Messages     :       1000000 *\r\n[52172] 2016/12/29 12:30:02.469397 [INF] STAN:   Bytes        :     976.56 MB *\r\n[52172] 2016/12/29 12:30:02.469399 [INF] STAN:   Age          :     unlimited *\r\n[52172] 2016/12/29 12:30:02.469400 [INF] STAN: --------------------------------\r\n```\r\n\r\nOf course, this could as well be a problem on our side. Can you give us a hint on how to debug this problem?\r\n"},"comment":{"url":"https://api.github.com/repos/nats-io/go-nats-streaming/issues/comments/269666612","html_url":"https://github.com/nats-io/go-nats-streaming/issues/129#issuecomment-269666612","issue_url":"https://api.github.com/repos/nats-io/go-nats-streaming/issues/129","id":269666612,"user":{"login":"kozlovic","id":13736691,"avatar_url":"https://avatars.githubusercontent.com/u/13736691?v=3","gravatar_id":"","url":"https://api.github.com/users/kozlovic","html_url":"https://github.com/kozlovic","followers_url":"https://api.github.com/users/kozlovic/followers","following_url":"https://api.github.com/users/kozlovic/following{/other_user}","gists_url":"https://api.github.com/users/kozlovic/gists{/gist_id}","starred_url":"https://api.github.com/users/kozlovic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kozlovic/subscriptions","organizations_url":"https://api.github.com/users/kozlovic/orgs","repos_url":"https://api.github.com/users/kozlovic/repos","events_url":"https://api.github.com/users/kozlovic/events{/privacy}","received_events_url":"https://api.github.com/users/kozlovic/received_events","type":"User","site_admin":false},"created_at":"2016-12-29T17:49:06Z","updated_at":"2016-12-29T17:49:06Z","body":"The reason you can't mix and match is that NATS Streaming uses NATS for communication. Streaming server and clients are not directly connected, they go through the NATS server. So subjects space is isolated with some unique ID. For instance, if in Streaming you are publishing on `foo`, the streaming server listens on say: `_STAN.pub.Xqj9tf4MUfadLp6xPp7vRg.>` and the streaming client will publish on `_STAN.pub.Xqj9tf4MUfadLp6xPp7vRg.foo`. Streaming clients send NATS message that are protocol buffers containing some meta data and the actual payload.\r\n\r\nThere is no plan to merge NATS and NATS Streaming."}}
{"actor":{"display_login":"kozlovic"}}