{"action":"created","issue":{"url":"https://api.github.com/repos/dotnet/orleans/issues/4842","repository_url":"https://api.github.com/repos/dotnet/orleans","labels_url":"https://api.github.com/repos/dotnet/orleans/issues/4842/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/orleans/issues/4842/comments","events_url":"https://api.github.com/repos/dotnet/orleans/issues/4842/events","html_url":"https://github.com/dotnet/orleans/issues/4842","id":349679316,"node_id":"MDU6SXNzdWUzNDk2NzkzMTY=","number":4842,"title":"Orleans.Runtime.OrleansException: Not connected to a gateway when reconnecting to a local silo","user":{"login":"skyflyer","id":415535,"node_id":"MDQ6VXNlcjQxNTUzNQ==","avatar_url":"https://avatars3.githubusercontent.com/u/415535?v=4","gravatar_id":"","url":"https://api.github.com/users/skyflyer","html_url":"https://github.com/skyflyer","followers_url":"https://api.github.com/users/skyflyer/followers","following_url":"https://api.github.com/users/skyflyer/following{/other_user}","gists_url":"https://api.github.com/users/skyflyer/gists{/gist_id}","starred_url":"https://api.github.com/users/skyflyer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/skyflyer/subscriptions","organizations_url":"https://api.github.com/users/skyflyer/orgs","repos_url":"https://api.github.com/users/skyflyer/repos","events_url":"https://api.github.com/users/skyflyer/events{/privacy}","received_events_url":"https://api.github.com/users/skyflyer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"benjaminpetit","id":20427417,"node_id":"MDQ6VXNlcjIwNDI3NDE3","avatar_url":"https://avatars0.githubusercontent.com/u/20427417?v=4","gravatar_id":"","url":"https://api.github.com/users/benjaminpetit","html_url":"https://github.com/benjaminpetit","followers_url":"https://api.github.com/users/benjaminpetit/followers","following_url":"https://api.github.com/users/benjaminpetit/following{/other_user}","gists_url":"https://api.github.com/users/benjaminpetit/gists{/gist_id}","starred_url":"https://api.github.com/users/benjaminpetit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benjaminpetit/subscriptions","organizations_url":"https://api.github.com/users/benjaminpetit/orgs","repos_url":"https://api.github.com/users/benjaminpetit/repos","events_url":"https://api.github.com/users/benjaminpetit/events{/privacy}","received_events_url":"https://api.github.com/users/benjaminpetit/received_events","type":"User","site_admin":false},"assignees":[{"login":"benjaminpetit","id":20427417,"node_id":"MDQ6VXNlcjIwNDI3NDE3","avatar_url":"https://avatars0.githubusercontent.com/u/20427417?v=4","gravatar_id":"","url":"https://api.github.com/users/benjaminpetit","html_url":"https://github.com/benjaminpetit","followers_url":"https://api.github.com/users/benjaminpetit/followers","following_url":"https://api.github.com/users/benjaminpetit/following{/other_user}","gists_url":"https://api.github.com/users/benjaminpetit/gists{/gist_id}","starred_url":"https://api.github.com/users/benjaminpetit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benjaminpetit/subscriptions","organizations_url":"https://api.github.com/users/benjaminpetit/orgs","repos_url":"https://api.github.com/users/benjaminpetit/repos","events_url":"https://api.github.com/users/benjaminpetit/events{/privacy}","received_events_url":"https://api.github.com/users/benjaminpetit/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/dotnet/orleans/milestones/17","html_url":"https://github.com/dotnet/orleans/milestone/17","labels_url":"https://api.github.com/repos/dotnet/orleans/milestones/17/labels","id":2111063,"node_id":"MDk6TWlsZXN0b25lMjExMTA2Mw==","number":17,"title":"Triage","description":"","creator":{"login":"sergeybykov","id":8248806,"node_id":"MDQ6VXNlcjgyNDg4MDY=","avatar_url":"https://avatars1.githubusercontent.com/u/8248806?v=4","gravatar_id":"","url":"https://api.github.com/users/sergeybykov","html_url":"https://github.com/sergeybykov","followers_url":"https://api.github.com/users/sergeybykov/followers","following_url":"https://api.github.com/users/sergeybykov/following{/other_user}","gists_url":"https://api.github.com/users/sergeybykov/gists{/gist_id}","starred_url":"https://api.github.com/users/sergeybykov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sergeybykov/subscriptions","organizations_url":"https://api.github.com/users/sergeybykov/orgs","repos_url":"https://api.github.com/users/sergeybykov/repos","events_url":"https://api.github.com/users/sergeybykov/events{/privacy}","received_events_url":"https://api.github.com/users/sergeybykov/received_events","type":"User","site_admin":false},"open_issues":68,"closed_issues":490,"state":"open","created_at":"2016-11-03T00:53:47Z","updated_at":"2018-08-16T20:52:07Z","due_on":null,"closed_at":null},"comments":0,"created_at":"2018-08-10T22:53:54Z","updated_at":"2018-08-16T20:58:09Z","closed_at":null,"author_association":"NONE","body":"I'm just playing with Orleans and one of the basics and I tried the 2.0 sample app.\r\n\r\nI started the client before the silo (intentionally), just to see if the retry policy, outlined there would work. The client connected, but then immediately failed with the error I'm pasting below.\r\n\r\nSteps to reproduce:\r\n\r\n1. Use the [Hello world sample](https://github.com/dotnet/orleans/tree/master/Samples/2.0/HelloWorld)\r\n1. Start the client\r\n1. Start the silo (after the client has been started)\r\n1. Observe the log output to see the exception\r\n\r\nExpected behaviour:\r\nI would expect the client to successfully reconnect and perform operations normally. Is this something that is supposed to work or am I expecting too much? Let me know if you need more details on the setup.\r\n\r\nLog output:\r\n\r\n<details>\r\n  <summary>Expand log content</summary>\r\n\r\n```\r\n$ dotnet run --project src/OrleansClient/OrleansClient.csproj\r\ninfo: Orleans.OutsideRuntimeClient[100313]\r\n      ---------- Initializing OutsideRuntimeClient on Petarda.local at 172.16.1.50 Client Id = *cli/c3e0439a ----------\r\ninfo: Orleans.OutsideRuntimeClient[100314]\r\n      ---------- Starting OutsideRuntimeClient with runtime Version='2.0.3. Commit Hash: c5432c1809a37f12142fb1c489fb0a4f77ed3bda (Release).' in AppDomain=<AppDomain.Id=1, AppDomain.FriendlyName=OrleansClient>\r\ninfo: Orleans.Messaging.GatewayManager[101309]\r\n      Found 1 knownGateways from Gateway listProvider [gwy.tcp://127.0.0.1:30000/0]\r\nwarn: Messaging.GatewayConnection/GatewayClientSender_gwy.tcp://127.0.0.1:30000/0[100178]\r\n      Unable to connect to gateway at address gwy.tcp://127.0.0.1:30000/0 on trial 0 (Exception: Could not connect to 127.0.0.1:30000: ConnectionRefused)\r\nwarn: Messaging.GatewayConnection/GatewayClientSender_gwy.tcp://127.0.0.1:30000/0[100913]\r\n      Marking gateway at address gwy.tcp://127.0.0.1:30000/0 as Disconnected\r\ninfo: Orleans.Messaging.GatewayManager[101309]\r\n      Refreshed the live Gateway list. Found 1 gateways from Gateway listProvider: [gwy.tcp://127.0.0.1:30000/0]. Picked only known live out of them. Now has 1 live Gateways: [gwy.tcp://127.0.0.1:30000/0]. Previous refresh time was = 08/10/2018 22:41:30\r\nwarn: Messaging.GatewayConnection/GatewayClientSender_gwy.tcp://127.0.0.1:30000/0[100178]\r\n      Unable to connect to gateway at address gwy.tcp://127.0.0.1:30000/0 on trial 1 (Exception: Could not connect to 127.0.0.1:30000: ConnectionRefused)\r\nwarn: Messaging.GatewayConnection/GatewayClientSender_gwy.tcp://127.0.0.1:30000/0[100913]\r\n      Marking gateway at address gwy.tcp://127.0.0.1:30000/0 as Disconnected\r\ninfo: Orleans.Messaging.GatewayManager[101309]\r\n      Refreshed the live Gateway list. Found 1 gateways from Gateway listProvider: [gwy.tcp://127.0.0.1:30000/0]. Picked only known live out of them. Now has 1 live Gateways: [gwy.tcp://127.0.0.1:30000/0]. Previous refresh time was = 08/10/2018 22:41:30\r\nwarn: Messaging.GatewayConnection/GatewayClientSender_gwy.tcp://127.0.0.1:30000/0[100912]\r\n      Marking gateway at address gwy.tcp://127.0.0.1:30000/0 as Dead in my client local gateway list.\r\nwarn: Orleans.Runtime.CallbackData[100157]\r\n      The target silo became unavailable for message: Request *cli/c3e0439a@28c5cba1->S127.0.0.1:30000:0*stg/17/00000011@S00000011 #2: . Target History is: <S127.0.0.1:30000:0:*stg/17/00000011:@S00000011>. See https://aka.ms/orleans-troubleshooting for troubleshooting help. About to break its promise.\r\nwarn: Orleans.OutsideRuntimeClient[100011]\r\n      No callback for response message: Unrecoverable Rejection (info: Target silo S127.0.0.1:30000:0 is unavailable) Response S127.0.0.1:30000:0*stg/17/00000011@S00000011->*cli/c3e0439a@28c5cba1 #2:\r\nCluster client attempt 1 of 5 failed to connect to cluster.  Exception: Orleans.Runtime.SiloUnavailableException: The target silo became unavailable for message: Request *cli/c3e0439a@28c5cba1->S127.0.0.1:30000:0*stg/17/00000011@S00000011 #2: . Target History is: <S127.0.0.1:30000:0:*stg/17/00000011:@S00000011>. See https://aka.ms/orleans-troubleshooting for troubleshooting help.\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>b__58_2()\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>g__ExecuteWithRetries58_3(Func`1 task, Func`2 shouldRetry)\r\nRetrying ...\r\nwarn: Orleans.Messaging.GatewayManager[101321]\r\n      All gateways have previously been marked as dead. Clearing the list of dead gateways to expedite reconnection.\r\ninfo: Orleans.Messaging.GatewayManager[101309]\r\n      Refreshed the live Gateway list. Found 1 gateways from Gateway listProvider: [gwy.tcp://127.0.0.1:30000/0]. Picked only known live out of them. Now has 1 live Gateways: [gwy.tcp://127.0.0.1:30000/0]. Previous refresh time was = 08/10/2018 22:41:30\r\nCluster client failed to connect to cluster with unexpected error.  Exception: Orleans.Runtime.OrleansException: Not connected to a gateway\r\n   at Orleans.Messaging.ClientMessageCenter.GetLiveGatewaySiloAddress()\r\n   at Orleans.Messaging.ClientMessageCenter.GetGrainTypeResolver(IInternalGrainFactory grainFactory)\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>b__58_2()\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>g__ExecuteWithRetries58_3(Func`1 task, Func`2 shouldRetry)\r\nException: Orleans.Runtime.OrleansException: Not connected to a gateway\r\n   at Orleans.Messaging.ClientMessageCenter.GetLiveGatewaySiloAddress()\r\n   at Orleans.Messaging.ClientMessageCenter.GetGrainTypeResolver(IInternalGrainFactory grainFactory)\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>b__58_2()\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>g__ExecuteWithRetries58_3(Func`1 task, Func`2 shouldRetry)\r\n   at Orleans.OutsideRuntimeClient.<StartInternal>g__ExecuteWithRetries58_3(Func`1 task, Func`2 shouldRetry)\r\n   at Orleans.OutsideRuntimeClient.StartInternal(Func`2 retryFilter)\r\n   at Orleans.OutsideRuntimeClient.Start(Func`2 retryFilter)\r\n   at Orleans.ClusterClient.Connect(Func`2 retryFilter)\r\n   at OrleansClient.Program.StartClientWithRetries() in /Users/miha/work/dev/Orleans-Docker/src/OrleansClient/Program.cs:line 59\r\n   at OrleansClient.Program.RunMainAsync() in /Users/miha/work/dev/Orleans-Docker/src/OrleansClient/Program.cs:line 29\r\ninfo: Orleans.Messaging.GatewayManager[101309]\r\n      Refreshed the live Gateway list. Found 1 gateways from Gateway listProvider: [gwy.tcp://127.0.0.1:30000/0]. Picked only known live out of them. Now has 1 live Gateways: [gwy.tcp://127.0.0.1:30000/0]. Previous refresh time was = 08/10/2018 22:41:40\r\n```\r\n</details>"},"comment":{"url":"https://api.github.com/repos/dotnet/orleans/issues/comments/413682516","html_url":"https://github.com/dotnet/orleans/issues/4842#issuecomment-413682516","issue_url":"https://api.github.com/repos/dotnet/orleans/issues/4842","id":413682516,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzY4MjUxNg==","user":{"login":"benjaminpetit","id":20427417,"node_id":"MDQ6VXNlcjIwNDI3NDE3","avatar_url":"https://avatars0.githubusercontent.com/u/20427417?v=4","gravatar_id":"","url":"https://api.github.com/users/benjaminpetit","html_url":"https://github.com/benjaminpetit","followers_url":"https://api.github.com/users/benjaminpetit/followers","following_url":"https://api.github.com/users/benjaminpetit/following{/other_user}","gists_url":"https://api.github.com/users/benjaminpetit/gists{/gist_id}","starred_url":"https://api.github.com/users/benjaminpetit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benjaminpetit/subscriptions","organizations_url":"https://api.github.com/users/benjaminpetit/orgs","repos_url":"https://api.github.com/users/benjaminpetit/repos","events_url":"https://api.github.com/users/benjaminpetit/events{/privacy}","received_events_url":"https://api.github.com/users/benjaminpetit/received_events","type":"User","site_admin":false},"created_at":"2018-08-16T20:58:09Z","updated_at":"2018-08-16T20:58:09Z","author_association":"CONTRIBUTOR","body":"I think what you are experiencing right now is only because you are using a static gateway provider. If you use multiple silos with something like the azure storage gateway/membership providers, the retry mechanism will works as intended."}}
{"actor":{"display_login":"benjaminpetit"}}