{"action":"created","issue":{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/issues/181","repository_url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer","labels_url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/issues/181/labels{/name}","comments_url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/issues/181/comments","events_url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/issues/181/events","html_url":"https://github.com/Particular/NServiceBus.SqlServer/issues/181","id":141578742,"number":181,"title":"V3 subscriber receives duplicated messages when subscribing to V2 publisher","user":{"login":"tmasternak","id":1092707,"avatar_url":"https://avatars.githubusercontent.com/u/1092707?v=3","gravatar_id":"","url":"https://api.github.com/users/tmasternak","html_url":"https://github.com/tmasternak","followers_url":"https://api.github.com/users/tmasternak/followers","following_url":"https://api.github.com/users/tmasternak/following{/other_user}","gists_url":"https://api.github.com/users/tmasternak/gists{/gist_id}","starred_url":"https://api.github.com/users/tmasternak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tmasternak/subscriptions","organizations_url":"https://api.github.com/users/tmasternak/orgs","repos_url":"https://api.github.com/users/tmasternak/repos","events_url":"https://api.github.com/users/tmasternak/events{/privacy}","received_events_url":"https://api.github.com/users/tmasternak/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/labels/Consider%20for%20Promotion","name":"Consider for Promotion","color":"bfd4f2"},{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/labels/Impact:%20L","name":"Impact: L","color":"e11d21"},{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/labels/Project:%20V6%20Launch","name":"Project: V6 Launch","color":"ffff00"},{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/labels/State:%20Next%20In%20Line","name":"State: Next In Line","color":"e6e6e6"},{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/labels/Type:%20Bug","name":"Type: Bug","color":"e10c02"}],"state":"open","locked":false,"assignee":{"login":"SzymonPobiega","id":163318,"avatar_url":"https://avatars.githubusercontent.com/u/163318?v=3","gravatar_id":"","url":"https://api.github.com/users/SzymonPobiega","html_url":"https://github.com/SzymonPobiega","followers_url":"https://api.github.com/users/SzymonPobiega/followers","following_url":"https://api.github.com/users/SzymonPobiega/following{/other_user}","gists_url":"https://api.github.com/users/SzymonPobiega/gists{/gist_id}","starred_url":"https://api.github.com/users/SzymonPobiega/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SzymonPobiega/subscriptions","organizations_url":"https://api.github.com/users/SzymonPobiega/orgs","repos_url":"https://api.github.com/users/SzymonPobiega/repos","events_url":"https://api.github.com/users/SzymonPobiega/events{/privacy}","received_events_url":"https://api.github.com/users/SzymonPobiega/received_events","type":"User","site_admin":false},"milestone":null,"comments":31,"created_at":"2016-03-17T13:09:53Z","updated_at":"2016-04-12T19:51:10Z","closed_at":null,"body":"Taskforce: @SzymonPobiega\r\nLet's assume we have Endpoint A and Endpoint B both running V2 of SqlServer. Let's assume further that A is subscribing to events published by B.\r\n\r\nIn situation when Endpoint A gets updated to V3 of SqlServer transport it will start receiving duplicated messages. The root cause of this behavior is duplicate entry in messages subscription storage that will be created at Endpoint B. Before ugrade B will have ` [\"EndpointA\", \"EventTypeX\"]`  entry in subscription storage. After subscribe message from A gets processed a second entry will be stored in form `[\"EndpointA@schemaA\", \"EventTypeX\"]`.\r\n\r\nIt's important to note the problem should be solved also for situation when publisher is scaled-out and instances are updated to V3 not all at once. \r\n\r\nThe reason for this is that:\r\n * In V3 the address sent in subscription messages has changed from `tableName` to `tableName@schema`\r\n * Subscription store holds unique constraint on endpoint address (it does not interpret it in any way) and will treat `EndpointA` and `EndpointA@scheamA` as two different addresses - this btw makes a lot of sense\r\n * V2 endpoint already has a subscription from when Endpoint A was still running on V2\r\n\r\n# Who's affected\r\nAll using SqlServer V2 and publish-subscribe that want to migrate to V3. \r\n\r\n# Impact\r\nBecause this is a blocker for V3 upgrade it is Large.\r\n\r\n# Potential solutions\r\n\r\n## Upgrade the subscription stores\r\n\r\nGuide users to run a SQL script that updates all addresses by appending the `@[dbo]` suffix to them. If the endpoints using these entries are still on V2, they would ignore anything that comes after `@`. The ones upgraded would take advantage of them.\r\n\r\n### Problems\r\n\r\n * Does not handle subscribers with custom schemas. If we upgrade an endpoint that uses a custom schema, after starting as a V3 endpoint he would send a subscribe message for address `MyEndpoint@[customeSchema]`. This entry would get added to the store and the subscriber would get a duplicate.\r\n\r\n## Provide legacy subscription mode in V3\r\n\r\nMake the `@[schema]` suffix an opt-in feature of V3. V3 would default to V2 behavior of not sending the schema name in subscribe messages. Users would be able to use the new mode (which fixes the coupling problems mentioned in #117) only if the V3 endpoint do not subscribe to any V2 endpoints.\r\n\r\n### Problems\r\n\r\n * It is hard to enforce \"no V2 subscriptions\" rule because subscriptions can be added later (after the upgrade) and there is no way we can detect the fact that such a subscription has been added.\r\n * We're taking one of the new features away from majority of users who will have mixed V2 and V3 deployments until they can upgrade everything to V3.\r\n \r\n## Send the schema information in a special header\r\n\r\nBefore dispatching the subscribe message, remove the schema part from the address and move it to a custom header. The V3 publisher which gets that message would be able to read that header and reconstitute the proper `endpoint@[schema]` address before passing it to the subscription infrastructure. The V2 publisher would just ignore the header.\r\n\r\n### Problems\r\n\r\n * The wire format is suboptimal and we are postponing the problem for another release instead of solving it\r\n * The header/address manipulation requires either a pair of behaviors (which we don't want in the transport) or hacks in the receiver and dispatcher. \r\n * If the publisher is scaled out and being upgraded from V2 to V3, while it is upgraded some instances might be still on V2 while the others might be on V3. If a subscribe message is picked up by a V5 endpoint, an entry **without** schema name will be inserted. If another subscribe comes (e.g. from another instance of the subscriber) and gets to a V3 instance, an entry **with** schema name will be inserted. It can be mitigated by migration process that deduplicates entries with and without the schema.\r\n\r\n## Use native pubsub in V3\r\n\r\nImplement a subscription store within the transport in V3 and provide `IManageSubscriptions` interface to the core. When V3 subscribes to an event it would both send a subscribe message (in V2 format, without schema name) and insert an entry (with schema name) to the transport-level subscription store. If the subscriber happens to be in V2, it would use the message. If it happens to be V3, it would use the store. \r\n\r\nWhen starting a V3 endpoint migrates subscription from a NH subscription store to the transport-level store.\r\n\r\n### Problems\r\n\r\nIf the publisher is scaled out and being upgraded from V2 to V3 and a subscribe message comes from a V2 endpoint and is picked up by an instance which is still on V2, the entry will get added to the NH store and will be not visible for the V3 instances until the next-in-line instance is upgraded and performs the migration process. To mitigate this the migration might be triggered either periodically or by publishing.\r\n\r\n## Teach previous versions to understand the schema name\r\n\r\nRelease a patch to NHibernate/SQL Server transport for version 4/1 and 5/2 to understand the schema name in the subscribe message. \r\n\r\n### Problems\r\n\r\n * Nobody knows how complex a task this is. One big source of complexity is usage of `Address` which is told to ignore everything after the `@` sign. \r\n * This would require telling people that in order to upgrade a given endpoint X to V3 they need to upgrade all their other endpoints communicating with X to the latest patch release of NServiceBus 4/5 and SQL Transport 1/2. The issue here is that some of these endpoints might no longer be controlled by the same group who owns X.\r\n \r\n# Plan Of Attack [Draft]\r\n - [ ] Update [ProductionTests](https://github.com/Particular/ProductionTests/issues/61) to newest core and implement reproduction of the error there\r\n/cc: @mauroservienti"},"comment":{"url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/issues/comments/209076745","html_url":"https://github.com/Particular/NServiceBus.SqlServer/issues/181#issuecomment-209076745","issue_url":"https://api.github.com/repos/Particular/NServiceBus.SqlServer/issues/181","id":209076745,"user":{"login":"SzymonPobiega","id":163318,"avatar_url":"https://avatars.githubusercontent.com/u/163318?v=3","gravatar_id":"","url":"https://api.github.com/users/SzymonPobiega","html_url":"https://github.com/SzymonPobiega","followers_url":"https://api.github.com/users/SzymonPobiega/followers","following_url":"https://api.github.com/users/SzymonPobiega/following{/other_user}","gists_url":"https://api.github.com/users/SzymonPobiega/gists{/gist_id}","starred_url":"https://api.github.com/users/SzymonPobiega/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SzymonPobiega/subscriptions","organizations_url":"https://api.github.com/users/SzymonPobiega/orgs","repos_url":"https://api.github.com/users/SzymonPobiega/repos","events_url":"https://api.github.com/users/SzymonPobiega/events{/privacy}","received_events_url":"https://api.github.com/users/SzymonPobiega/received_events","type":"User","site_admin":false},"created_at":"2016-04-12T19:51:10Z","updated_at":"2016-04-12T19:51:10Z","body":"@danielmarbach if I understand this correctly, this is more a matter of taste. I don't blame the squad, they are doing what they are expected to. It is clear that I won't be able to make a decision myself and there will be *a group* of people involved. Because calling that group a *task force* carries a lot of assumptions, constraints etc. I am fine with calling it a group. You are more than welcome to be part of that group."}}