{"action":"created","issue":{"url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/issues/35","repository_url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS","labels_url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/issues/35/labels{/name}","comments_url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/issues/35/comments","events_url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/issues/35/events","html_url":"https://github.com/BiologicalRecordsCentre/NPMS/issues/35","id":285254122,"node_id":"MDU6SXNzdWUyODUyNTQxMjI=","number":35,"title":"Sync job created extra Data Access Policy attributes during live run","user":{"login":"andrewvanbreda","id":2463614,"node_id":"MDQ6VXNlcjI0NjM2MTQ=","avatar_url":"https://avatars2.githubusercontent.com/u/2463614?v=4","gravatar_id":"","url":"https://api.github.com/users/andrewvanbreda","html_url":"https://github.com/andrewvanbreda","followers_url":"https://api.github.com/users/andrewvanbreda/followers","following_url":"https://api.github.com/users/andrewvanbreda/following{/other_user}","gists_url":"https://api.github.com/users/andrewvanbreda/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewvanbreda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewvanbreda/subscriptions","organizations_url":"https://api.github.com/users/andrewvanbreda/orgs","repos_url":"https://api.github.com/users/andrewvanbreda/repos","events_url":"https://api.github.com/users/andrewvanbreda/events{/privacy}","received_events_url":"https://api.github.com/users/andrewvanbreda/received_events","type":"User","site_admin":false},"labels":[{"id":703268432,"node_id":"MDU6TGFiZWw3MDMyNjg0MzI=","url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/labels/Medium","name":"Medium","color":"c5def5","default":false},{"id":572676418,"node_id":"MDU6TGFiZWw1NzI2NzY0MTg=","url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"closed","locked":false,"assignee":{"login":"andrewvanbreda","id":2463614,"node_id":"MDQ6VXNlcjI0NjM2MTQ=","avatar_url":"https://avatars2.githubusercontent.com/u/2463614?v=4","gravatar_id":"","url":"https://api.github.com/users/andrewvanbreda","html_url":"https://github.com/andrewvanbreda","followers_url":"https://api.github.com/users/andrewvanbreda/followers","following_url":"https://api.github.com/users/andrewvanbreda/following{/other_user}","gists_url":"https://api.github.com/users/andrewvanbreda/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewvanbreda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewvanbreda/subscriptions","organizations_url":"https://api.github.com/users/andrewvanbreda/orgs","repos_url":"https://api.github.com/users/andrewvanbreda/repos","events_url":"https://api.github.com/users/andrewvanbreda/events{/privacy}","received_events_url":"https://api.github.com/users/andrewvanbreda/received_events","type":"User","site_admin":false},"assignees":[{"login":"andrewvanbreda","id":2463614,"node_id":"MDQ6VXNlcjI0NjM2MTQ=","avatar_url":"https://avatars2.githubusercontent.com/u/2463614?v=4","gravatar_id":"","url":"https://api.github.com/users/andrewvanbreda","html_url":"https://github.com/andrewvanbreda","followers_url":"https://api.github.com/users/andrewvanbreda/followers","following_url":"https://api.github.com/users/andrewvanbreda/following{/other_user}","gists_url":"https://api.github.com/users/andrewvanbreda/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewvanbreda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewvanbreda/subscriptions","organizations_url":"https://api.github.com/users/andrewvanbreda/orgs","repos_url":"https://api.github.com/users/andrewvanbreda/repos","events_url":"https://api.github.com/users/andrewvanbreda/events{/privacy}","received_events_url":"https://api.github.com/users/andrewvanbreda/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2017-12-31T12:02:27Z","updated_at":"2019-05-21T13:55:45Z","closed_at":"2019-05-21T13:55:45Z","author_association":"COLLABORATOR","body":"When I ran the Address Sync job on NPMS live, lots of extra duplicate person_attribute_values were created for when users had selected the Data Access Policy checkbox (noticed by Jim, typically 3 per person)\r\nThis was not picked up during testing on either my machine or the training site tests.\r\nFurther tests on my machine and the training site have not caused the issue either.\r\nIn fact, a re-run for just 1 affected user on live showed that the sync job was even able to correct the problem when the following steps were taken\r\n- Set Data Access Policy Agreement to be not mandatory\r\n- Switch off the Agreement checkbox for the affected user.\r\n- Run the sync job for that user only (seemed to need a couple of runs to get rid of the duplicates)\r\n- Switch on the Agreement checkbox for affected user again (and make mandatory field again)\r\n- Run sync job and the problem is corrected.\r\nEven further runs does not cause duplication issues.\r\n\r\nSo my guess is that in fact the problem has happened at times when I received a timeout error (the sync job was always know to be slow). My guess is the problems occurred then,  and I simply didn't realise the problem was more that a timeout when I saw it.\r\nAVB: so the question is now how to proceed.\r\nThe duplicates don't seem to be damaging the warehouse, however they have the potential to cause duplicate rows on some reports. Might need manual correction.\r\nThe sync job clearly needs speeding up as it means if we do needs to run it for all users, it currently isn't easily achievable"},"comment":{"url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/issues/comments/494401933","html_url":"https://github.com/BiologicalRecordsCentre/NPMS/issues/35#issuecomment-494401933","issue_url":"https://api.github.com/repos/BiologicalRecordsCentre/NPMS/issues/35","id":494401933,"node_id":"MDEyOklzc3VlQ29tbWVudDQ5NDQwMTkzMw==","user":{"login":"andrewvanbreda","id":2463614,"node_id":"MDQ6VXNlcjI0NjM2MTQ=","avatar_url":"https://avatars2.githubusercontent.com/u/2463614?v=4","gravatar_id":"","url":"https://api.github.com/users/andrewvanbreda","html_url":"https://github.com/andrewvanbreda","followers_url":"https://api.github.com/users/andrewvanbreda/followers","following_url":"https://api.github.com/users/andrewvanbreda/following{/other_user}","gists_url":"https://api.github.com/users/andrewvanbreda/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewvanbreda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewvanbreda/subscriptions","organizations_url":"https://api.github.com/users/andrewvanbreda/orgs","repos_url":"https://api.github.com/users/andrewvanbreda/repos","events_url":"https://api.github.com/users/andrewvanbreda/events{/privacy}","received_events_url":"https://api.github.com/users/andrewvanbreda/received_events","type":"User","site_admin":false},"created_at":"2019-05-21T13:55:45Z","updated_at":"2019-05-21T13:55:45Z","author_association":"COLLABORATOR","body":"@BirenRathod \r\nOk, I think I made a mistake in the 2nd second as it should of been count(*).\r\nIt doesn't matter I think anyway, as 2000 attributes sounds about right for the number of users so i am happy with that. I was only concerned if the number was huge.\r\nThanks. Will close now.\r\nAndy"}}
{"actor":{"display_login":"andrewvanbreda"}}