{"action":"created","issue":{"url":"https://api.github.com/repos/dmwm/CRABServer/issues/5191","repository_url":"https://api.github.com/repos/dmwm/CRABServer","labels_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5191/labels{/name}","comments_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5191/comments","events_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5191/events","html_url":"https://github.com/dmwm/CRABServer/issues/5191","id":148578990,"number":5191,"title":"Adjust the list of sites when ignoreLocality is used","user":{"login":"mmascher","id":1586246,"avatar_url":"https://avatars.githubusercontent.com/u/1586246?v=3","gravatar_id":"","url":"https://api.github.com/users/mmascher","html_url":"https://github.com/mmascher","followers_url":"https://api.github.com/users/mmascher/followers","following_url":"https://api.github.com/users/mmascher/following{/other_user}","gists_url":"https://api.github.com/users/mmascher/gists{/gist_id}","starred_url":"https://api.github.com/users/mmascher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmascher/subscriptions","organizations_url":"https://api.github.com/users/mmascher/orgs","repos_url":"https://api.github.com/users/mmascher/repos","events_url":"https://api.github.com/users/mmascher/events{/privacy}","received_events_url":"https://api.github.com/users/mmascher/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"mmascher","id":1586246,"avatar_url":"https://avatars.githubusercontent.com/u/1586246?v=3","gravatar_id":"","url":"https://api.github.com/users/mmascher","html_url":"https://github.com/mmascher","followers_url":"https://api.github.com/users/mmascher/followers","following_url":"https://api.github.com/users/mmascher/following{/other_user}","gists_url":"https://api.github.com/users/mmascher/gists{/gist_id}","starred_url":"https://api.github.com/users/mmascher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmascher/subscriptions","organizations_url":"https://api.github.com/users/mmascher/orgs","repos_url":"https://api.github.com/users/mmascher/repos","events_url":"https://api.github.com/users/mmascher/events{/privacy}","received_events_url":"https://api.github.com/users/mmascher/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/dmwm/CRABServer/milestones/78","html_url":"https://github.com/dmwm/CRABServer/milestones/CRAB3%20April%202016","labels_url":"https://api.github.com/repos/dmwm/CRABServer/milestones/78/labels","id":1454215,"number":78,"title":"CRAB3 April 2016","description":"maintenance release","creator":{"login":"belforte","id":1837785,"avatar_url":"https://avatars.githubusercontent.com/u/1837785?v=3","gravatar_id":"","url":"https://api.github.com/users/belforte","html_url":"https://github.com/belforte","followers_url":"https://api.github.com/users/belforte/followers","following_url":"https://api.github.com/users/belforte/following{/other_user}","gists_url":"https://api.github.com/users/belforte/gists{/gist_id}","starred_url":"https://api.github.com/users/belforte/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/belforte/subscriptions","organizations_url":"https://api.github.com/users/belforte/orgs","repos_url":"https://api.github.com/users/belforte/repos","events_url":"https://api.github.com/users/belforte/events{/privacy}","received_events_url":"https://api.github.com/users/belforte/received_events","type":"User","site_admin":false},"open_issues":12,"closed_issues":12,"state":"open","created_at":"2015-12-10T14:46:08Z","updated_at":"2016-04-15T07:26:19Z","due_on":null,"closed_at":null},"comments":1,"created_at":"2016-04-15T07:26:19Z","updated_at":"2016-04-15T07:35:39Z","closed_at":null,"body":"When ignoreLocality is used we simply take the whole list of sites from sitedb: https://github.com/dmwm/CRABServer/blob/master/src/python/TaskWorker/Actions/DagmanCreator.py#L642\r\n\r\nIMHO this is wrong, if anything we should do the same thing we do for PrivateMC jobs where we take out T1s (https://github.com/dmwm/CRABServer/blob/master/src/python/TaskWorker/Actions/MakeFakeFileSet.py#L24-L33). I know we need something better in the SSB, but we can start with this.\r\n\r\nBTW, this is another reason (in addition to PrivateMC) that caused the T0 to be in the DESIRED_Sites for CRAB3 jobs. That specific problem is now solved by manually blacklisting the T0 in the SSB. But I wonder what would happen if SSB is down, do we ignore it and simply do not apply the \"global blacklist\"? Also, we said we should provide a way for users to ignore the global blacklist, we may want to think about that twice if we used the global blacklist for avoid to put the T0 in the DESIRED_Sites."},"comment":{"url":"https://api.github.com/repos/dmwm/CRABServer/issues/comments/210330631","html_url":"https://github.com/dmwm/CRABServer/issues/5191#issuecomment-210330631","issue_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5191","id":210330631,"user":{"login":"belforte","id":1837785,"avatar_url":"https://avatars.githubusercontent.com/u/1837785?v=3","gravatar_id":"","url":"https://api.github.com/users/belforte","html_url":"https://github.com/belforte","followers_url":"https://api.github.com/users/belforte/followers","following_url":"https://api.github.com/users/belforte/following{/other_user}","gists_url":"https://api.github.com/users/belforte/gists{/gist_id}","starred_url":"https://api.github.com/users/belforte/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/belforte/subscriptions","organizations_url":"https://api.github.com/users/belforte/orgs","repos_url":"https://api.github.com/users/belforte/repos","events_url":"https://api.github.com/users/belforte/events{/privacy}","received_events_url":"https://api.github.com/users/belforte/received_events","type":"User","site_admin":false},"created_at":"2016-04-15T07:35:39Z","updated_at":"2016-04-15T07:35:39Z","body":"The T0 was not a problem, only aesthetic, as I tried to explain. Here's how I'd approach this:\r\n1. change the semantic of \"ignoreLocality\" to the Crab2 style of \"overrideDataLocation\"\r\n   if user do not like our list, they provide theirs and take their risk and responsibility\r\n2. if CRAB jobs are harmful to a PSN (HLT e.g.) they must be blocked in the infrastructure, not relying on users, nor on CRAB classAds\r\n3. you have a good point that once we take our list from SSB we have to worry about the http fetching to fail and a cache looks a good way to me. But this is general for many things, like getting the list of PSN from SiteDB etc. So either we cache everything, or take the hit and fail submission. I see no evidence for SSB to be less available than SiteDB, and long term if we go on with \"CRIC\" we will take both PSN/PNN and blacklists from the same server\r\n"}}