{"action":"created","issue":{"url":"https://api.github.com/repos/cms-l1t-offline/cmssw/issues/423","repository_url":"https://api.github.com/repos/cms-l1t-offline/cmssw","labels_url":"https://api.github.com/repos/cms-l1t-offline/cmssw/issues/423/labels{/name}","comments_url":"https://api.github.com/repos/cms-l1t-offline/cmssw/issues/423/comments","events_url":"https://api.github.com/repos/cms-l1t-offline/cmssw/issues/423/events","html_url":"https://github.com/cms-l1t-offline/cmssw/issues/423","id":182268347,"number":423,"title":"Memory leaks - L1T code in CMSSW for Sept 2016 MC Production","user":{"login":"rekovic","id":4990487,"avatar_url":"https://avatars.githubusercontent.com/u/4990487?v=3","gravatar_id":"","url":"https://api.github.com/users/rekovic","html_url":"https://github.com/rekovic","followers_url":"https://api.github.com/users/rekovic/followers","following_url":"https://api.github.com/users/rekovic/following{/other_user}","gists_url":"https://api.github.com/users/rekovic/gists{/gist_id}","starred_url":"https://api.github.com/users/rekovic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rekovic/subscriptions","organizations_url":"https://api.github.com/users/rekovic/orgs","repos_url":"https://api.github.com/users/rekovic/repos","events_url":"https://api.github.com/users/rekovic/events{/privacy}","received_events_url":"https://api.github.com/users/rekovic/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2016-10-11T13:47:03Z","updated_at":"2016-10-12T16:16:00Z","closed_at":null,"body":"This issue is to track memory leaks from L1Trigger Offline software tagged l1t-tsg-v9.2\r\nafter it has been merged in CMSSW together with the big O2O PR.\r\nBoth the core L1T software and O2O L1T software have been merged by the IB CMSSW_8_0_X_2016-10-06-2300. \r\n\r\nSince then, a PR #16085 reduced the big memory blob allocated by the OMTF XMLConfigReader, by 80%, from about 500MB to 100MB.\r\n\r\nTo track the memory leaks used igprof. \r\n\r\n<pre>\r\ncmsrel CMSSW_8_0_X_2016-10-06-2300\r\ncd CMSSW_8_0_X_2016-10-06-2300/src/\r\ncmsenv\r\ngit cms-init\r\ngit checkout -b mydevel\r\ngit cms-addpkg L1Trigger/L1TCommon\r\ngit cms-merge-topic 16085\r\ngit cms-addpkg L1Trigger/L1TMuon\r\ngit cms-addpkg L1Trigger/L1Calorimeter\r\ngit cms-addpkg L1Trigger/L1TCalorimeter\r\ngit cms-addpkg L1Trigger/L1TMuonEndcap\r\ngit cms-addpkg L1Trigger/L1TMuonEndCap\r\ngit cms-addpkg L1Trigger/L1TGlobal\r\ngit clone https://github.com/cms-data/L1Trigger-L1TGlobal.git\r\nL1Trigger/L1TGlobal/data\r\ngit clone https://github.com/cms-data/L1Trigger-L1TMuon.git\r\nL1Trigger/L1TMuon/data\r\ngit clone https://github.com/cms-data/L1Trigger-L1TCalorimeter.git\r\nL1Trigger/L1TCalorimeter/data\r\nscram b -j7\r\ncmsDriver.py l1Ntuple -s RAW2DIGI\r\n--python_filename=l1Ntuple_RAW_wRECO.py -n 100 --no_output --no_exec\r\n--era=Run2_2016 --data --conditions=80X_dataRun2_Prompt_v11\r\n--filein=/store/express/Run2016G/ExpressPhysics/FEVT/Express-v1/000/280/384/00000/0AC379C2-7976-E611-A9A7-02163E0118D8.root\r\n--customise=L1Trigger/Configuration/customiseReEmul.L1TReEmulFromRAW\r\n--customise=L1Trigger/L1TNtuples/customiseL1Ntuple.L1NtupleAODRAWEMU\r\n</pre>\r\n\r\n**Igprof results  CMSSW_8_0_X_2016-10-06-2300 + PR #16085**: \r\nhttps://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085/self showed remaining leaks:\r\n\r\n* OMTF XMLConfigReader has still a leak after the big one was fixed (const):\r\nhttps://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085/68\r\n\r\n* TwinMux (10kB/event):\r\nhttps://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085/472\r\nhttps://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085/1285\r\n\r\n* BMTF unpacker (8kB/event):\r\nhttps://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085/550\r\n\r\n* EMTF forest (const):\r\nhttps://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085/1246\r\n\r\n\r\n**Igprof results TwinMux fix (+ above)**:\r\nhttps://rekovic.web.cern.ch/rekovic/cms/cgi-bin/igprof-navigator/igreport_live_memoryManage-TwinMux-only/self does not contain leaks from TwinMux any more.  Code in branch:  <pre> cms-l1t-offline/pr80x-l1t-fixed-memoryManage-TwinMux-only</pre>"},"comment":{"url":"https://api.github.com/repos/cms-l1t-offline/cmssw/issues/comments/253261181","html_url":"https://github.com/cms-l1t-offline/cmssw/issues/423#issuecomment-253261181","issue_url":"https://api.github.com/repos/cms-l1t-offline/cmssw/issues/423","id":253261181,"user":{"login":"eparadas","id":6408474,"avatar_url":"https://avatars.githubusercontent.com/u/6408474?v=3","gravatar_id":"","url":"https://api.github.com/users/eparadas","html_url":"https://github.com/eparadas","followers_url":"https://api.github.com/users/eparadas/followers","following_url":"https://api.github.com/users/eparadas/following{/other_user}","gists_url":"https://api.github.com/users/eparadas/gists{/gist_id}","starred_url":"https://api.github.com/users/eparadas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eparadas/subscriptions","organizations_url":"https://api.github.com/users/eparadas/orgs","repos_url":"https://api.github.com/users/eparadas/repos","events_url":"https://api.github.com/users/eparadas/events{/privacy}","received_events_url":"https://api.github.com/users/eparadas/received_events","type":"User","site_admin":false},"created_at":"2016-10-12T16:16:00Z","updated_at":"2016-10-12T16:16:00Z","body":"Hi,\n\nAndrea thanks a lot for the comments.\nThomas, the linkAndQual is used inside the unpacking function and it \nkeeps information from different links (blocks).\n\nthe linkAndQual should live outside the unpacking so that it keeps all \nthe information of an event. But it is defined in the header as a \nnon-pointer so I don't understand how this could be the problem. It \nshould be deleted during every event, right?\n\nCheers,\nVangelis\n\nOn 10/12/2016 06:14 PM, Thomas Reis wrote:\n>\n> Where is linkAndQual_ actually set/used outside of the unpacking \n> function? I do not find anything about it.\n>\n> I have tried to make it a local variable in the unpacking function \n> only and the memory leak vanishes.\n> thomreis@698b425 \n> <https://github.com/thomreis/cmssw/commit/698b425f9f8c6ce6716f48f916590b0611cc2e2b>\n> igprof results:\n> https://treis.web.cern.ch/treis/cgi-bin/igprof-navigator/igreport_l1ntuple_mp_live_CMSSW_8_0_X_2016-10-06-2300_PR16085_bmtftest/self\n> However, I am not sure if that changes the behaviour of the unpacker.\n>\n> —\n> You are receiving this because you commented.\n> Reply to this email directly, view it on GitHub \n> <https://github.com/cms-l1t-offline/cmssw/issues/423#issuecomment-253243023>, \n> or mute the thread \n> <https://github.com/notifications/unsubscribe-auth/AGHJGijjpEWDhHDJzCcPQROket_3JO61ks5qzPlZgaJpZM4KTpdi>.\n>\n\n"}}
{"actor":{"display_login":"eparadas"}}