{"action":"opened","issue":{"url":"https://api.github.com/repos/sosreport/sos/issues/804","repository_url":"https://api.github.com/repos/sosreport/sos","labels_url":"https://api.github.com/repos/sosreport/sos/issues/804/labels{/name}","comments_url":"https://api.github.com/repos/sosreport/sos/issues/804/comments","events_url":"https://api.github.com/repos/sosreport/sos/issues/804/events","html_url":"https://github.com/sosreport/sos/issues/804","id":148276671,"number":804,"title":"Caught exception in sosreport at plugin method \"logs.collect()\"","user":{"login":"anshulbehl","id":6129262,"avatar_url":"https://avatars.githubusercontent.com/u/6129262?v=3","gravatar_id":"","url":"https://api.github.com/users/anshulbehl","html_url":"https://github.com/anshulbehl","followers_url":"https://api.github.com/users/anshulbehl/followers","following_url":"https://api.github.com/users/anshulbehl/following{/other_user}","gists_url":"https://api.github.com/users/anshulbehl/gists{/gist_id}","starred_url":"https://api.github.com/users/anshulbehl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anshulbehl/subscriptions","organizations_url":"https://api.github.com/users/anshulbehl/orgs","repos_url":"https://api.github.com/users/anshulbehl/repos","events_url":"https://api.github.com/users/anshulbehl/events{/privacy}","received_events_url":"https://api.github.com/users/anshulbehl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-04-14T07:31:30Z","updated_at":"2016-04-14T07:31:30Z","closed_at":null,"body":"While running sosreport, I came across an issue where I see that logs and lsbrelease plugins are failing with the below error:\r\n\r\nRunning 43/103: logs...        caught exception in plugin method \"logs.collect()\"\r\nwriting traceback to sos_logs/logs-plugin-errors.txt\r\n  Running 44/103: lsbrelease...        caught exception in plugin method \"lsbrelease.collect()\"\r\nwriting traceback to sos_logs/lsbrelease-plugin-errors.txt\r\n  Running 103/103: yum...                        \r\nCreating compressed archive...\r\n\r\nWhile looking at these files I can see that host machine is under a lot of pressure while running these plugins and hence throw an MemoryError, after doing some debugging I have the below observations.\r\n \r\nApparently the output of \"journalctl --all --this-boot --no-pager\" is too large for python to handle which is causing the exception, After doing some debugging I can see the following data:\r\n\r\n179  \t    def add_string(self, content, dest):\r\n180  \t        import pdb; pdb.set_trace()\r\n181  ->\t\tsrc = dest\r\n182  \t        dest = self.dest_path(dest)\r\n183  \t        self._check_path(dest)\r\n184  \t        f = codecs.open(dest, 'w', encoding='utf-8')\r\n185  \t\tf.write(content)\r\n(Pdb) sys.getsizeof(content)\r\n253876652\r\n\r\nsize of content is around 2GB+ which is being written to destination file as whole.\r\nPlease let me know if this issue is being worked at and what are the possible breakages if we try read the object properly using specific buffer sizes and not dumping all the data simultaneously to the file.\r\n\r\nThanks\r\nAnshul Behl"}}