{"action":"created","issue":{"url":"https://api.github.com/repos/buriy/python-readability/issues/102","repository_url":"https://api.github.com/repos/buriy/python-readability","labels_url":"https://api.github.com/repos/buriy/python-readability/issues/102/labels{/name}","comments_url":"https://api.github.com/repos/buriy/python-readability/issues/102/comments","events_url":"https://api.github.com/repos/buriy/python-readability/issues/102/events","html_url":"https://github.com/buriy/python-readability/issues/102","id":293821667,"number":102,"title":"Don't use drop_tree() while iterating","user":{"login":"P1zz4br0etch3n","id":9010623,"avatar_url":"https://avatars0.githubusercontent.com/u/9010623?v=4","gravatar_id":"","url":"https://api.github.com/users/P1zz4br0etch3n","html_url":"https://github.com/P1zz4br0etch3n","followers_url":"https://api.github.com/users/P1zz4br0etch3n/followers","following_url":"https://api.github.com/users/P1zz4br0etch3n/following{/other_user}","gists_url":"https://api.github.com/users/P1zz4br0etch3n/gists{/gist_id}","starred_url":"https://api.github.com/users/P1zz4br0etch3n/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/P1zz4br0etch3n/subscriptions","organizations_url":"https://api.github.com/users/P1zz4br0etch3n/orgs","repos_url":"https://api.github.com/users/P1zz4br0etch3n/repos","events_url":"https://api.github.com/users/P1zz4br0etch3n/events{/privacy}","received_events_url":"https://api.github.com/users/P1zz4br0etch3n/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-02T09:18:52Z","updated_at":"2018-05-07T10:54:29Z","closed_at":"2018-05-07T10:54:29Z","author_association":"NONE","body":"I found that using drop_tree() while iterating through all nodes in the method remove_unlikely_candidates() breaks the iterator if the dropped element has children.\r\n\r\na simple example:\r\n```\r\nfrom lxml.etree import tostring\r\nfrom readability import Document\r\n\r\nHTML = '''\r\n    <html>\r\n        <body>\r\n            <aside id=\"1\">\r\n                    <a>\r\n            </aside>\r\n            <aside id=\"2\"/>\r\n            <aside id=\"3\"/>\r\n        </body>\r\n    </html>\r\n'''\r\n\r\n# parse html\r\nDOC = Document(HTML)\r\nhtml = DOC._html()\r\n\r\n# iter through all elements\r\nfor elem in html.iter():\r\n    if elem.tag == 'aside':\r\n        elem.drop_tree()\r\n\r\n# print resulting html\r\nprint(tostring(DOC.html, pretty_print=True))\r\n```\r\noutput:\r\n```\r\n<html>\r\n        <body>\r\n\r\n            <aside id=\"2\"/>\r\n            <aside id=\"3\"/>\r\n        </body>\r\n    </html>\r\n```\r\n\r\nActually every **aside** tag should be deleted, but it didn't.\r\nIf you leave out the **a** tag, everything works fine. \r\n\r\nA solution to this could be to wrap the iter statement in a list:\r\n`for elem in list(self.html.iter()):`\r\nOr to collect the elements that should be removed in a list, and drop them after iterating."},"comment":{"url":"https://api.github.com/repos/buriy/python-readability/issues/comments/387029785","html_url":"https://github.com/buriy/python-readability/issues/102#issuecomment-387029785","issue_url":"https://api.github.com/repos/buriy/python-readability/issues/102","id":387029785,"user":{"login":"buriy","id":21124,"avatar_url":"https://avatars1.githubusercontent.com/u/21124?v=4","gravatar_id":"","url":"https://api.github.com/users/buriy","html_url":"https://github.com/buriy","followers_url":"https://api.github.com/users/buriy/followers","following_url":"https://api.github.com/users/buriy/following{/other_user}","gists_url":"https://api.github.com/users/buriy/gists{/gist_id}","starred_url":"https://api.github.com/users/buriy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/buriy/subscriptions","organizations_url":"https://api.github.com/users/buriy/orgs","repos_url":"https://api.github.com/users/buriy/repos","events_url":"https://api.github.com/users/buriy/events{/privacy}","received_events_url":"https://api.github.com/users/buriy/received_events","type":"User","site_admin":false},"created_at":"2018-05-07T10:54:29Z","updated_at":"2018-05-07T10:54:29Z","author_association":"OWNER","body":"Thanks a lot! Fixed in version 0.7."}}
{"actor":{"display_login":"buriy"}}