{"action":"created","issue":{"url":"https://api.github.com/repos/HazyResearch/ddlite/issues/104","repository_url":"https://api.github.com/repos/HazyResearch/ddlite","labels_url":"https://api.github.com/repos/HazyResearch/ddlite/issues/104/labels{/name}","comments_url":"https://api.github.com/repos/HazyResearch/ddlite/issues/104/comments","events_url":"https://api.github.com/repos/HazyResearch/ddlite/issues/104/events","html_url":"https://github.com/HazyResearch/ddlite/issues/104","id":146791349,"number":104,"title":"More parsing issues","user":{"login":"jason-fries","id":348682,"avatar_url":"https://avatars.githubusercontent.com/u/348682?v=3","gravatar_id":"","url":"https://api.github.com/users/jason-fries","html_url":"https://github.com/jason-fries","followers_url":"https://api.github.com/users/jason-fries/followers","following_url":"https://api.github.com/users/jason-fries/following{/other_user}","gists_url":"https://api.github.com/users/jason-fries/gists{/gist_id}","starred_url":"https://api.github.com/users/jason-fries/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jason-fries/subscriptions","organizations_url":"https://api.github.com/users/jason-fries/orgs","repos_url":"https://api.github.com/users/jason-fries/repos","events_url":"https://api.github.com/users/jason-fries/events{/privacy}","received_events_url":"https://api.github.com/users/jason-fries/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/HazyResearch/ddlite/labels/bug","name":"bug","color":"fc2929"}],"state":"closed","locked":false,"assignee":{"login":"henryre","id":7783678,"avatar_url":"https://avatars.githubusercontent.com/u/7783678?v=3","gravatar_id":"","url":"https://api.github.com/users/henryre","html_url":"https://github.com/henryre","followers_url":"https://api.github.com/users/henryre/followers","following_url":"https://api.github.com/users/henryre/following{/other_user}","gists_url":"https://api.github.com/users/henryre/gists{/gist_id}","starred_url":"https://api.github.com/users/henryre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/henryre/subscriptions","organizations_url":"https://api.github.com/users/henryre/orgs","repos_url":"https://api.github.com/users/henryre/repos","events_url":"https://api.github.com/users/henryre/events{/privacy}","received_events_url":"https://api.github.com/users/henryre/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/HazyResearch/ddlite/milestones/1","html_url":"https://github.com/HazyResearch/ddlite/milestones/DeepDive%20Lite%200.1","labels_url":"https://api.github.com/repos/HazyResearch/ddlite/milestones/1/labels","id":1607442,"number":1,"title":"DeepDive Lite 0.1","description":"Preparation for the bio entity hackathon","creator":{"login":"ajratner","id":1585405,"avatar_url":"https://avatars.githubusercontent.com/u/1585405?v=3","gravatar_id":"","url":"https://api.github.com/users/ajratner","html_url":"https://github.com/ajratner","followers_url":"https://api.github.com/users/ajratner/followers","following_url":"https://api.github.com/users/ajratner/following{/other_user}","gists_url":"https://api.github.com/users/ajratner/gists{/gist_id}","starred_url":"https://api.github.com/users/ajratner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajratner/subscriptions","organizations_url":"https://api.github.com/users/ajratner/orgs","repos_url":"https://api.github.com/users/ajratner/repos","events_url":"https://api.github.com/users/ajratner/events{/privacy}","received_events_url":"https://api.github.com/users/ajratner/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":46,"state":"open","created_at":"2016-02-26T05:55:22Z","updated_at":"2016-04-08T15:44:18Z","due_on":"2016-04-08T07:00:00Z","closed_at":null},"comments":7,"created_at":"2016-04-08T02:00:51Z","updated_at":"2016-04-08T15:44:18Z","closed_at":"2016-04-08T15:44:18Z","body":"**UPDATE: Checking lxml to see if there is some weirdness going on -- I might be able to fix things.**\r\n\r\nMore parsing weirdness using the latest version of ddlite. Parsing a subset of the PMC corpus results in \r\nthis error (I've attached the offending file). The XML itself is well-formed. \r\n\r\n`Traceback (most recent call last):\r\n  File \"/Users/fries/Code/dd-biolib/examples/pmc_corpus_demo.py\", line 21, in <module>\r\n    candidates = Entities(sentences, matcher)\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/ddlite.py\", line 279, in __init__\r\n    super(Entities, self).__init__(content)\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/ddlite.py\", line 126, in __init__\r\n    self._candidates = list(self._extract_candidates(C))\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/ddlite.py\", line 147, in _extract_candidates\r\n    for cand in self._apply(sent):\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/ddlite.py\", line 285, in _apply\r\n    xt = corenlp_to_xmltree(sent)\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/tree_structs.py\", line 88, in corenlp_to_xmltree\r\n    root = corenlp_to_xmltree_sub(s, dep_parents, 0)\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/tree_structs.py\", line 121, in corenlp_to_xmltree_sub\r\n    root.append(corenlp_to_xmltree_sub(s, dep_parents, i+1))\r\n  File \"/Users/fries/Code/dd-biolib/ddlite/tree_structs.py\", line 118, in corenlp_to_xmltree_sub\r\n    root = et.Element('node', attrib=attrib)\r\n  File \"src/lxml/lxml.etree.pyx\", line 2998, in lxml.etree.Element (src/lxml/lxml.etree.c:80597)\r\n  File \"src/lxml/apihelpers.pxi\", line 135, in lxml.etree._makeElement (src/lxml/lxml.etree.c:17594)\r\n  File \"src/lxml/apihelpers.pxi\", line 123, in lxml.etree._makeElement (src/lxml/lxml.etree.c:17442)\r\n  File \"src/lxml/apihelpers.pxi\", line 305, in lxml.etree._initNodeAttributes (src/lxml/lxml.etree.c:19976)\r\n  File \"src/lxml/apihelpers.pxi\", line 316, in lxml.etree._addAttributeToNode (src/lxml/lxml.etree.c:20186)\r\n  File \"src/lxml/apihelpers.pxi\", line 1439, in lxml.etree._utf8 (src/lxml/lxml.etree.c:32431)\r\nValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters`\r\n\r\n[Arthritis_2012_Jul_25_2012_943156.nxml.zip](https://github.com/HazyResearch/ddlite/files/209502/Arthritis_2012_Jul_25_2012_943156.nxml.zip)\r\n"},"comment":{"url":"https://api.github.com/repos/HazyResearch/ddlite/issues/comments/207487335","html_url":"https://github.com/HazyResearch/ddlite/issues/104#issuecomment-207487335","issue_url":"https://api.github.com/repos/HazyResearch/ddlite/issues/104","id":207487335,"user":{"login":"henryre","id":7783678,"avatar_url":"https://avatars.githubusercontent.com/u/7783678?v=3","gravatar_id":"","url":"https://api.github.com/users/henryre","html_url":"https://github.com/henryre","followers_url":"https://api.github.com/users/henryre/followers","following_url":"https://api.github.com/users/henryre/following{/other_user}","gists_url":"https://api.github.com/users/henryre/gists{/gist_id}","starred_url":"https://api.github.com/users/henryre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/henryre/subscriptions","organizations_url":"https://api.github.com/users/henryre/orgs","repos_url":"https://api.github.com/users/henryre/repos","events_url":"https://api.github.com/users/henryre/events{/privacy}","received_events_url":"https://api.github.com/users/henryre/received_events","type":"User","site_admin":false},"created_at":"2016-04-08T15:44:18Z","updated_at":"2016-04-08T15:44:18Z","body":"Closing this for now. The issue seems to be with tables parsed as sentences. Right now, it doesn't make sense to engineer a clever ddlite utility to catch them before parsing. Once parsed, tables can be removed by thresholding the sentence length as @jason-fries suggested above. "}}