{"action":"created","issue":{"url":"https://api.github.com/repos/fecgov/fec-eregs/issues/426","repository_url":"https://api.github.com/repos/fecgov/fec-eregs","labels_url":"https://api.github.com/repos/fecgov/fec-eregs/issues/426/labels{/name}","comments_url":"https://api.github.com/repos/fecgov/fec-eregs/issues/426/comments","events_url":"https://api.github.com/repos/fecgov/fec-eregs/issues/426/events","html_url":"https://github.com/fecgov/fec-eregs/issues/426","id":391262728,"node_id":"MDU6SXNzdWUzOTEyNjI3Mjg=","number":426,"title":"unable to parse 2018 eregs on local.","user":{"login":"pkfec","id":11650355,"node_id":"MDQ6VXNlcjExNjUwMzU1","avatar_url":"https://avatars1.githubusercontent.com/u/11650355?v=4","gravatar_id":"","url":"https://api.github.com/users/pkfec","html_url":"https://github.com/pkfec","followers_url":"https://api.github.com/users/pkfec/followers","following_url":"https://api.github.com/users/pkfec/following{/other_user}","gists_url":"https://api.github.com/users/pkfec/gists{/gist_id}","starred_url":"https://api.github.com/users/pkfec/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pkfec/subscriptions","organizations_url":"https://api.github.com/users/pkfec/orgs","repos_url":"https://api.github.com/users/pkfec/repos","events_url":"https://api.github.com/users/pkfec/events{/privacy}","received_events_url":"https://api.github.com/users/pkfec/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"qqss88","id":7330555,"node_id":"MDQ6VXNlcjczMzA1NTU=","avatar_url":"https://avatars2.githubusercontent.com/u/7330555?v=4","gravatar_id":"","url":"https://api.github.com/users/qqss88","html_url":"https://github.com/qqss88","followers_url":"https://api.github.com/users/qqss88/followers","following_url":"https://api.github.com/users/qqss88/following{/other_user}","gists_url":"https://api.github.com/users/qqss88/gists{/gist_id}","starred_url":"https://api.github.com/users/qqss88/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qqss88/subscriptions","organizations_url":"https://api.github.com/users/qqss88/orgs","repos_url":"https://api.github.com/users/qqss88/repos","events_url":"https://api.github.com/users/qqss88/events{/privacy}","received_events_url":"https://api.github.com/users/qqss88/received_events","type":"User","site_admin":false},"assignees":[{"login":"qqss88","id":7330555,"node_id":"MDQ6VXNlcjczMzA1NTU=","avatar_url":"https://avatars2.githubusercontent.com/u/7330555?v=4","gravatar_id":"","url":"https://api.github.com/users/qqss88","html_url":"https://github.com/qqss88","followers_url":"https://api.github.com/users/qqss88/followers","following_url":"https://api.github.com/users/qqss88/following{/other_user}","gists_url":"https://api.github.com/users/qqss88/gists{/gist_id}","starred_url":"https://api.github.com/users/qqss88/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qqss88/subscriptions","organizations_url":"https://api.github.com/users/qqss88/orgs","repos_url":"https://api.github.com/users/qqss88/repos","events_url":"https://api.github.com/users/qqss88/events{/privacy}","received_events_url":"https://api.github.com/users/qqss88/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/fecgov/fec-eregs/milestones/52","html_url":"https://github.com/fecgov/fec-eregs/milestone/52","labels_url":"https://api.github.com/repos/fecgov/fec-eregs/milestones/52/labels","id":3849821,"node_id":"MDk6TWlsZXN0b25lMzg0OTgyMQ==","number":52,"title":"Sprint 7.7","description":"","creator":{"login":"lbeaufort","id":31420082,"node_id":"MDQ6VXNlcjMxNDIwMDgy","avatar_url":"https://avatars1.githubusercontent.com/u/31420082?v=4","gravatar_id":"","url":"https://api.github.com/users/lbeaufort","html_url":"https://github.com/lbeaufort","followers_url":"https://api.github.com/users/lbeaufort/followers","following_url":"https://api.github.com/users/lbeaufort/following{/other_user}","gists_url":"https://api.github.com/users/lbeaufort/gists{/gist_id}","starred_url":"https://api.github.com/users/lbeaufort/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lbeaufort/subscriptions","organizations_url":"https://api.github.com/users/lbeaufort/orgs","repos_url":"https://api.github.com/users/lbeaufort/repos","events_url":"https://api.github.com/users/lbeaufort/events{/privacy}","received_events_url":"https://api.github.com/users/lbeaufort/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":2,"state":"open","created_at":"2018-11-26T20:40:12Z","updated_at":"2018-12-21T16:19:44Z","due_on":"2018-12-17T08:00:00Z","closed_at":null},"comments":2,"created_at":"2018-12-14T20:09:23Z","updated_at":"2018-12-22T00:42:01Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Tried to parse Title 11 FEC 2018 eregs on local. Out of 45 eregs only 7 parsed  successfully and rest of them failed with errors. Noticed two different type of errors in the logs.\r\n\r\n**Here is the list of 7  eregs that parsed successfully:**\r\n![screen shot 2018-12-14 at 2 52 55 pm](https://user-images.githubusercontent.com/11650355/50024741-5b3fc400-ffb1-11e8-8dec-eaa3366d1303.png)\r\n\r\nregs with **single citations** failed with an  **IndexError: string index out of range error** error\r\nregs with **multiple citations** failed with an **AttributeError: 'str' object has no attribute 'start'`** error.\r\n \r\n**Example 1:**\r\n**Parsing 11 CFR 9038 failed with an IndexError:**\r\n2018-12-14 19:01:09 regparser.history.annual                 No <PARTS> in https://www.gpo.gov/fdsys/bulkdata/CFR/2018/title-11/CFR-2018-title11-vol1.xml. Assuming this volume contains all of the regs\r\n2018-12-14 19:01:09 regparser.commands.annual_version        Getting annual version - 11 CFR 9038, Year: 2018\r\n2018-12-14 19:01:10 regparser.tree.gpo_cfr.builder           Build tree <Element CFRGRANULE at 0x10c1cbf88>\r\n2018-12-14 19:01:10 regparser.tree.gpo_cfr.builder           Preprocessing XML <Element CFRGRANULE at 0x10c1cbf88>\r\n2018-12-14 19:01:10 regparser.content                        Could not load macros from regcontent.macros\r\nTraceback (most recent call last):\r\n  File \"/Users/pkasireddy/.pyenv/versions/eregs365/bin/eregs\", line 11, in <module>\r\n    sys.exit(eregs())\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/web/management/runner.py\", line 22, in eregs\r\n    runner(args)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/web/management/runner.py\", line 14, in runner\r\n    execute_from_command_line(argv)\r\n  File \"/Users/\r\npkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line\r\n    utility.execute()\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/django/core/management/__init__.py\", line 356, in execute\r\n    self.fetch_command(subcommand).run_from_argv(self.argv)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/djclick/adapter.py\", line 62, in run_from_argv\r\n    return self.main(args=argv[2:], standalone_mode=False)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 717, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/djclick/adapter.py\", line 49, in invoke\r\n    return super(DjangoCommandMixin, self).invoke(ctx)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/retry.py\", line 47, in invoke\r\n    lambda: super(RetryingCommand, self).invoke(deepcopy(ctx)))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/retry.py\", line 57, in run_or_resolve\r\n    cmd()\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/retry.py\", line 47, in <lambda>\r\n    lambda: super(RetryingCommand, self).invoke(deepcopy(ctx)))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 1137, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 956, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 555, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/decorators.py\", line 17, in new_func\r\n    return f(get_current_context(), *args, **kwargs)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/pipeline.py\", line 32, in pipeline\r\n    ctx.invoke(annual_version, **params)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 555, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/annual_version.py\", line 73, in annual_version\r\n    process_if_needed(vol, cfr_part)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/annual_version.py\", line 29, in process_if_needed\r\n    tree = builder.build_tree(annual_entry.read().xml)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/builder.py\", line 98, in build_tree\r\n    plugin(tree, xml_node)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/section.py\", line 258, in __call__\r\n    sections = build_from_section(parent.cfr_part, xml_node)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/section.py\", line 159, in build_from_section\r\n    RegtextParagraphProcessor().process(section_xml, sect_node)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/xml_parser/paragraph_processor.py\", line 122, in process\r\n    nodes = self.parse_nodes(xml)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/xml_parser/paragraph_processor.py\", line 48, in parse_nodes\r\n    nodes.extend(tag_matcher.derive_nodes(child, processor=self))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/section.py\", line 208, in derive_nodes\r\n    for marker, plain_text, tagged_text in split_by_markers(xml):\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/section.py\", line 188, in split_by_markers\r\n    markers_list = get_markers(tagged_text, next_marker(xml))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/section.py\", line 52, in get_markers\r\n    collapsed = collapsed_markers(text)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/tree/gpo_cfr/section.py\", line 101, in collapsed_markers\r\n    potential = [trip for trip in remove_citation_overlaps(text, potential)]\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 302, in remove_citation_overlaps\r\n    return [(m, start, end) for m, start, end in possible_markers\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 306, in <listcomp>\r\n    for e in internal_citations(text))]\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 258, in internal_citations\r\n    single(grammar.marker_paragraph, False)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 242, in single\r\n    initial_label, comment))\r\n  **`File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 201, in single_citations\r\n    start = match.marker.pos[1]\r\nIndexError: string index out of range`**\r\n\r\n**Example #2** \r\n**Parsing 11 CFR 9039 regulation failed with a AttributeError:**\r\n2018-12-14 19:01:12 regparser.history.annual                 No <PARTS> in https://www.gpo.gov/fdsys/bulkdata/CFR/2018/title-11/CFR-2018-title11-vol1.xml. Assuming this volume contains all of the regs\r\n2018-12-14 19:01:12 regparser.commands.annual_version        Getting annual version - 11 CFR 9039, Year: 2018\r\n2018-12-14 19:01:13 regparser.commands.layers                Build layers - 11 CFR 9039\r\nTraceback (most recent call last):\r\n  File \"/Users/pkasireddy/.pyenv/versions/eregs365/bin/eregs\", line 11, in <module>\r\n    sys.exit(eregs())\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/web/management/runner.py\", line 22, in eregs\r\n    runner(args)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/web/management/runner.py\", line 14, in runner\r\n    execute_from_command_line(argv)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line\r\n    utility.execute()\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/django/core/management/__init__.py\", line 356, in execute\r\n    self.fetch_command(subcommand).run_from_argv(self.argv)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/djclick/adapter.py\", line 62, in run_from_argv\r\n    return self.main(args=argv[2:], standalone_mode=False)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 717, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/djclick/adapter.py\", line 49, in invoke\r\n    return super(DjangoCommandMixin, self).invoke(ctx)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/retry.py\", line 47, in invoke\r\n    lambda: super(RetryingCommand, self).invoke(deepcopy(ctx)))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/retry.py\", line 57, in run_or_resolve\r\n    cmd()\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/retry.py\", line 47, in <lambda>\r\n    lambda: super(RetryingCommand, self).invoke(deepcopy(ctx)))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 1137, in invoke\r\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 956, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 555, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/decorators.py\", line 17, in new_func\r\n    return f(get_current_context(), *args, **kwargs)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/pipeline.py\", line 37, in pipeline\r\n    ctx.invoke(layers, **params)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/click/core.py\", line 555, in invoke\r\n    return callback(*args, **kwargs)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/layers.py\", line 91, in layers\r\n    process_cfr_layers(stale, tree_title, version_entry)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/commands/layers.py\", line 64, in process_cfr_layers\r\n    tree, cfr_title=int(cfr_title), version=version).build()\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/layer.py\", line 52, in build\r\n    self.builder(self.tree, cache)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/layer.py\", line 48, in builder\r\n    self.builder(c, cache)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/layer.py\", line 48, in builder\r\n    self.builder(c, cache)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/layer.py\", line 43, in builder\r\n    layer_element = self.process(node)\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/external_citations.py\", line 13, in process\r\n    for finder in external_types.ALL\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/external_citations.py\", line 14, in <listcomp>\r\n    for cite in finder().find(node)]\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/layer/external_types.py\", line 47, in find\r\n    for cit in cfr_citations(node.text):\r\n  File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 319, in cfr_citations\r\n    include_fill=include_fill))\r\n  **`File \"/Users/pkasireddy/.pyenv/versions/3.6.5/envs/eregs365/lib/python3.6/site-packages/regparser/citations.py\", line 223, in multiple_citations\r\n    match.pos.start, match.pos.end, new_label,\r\nAttributeError: 'str' object has no attribute 'start'`**"},"comment":{"url":"https://api.github.com/repos/fecgov/fec-eregs/issues/comments/449531240","html_url":"https://github.com/fecgov/fec-eregs/issues/426#issuecomment-449531240","issue_url":"https://api.github.com/repos/fecgov/fec-eregs/issues/426","id":449531240,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0OTUzMTI0MA==","user":{"login":"fec-jli","id":24395751,"node_id":"MDQ6VXNlcjI0Mzk1NzUx","avatar_url":"https://avatars0.githubusercontent.com/u/24395751?v=4","gravatar_id":"","url":"https://api.github.com/users/fec-jli","html_url":"https://github.com/fec-jli","followers_url":"https://api.github.com/users/fec-jli/followers","following_url":"https://api.github.com/users/fec-jli/following{/other_user}","gists_url":"https://api.github.com/users/fec-jli/gists{/gist_id}","starred_url":"https://api.github.com/users/fec-jli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fec-jli/subscriptions","organizations_url":"https://api.github.com/users/fec-jli/orgs","repos_url":"https://api.github.com/users/fec-jli/repos","events_url":"https://api.github.com/users/fec-jli/events{/privacy}","received_events_url":"https://api.github.com/users/fec-jli/received_events","type":"User","site_admin":false},"created_at":"2018-12-22T00:42:01Z","updated_at":"2018-12-22T00:42:01Z","author_association":"CONTRIBUTOR","body":"We may need Modify runtime.txt set python version = 3.6.1\r\n3.6.5 for GET method works well."}}
{"actor":{"display_login":"fec-jli"}}