{"action":"created","issue":{"url":"https://api.github.com/repos/jgm/cmark/issues/169","repository_url":"https://api.github.com/repos/jgm/cmark","labels_url":"https://api.github.com/repos/jgm/cmark/issues/169/labels{/name}","comments_url":"https://api.github.com/repos/jgm/cmark/issues/169/comments","events_url":"https://api.github.com/repos/jgm/cmark/issues/169/events","html_url":"https://github.com/jgm/cmark/pull/169","id":194502685,"number":169,"title":"More sourcepos!","user":{"login":"MathieuDuponchelle","id":1278273,"avatar_url":"https://avatars.githubusercontent.com/u/1278273?v=3","gravatar_id":"","url":"https://api.github.com/users/MathieuDuponchelle","html_url":"https://github.com/MathieuDuponchelle","followers_url":"https://api.github.com/users/MathieuDuponchelle/followers","following_url":"https://api.github.com/users/MathieuDuponchelle/following{/other_user}","gists_url":"https://api.github.com/users/MathieuDuponchelle/gists{/gist_id}","starred_url":"https://api.github.com/users/MathieuDuponchelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MathieuDuponchelle/subscriptions","organizations_url":"https://api.github.com/users/MathieuDuponchelle/orgs","repos_url":"https://api.github.com/users/MathieuDuponchelle/repos","events_url":"https://api.github.com/users/MathieuDuponchelle/events{/privacy}","received_events_url":"https://api.github.com/users/MathieuDuponchelle/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-12-09T03:04:42Z","updated_at":"2016-12-09T09:12:21Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/jgm/cmark/pulls/169","html_url":"https://github.com/jgm/cmark/pull/169","diff_url":"https://github.com/jgm/cmark/pull/169.diff","patch_url":"https://github.com/jgm/cmark/pull/169.patch"},"body":"Found time to work a bit more on https://github.com/jgm/cmark/issues/26 , these commits solely implement the definition of a gap-free source map with no empty extents, parsed in the same pass as the nodes.\r\n\r\nA few things to discuss:\r\n\r\n* Ownership of line-termination characters and trailing whitespace at the end of blocks, I make them belong to the root\r\n* Ownership of reference definitions, which I also attribute to the root, as there's no proper AST node for them\r\n* Performance, my initial implementation for this approach doubled make bench time (erg), I now observe a 25 % difference, which seems reasonable to me, not sure how much can be ironed out still, the task is inherently a bit complex, due to potential discontinuities between the n extents constituting a block. Either we consider this a reasonable enough performance hit and hope we find more smart ways to reduce it, or we make it conditional to the SOURCEPOS option as currently.\r\n\r\nThings that are in my opinion out of scope and can be discussed later:\r\n\r\n* Actual API for this\r\n* \"Reverse source map\", ie node to extents\r\n\r\nTo test this, just enable the call to \"print_parser_source_map\" in blocks.c:finalize_document , example output for the case that made me revise my approach:\r\n\r\n```\r\n>     code\r\n>     more code\r\n```\r\n\r\n```\r\n0:1 - block_quote (0x9485f0)\r\n1:2 - block_quote (0x9485f0)\r\n2:6 - code_block (0x948740)\r\n6:14 - code_block (0x948740)\r\n14:15 - block_quote (0x9485f0)\r\n15:16 - block_quote (0x9485f0)\r\n16:20 - code_block (0x948740)\r\n20:30 - code_block (0x948740)\r\n```\r\n\r\nI'm sure there's more to say, but let's keep this short :)"},"comment":{"url":"https://api.github.com/repos/jgm/cmark/issues/comments/265967512","html_url":"https://github.com/jgm/cmark/pull/169#issuecomment-265967512","issue_url":"https://api.github.com/repos/jgm/cmark/issues/169","id":265967512,"user":{"login":"jgm","id":3044,"avatar_url":"https://avatars.githubusercontent.com/u/3044?v=3","gravatar_id":"","url":"https://api.github.com/users/jgm","html_url":"https://github.com/jgm","followers_url":"https://api.github.com/users/jgm/followers","following_url":"https://api.github.com/users/jgm/following{/other_user}","gists_url":"https://api.github.com/users/jgm/gists{/gist_id}","starred_url":"https://api.github.com/users/jgm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jgm/subscriptions","organizations_url":"https://api.github.com/users/jgm/orgs","repos_url":"https://api.github.com/users/jgm/repos","events_url":"https://api.github.com/users/jgm/events{/privacy}","received_events_url":"https://api.github.com/users/jgm/received_events","type":"User","site_admin":false},"created_at":"2016-12-09T09:12:21Z","updated_at":"2016-12-09T09:12:21Z","body":"A note on syntax highlighting:  the current system will make it easy to highlight the markers for a block quote, but the paragraph inside the block quote will be highlighted as a regular paragraph, not e.g. put in a special color.  I don't know if that's always going to be what is wanted, but it's something to think about.\r\n\r\nI think a lot of people who care about syntax highlighting may want, e.g. to highlight the TEXT of a header, not just the opening and closing `#` marks.  And it's hard to see how to do that with this system, since there's no explicit marker for the end of a block.\r\n\r\nHere's an idea. Instead of a linked list of nodes for extents, which brings in lots of memory allocation (that is probably what is responsible for the performance drop), why not have an array of structs, of length equal to the length of the source document?  (It could be allocated once at the beginning.)\r\n\r\nEach element of the array would contain:\r\n- a pointer to a node\r\n- an indication BEGIN, END, or CONTINUED\r\n\r\nThen we'd really have all the information that could conceivably be needed.  This array might end up taking more memory than your linked list, but it shouldn't be more than 10x the size of the source document.\r\n\r\n"}}
{"actor":{"display_login":"jgm"}}