{"action":"created","issue":{"url":"https://api.github.com/repos/taocpp/PEGTL/issues/129","repository_url":"https://api.github.com/repos/taocpp/PEGTL","labels_url":"https://api.github.com/repos/taocpp/PEGTL/issues/129/labels{/name}","comments_url":"https://api.github.com/repos/taocpp/PEGTL/issues/129/comments","events_url":"https://api.github.com/repos/taocpp/PEGTL/issues/129/events","html_url":"https://github.com/taocpp/PEGTL/issues/129","id":364343404,"node_id":"MDU6SXNzdWUzNjQzNDM0MDQ=","number":129,"title":"Dynamic matching and parse tree","user":{"login":"petersteneteg","id":3638222,"node_id":"MDQ6VXNlcjM2MzgyMjI=","avatar_url":"https://avatars3.githubusercontent.com/u/3638222?v=4","gravatar_id":"","url":"https://api.github.com/users/petersteneteg","html_url":"https://github.com/petersteneteg","followers_url":"https://api.github.com/users/petersteneteg/followers","following_url":"https://api.github.com/users/petersteneteg/following{/other_user}","gists_url":"https://api.github.com/users/petersteneteg/gists{/gist_id}","starred_url":"https://api.github.com/users/petersteneteg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petersteneteg/subscriptions","organizations_url":"https://api.github.com/users/petersteneteg/orgs","repos_url":"https://api.github.com/users/petersteneteg/repos","events_url":"https://api.github.com/users/petersteneteg/events{/privacy}","received_events_url":"https://api.github.com/users/petersteneteg/received_events","type":"User","site_admin":false},"labels":[{"id":160259192,"node_id":"MDU6TGFiZWwxNjAyNTkxOTI=","url":"https://api.github.com/repos/taocpp/PEGTL/labels/bug","name":"bug","color":"fc2929","default":true},{"id":160259197,"node_id":"MDU6TGFiZWwxNjAyNTkxOTc=","url":"https://api.github.com/repos/taocpp/PEGTL/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":{"login":"d-frey","id":3956325,"node_id":"MDQ6VXNlcjM5NTYzMjU=","avatar_url":"https://avatars2.githubusercontent.com/u/3956325?v=4","gravatar_id":"","url":"https://api.github.com/users/d-frey","html_url":"https://github.com/d-frey","followers_url":"https://api.github.com/users/d-frey/followers","following_url":"https://api.github.com/users/d-frey/following{/other_user}","gists_url":"https://api.github.com/users/d-frey/gists{/gist_id}","starred_url":"https://api.github.com/users/d-frey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/d-frey/subscriptions","organizations_url":"https://api.github.com/users/d-frey/orgs","repos_url":"https://api.github.com/users/d-frey/repos","events_url":"https://api.github.com/users/d-frey/events{/privacy}","received_events_url":"https://api.github.com/users/d-frey/received_events","type":"User","site_admin":false},"assignees":[{"login":"d-frey","id":3956325,"node_id":"MDQ6VXNlcjM5NTYzMjU=","avatar_url":"https://avatars2.githubusercontent.com/u/3956325?v=4","gravatar_id":"","url":"https://api.github.com/users/d-frey","html_url":"https://github.com/d-frey","followers_url":"https://api.github.com/users/d-frey/followers","following_url":"https://api.github.com/users/d-frey/following{/other_user}","gists_url":"https://api.github.com/users/d-frey/gists{/gist_id}","starred_url":"https://api.github.com/users/d-frey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/d-frey/subscriptions","organizations_url":"https://api.github.com/users/d-frey/orgs","repos_url":"https://api.github.com/users/d-frey/repos","events_url":"https://api.github.com/users/d-frey/events{/privacy}","received_events_url":"https://api.github.com/users/d-frey/received_events","type":"User","site_admin":false}],"milestone":null,"comments":10,"created_at":"2018-09-27T07:40:45Z","updated_at":"2018-10-03T20:43:52Z","closed_at":null,"author_association":"NONE","body":"Hi\r\n\r\nI was trying to use the parse_tree functionality together with the dynamic matching. \r\nI tried your dynamic match example with parse_tree::parse. but was not able to compile.\r\nFirstly the long_literal_mark struct needs a analyze_t type. \r\nthen I get errors about long_literal_id not having a apply or apply0.\r\n\r\nAny suggestions? \r\n"},"comment":{"url":"https://api.github.com/repos/taocpp/PEGTL/issues/comments/426795248","html_url":"https://github.com/taocpp/PEGTL/issues/129#issuecomment-426795248","issue_url":"https://api.github.com/repos/taocpp/PEGTL/issues/129","id":426795248,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNjc5NTI0OA==","user":{"login":"mkrupcale","id":13936020,"node_id":"MDQ6VXNlcjEzOTM2MDIw","avatar_url":"https://avatars1.githubusercontent.com/u/13936020?v=4","gravatar_id":"","url":"https://api.github.com/users/mkrupcale","html_url":"https://github.com/mkrupcale","followers_url":"https://api.github.com/users/mkrupcale/followers","following_url":"https://api.github.com/users/mkrupcale/following{/other_user}","gists_url":"https://api.github.com/users/mkrupcale/gists{/gist_id}","starred_url":"https://api.github.com/users/mkrupcale/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mkrupcale/subscriptions","organizations_url":"https://api.github.com/users/mkrupcale/orgs","repos_url":"https://api.github.com/users/mkrupcale/repos","events_url":"https://api.github.com/users/mkrupcale/events{/privacy}","received_events_url":"https://api.github.com/users/mkrupcale/received_events","type":"User","site_admin":false},"created_at":"2018-10-03T20:43:52Z","updated_at":"2018-10-03T20:43:52Z","author_association":"NONE","body":"> I'm not worried about an extra pointer in the input classes, they are not constructed/destructed during a parsing run (unlike a marker, etc.).\r\n\r\nFair enough, but the concept of the `Input` classes containing an internal state variable does not make much sense since this internal state has nothing to do with the `Input` class at all, and it (currently) only applies to `parse_tree` code.\r\n\r\n> I also think that from a user's point of view building a parse tree should be as consistent and as noninvasive as possible\r\n\r\nIt is more consistent with previous custom `match` implementations when one considers that the `match` function must accept all `States...` (at least for complex rules--simple rules need not worry about them at all). I'm not entirely sure what noninvasive means here because either way one must implement the custom `match` function to perform advanced matching, and using a `Rule` in a `parse_tree::parse` context is not the same as a default PEGTL `parse` because the expected output is different. Even so, the `parse_tree` `Rule`'s `match` function need only call the non-`parse_tree` `Rule`'s `match` function if they do not need access to the `parse_tree` state, and there will be no duplication of code.\r\n\r\n> the parse tree state is an internal detail.\r\n\r\nI'm not sure it's fair to consider the state an internal detail when implementing custom `match`, which as always must accept all `States...`. Sure the implementation of the state is an internal detail to the `parse_tree`, but the fact that it is passed to `match` is not so much an internal detail. This is apparent in the implementation of `parse_tree::parse` and can be simply documented in the usage of `parse_tree` since it's not expected to change how many internal state parameters are required.\r\n\r\n> Hiding it in the input class leaves a lot of general rules intact that are otherwise broken, which previously lead to all sort of hacks and work arounds.\r\n\r\nI'd consider hiding it in the `Input` class a hack of its own. What hacks were previously necessary with `parse_tree`? What general rules are broken without hiding the `parse_tree` state? I've used `parse_tree` with many different general rules without issue, and on a quick perusal, I don't see any built-in rules which expect a specific number of `States...`. Or do you mean custom, general rules external to PEGTL?\r\n\r\n> If you look at the commit where I added the internal state, it also simplifies the control class of the parse tree.\r\n\r\nYou mean by removing the implementation of `start`, `success`, `failure`, etc. functions for the `make_control` specialization when the `Rule` `Node` should not be stored and it is a leaf or parent of unselected leaves? This could already be done prior to 0346e101e0b5e101c8295bc6a17e922d3c4d1d2d.\r\n\r\nOr do you mean simplified relative to 057db2cec87ceb47bd8fef36257a9e48fc43cd90..d02c14107992cdc0edb558acb4b609d408cf2e8a? I don't really understand why these commits were even necessary because as far as I can tell they only implement the `match` function just like the `normal` `Control` while still passing the internal state argument. Wouldn't this already be achieved by using the `normal` `match` function, which is the default? Is not `Control2` template template parameter the same as `Control`? Doesn't this just come from `parse`,\r\n\r\n```C++\r\ntemplate< typename Rule,\r\n          template< typename... > class Action = nothing,\r\n          template< typename... > class Control = normal,\r\n          apply_mode A = apply_mode::ACTION,\r\n          rewind_mode M = rewind_mode::REQUIRED,\r\n          typename Input,\r\n          typename... States >\r\nbool parse( Input&& in, States&&... st )\r\n{\r\n   return Control< Rule >::template match< A, M, Action, Control >( in, st... );\r\n}\r\n```\r\n\r\nand thus `Control = Control2 = make_control` from `parse_tree`?\r\n\r\n> And, importantly, it allows a single grammar to be used for both basic parsing (possibly with other actions) without rules and actions knowing about the parse-tree-state, and for a parsing run that builds a parse tree with everything that is connected.\r\n\r\nActions already do not need to know about the `parse_tree` state, both prior to 057db2cec87ceb47bd8fef36257a9e48fc43cd90 and in my own implementation of actions. It is only the `match` function which is special, and as I've argued, it's a rather advanced yet integral part of PEGTL which I think requires some awareness by the (advanced) user that tries to write custom `Rule` or `Control` `match`. There is nothing preventing a single grammar from being used for both basic parsing and parsing with `parse_tree`--all that is required is to overload the `match` function inside of any custom `Rule`s which will be used in both. For simple `Rule`s, no overload is even necessary, and it will work with both.\r\n\r\n> Concerning PR #127...\r\n\r\nOkay, will do.\r\n\r\n## Possible alternative solution\r\n\r\nOne alternative solution which could make this explicit overload unnecessary is to instead move the `parse_tree` internal state argument to the end, after the user-provided `States...`, i.e.\r\n\r\n```C++\r\ntemplate< typename Rule,\r\n          typename Node,\r\n          template< typename... > class Selector = internal::store_all,\r\n          template< typename... > class Action = nothing,\r\n          template< typename... > class Control = normal,\r\n          typename Input,\r\n          typename... States >\r\nstd::unique_ptr< Node > parse( Input&& in, States&&... st )\r\n{\r\n   internal::state< Node > state;\r\n   if( !TAO_PEGTL_NAMESPACE::parse< Rule, Action, internal::make_control< Selector, Control >::template type >( in, st..., state ) ) {\r\n      return nullptr;\r\n   }\r\n...\r\n}\r\n```\r\n\r\nObviously also the order would have to be updated in `make_control` functions, but this is an internal detail which shouldn't be a problem.\r\n\r\nThen the user-defined custom `match` function which accepts an exact number of arguments need only provide a template parameter `ExtraStates...` at the end of the `match` function to take any additional arguments not needed by the user.\r\n\r\nSo instead of\r\n\r\n```C++\r\nstruct complex_custom_rule\r\n{\r\n   template< pegtl::apply_mode,\r\n             pegtl::rewind_mode,\r\n             template < typename... > class Action,\r\n             template < typename... > class Control,\r\n             typename InternalState,\r\n             typename Input >\r\n   static bool match( Input& in, const InternalState&, const std::string& id, const std::string& ) {...}\r\n}\r\n```\r\n\r\nthe user would only write the templated `match` function\r\n\r\n```C++\r\nstruct complex_custom_rule\r\n{\r\n   template< pegtl::apply_mode,\r\n             pegtl::rewind_mode,\r\n             template < typename... > class Action,\r\n             template < typename... > class Control,\r\n             typename Input,\r\n             typename... ExtraStates >\r\n   static bool match( Input& in, const std::string& id, const std::string&, ExtraStates&&... ) {...}\r\n}\r\n```\r\n\r\nwhich should work with both basic parse and `parse_tree` parse. Simple rules again will still work with both."}}
{"actor":{"display_login":"mkrupcale"}}