{"action":"created","issue":{"url":"https://api.github.com/repos/garu/Data-Printer/issues/75","labels_url":"https://api.github.com/repos/garu/Data-Printer/issues/75/labels{/name}","comments_url":"https://api.github.com/repos/garu/Data-Printer/issues/75/comments","events_url":"https://api.github.com/repos/garu/Data-Printer/issues/75/events","html_url":"https://github.com/garu/Data-Printer/issues/75","id":117679133,"number":75,"title":"DDP adding too much magic to hash containers?","user":{"login":"TiMBuS","id":43602,"avatar_url":"https://avatars.githubusercontent.com/u/43602?v=3","gravatar_id":"","url":"https://api.github.com/users/TiMBuS","html_url":"https://github.com/TiMBuS","followers_url":"https://api.github.com/users/TiMBuS/followers","following_url":"https://api.github.com/users/TiMBuS/following{/other_user}","gists_url":"https://api.github.com/users/TiMBuS/gists{/gist_id}","starred_url":"https://api.github.com/users/TiMBuS/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TiMBuS/subscriptions","organizations_url":"https://api.github.com/users/TiMBuS/orgs","repos_url":"https://api.github.com/users/TiMBuS/repos","events_url":"https://api.github.com/users/TiMBuS/events{/privacy}","received_events_url":"https://api.github.com/users/TiMBuS/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-11-18T20:54:00Z","updated_at":"2016-01-26T13:38:18Z","closed_at":"2015-11-19T04:06:28Z","body":"So, I have a strange issue using DDP in combination with JSON::XS. I'm not entirely sure who is at fault here but I thought I'd have a better chance bugging you and not MLehmann..\r\n\r\nHere's the issue:\r\n```perl\r\nuse DDP;\r\nuse JSON; \r\nmy %teh = map {$_,1} qw(apple banana xylophone zephyr); \r\nnp %teh; \r\nsay JSON->new->canonical->encode(\\%teh);\r\n\r\n#Output:\r\n{\"banana\":1,\"xylophone\":1,\"apple\":1,\"zephyr\":1}\r\n```\r\nNote the hash is unordered.. even though 'canonical' should order the keys alphabetically. This problem does not occur if I don't pass the hash to DDP, which means something in DDP is changing it (and Devel::Peek confirms that DDP adds a ton of magic to the hash).\r\n\r\nLooking at JSON::XS source shows that it sorts differently based on SV magic, but I'm not enough of a wizard to know what it's actually doing. All I know is that someone is doing something wrong, somewhere."},"comment":{"url":"https://api.github.com/repos/garu/Data-Printer/issues/comments/175016780","html_url":"https://github.com/garu/Data-Printer/issues/75#issuecomment-175016780","issue_url":"https://api.github.com/repos/garu/Data-Printer/issues/75","id":175016780,"user":{"login":"dur-randir","id":823802,"avatar_url":"https://avatars.githubusercontent.com/u/823802?v=3","gravatar_id":"","url":"https://api.github.com/users/dur-randir","html_url":"https://github.com/dur-randir","followers_url":"https://api.github.com/users/dur-randir/followers","following_url":"https://api.github.com/users/dur-randir/following{/other_user}","gists_url":"https://api.github.com/users/dur-randir/gists{/gist_id}","starred_url":"https://api.github.com/users/dur-randir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dur-randir/subscriptions","organizations_url":"https://api.github.com/users/dur-randir/orgs","repos_url":"https://api.github.com/users/dur-randir/repos","events_url":"https://api.github.com/users/dur-randir/events{/privacy}","received_events_url":"https://api.github.com/users/dur-randir/received_events","type":"User","site_admin":false},"created_at":"2016-01-26T13:38:18Z","updated_at":"2016-01-26T13:38:18Z","body":"@garu, it's great that you're concerned about a quality of your modules. I'll try to summarize what's wrong with fieldhash().\r\n\r\n* While it's true that internal representation of a variable can change, perl itself almost exclusively looks at the variable's public flags to determine it's content. So, most XS modules won't care about internal representation, only about public flags.\r\n* fieldhash() not only changes internal representation, but also sets SVs_RMG flag for the SV, thus pessimising core codepaths related to hash/array access and confusing XS modules (like you've seen in Clone and JSON::XS). But setting this flag can easily be skipped - as the attached magic vtable has no valid callbacks and is used only as a private storage.\r\n* fieldhash() increased memory consumption comes not only from the magic marker attached to the variable, but it also stores weak backref - so, after call to p(), for each reference in the structure printed, there are additional 2*MAGIC + HE + 3*SV allocated and hanging around forever. That seems MAGIC+HE+3*SV too much for me)\r\n* Even after you remove a variable from fieldhash'd hash (or completely destroy that hash), variable still retains all MAGIC pointers with their additional data, so you free just HE+SV.\r\n\r\nI can write a custom module that will be much more like a direct 'exists $p->{_seen}->{_object_id($item)}' equivalent with two available trade-offs:\r\n* forever store MAGIC pointer in the variable, but avoid garbage collection pass afterwards\r\n* clean MAGIC pointer after p() traversal, consuming a bit more cpu+memory in the process, but releasing it afterwards\r\nReferences will still be extended to PVMG type, but without any flags set/other observable effects.\r\n\r\nBut this has one problem - this will be a XS module, and DDP currently has only pure-perl or core deps. I can make it optional and provide a common layer for both it and fieldhash() usage, if you approve this path."}}