{"action":"created","issue":{"url":"https://api.github.com/repos/stleary/JSON-java/issues/288","repository_url":"https://api.github.com/repos/stleary/JSON-java","labels_url":"https://api.github.com/repos/stleary/JSON-java/issues/288/labels{/name}","comments_url":"https://api.github.com/repos/stleary/JSON-java/issues/288/comments","events_url":"https://api.github.com/repos/stleary/JSON-java/issues/288/events","html_url":"https://github.com/stleary/JSON-java/pull/288","id":178665095,"number":288,"title":"Bug fixes for XML Encoding and Decoding","user":{"login":"johnjaylward","id":383703,"avatar_url":"https://avatars.githubusercontent.com/u/383703?v=3","gravatar_id":"","url":"https://api.github.com/users/johnjaylward","html_url":"https://github.com/johnjaylward","followers_url":"https://api.github.com/users/johnjaylward/followers","following_url":"https://api.github.com/users/johnjaylward/following{/other_user}","gists_url":"https://api.github.com/users/johnjaylward/gists{/gist_id}","starred_url":"https://api.github.com/users/johnjaylward/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johnjaylward/subscriptions","organizations_url":"https://api.github.com/users/johnjaylward/orgs","repos_url":"https://api.github.com/users/johnjaylward/repos","events_url":"https://api.github.com/users/johnjaylward/events{/privacy}","received_events_url":"https://api.github.com/users/johnjaylward/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2016-09-22T16:56:27Z","updated_at":"2016-09-23T15:33:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/stleary/JSON-java/pulls/288","html_url":"https://github.com/stleary/JSON-java/pull/288","diff_url":"https://github.com/stleary/JSON-java/pull/288.diff","patch_url":"https://github.com/stleary/JSON-java/pull/288.patch"},"body":"This pull request is in reference to issue #285. It modifies how the XML parser handles string values and also updates the `escape` function to properly encode control characters using XML entities. Better reversibility is now available that entities are decoded on read. Previously they were encoded but not decoded.\r\n\r\n**What problem does this code solve?**\r\nFixes bug where control characters were being placed in XML as unescaped values. Also unescapes encoded values when reading XML into a JSONObject to support full reversibility of XML -> JSON -> XML.\r\n\r\n**Risks**\r\n_Control Character Changes_\r\nLow risk bug fix. No users should be expecting control characters output as unicode characters. They will now see ISO Control characters encoded using the &#xXXXX; notation as defined by the running JVMs `Character.isISOControl(int)` method.\r\n\r\n_Reversibility Changes_\r\nLow risk. Full reversibility is not offered with the XML class but improvements have been made in recent releases for Arrays (#170) and other minor items. Full reversibility is supposed to be offered through there JSONML class as it uses a normalized XML structure.\r\n\r\nHTML Entities are still not supported and will still break reversibility in both XML and JSONML classes, however regular XML entities are now fully supported and are properly converted between the two object types. Most XML parsers will fail if they see an HTML specific entity when run in strict mode, so this is not a major concern here.\r\n\r\nExample of broken HTML entities (not valid XML and should break most parsers):\r\n```xml\r\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<root><amount>10,00 &#x20ac;</amount><description>A&ccedil;&atilde;o V&aacute;lida</description></root>\r\n```\r\nThis would turn into JSON that looks like this:\r\n```json\r\n{\"root\":{\"amount\":\"10,00 €\",\"description\":\"A&ccedil;&atilde;o V&aacute;lida\"}}\r\n```\r\nIf the JSON is converted back to XML, proper XML would be generated like so:\r\n```xml\r\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<root><amount>10,00 €</amount><description>A&amp;ccedil;&amp;atilde;o V&amp;aacute;lida</description></root>\r\n```\r\nThe resulting XML does not match the original. This is probably the desired behavior, but should be noted. This has not changed in this pull request. This is current behaviour that is being noted here.\r\n\r\nThe parsing that has changed is in the case of valid XML entities. Now when valid XML entities are encountered in an XML document, they will be decoded upon conversion to JSON. The JSON -> XML conversion will still encode the entities (< > ' \" &) as well as now encoding control characters. The XML -> JSON -> XML -> JSON will have the proper values over multiple iterations.\r\n\r\n**Changes to the API?**\r\nYes, I removed the deprecation that I previously added to `XML.stringToValue`. I also added a new public static method to the XML class called `unescape` to complement the existing public static method `escape`.\r\n\r\n**Will this require a new release?**\r\nCan be rolled into the next release, pending comments.\r\n\r\n**Should the documentation be updated?**\r\nNo documentation updates required.\r\n\r\n**Does it break the unit tests?**\r\nNo, but new unit tests have been added in a separate pull request."},"comment":{"url":"https://api.github.com/repos/stleary/JSON-java/issues/comments/249224983","html_url":"https://github.com/stleary/JSON-java/pull/288#issuecomment-249224983","issue_url":"https://api.github.com/repos/stleary/JSON-java/issues/288","id":249224983,"user":{"login":"eduveks","id":5670920,"avatar_url":"https://avatars.githubusercontent.com/u/5670920?v=3","gravatar_id":"","url":"https://api.github.com/users/eduveks","html_url":"https://github.com/eduveks","followers_url":"https://api.github.com/users/eduveks/followers","following_url":"https://api.github.com/users/eduveks/following{/other_user}","gists_url":"https://api.github.com/users/eduveks/gists{/gist_id}","starred_url":"https://api.github.com/users/eduveks/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eduveks/subscriptions","organizations_url":"https://api.github.com/users/eduveks/orgs","repos_url":"https://api.github.com/users/eduveks/repos","events_url":"https://api.github.com/users/eduveks/events{/privacy}","received_events_url":"https://api.github.com/users/eduveks/received_events","type":"User","site_admin":false},"created_at":"2016-09-23T15:33:18Z","updated_at":"2016-09-23T15:33:18Z","body":"Good news!\r\n\r\n johnjaylward, to work correctly I was forced to change your code:\r\n\r\n`if (Character.isISOControl(cp)) ...`\r\n\r\nDo not works properly because the latin character with accents is not detected as special.\r\n\r\nBut if you use:\r\n\r\n`if (!asciiEncoder.canEncode((char)cp)) ...`\r\n\r\nWorks very well!\r\n\r\nBefore the for I declared the asciiEncoder:\r\n\r\n```\r\nCharsetEncoder asciiEncoder = Charset.forName(\"US-ASCII\").newEncoder();\r\nfor (final int cp : codePointIterator(string)) {\r\n...\r\n}\r\n```\r\nBy this way in the XMLTest.testJsonToXmlEscape now the result of XML.toString(json) will be:\r\n\r\n`<amount>10,00 &#x20ac;</amount><description>A&#xe7;&#xe3;o V&#xe1;lida&#x85;</description><xmlEntities>&quot; &apos; &amp; &lt; &gt;</xmlEntities>`\r\n\r\nIt's is perfect because ç, ã, á, ... will be now very well encoded!\r\n\r\nRight?"}}
{"actor":{"display_login":"eduveks"}}