{"action":"created","comment":{"url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/comments/195699652","pull_request_review_id":129121114,"id":195699652,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5NTY5OTY1Mg==","diff_hunk":"@@ -0,0 +1,85 @@\n+package com.gu.util\n+\n+import com.gu.util.zuora.RestRequestMaker.{ClientFailableOp, GenericError}\n+import com.gu.util.zuora.SafeQueryBuilder.Implicits._\n+import com.gu.util.zuora.SafeQueryBuilder.{OrTraverse, SafeQuery}\n+import org.scalatest._\n+import scalaz.{-\\/, \\/-}\n+\n+class SafeQueryBuilderEscapeTest extends FlatSpec with Matchers {\n+\n+  it should \"escape single quotes\" in {\n+    val actual = makeSafeStringIntoQueryLiteral(\"\"\"bobby tables'drop database students\"\"\")\n+    actual should be(\\/-(\"\"\"'bobby tables\\'drop database students'\"\"\"))\n+  }\n+\n+  it should \"escape double quotes\" in {\n+    val actual = makeSafeStringIntoQueryLiteral(\"\"\"a very \"nice\" query\"\"\")\n+    actual should be(\\/-(\"\"\"'a very \\\"nice\\\" query'\"\"\"))\n+  }\n+\n+  it should \"escape backslashes\" in {\n+    val actual = makeSafeStringIntoQueryLiteral(\"\"\"a very \\ query\"\"\")\n+    actual should be(\\/-(\"\"\"'a very \\\\ query'\"\"\"))\n+  }\n+\n+  it should \"escape single quotes double check the length\" in {\n+    val actual = makeSafeStringIntoQueryLiteral(\"\"\"'\"\"\")\n+    actual.map(_.length) should be(\\/-(4))\n+  }\n+\n+  it should \"escape double quotes double check the length\" in {\n+    val actual = makeSafeStringIntoQueryLiteral(\"\"\"\"\"\"\")\n+    actual.map(_.length) should be(\\/-(4))\n+  }\n+\n+  it should \"escape backslash double check the length\" in {\n+    val actual = makeSafeStringIntoQueryLiteral(\"\"\"\\\"\"\")\n+    actual.map(_.length) should be(\\/-(4))\n+  }\n+\n+  it should \"remove control chars - this is not 100% safe - we should reject completely\" in {","path":"lib/zuora/src/test/scala/com/gu/util/SafeQueryBuilderTest.scala","position":41,"original_position":41,"commit_id":"13d93a64b1c308ab8037e5cef8027c4515abe839","original_commit_id":"13d93a64b1c308ab8037e5cef8027c4515abe839","user":{"login":"johnduffell","id":7304387,"node_id":"MDQ6VXNlcjczMDQzODc=","avatar_url":"https://avatars1.githubusercontent.com/u/7304387?v=4","gravatar_id":"","url":"https://api.github.com/users/johnduffell","html_url":"https://github.com/johnduffell","followers_url":"https://api.github.com/users/johnduffell/followers","following_url":"https://api.github.com/users/johnduffell/following{/other_user}","gists_url":"https://api.github.com/users/johnduffell/gists{/gist_id}","starred_url":"https://api.github.com/users/johnduffell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johnduffell/subscriptions","organizations_url":"https://api.github.com/users/johnduffell/orgs","repos_url":"https://api.github.com/users/johnduffell/repos","events_url":"https://api.github.com/users/johnduffell/events{/privacy}","received_events_url":"https://api.github.com/users/johnduffell/received_events","type":"User","site_admin":false},"body":"not only that but the test shoudl check each one separately","created_at":"2018-06-15T10:59:24Z","updated_at":"2018-06-15T10:59:24Z","html_url":"https://github.com/guardian/zuora-auto-cancel/pull/114#discussion_r195699652","pull_request_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/comments/195699652"},"html":{"href":"https://github.com/guardian/zuora-auto-cancel/pull/114#discussion_r195699652"},"pull_request":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114"}},"in_reply_to_id":195687378},"pull_request":{"url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114","id":194551981,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk0NTUxOTgx","html_url":"https://github.com/guardian/zuora-auto-cancel/pull/114","diff_url":"https://github.com/guardian/zuora-auto-cancel/pull/114.diff","patch_url":"https://github.com/guardian/zuora-auto-cancel/pull/114.patch","issue_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/114","number":114,"state":"open","locked":false,"title":"build zuora queries in a fundamentally safe way","user":{"login":"johnduffell","id":7304387,"node_id":"MDQ6VXNlcjczMDQzODc=","avatar_url":"https://avatars1.githubusercontent.com/u/7304387?v=4","gravatar_id":"","url":"https://api.github.com/users/johnduffell","html_url":"https://github.com/johnduffell","followers_url":"https://api.github.com/users/johnduffell/followers","following_url":"https://api.github.com/users/johnduffell/following{/other_user}","gists_url":"https://api.github.com/users/johnduffell/gists{/gist_id}","starred_url":"https://api.github.com/users/johnduffell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johnduffell/subscriptions","organizations_url":"https://api.github.com/users/johnduffell/orgs","repos_url":"https://api.github.com/users/johnduffell/repos","events_url":"https://api.github.com/users/johnduffell/events{/privacy}","received_events_url":"https://api.github.com/users/johnduffell/received_events","type":"User","site_admin":false},"body":"At the moment we leave it up to callers to construct a zuora query by concatenating together strings.  This is a fundamentally broken and dangerous way to work - no application code should build \"strings\" which have higher level meaning, it should only happen in specifically written serialisation code, and well tested as such.\r\n\r\nThis PR introduces a query serialiser as a custom string interpolator.  It builds a Safe query object which can then be further serialised into Json by the normal js writes.  All of the inserts are classed as string literals, and as such it will insert the appropriate single quotes, plus escaping of such.\r\n\r\nAs an interesting aside which would be particularly relevant to @pvighi , this code is designed for the zoql query interface, where backslash is used to escape any special characters: https://knowledgecenter.zuora.com/DC_Developers/K_Zuora_Object_Query_Language/Filter_Statements#Reserved_and_Escaped_Characters\r\nHowever, in the \"zoql export\" api, which takes pretty much the same types of queries, in typical zuora style they have gone for escaping single quotes by doubling them up.  So we will need a separate serialiser for that at some point https://knowledgecenter.zuora.com/DC_Developers/M_Export_ZOQL/A_Select_Statement#Reserved_and_Escaped_Characters \r\n\r\nThis is about ready to go, please take a (another) look @pvighi @paulbrown1982 @twrichards @jacobwinch   Expecially interested in ease of use/clarity and safety.  There are a fair few concepts in there I'd rather avoid, apart from the readabilirty.  Custom string interpolation, varargs extracted separately with type classes on each arg, there's a lot of weirdness going on.","created_at":"2018-06-13T11:17:06Z","updated_at":"2018-06-15T10:59:24Z","closed_at":null,"merged_at":null,"merge_commit_sha":"01a66bbb2a05f2f7efb072c0c95027c2dce0d069","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114/commits","review_comments_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114/comments","review_comment_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/comments{/number}","comments_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/114/comments","statuses_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/statuses/13d93a64b1c308ab8037e5cef8027c4515abe839","head":{"label":"guardian:safe-zuora-queries","ref":"safe-zuora-queries","sha":"13d93a64b1c308ab8037e5cef8027c4515abe839","user":{"login":"guardian","id":164318,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE2NDMxOA==","avatar_url":"https://avatars0.githubusercontent.com/u/164318?v=4","gravatar_id":"","url":"https://api.github.com/users/guardian","html_url":"https://github.com/guardian","followers_url":"https://api.github.com/users/guardian/followers","following_url":"https://api.github.com/users/guardian/following{/other_user}","gists_url":"https://api.github.com/users/guardian/gists{/gist_id}","starred_url":"https://api.github.com/users/guardian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guardian/subscriptions","organizations_url":"https://api.github.com/users/guardian/orgs","repos_url":"https://api.github.com/users/guardian/repos","events_url":"https://api.github.com/users/guardian/events{/privacy}","received_events_url":"https://api.github.com/users/guardian/received_events","type":"Organization","site_admin":false},"repo":{"id":80627505,"node_id":"MDEwOlJlcG9zaXRvcnk4MDYyNzUwNQ==","name":"zuora-auto-cancel","full_name":"guardian/zuora-auto-cancel","owner":{"login":"guardian","id":164318,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE2NDMxOA==","avatar_url":"https://avatars0.githubusercontent.com/u/164318?v=4","gravatar_id":"","url":"https://api.github.com/users/guardian","html_url":"https://github.com/guardian","followers_url":"https://api.github.com/users/guardian/followers","following_url":"https://api.github.com/users/guardian/following{/other_user}","gists_url":"https://api.github.com/users/guardian/gists{/gist_id}","starred_url":"https://api.github.com/users/guardian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guardian/subscriptions","organizations_url":"https://api.github.com/users/guardian/orgs","repos_url":"https://api.github.com/users/guardian/repos","events_url":"https://api.github.com/users/guardian/events{/privacy}","received_events_url":"https://api.github.com/users/guardian/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/guardian/zuora-auto-cancel","description":"Lambdas covering supporter operations, mostly in life operations","fork":false,"url":"https://api.github.com/repos/guardian/zuora-auto-cancel","forks_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/forks","keys_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/keys{/key_id}","collaborators_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/teams","hooks_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/hooks","issue_events_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/events{/number}","events_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/events","assignees_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/assignees{/user}","branches_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/branches{/branch}","tags_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/tags","blobs_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/refs{/sha}","trees_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/trees{/sha}","statuses_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/statuses/{sha}","languages_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/languages","stargazers_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/stargazers","contributors_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/contributors","subscribers_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/subscribers","subscription_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/subscription","commits_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/commits{/sha}","git_commits_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/commits{/sha}","comments_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/comments{/number}","issue_comment_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/comments{/number}","contents_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/contents/{+path}","compare_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/compare/{base}...{head}","merges_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/merges","archive_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/downloads","issues_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues{/number}","pulls_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls{/number}","milestones_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/milestones{/number}","notifications_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/labels{/name}","releases_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/releases{/id}","deployments_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/deployments","created_at":"2017-02-01T14:25:47Z","updated_at":"2018-06-13T09:14:30Z","pushed_at":"2018-06-15T10:04:59Z","git_url":"git://github.com/guardian/zuora-auto-cancel.git","ssh_url":"git@github.com:guardian/zuora-auto-cancel.git","clone_url":"https://github.com/guardian/zuora-auto-cancel.git","svn_url":"https://github.com/guardian/zuora-auto-cancel","homepage":"","size":1179,"stargazers_count":3,"watchers_count":3,"language":"Scala","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":2,"license":null,"forks":1,"open_issues":2,"watchers":3,"default_branch":"master"}},"base":{"label":"guardian:master","ref":"master","sha":"ecc1de73006fc665b2c522205906e254935eb455","user":{"login":"guardian","id":164318,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE2NDMxOA==","avatar_url":"https://avatars0.githubusercontent.com/u/164318?v=4","gravatar_id":"","url":"https://api.github.com/users/guardian","html_url":"https://github.com/guardian","followers_url":"https://api.github.com/users/guardian/followers","following_url":"https://api.github.com/users/guardian/following{/other_user}","gists_url":"https://api.github.com/users/guardian/gists{/gist_id}","starred_url":"https://api.github.com/users/guardian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guardian/subscriptions","organizations_url":"https://api.github.com/users/guardian/orgs","repos_url":"https://api.github.com/users/guardian/repos","events_url":"https://api.github.com/users/guardian/events{/privacy}","received_events_url":"https://api.github.com/users/guardian/received_events","type":"Organization","site_admin":false},"repo":{"id":80627505,"node_id":"MDEwOlJlcG9zaXRvcnk4MDYyNzUwNQ==","name":"zuora-auto-cancel","full_name":"guardian/zuora-auto-cancel","owner":{"login":"guardian","id":164318,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE2NDMxOA==","avatar_url":"https://avatars0.githubusercontent.com/u/164318?v=4","gravatar_id":"","url":"https://api.github.com/users/guardian","html_url":"https://github.com/guardian","followers_url":"https://api.github.com/users/guardian/followers","following_url":"https://api.github.com/users/guardian/following{/other_user}","gists_url":"https://api.github.com/users/guardian/gists{/gist_id}","starred_url":"https://api.github.com/users/guardian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guardian/subscriptions","organizations_url":"https://api.github.com/users/guardian/orgs","repos_url":"https://api.github.com/users/guardian/repos","events_url":"https://api.github.com/users/guardian/events{/privacy}","received_events_url":"https://api.github.com/users/guardian/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/guardian/zuora-auto-cancel","description":"Lambdas covering supporter operations, mostly in life operations","fork":false,"url":"https://api.github.com/repos/guardian/zuora-auto-cancel","forks_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/forks","keys_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/keys{/key_id}","collaborators_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/teams","hooks_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/hooks","issue_events_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/events{/number}","events_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/events","assignees_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/assignees{/user}","branches_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/branches{/branch}","tags_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/tags","blobs_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/refs{/sha}","trees_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/trees{/sha}","statuses_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/statuses/{sha}","languages_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/languages","stargazers_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/stargazers","contributors_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/contributors","subscribers_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/subscribers","subscription_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/subscription","commits_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/commits{/sha}","git_commits_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/git/commits{/sha}","comments_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/comments{/number}","issue_comment_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/comments{/number}","contents_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/contents/{+path}","compare_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/compare/{base}...{head}","merges_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/merges","archive_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/downloads","issues_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues{/number}","pulls_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls{/number}","milestones_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/milestones{/number}","notifications_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/labels{/name}","releases_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/releases{/id}","deployments_url":"https://api.github.com/repos/guardian/zuora-auto-cancel/deployments","created_at":"2017-02-01T14:25:47Z","updated_at":"2018-06-13T09:14:30Z","pushed_at":"2018-06-15T10:04:59Z","git_url":"git://github.com/guardian/zuora-auto-cancel.git","ssh_url":"git@github.com:guardian/zuora-auto-cancel.git","clone_url":"https://github.com/guardian/zuora-auto-cancel.git","svn_url":"https://github.com/guardian/zuora-auto-cancel","homepage":"","size":1179,"stargazers_count":3,"watchers_count":3,"language":"Scala","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":2,"license":null,"forks":1,"open_issues":2,"watchers":3,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114"},"html":{"href":"https://github.com/guardian/zuora-auto-cancel/pull/114"},"issue":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/114"},"comments":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/issues/114/comments"},"review_comments":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114/comments"},"review_comment":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/pulls/114/commits"},"statuses":{"href":"https://api.github.com/repos/guardian/zuora-auto-cancel/statuses/13d93a64b1c308ab8037e5cef8027c4515abe839"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"johnduffell"}}