{"action":"created","issue":{"url":"https://api.github.com/repos/bazaarvoice/jolt/issues/680","repository_url":"https://api.github.com/repos/bazaarvoice/jolt","labels_url":"https://api.github.com/repos/bazaarvoice/jolt/issues/680/labels{/name}","comments_url":"https://api.github.com/repos/bazaarvoice/jolt/issues/680/comments","events_url":"https://api.github.com/repos/bazaarvoice/jolt/issues/680/events","html_url":"https://github.com/bazaarvoice/jolt/issues/680","id":368282898,"node_id":"MDU6SXNzdWUzNjgyODI4OTg=","number":680,"title":"Slight modification to existing code, need help","user":{"login":"steveLandes","id":43789429,"node_id":"MDQ6VXNlcjQzNzg5NDI5","avatar_url":"https://avatars1.githubusercontent.com/u/43789429?v=4","gravatar_id":"","url":"https://api.github.com/users/steveLandes","html_url":"https://github.com/steveLandes","followers_url":"https://api.github.com/users/steveLandes/followers","following_url":"https://api.github.com/users/steveLandes/following{/other_user}","gists_url":"https://api.github.com/users/steveLandes/gists{/gist_id}","starred_url":"https://api.github.com/users/steveLandes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/steveLandes/subscriptions","organizations_url":"https://api.github.com/users/steveLandes/orgs","repos_url":"https://api.github.com/users/steveLandes/repos","events_url":"https://api.github.com/users/steveLandes/events{/privacy}","received_events_url":"https://api.github.com/users/steveLandes/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-10-09T15:58:48Z","updated_at":"2018-10-09T17:06:42Z","closed_at":null,"author_association":"NONE","body":"You helped me a lot with https://github.com/bazaarvoice/jolt/issues/673\r\n\r\nI have had a request to slightly modify the output from that spec.  I'll repost the relevant stuff here for convenience.\r\n\r\nInput: \r\n```\r\n{\r\n  \"data\": {\r\n    \"user\": {\r\n      \"active\": false,\r\n      \"displayName\": \"string\",\r\n      \"userId\": \"d3459481-fb1c-48bb-8685-40eb629e2ae1\",\r\n      \"externalId\": \"d3459481-fb1c-48bb-8685-40eb629e2ae1\",\r\n      \"id\": \"string\",\r\n      \"locale\": \"en-US\",\r\n      \"meta\": {\r\n        \"attributes\": [\r\n          \"string\"\r\n        ],\r\n        \"created\": \"2018-09-26T20:29:12.374Z\",\r\n        \"lastModified\": \"2018-09-26T20:29:12.374Z\",\r\n        \"location\": \"string\",\r\n        \"resourceType\": \"string\",\r\n        \"version\": \"string\"\r\n      },\r\n      \"nickName\": \"string\",\r\n      \"password\": \"string\",\r\n      \"preferredLanguage\": \"en\",\r\n      \"profileUrl\": \"string\",\r\n      \"timezone\": \"string\",\r\n      \"title\": \"string\",\r\n      \"userName\": \"username\",\r\n      \"userType\": \"string\"\r\n    },\r\n    \"products\": [\r\n      {\r\n        \"product\": \"product1\",\r\n        \"consents\": [\r\n          {\r\n            \"uuid\": \"d3459481-fb1c-48bb-8685-40eb629e2ae1\",\r\n            \"useCategoryCode\": \"tc\",\r\n            \"serviceCode\": \"pc\",\r\n            \"serviceFunctionCode\": \"pc-reg\",\r\n            \"country\": \"US\",\r\n            \"language\": \"en\",\r\n            \"status\": \"OptIn\",\r\n            \"consentDate\": \"2019-02-03\"\r\n          }\r\n        ],\r\n        \"object\": {}\r\n      },\r\n      {\r\n        \"product\": \"product2\",\r\n        \"consents\": [\r\n          {\r\n            \"uuid\": \"d3459481-fb1c-48bb-8685-40eb629e2ae2\",\r\n            \"useCategoryCode\": \"tc\",\r\n            \"serviceCode\": \"pc\",\r\n            \"serviceFunctionCode\": \"pc-reg\",\r\n            \"country\": \"US\",\r\n            \"language\": \"en\",\r\n            \"status\": \"OptIn\",\r\n            \"consentDate\": \"2019-02-03\"\r\n          }\r\n        ],\r\n        \"object\": {}\r\n      }\r\n    ]\r\n  }\r\n}\r\n```\r\n\r\nOld Spec:\r\n```\r\n[\r\n  {\r\n    // Build the \"final-ish\" output objects, but maintain the \r\n    //  doubly nested lists.\r\n    // Have to do it this way so that all the properties line \r\n    //  up / are right.\r\n    \"operation\": \"shift\",\r\n    \"spec\": {\r\n      \"data\": {\r\n        \"products\": {\r\n          \"*\": { // array index\r\n            \"consents\": {\r\n              \"*\": { // array index\r\n                //\r\n                // things that pass thru as is\r\n                \"consentDate\": \"outer[&3].inner[&1].consentedDate\",\r\n                \"serviceCode\": \"outer[&3].inner[&1].serviceCode\",\r\n                \"serviceFunctionCode\": \"outer[&3].inner[&1].serviceFunctionCode\",\r\n                \"useCategoryCode\": \"outer[&3].inner[&1].useCategoryCode\",\r\n                \"uuid\": \"outer[&3].inner[&1].uuid\",\r\n                //\r\n                // things that will require fixing later\r\n                \"status\": \"outer[&3].inner[&1].agreedRaw\",\r\n                \"@(2,product)\": \"outer[&3].inner[&1].requiredRaw\",\r\n                //\r\n                // things we have to pull from outside the \r\n                //  input products[] and consets[] arrays\r\n                \"@(4,user.meta.lastModified)\": \"outer[&3].inner[&1].lastModifiedDate\",\r\n                \"@(4,user.locale)\": \"outer[&3].inner[&1].locale\"\r\n              }\r\n            }\r\n          }\r\n        }\r\n      }\r\n    }\r\n  },\r\n  {\r\n    // Accumulate the \"final-ish\" objects into a consentData \r\n    //  array.\r\n    \"operation\": \"shift\",\r\n    \"spec\": {\r\n      \"outer\": {\r\n        \"*\": {\r\n          \"inner\": {\r\n            \"*\": \"consentData[]\"\r\n          }\r\n        }\r\n      }\r\n    }\r\n  },\r\n  {\r\n    \"operation\": \"modify-define-beta\",\r\n    \"spec\": {\r\n      // add the lookup values for \"agreed\" and \"required\" \r\n      //  the document, will use them later\r\n      \"lookups\": {\r\n        \"agreed\": {\r\n          \"OptIn\": true\r\n        },\r\n        \"required\": {\r\n          \"product1\": true\r\n        }\r\n      },\r\n      \"consentData\": {\r\n        \"*\": {\r\n          // hardcode the things that need to be hardcoded\r\n          \"consentType\": \"string\",\r\n          \"currentVersion\": \"v1\"\r\n        }\r\n      }\r\n    }\r\n  },\r\n  {\r\n    \"operation\": \"shift\",\r\n    \"spec\": {\r\n      \"consentData\": {\r\n        \"*\": {\r\n          // do the lookups for required and agreed\r\n          \"requiredRaw\": {\r\n            \"*\": { // \"product1\"?\r\n              // this will either lookup to true or do nothing\r\n              \"@(4,lookups.required.&)\": \"consentData[&3].required\"\r\n            }\r\n          },\r\n          \"agreedRaw\": {\r\n            \"*\": { // \"OptIn\" ?\r\n              // this will either lookup to true or do nothing\r\n              \"@(4,lookups.agreed.&)\": \"consentData[&3].agreed\"\r\n            }\r\n          },\r\n          // pass everything else thru\r\n          \"*\": \"consentData[&1].&\"\r\n        }\r\n      }\r\n    }\r\n  },\r\n  {\r\n    // If required and agreed did not evaluate to 'true', then\r\n    //  default them to false\r\n    \"operation\": \"modify-default-beta\",\r\n    \"spec\": {\r\n      \"consentData\": {\r\n        \"*\": {\r\n          \"required\": false,\r\n          \"agreed\": false\r\n        }\r\n      }\r\n    }\r\n  }\r\n]\r\n```\r\n\r\nOLD Desired output:\r\n```\r\n{\r\n  \"consentData\": [\r\n    {\r\n      \"agreed\": true, \r\n      \"consentType\": \"string\", \r\n      \"consentedDate\": \"2019-02-03\",\r\n      \"currentVersion\": \"v1\",\r\n      \"lastModifiedDate\": \"2018-10-01T15:31:19.429Z\", \r\n      \"locale\": \"string\", \r\n      \"required\": true,\r\n      \"serviceCode\": \"string\", \r\n      \"serviceFunctionCode\": \"string\", \r\n      \"useCategoryCode\": \"string\", \r\n      \"uuid\": \"d3459481-fb1c-48bb-8685-40eb629e2ae1\"\r\n    },\r\n    {\r\n      \"agreed\": true,\r\n      \"consentType\": \"string\",\r\n      \"consentedDate\": \"2019-02-03\",\r\n      \"currentVersion\": \"v1\",\r\n      \"lastModifiedDate\": \"2018-10-01T15:31:19.429Z\",\r\n      \"locale\": \"string\",\r\n      \"required\": false,\r\n      \"serviceCode\": \"string\",\r\n      \"serviceFunctionCode\": \"string\",\r\n      \"useCategoryCode\": \"string\",\r\n      \"uuid\": \"d3459481-fb1c-48bb-8685-40eb629e2ae2\"\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\nNEW desired output (new lines have comments to clearly show them):\r\n```\r\n{\r\n  \"userConsents\": {     //NEW LEVEL OF ORGANIZATION\r\n    \"userId\": \"d3459481-fb1c-48bb-8685-40eb629e2ae1\",  //NEW FIELD (data.user.userId)\r\n    \"userName\":  \"username\",  //NEW FIELD (data.user.username)\r\n    \"consentData\": [\r\n      {\r\n        \"agreed\": true,   \r\n        \"consentType\": \"string\", \r\n        \"consentedDate\": \"2019-02-03\",  \r\n        \"currentVersion\": \"v1\",  \r\n        \"lastModifiedDate\": \"2018-10-01T15:31:19.429Z\",\r\n        \"locale\": \"string\", \r\n        \"required\": true, \r\n        \"serviceCode\": \"string\", \r\n        \"serviceFunctionCode\": \"string\", \r\n        \"useCategoryCode\": \"string\",  \r\n        \"uuid\": \"d3459481-fb1c-48bb-8685-40eb629e2ae1\"\r\n      },\r\n      {\r\n        \"agreed\": true,\r\n        \"consentType\": \"string\",\r\n        \"consentedDate\": \"2019-02-03\",\r\n        \"currentVersion\": \"v1\",\r\n        \"lastModifiedDate\": \"2018-10-01T15:31:19.429Z\",\r\n        \"locale\": \"string\",\r\n        \"required\": false,\r\n        \"serviceCode\": \"string\",\r\n        \"serviceFunctionCode\": \"string\",\r\n        \"useCategoryCode\": \"string\",\r\n        \"uuid\": \"d3459481-fb1c-48bb-8685-40eb629e2ae2\"\r\n      }\r\n    ]\r\n  }\r\n}\r\n```\r\n\r\nBasically, I need to add the userConsents field, and shift everything up underneath it while adding 2 new fields underneath that.  No idea how to do it.\r\n"},"comment":{"url":"https://api.github.com/repos/bazaarvoice/jolt/issues/comments/428272654","html_url":"https://github.com/bazaarvoice/jolt/issues/680#issuecomment-428272654","issue_url":"https://api.github.com/repos/bazaarvoice/jolt/issues/680","id":428272654,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODI3MjY1NA==","user":{"login":"jashgala","id":16336864,"node_id":"MDQ6VXNlcjE2MzM2ODY0","avatar_url":"https://avatars3.githubusercontent.com/u/16336864?v=4","gravatar_id":"","url":"https://api.github.com/users/jashgala","html_url":"https://github.com/jashgala","followers_url":"https://api.github.com/users/jashgala/followers","following_url":"https://api.github.com/users/jashgala/following{/other_user}","gists_url":"https://api.github.com/users/jashgala/gists{/gist_id}","starred_url":"https://api.github.com/users/jashgala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jashgala/subscriptions","organizations_url":"https://api.github.com/users/jashgala/orgs","repos_url":"https://api.github.com/users/jashgala/repos","events_url":"https://api.github.com/users/jashgala/events{/privacy}","received_events_url":"https://api.github.com/users/jashgala/received_events","type":"User","site_admin":false},"created_at":"2018-10-09T17:06:42Z","updated_at":"2018-10-09T17:06:42Z","author_association":"NONE","body":"\r\nGiven the complexity of the operations you are performing, perhaps it may be simpler to write a new step. Here are a few pointers in the right direction:\r\n\r\n- As far as the commented portion is concerned, you would need to push the `data.user.userId` field into `userConsents.userId` and similar for `userName`. One problem you will face here is that in the first step of your spec, you are only preserving fields from `data.products.*` and not `data.user.*`. By default, the Shift transform removes stuff you don't move so you can add a block perserving it:\r\n\r\n```\r\n...\r\n\"data\": {\r\n   \"user\": {\r\n        \"*\": \"data.user.&\"\r\n    }\r\n}\r\n```\r\nKeep these preserved until the last step.\r\n\r\n- For moving consentData and the above data at the end you can then use the following spec:\r\n\r\n```\r\n{\r\n    \"operation\": \"shift\",\r\n    \"spec\": {\r\n        \"data\": {\r\n            \"user\": {\r\n                \"userId\" : \"userConsents.userId\",\r\n                \"userName\": \"userConsents.userName\"\r\n            }\r\n        },\r\n        \"consentData\": {\r\n            \"userConsents.consentData\"\r\n        }\r\n    }\r\n}\r\n```"}}
{"actor":{"display_login":"jashgala"}}