{"action":"created","issue":{"url":"https://api.github.com/repos/adomokos/light-service/issues/65","repository_url":"https://api.github.com/repos/adomokos/light-service","labels_url":"https://api.github.com/repos/adomokos/light-service/issues/65/labels{/name}","comments_url":"https://api.github.com/repos/adomokos/light-service/issues/65/comments","events_url":"https://api.github.com/repos/adomokos/light-service/issues/65/events","html_url":"https://github.com/adomokos/light-service/issues/65","id":88745820,"number":65,"title":"The Need for Super-Organizers","user":{"login":"adomokos","id":80530,"avatar_url":"https://avatars.githubusercontent.com/u/80530?v=3","gravatar_id":"","url":"https://api.github.com/users/adomokos","html_url":"https://github.com/adomokos","followers_url":"https://api.github.com/users/adomokos/followers","following_url":"https://api.github.com/users/adomokos/following{/other_user}","gists_url":"https://api.github.com/users/adomokos/gists{/gist_id}","starred_url":"https://api.github.com/users/adomokos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adomokos/subscriptions","organizations_url":"https://api.github.com/users/adomokos/orgs","repos_url":"https://api.github.com/users/adomokos/repos","events_url":"https://api.github.com/users/adomokos/events{/privacy}","received_events_url":"https://api.github.com/users/adomokos/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-06-16T14:37:52Z","updated_at":"2016-02-26T18:19:29Z","closed_at":null,"body":"As our business requirements and code complexity grows, we see the need of weaving together Organizers just like we do with Actions. We have conventions: an Action can not call other Action(s), an Organizer can not invoke other Organizer(s). This is how we try to keep our code simple and easy to follow.\r\n\r\nWe have been calling multiple Organizers from controller actions which is a nice way of pushing out logic from Rails, but it seems it's not enough. We could build new organizers that reuse all the actions from the other organizers, but I am not sure that's the best solution, as the Organizer is responsible for one specific logic. For example, one Organizer handles Payment, the other communicates to an external API if the Payment was successful.\r\n\r\nWe've been considering \"Orchestrators\" or \"Super Organizers\", that would treat Organizers as higher level Actions. With this change the Organizers have a uniform interface, they might express `expects` and `promises` as well.\r\n\r\nDo you have any thoughts on this?"},"comment":{"url":"https://api.github.com/repos/adomokos/light-service/issues/comments/189409257","html_url":"https://github.com/adomokos/light-service/issues/65#issuecomment-189409257","issue_url":"https://api.github.com/repos/adomokos/light-service/issues/65","id":189409257,"user":{"login":"adomokos","id":80530,"avatar_url":"https://avatars.githubusercontent.com/u/80530?v=3","gravatar_id":"","url":"https://api.github.com/users/adomokos","html_url":"https://github.com/adomokos","followers_url":"https://api.github.com/users/adomokos/followers","following_url":"https://api.github.com/users/adomokos/following{/other_user}","gists_url":"https://api.github.com/users/adomokos/gists{/gist_id}","starred_url":"https://api.github.com/users/adomokos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adomokos/subscriptions","organizations_url":"https://api.github.com/users/adomokos/orgs","repos_url":"https://api.github.com/users/adomokos/repos","events_url":"https://api.github.com/users/adomokos/events{/privacy}","received_events_url":"https://api.github.com/users/adomokos/received_events","type":"User","site_admin":false},"created_at":"2016-02-26T18:19:28Z","updated_at":"2016-02-26T18:19:28Z","body":"As we are trying to build more complex ETLs with LightService, we started narrowing down the scope of \"orchestrators\".\r\n\r\nHere are the goals that I think is needed for a PR to close this issue:\r\n\r\n1. Regardless of complexity, we should be able to map the entire workflow in 1 organizer\r\n2. We need to handle iterators\r\n3. We need to handle conditionals\r\n\r\n<b>1. One workflow</b>\r\nAs complexity grows, we started creating child organizers with its own set of actions. We put them in separate files. When we wanted to check what's happening where, we had to open the spaghetti of organizers. Even worse, we wrapped an organizer call into an action as we iterated through a collection and called an organizer with each item.\r\n\r\n<b>2. Handle Iterators</b>\r\nI described that above already, we had to iterate over a collection and call a child organizer to deal with it. Big no-no.\r\n\r\n<b>3. Handle Conditionals</b>\r\nDepending on the state of the items in the context, I may or may not want to call and action or a set of actions from an organizer. Obviously I could use an action to make that decision, but that check is then hidden from the organizer. I have to dig to find that information.\r\n\r\nHere is what I came up with as a starting point:\r\n\r\n```ruby\r\nmodule ETL\r\n  class RunsProcess\r\n    extend LightService::Organizer\r\n\r\n    def self.call(run_id)\r\n      with(:run_id => run_id)\r\n        .reduce(actions)\r\n    end\r\n\r\n    def self.actions\r\n      [\r\n        Action1,\r\n        Action2,\r\n        Action3,\r\n        Class.new do\r\n          extend LightService::Action\r\n          extend LightService::Organizer\r\n          expects :run_id, :items\r\n\r\n          executed do |ctx|\r\n            ctx.items.each do |item|\r\n              with(:run_id => ctx.run_id,\r\n                     :item => item)\r\n                .reduce([\r\n                          SubAction1,\r\n                          SubAction2,\r\n                          SubAction3,\r\n                          SubAction4])\r\n            end\r\n          end\r\n        end\r\n      ]\r\n    end\r\n  end\r\nend\r\n```\r\n\r\nThis works, but it's very noisy. It's hard to see the what's really going on here.\r\n\r\nThis was proposed to make it better:\r\n```ruby\r\nmodule ETL\r\n  class RunsProcess\r\n    ...\r\n    def self.actions\r\n      [\r\n        Action1,\r\n        Action2,\r\n        Action3,\r\n        iterate(:documents).over([\r\n            SubAction1,\r\n            SubAction2,\r\n            SubAction3,\r\n            SubAction4])\r\n      ]\r\n    end\r\n  end\r\nend\r\n```\r\n\r\nThis is much better, and I think it addresses point 1 and 2 above. I think we would build this and add it to LS if others think it's a good idea.\r\n\r\nBut what are we doing with the conditionals?\r\n\r\nWould something like this work?\r\n```ruby\r\nmodule ETL\r\n  class RunsProcess\r\n    ...\r\n    def self.actions\r\n      [\r\n        Action1,\r\n        Action2,\r\n        Action3,\r\n        iterate(:documents).over([\r\n            SubAction1,\r\n            SubAction2,\r\n            SubAction3,\r\n            SubAction4]),\r\n        condition(-> { ctx.something.count == 2 }).with([\r\n            SubAction5,\r\n            SubAction6,\r\n            SubAction7\r\n        ]),\r\n        Action4,\r\n        Action5\r\n      ]\r\n    end\r\n  end\r\nend\r\n```\r\nWhat do people think about this? The lambda encapsulates the condition and it's invoked when that particular step is being processed.\r\n\r\nOr, we could just create an action that acts like a guard for a series of child actions like this:\r\n```ruby\r\nmodule ETL\r\n  class RunsProcess\r\n    ...\r\n    def self.actions\r\n      [\r\n        Action1,\r\n        Action2,\r\n        Action3,\r\n        iterate(:documents).over([\r\n            SubAction1,\r\n            SubAction2,\r\n            SubAction3,\r\n            SubAction4]),\r\n        [SkipThisIfBooleanLogicIsFalseAction,\r\n         SubAction5,\r\n         SubAction6,\r\n         SubAction7],\r\n        Action4,\r\n        Action5\r\n      ]\r\n    end\r\n  end\r\nend\r\n```\r\n\r\nAny feedback, comment is much appreciated."}}