{"action":"created","issue":{"url":"https://api.github.com/repos/propershark/shark/issues/1","repository_url":"https://api.github.com/repos/propershark/shark","labels_url":"https://api.github.com/repos/propershark/shark/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/propershark/shark/issues/1/comments","events_url":"https://api.github.com/repos/propershark/shark/issues/1/events","html_url":"https://github.com/propershark/shark/issues/1","id":162559966,"number":1,"title":"Middleware?","user":{"login":"faultyserver","id":783733,"avatar_url":"https://avatars.githubusercontent.com/u/783733?v=3","gravatar_id":"","url":"https://api.github.com/users/faultyserver","html_url":"https://github.com/faultyserver","followers_url":"https://api.github.com/users/faultyserver/followers","following_url":"https://api.github.com/users/faultyserver/following{/other_user}","gists_url":"https://api.github.com/users/faultyserver/gists{/gist_id}","starred_url":"https://api.github.com/users/faultyserver/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/faultyserver/subscriptions","organizations_url":"https://api.github.com/users/faultyserver/orgs","repos_url":"https://api.github.com/users/faultyserver/repos","events_url":"https://api.github.com/users/faultyserver/events{/privacy}","received_events_url":"https://api.github.com/users/faultyserver/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-06-27T22:09:19Z","updated_at":"2016-06-29T05:58:30Z","closed_at":null,"body":"Middleware would work similarly to [Rack middleware](http://railscasts.com/episodes/151-rack-middleware), enabling runtime-configured event handling and data modification.\r\n\r\nOne use case for this feature would be to associate `Stops` and `Vehicles` to `Routes`. While this could also be done in `Source#update`, doing so in middleware would avoid the need to couple different `Source` instances together or have cache storage to perform object lookups."},"comment":{"url":"https://api.github.com/repos/propershark/shark/issues/comments/229263865","html_url":"https://github.com/propershark/shark/issues/1#issuecomment-229263865","issue_url":"https://api.github.com/repos/propershark/shark/issues/1","id":229263865,"user":{"login":"faultyserver","id":783733,"avatar_url":"https://avatars.githubusercontent.com/u/783733?v=3","gravatar_id":"","url":"https://api.github.com/users/faultyserver","html_url":"https://github.com/faultyserver","followers_url":"https://api.github.com/users/faultyserver/followers","following_url":"https://api.github.com/users/faultyserver/following{/other_user}","gists_url":"https://api.github.com/users/faultyserver/gists{/gist_id}","starred_url":"https://api.github.com/users/faultyserver/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/faultyserver/subscriptions","organizations_url":"https://api.github.com/users/faultyserver/orgs","repos_url":"https://api.github.com/users/faultyserver/repos","events_url":"https://api.github.com/users/faultyserver/events{/privacy}","received_events_url":"https://api.github.com/users/faultyserver/received_events","type":"User","site_admin":false},"created_at":"2016-06-29T05:58:30Z","updated_at":"2016-06-29T05:58:30Z","body":"Eh, not really. But the thought behind why is kind of complicated. I also pretty quickly changed my mind about where association should happen per #2, both of which I'll try to explain here.\r\n\r\n### Design\r\nThere are essentially three layers of responsibility in the system:\r\n\r\n- **Sourcing:** loading data from different sources (duh) and normalizing it to a consistent data structure (renaming fields, choosing primary keys, etc.).\r\n- **Management:** determining an \"active set\" of objects, persisting data (see #2), scheduling updates, triggering events.\r\n- **Publishing:** sending events over the network (just WAMP for now), logging, error handling (errors will be published as events), etc.\r\n\r\nMiddleware essentially generalizes the Publishing layer, creating an upward growing stack. This is a little backwards compared to Rack, where middleware is considered \"below\" the application:\r\n\r\n```\r\nShark:                            Rack:\r\n------                            -----\r\n...             ∧                 Rails/Sinatra\r\nWAMP Transport  ∧              ∨  Logger\r\nError Handler   ∧- Middleware -∨  Error Handler\r\nLogger          ∧              ∨  HTTP Transport\r\nManagement                     ∨  ...\r\nSourcing                          ...\r\n```\r\n\r\nI think of it this way because it better mirrors an Operating System. Sourcing like the kernel, dealing mostly with single objects and modifying their properties. Management is the standard library, dealing with multiple objects and triggering events. Middleware/Publishing is the user space, where interaction with the system can occur. Rack can use the same metaphor with its stack, it just happens to work the other way around.\r\n\r\n### Solutions\r\nNow, the principle behind why \"core shimming\" wouldn't really work in middleware: everything from Management up should be idempotent. Creating associations and shimming attributes both violate that principle.\r\n\r\nThe solution is to do these things in the Sourcing layer. Since Sources are pretty much agency-specific already, any agency that doesn't provide an attribute can shim that attribute when creating the data object in the Source.\r\n\r\nFor creating associations, a consequence of #2 means that Sources will have access to *all* objects, and associations can be created by polling the data storage adapter. Here, the associations should not embed objects, but rather include just the unique identifier of the associated object. Then, Middleware classes will also be able to poll the data storage adapter to embed objects when necessary. This reduces memory costs for highly-connected objects like Routes with Stops.\r\n\r\nThe data storage adapter is thus a transient module (not directly tied to any layer), acting as a static information mediary that can be guaranteed to exist, and does not need to passed around."}}
{"actor":{"display_login":"faultyserver"}}