{"action":"created","issue":{"url":"https://api.github.com/repos/open-craft/blockstore/issues/16","repository_url":"https://api.github.com/repos/open-craft/blockstore","labels_url":"https://api.github.com/repos/open-craft/blockstore/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/open-craft/blockstore/issues/16/comments","events_url":"https://api.github.com/repos/open-craft/blockstore/issues/16/events","html_url":"https://github.com/open-craft/blockstore/issues/16","id":378888685,"node_id":"MDU6SXNzdWUzNzg4ODg2ODU=","number":16,"title":"Course publish granularity and implications for Bundle modeling","user":{"login":"ormsbee","id":41625,"node_id":"MDQ6VXNlcjQxNjI1","avatar_url":"https://avatars1.githubusercontent.com/u/41625?v=4","gravatar_id":"","url":"https://api.github.com/users/ormsbee","html_url":"https://github.com/ormsbee","followers_url":"https://api.github.com/users/ormsbee/followers","following_url":"https://api.github.com/users/ormsbee/following{/other_user}","gists_url":"https://api.github.com/users/ormsbee/gists{/gist_id}","starred_url":"https://api.github.com/users/ormsbee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ormsbee/subscriptions","organizations_url":"https://api.github.com/users/ormsbee/orgs","repos_url":"https://api.github.com/users/ormsbee/repos","events_url":"https://api.github.com/users/ormsbee/events{/privacy}","received_events_url":"https://api.github.com/users/ormsbee/received_events","type":"User","site_admin":false},"labels":[{"id":1135555593,"node_id":"MDU6TGFiZWwxMTM1NTU1NTkz","url":"https://api.github.com/repos/open-craft/blockstore/labels/arch","name":"arch","color":"efaac7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2018-11-08T19:46:19Z","updated_at":"2018-11-21T15:47:51Z","closed_at":null,"author_association":"COLLABORATOR","body":"Capturing notes from the conversation @bradenmacdonald, @symbolist, and I had with @JAAkana on the subject of course publishing granularity.\r\n\r\nThe types of publish that exist today:\r\n* publish unit (most commonly used)\r\n* publish section (on outline page)\r\n* publish subsection (on outline page)\r\n* publish entire course (only via import)\r\n\r\nThe primitives we have to play with are BundleVersions, Links, and Files. We want to model the data so that:\r\n\r\n1. Common use cases with existing courses are operationally simple.\r\n2. We are set up for a long term vision of more dynamic courses content.\r\n3. Import/export is still straightforward.\r\n\r\nMaking publishes happen at a level more granular than a Bundle is possible but might introduce complications -- we could make it so that you partially commit a Draft, e.g. one file that represents the Unit, but that could get complicated with dependencies.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/open-craft/blockstore/issues/comments/440713556","html_url":"https://github.com/open-craft/blockstore/issues/16#issuecomment-440713556","issue_url":"https://api.github.com/repos/open-craft/blockstore/issues/16","id":440713556,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MDcxMzU1Ng==","user":{"login":"symbolist","id":5305552,"node_id":"MDQ6VXNlcjUzMDU1NTI=","avatar_url":"https://avatars2.githubusercontent.com/u/5305552?v=4","gravatar_id":"","url":"https://api.github.com/users/symbolist","html_url":"https://github.com/symbolist","followers_url":"https://api.github.com/users/symbolist/followers","following_url":"https://api.github.com/users/symbolist/following{/other_user}","gists_url":"https://api.github.com/users/symbolist/gists{/gist_id}","starred_url":"https://api.github.com/users/symbolist/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/symbolist/subscriptions","organizations_url":"https://api.github.com/users/symbolist/orgs","repos_url":"https://api.github.com/users/symbolist/repos","events_url":"https://api.github.com/users/symbolist/events{/privacy}","received_events_url":"https://api.github.com/users/symbolist/received_events","type":"User","site_admin":false},"created_at":"2018-11-21T15:47:51Z","updated_at":"2018-11-21T15:47:51Z","author_association":"MEMBER","body":"@bradenmacdonald \r\n\r\n> What about conceptually thinking of one XBlock per file/folder, with no bundle.json file or other shared state among XBlocks in the bundle? Then you can easily publish one file/folder at a time with the existing architecture and still have bundles that are at a reasonable size.\r\n\r\nHmm. Isn't this conceptually similar to \"why not do the organization at the collection level and import/export happens at the collection level too? So in this model, one bundle will have at most one xblock/course outline/pathway type entity.\"? where XBlock folder == Bundle and Bundle == Collection. I think we have already talked about moving the data out from bundle.json to a higher level so we can definitely make the Bundle lighter.\r\n\r\n> An idea that @bradenmacdonald had while we were hashing things out on a whiteboard yesterday was to:\r\n> - Break all leaf blocks into their own Bundles, as you were advocating for.\r\n> - Consolidate all containers together as separate files in a single Bundle.\r\n\r\nIf we do have to group things together I do like this grouping a lot more because higher-level entities are going to be more course specific and less likely to be \"shareable\"? They also likely will not have static assets associated with them which simplifies versioning and partial publishing of the Bundle.\r\n\r\n@ormsbee \r\n\r\n> 360K data isn't great for the network or CPU, but it's manageable (say ~10-20ms range) if we're careful about our representation. But we start to enter disk usage patterns that echo Split's structure doc.\r\n\r\nIs this because Studio needs to load the structure of the whole course frequently? (My assumption was that it would only need to load subsets of the tree at a time.)\r\n\r\nAlso is it correct that unlike with Split this will only impact Studio and not LMS which will use an optimized/compiled version of the data?\r\n\r\n>  The life of a course easily sees hundreds and possibly thousands of small edits. Pretty soon we're at > 1 GB for just the accumulated Link data about this Course Bundle.\r\n\r\nHmm. Lets say I import a new large course with 10K XBlocks. If I go into Studio and edit a few XBlocks in a Unit and publish it, the total number of new BundleVersions is going to be number of XBlocks updated + the nodes up the tree. So the average new BundlesVersions per publish should be 4-5 (and a few KB of metadata)? I think the 1GB number assumes we are updating all the nodes in the tree; do we need to do that?"}}
{"actor":{"display_login":"symbolist"}}