{"action":"opened","issue":{"url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/issues/12","repository_url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse","labels_url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/issues/12/labels{/name}","comments_url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/issues/12/comments","events_url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/issues/12/events","html_url":"https://github.com/DistrictDataLabs/partisan-discourse/issues/12","id":169999657,"number":12,"title":"Get a model build process going for multiple models in the database","user":{"login":"lauralorenz","id":5643703,"avatar_url":"https://avatars.githubusercontent.com/u/5643703?v=3","gravatar_id":"","url":"https://api.github.com/users/lauralorenz","html_url":"https://github.com/lauralorenz","followers_url":"https://api.github.com/users/lauralorenz/followers","following_url":"https://api.github.com/users/lauralorenz/following{/other_user}","gists_url":"https://api.github.com/users/lauralorenz/gists{/gist_id}","starred_url":"https://api.github.com/users/lauralorenz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lauralorenz/subscriptions","organizations_url":"https://api.github.com/users/lauralorenz/orgs","repos_url":"https://api.github.com/users/lauralorenz/repos","events_url":"https://api.github.com/users/lauralorenz/events{/privacy}","received_events_url":"https://api.github.com/users/lauralorenz/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/labels/priority:%20high","name":"priority: high","color":"d93f0b"},{"url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/labels/ready","name":"ready","color":"cccccc"},{"url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/labels/type:%20feature","name":"type: feature","color":"c2e0c6"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/milestones/2","html_url":"https://github.com/DistrictDataLabs/partisan-discourse/milestones/PyData%20DC%202016%20Release","labels_url":"https://api.github.com/repos/DistrictDataLabs/partisan-discourse/milestones/2/labels","id":1930253,"number":2,"title":"PyData DC 2016 Release","description":"Milestone for new demo features for PyData DC 2016.","creator":{"login":"lauralorenz","id":5643703,"avatar_url":"https://avatars.githubusercontent.com/u/5643703?v=3","gravatar_id":"","url":"https://api.github.com/users/lauralorenz","html_url":"https://github.com/lauralorenz","followers_url":"https://api.github.com/users/lauralorenz/followers","following_url":"https://api.github.com/users/lauralorenz/following{/other_user}","gists_url":"https://api.github.com/users/lauralorenz/gists{/gist_id}","starred_url":"https://api.github.com/users/lauralorenz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lauralorenz/subscriptions","organizations_url":"https://api.github.com/users/lauralorenz/orgs","repos_url":"https://api.github.com/users/lauralorenz/repos","events_url":"https://api.github.com/users/lauralorenz/events{/privacy}","received_events_url":"https://api.github.com/users/lauralorenz/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":0,"state":"open","created_at":"2016-08-08T18:27:50Z","updated_at":"2016-08-08T19:04:47Z","due_on":"2016-10-07T04:00:00Z","closed_at":null},"comments":0,"created_at":"2016-08-08T19:04:47Z","updated_at":"2016-08-08T19:04:47Z","closed_at":null,"body":"Right now we have a management command [manage.py train](https://github.com/DistrictDataLabs/partisan-discourse/blob/2fb7a5ff89ab50cc6b6d309c2b25cdafaf6a6f47/arbiter/management/commands/train.py#L37) that supports building a model [against a corpus from disk](https://github.com/DistrictDataLabs/partisan-discourse/blob/2fb7a5ff89ab50cc6b6d309c2b25cdafaf6a6f47/arbiter/management/commands/train.py#L70) and saving it to the database using the `arbiter` `Estimator` and `Score` models. With this issue, we want to be able to create models against documents in the database, and support the ability for multiple models to use the same documents and know which documents they used.\r\n\r\nI think the intended implementation strategy for this is to \r\n- build a reader like/subclassed from/expanding [`TranscriptCorpusReader`](https://github.com/DistrictDataLabs/partisan-discourse/blob/2fb7a5ff89ab50cc6b6d309c2b25cdafaf6a6f47/corpus/reader.py#L38) that can pull a corpus from the database via a queryset as opposed to from disk\r\n- provide a way for the model management command to ingest a queryset specification and utilize it [when constructing the corpus](https://github.com/DistrictDataLabs/partisan-discourse/blob/2fb7a5ff89ab50cc6b6d309c2b25cdafaf6a6f47/arbiter/management/commands/train.py#L82-L83)\r\n- add necessary features to the build process and attributes to the `Estimator` model to be able to track an `Estimator`s dependent documents\r\n    - may want to consider what happens if the documents change underneath the `Estimator` instance; do we version the documents or each `Estimator`s input data? Do we care about reproducibility of each `Estimator` at this granularity?\r\n\r\nThis issue is closed when a model build process can be run against the stored documents in the database and track which documents were used for each model."}}
{"actor":{"display_login":"lauralorenz"}}