{"action":"opened","issue":{"url":"https://api.github.com/repos/mozilla/voice-web/issues/1049","repository_url":"https://api.github.com/repos/mozilla/voice-web","labels_url":"https://api.github.com/repos/mozilla/voice-web/issues/1049/labels{/name}","comments_url":"https://api.github.com/repos/mozilla/voice-web/issues/1049/comments","events_url":"https://api.github.com/repos/mozilla/voice-web/issues/1049/events","html_url":"https://github.com/mozilla/voice-web/issues/1049","id":328733628,"node_id":"MDU6SXNzdWUzMjg3MzM2Mjg=","number":1049,"title":"How to best deal with corrections with regards to train/validation/test split","user":{"login":"jf99","id":7962377,"node_id":"MDQ6VXNlcjc5NjIzNzc=","avatar_url":"https://avatars0.githubusercontent.com/u/7962377?v=4","gravatar_id":"","url":"https://api.github.com/users/jf99","html_url":"https://github.com/jf99","followers_url":"https://api.github.com/users/jf99/followers","following_url":"https://api.github.com/users/jf99/following{/other_user}","gists_url":"https://api.github.com/users/jf99/gists{/gist_id}","starred_url":"https://api.github.com/users/jf99/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jf99/subscriptions","organizations_url":"https://api.github.com/users/jf99/orgs","repos_url":"https://api.github.com/users/jf99/repos","events_url":"https://api.github.com/users/jf99/events{/privacy}","received_events_url":"https://api.github.com/users/jf99/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-06-02T10:38:34Z","updated_at":"2018-06-02T10:38:34Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"According to #806 the data will be split into disjunct sets, so that\r\n- each reader is in no more than one set and\r\n- each sentence is in no more than one set (focus of this issue).\r\n\r\nI don't know how this is currently implemented, but it might be worth to spend some thoughts on it. Essentially, I see two possible approaches:\r\n- 1.) Split the sentences based on line number. E.g. numbers ending on 9 are test set, ending on 8 is validation set and the like.\r\n- 2.) Compute a hash value out of the sentence string and use it to separate the sets.\r\n\r\nThe choice for either one of the possibilities has consequences when it comes to editing the sentence database. Obviously, despite all proofreading, belated corrections will be necessary here and there. If you choose possibility 1, then it must be prohibited to insert or delete lines before existing ones. Otherwise the following sentences would move to another dataset.\r\n\r\nOn the other hand, if you choose possibility 2, even a minor correction (e.g. inserting a comma) might assign the corrected sentence to a new dataset. As a result, you would have the (almost) same sentences in two different datasets. To get around this problem, one could delete all recordings based on the uncorrected sentence. However, a drawback of this measure would be that even minor corrections could destroy a hundred of valuable recordings. So at least the corrector should be warned.\r\n\r\nIs there a better solution than the proposed?"}}
{"actor":{"display_login":"jf99"}}