{"action":"created","issue":{"url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/issues/1","repository_url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN","labels_url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/issues/1/comments","events_url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/issues/1/events","html_url":"https://github.com/vanlummelhuizen/ASL-ELAN/issues/1","id":132143397,"number":1,"title":"Enhance corpus updater script","user":{"login":"richardbank","id":10399929,"avatar_url":"https://avatars.githubusercontent.com/u/10399929?v=3","gravatar_id":"","url":"https://api.github.com/users/richardbank","html_url":"https://github.com/richardbank","followers_url":"https://api.github.com/users/richardbank/followers","following_url":"https://api.github.com/users/richardbank/following{/other_user}","gists_url":"https://api.github.com/users/richardbank/gists{/gist_id}","starred_url":"https://api.github.com/users/richardbank/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/richardbank/subscriptions","organizations_url":"https://api.github.com/users/richardbank/orgs","repos_url":"https://api.github.com/users/richardbank/repos","events_url":"https://api.github.com/users/richardbank/events{/privacy}","received_events_url":"https://api.github.com/users/richardbank/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/labels/Top%20priority","name":"Top priority","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2016-02-08T13:36:46Z","updated_at":"2016-04-05T07:51:21Z","closed_at":null,"body":"Every night, Signbank creates a fresh ECV to reflect any changes in the lexical database. Changes in glosses in the corpus, however, are only updated when eaf files are opened and ELAN downloads the new ECV. This can lead to outdated search results and frequency counts.\r\n\r\n[elan-ecv-updater.zip](https://github.com/vanlummelhuizen/ASL-ELAN/files/100717/elan-ecv-updater.zip) (source code with Han) does two things:\r\n\r\n* if a gloss value differs from an ECV value (on the basis of the ECV-link in the annotation), the ECV value is adopted.\r\n* if an annotation does not contain an ECV-link, an ECV-link is created (on the basis of the gloss value; if no corresponding value is found in the ECV, no link is created)\r\n\r\n`Elan-ecv-updater.jar` depends on `xercesImpl-2.11.0.jar` and `xml-apis-1.4.01.jar`. It is called by\r\n\r\n`java -Xmx600m -cp elan-ecv-updater.jar;xercesImpl-2.11.0.jar;xml-apis-1.4.01.jar mpi.eudico.client.annotator.util.CorpusECVUpdater -L nld Inputfolder Outputfolder`\r\n\r\nwhere `-Xmx600m` specifies the java heap space at 600MB, and `-L nld` enforces Dutch as the content language.\r\n\r\nThis works well, but there is room for improvement. Ideally this script should run unsupervised every night, so that the whole corpus reflects the current state of the information in the lexical database. There already is an svn-checkout on the Signbank server. \r\n\r\nIssues/enhancements:\r\n* eaf files are now completely rewritten, regardless whether anything has changed in a file. This results in a complete renewal of the whole corpus every night, making it harder to track specific changes in the svn-history. Files should only be written when something actually changes.\r\n* changes in document date (in the ANNOTATION_DOCUMENT tag) should not be regarded as changes with regard to the previous point.\r\n* In case this script will be incorporated in ELAN at a certain point, make sure the java space heap is user configurable. 600MB is currently fine for the Corpus NGT, but this may need to be upgraded when the corpus further grows."},"comment":{"url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/issues/comments/205707340","html_url":"https://github.com/vanlummelhuizen/ASL-ELAN/issues/1#issuecomment-205707340","issue_url":"https://api.github.com/repos/vanlummelhuizen/ASL-ELAN/issues/1","id":205707340,"user":{"login":"richardbank","id":10399929,"avatar_url":"https://avatars.githubusercontent.com/u/10399929?v=3","gravatar_id":"","url":"https://api.github.com/users/richardbank","html_url":"https://github.com/richardbank","followers_url":"https://api.github.com/users/richardbank/followers","following_url":"https://api.github.com/users/richardbank/following{/other_user}","gists_url":"https://api.github.com/users/richardbank/gists{/gist_id}","starred_url":"https://api.github.com/users/richardbank/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/richardbank/subscriptions","organizations_url":"https://api.github.com/users/richardbank/orgs","repos_url":"https://api.github.com/users/richardbank/repos","events_url":"https://api.github.com/users/richardbank/events{/privacy}","received_events_url":"https://api.github.com/users/richardbank/received_events","type":"User","site_admin":false},"created_at":"2016-04-05T07:51:21Z","updated_at":"2016-04-05T07:51:21Z","body":"Some comments on the GUI:\r\n* in the first window ('Update transcriptions for ECVs'), it says 'in the same folder as the media files'. This should read 'in the same folder as the source files', right?\r\n* add a progress bar of some kind. Now, after pressing 'start', nothing happens, and there's no way of telling whether the script is running or not.\r\n* after quite a while, a 'Process Report' window opens with the message 'Finished processing. Exit status: 0'. What is Exit status: 0? Maybe just include here what would be written to a text file when using '>' on a command line? (i.e. the list of changes, including the file names)\r\n\r\n"}}