{"action":"created","issue":{"url":"https://api.github.com/repos/covarep/covarep/issues/105","repository_url":"https://api.github.com/repos/covarep/covarep","labels_url":"https://api.github.com/repos/covarep/covarep/issues/105/labels{/name}","comments_url":"https://api.github.com/repos/covarep/covarep/issues/105/comments","events_url":"https://api.github.com/repos/covarep/covarep/issues/105/events","html_url":"https://github.com/covarep/covarep/issues/105","id":296573341,"number":105,"title":"Question: Memory-dependent vectorization of pitch_srh ","user":{"login":"twoertwein","id":6618166,"avatar_url":"https://avatars0.githubusercontent.com/u/6618166?v=4","gravatar_id":"","url":"https://api.github.com/users/twoertwein","html_url":"https://github.com/twoertwein","followers_url":"https://api.github.com/users/twoertwein/followers","following_url":"https://api.github.com/users/twoertwein/following{/other_user}","gists_url":"https://api.github.com/users/twoertwein/gists{/gist_id}","starred_url":"https://api.github.com/users/twoertwein/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twoertwein/subscriptions","organizations_url":"https://api.github.com/users/twoertwein/orgs","repos_url":"https://api.github.com/users/twoertwein/repos","events_url":"https://api.github.com/users/twoertwein/events{/privacy}","received_events_url":"https://api.github.com/users/twoertwein/received_events","type":"User","site_admin":false},"labels":[{"id":49152367,"url":"https://api.github.com/repos/covarep/covarep/labels/Enhancement","name":"Enhancement","color":"84b6eb","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-02-13T00:29:21Z","updated_at":"2018-03-15T16:53:05Z","closed_at":"2018-03-15T16:53:05Z","author_association":"CONTRIBUTOR","body":"would it be worth it detecting the available memory and estimating the expected memory usage to vectorize the fft in pitch_srh (again)?https://github.com/covarep/covarep/blob/4d25cb1e963f9ba4f03c422cede32af9875d7591/glottalsource/pitch_srh.m#L130\r\n\r\nIf  not enough memory is available, we could have a loop (with a few iterations) over vectoricable blocks which fit into memory. This should (1) be faster than processing every frame individually and (2) it will make it possible to process \"really\" long audio files with little memory. To achieve (2) the new loop would need to span until this line https://github.com/covarep/covarep/blob/4d25cb1e963f9ba4f03c422cede32af9875d7591/glottalsource/pitch_srh.m#L159 I have audio files which when processed with the current pitch_srh exceed 16GB of memory.\r\n\r\nDetecting the available memory is platform-depended but we could use a solution similar to https://www.mathworks.com/matlabcentral/fileexchange/45598-memoryinfo (using matlab's built-in function `memory` on Windows and parse the output of `free` on unix-like OSes).\r\n\r\nI could implement that but first wanted to ask whether that would be welcomed (`free` should exist on probably all platforms but there might be cases where it doesn't exist). In cases we cannot estimate the available memory, we could assume that everything fits in the memory (most people process short files). \r\n\r\nIndependent of this, there should be a 'else break'-block in following line to avoid a second iteration which does exactly the same: https://github.com/covarep/covarep/blob/4d25cb1e963f9ba4f03c422cede32af9875d7591/glottalsource/pitch_srh.m#L156 "},"comment":{"url":"https://api.github.com/repos/covarep/covarep/issues/comments/373446837","html_url":"https://github.com/covarep/covarep/issues/105#issuecomment-373446837","issue_url":"https://api.github.com/repos/covarep/covarep/issues/105","id":373446837,"user":{"login":"twoertwein","id":6618166,"avatar_url":"https://avatars0.githubusercontent.com/u/6618166?v=4","gravatar_id":"","url":"https://api.github.com/users/twoertwein","html_url":"https://github.com/twoertwein","followers_url":"https://api.github.com/users/twoertwein/followers","following_url":"https://api.github.com/users/twoertwein/following{/other_user}","gists_url":"https://api.github.com/users/twoertwein/gists{/gist_id}","starred_url":"https://api.github.com/users/twoertwein/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twoertwein/subscriptions","organizations_url":"https://api.github.com/users/twoertwein/orgs","repos_url":"https://api.github.com/users/twoertwein/repos","events_url":"https://api.github.com/users/twoertwein/events{/privacy}","received_events_url":"https://api.github.com/users/twoertwein/received_events","type":"User","site_admin":false},"created_at":"2018-03-15T16:53:04Z","updated_at":"2018-03-15T16:53:04Z","author_association":"CONTRIBUTOR","body":"if each frame (or a block of frames) would be processed at a time, the 2nd iteration of pitch track estimation is not possible (`specMat` would not contain the entire data).\r\n\r\nI will shortly create two pull request:\r\n1) Add a start/stop option to `COVAREP_feature_formant_extraction_perfile` to only extract features in a specified interval. Many long audio files have 'nothing' in the beginning and end. That helps to easily shorten the audio signal.\r\n2) Re-use variables to allow MATLAB to use in-place operations in `pitch_srh`. Slightly fewer memory allocations need to be done. Unfortunately, it doesn't reduce the peak memory usage of `pitch_srh` (`specMat` is the memory bottleneck)."}}
{"actor":{"display_login":"twoertwein"}}