{"action":"created","issue":{"url":"https://api.github.com/repos/privefl/bigstatsr/issues/23","repository_url":"https://api.github.com/repos/privefl/bigstatsr","labels_url":"https://api.github.com/repos/privefl/bigstatsr/issues/23/labels{/name}","comments_url":"https://api.github.com/repos/privefl/bigstatsr/issues/23/comments","events_url":"https://api.github.com/repos/privefl/bigstatsr/issues/23/events","html_url":"https://github.com/privefl/bigstatsr/issues/23","id":333505462,"node_id":"MDU6SXNzdWUzMzM1MDU0NjI=","number":23,"title":"fast method to write sparse matrix to file","user":{"login":"ChiWPak","id":17429954,"node_id":"MDQ6VXNlcjE3NDI5OTU0","avatar_url":"https://avatars3.githubusercontent.com/u/17429954?v=4","gravatar_id":"","url":"https://api.github.com/users/ChiWPak","html_url":"https://github.com/ChiWPak","followers_url":"https://api.github.com/users/ChiWPak/followers","following_url":"https://api.github.com/users/ChiWPak/following{/other_user}","gists_url":"https://api.github.com/users/ChiWPak/gists{/gist_id}","starred_url":"https://api.github.com/users/ChiWPak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ChiWPak/subscriptions","organizations_url":"https://api.github.com/users/ChiWPak/orgs","repos_url":"https://api.github.com/users/ChiWPak/repos","events_url":"https://api.github.com/users/ChiWPak/events{/privacy}","received_events_url":"https://api.github.com/users/ChiWPak/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2018-06-19T03:19:31Z","updated_at":"2018-06-19T22:08:27Z","closed_at":null,"author_association":"NONE","body":"`bigstatsr` is super convenient if a large matrix is already on file but this is often not the case for me. I typically want to use `bigstatsr` after I've created a larger-than-memory matrix in R - which is usually in the form of a very sparse matrix (>95% sparsity). Writing to disk is a bottleneck because I have to iterate through the spares matrix by row-chunks (using `lapply` or equivalent), convert to non-sparse matrix, and write each chunk to file using `append=TRUE`. A faster method for doing this would be really helpful. Thanks!"},"comment":{"url":"https://api.github.com/repos/privefl/bigstatsr/issues/comments/398562092","html_url":"https://github.com/privefl/bigstatsr/issues/23#issuecomment-398562092","issue_url":"https://api.github.com/repos/privefl/bigstatsr/issues/23","id":398562092,"node_id":"MDEyOklzc3VlQ29tbWVudDM5ODU2MjA5Mg==","user":{"login":"privefl","id":11161036,"node_id":"MDQ6VXNlcjExMTYxMDM2","avatar_url":"https://avatars3.githubusercontent.com/u/11161036?v=4","gravatar_id":"","url":"https://api.github.com/users/privefl","html_url":"https://github.com/privefl","followers_url":"https://api.github.com/users/privefl/followers","following_url":"https://api.github.com/users/privefl/following{/other_user}","gists_url":"https://api.github.com/users/privefl/gists{/gist_id}","starred_url":"https://api.github.com/users/privefl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/privefl/subscriptions","organizations_url":"https://api.github.com/users/privefl/orgs","repos_url":"https://api.github.com/users/privefl/repos","events_url":"https://api.github.com/users/privefl/events{/privacy}","received_events_url":"https://api.github.com/users/privefl/received_events","type":"User","site_admin":false},"created_at":"2018-06-19T22:08:27Z","updated_at":"2018-06-19T22:08:27Z","author_association":"OWNER","body":"`big_read()` is meant to read text files, and is not very efficient.\r\n\r\nHere, if you want a partial SVD, you should really use package {RSpectra} directly:\r\n`svds(spMat, k = 10)`"}}
{"actor":{"display_login":"privefl"}}