{"action":"created","issue":{"url":"https://api.github.com/repos/tagspaces/tagspaces/issues/738","repository_url":"https://api.github.com/repos/tagspaces/tagspaces","labels_url":"https://api.github.com/repos/tagspaces/tagspaces/issues/738/labels{/name}","comments_url":"https://api.github.com/repos/tagspaces/tagspaces/issues/738/comments","events_url":"https://api.github.com/repos/tagspaces/tagspaces/issues/738/events","html_url":"https://github.com/tagspaces/tagspaces/issues/738","id":366978028,"node_id":"MDU6SXNzdWUzNjY5NzgwMjg=","number":738,"title":"Freezing in folders with too many files","user":{"login":"EfronC","id":5926318,"node_id":"MDQ6VXNlcjU5MjYzMTg=","avatar_url":"https://avatars3.githubusercontent.com/u/5926318?v=4","gravatar_id":"","url":"https://api.github.com/users/EfronC","html_url":"https://github.com/EfronC","followers_url":"https://api.github.com/users/EfronC/followers","following_url":"https://api.github.com/users/EfronC/following{/other_user}","gists_url":"https://api.github.com/users/EfronC/gists{/gist_id}","starred_url":"https://api.github.com/users/EfronC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EfronC/subscriptions","organizations_url":"https://api.github.com/users/EfronC/orgs","repos_url":"https://api.github.com/users/EfronC/repos","events_url":"https://api.github.com/users/EfronC/events{/privacy}","received_events_url":"https://api.github.com/users/EfronC/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-10-04T21:28:50Z","updated_at":"2018-10-05T14:37:33Z","closed_at":null,"author_association":"NONE","body":"While testing the app, i found that on folder with thousands of images, the app freeze the computer, and is necessary to do an emergency restart.\r\n\r\nSince i'm working on a contribution for the app(Support for XMP sidecar files), i investigated a little, and found that the problem is that you are using native fs.readdir(Inherited to fs-extra) to read the directory(checked on electron-io.js), and while this is not a problem with directories with a few files, in directories with thousands of files is a problem since this command loads all files to memory, and block the whole system(I think that the exact number of files to cause this varies depending on the requirements of your computer) [Issue reference](https://github.com/nodejs/node-v0.x-archive/issues/388)\r\n\r\nWhile searching a solution, i found the [readdir-enhanced](https://www.npmjs.com/package/readdir-enhanced) package, which has an implementation to read a directory by streaming, something like this:\r\n```\r\nvar readdir = require('readdir-enhanced');\r\nvar through2 = require('through2');\r\n// EventEmitter API\r\nreaddir.stream('my/directory')\r\n  .on('data', function(path) { ... })\r\n  .on('file', function(path) { ... })\r\n  .on('directory', function(path) { ... })\r\n  .on('symlink', function(path) { ... })\r\n  .on('error', function(err) { ... });\r\n```\r\nThis is a package that is not installed in the project, but in case you don't want to install anything new, there seems to be already installed packages with implementations [readdirp](https://www.npmjs.com/package/readdirp), but i haven't tested them since i'm not able to import the event-stream package.\r\n\r\nBut the problem with these is that the function is asynchronous, which causes the view to load before the process has ended. I'm not an expert in NodeJS, and this probably has a very simple solution, but that's why i'm asking if you could change that way to read directories for an streamed one, so that big folders can be loaded without freezing the app.\r\n\r\nThanks for your attention, hope this can be solved.\r\n"},"comment":{"url":"https://api.github.com/repos/tagspaces/tagspaces/issues/comments/427388032","html_url":"https://github.com/tagspaces/tagspaces/issues/738#issuecomment-427388032","issue_url":"https://api.github.com/repos/tagspaces/tagspaces/issues/738","id":427388032,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNzM4ODAzMg==","user":{"login":"EfronC","id":5926318,"node_id":"MDQ6VXNlcjU5MjYzMTg=","avatar_url":"https://avatars3.githubusercontent.com/u/5926318?v=4","gravatar_id":"","url":"https://api.github.com/users/EfronC","html_url":"https://github.com/EfronC","followers_url":"https://api.github.com/users/EfronC/followers","following_url":"https://api.github.com/users/EfronC/following{/other_user}","gists_url":"https://api.github.com/users/EfronC/gists{/gist_id}","starred_url":"https://api.github.com/users/EfronC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EfronC/subscriptions","organizations_url":"https://api.github.com/users/EfronC/orgs","repos_url":"https://api.github.com/users/EfronC/repos","events_url":"https://api.github.com/users/EfronC/events{/privacy}","received_events_url":"https://api.github.com/users/EfronC/received_events","type":"User","site_admin":false},"created_at":"2018-10-05T14:37:33Z","updated_at":"2018-10-05T14:37:33Z","author_association":"NONE","body":"When you say the last beta release, you mean the one is up on the code? Checking the code, i see the last commit in develop branch is from July 15, so if you are saying that this was changed after that commit, then it's probably a matter of waiting until you upload the next commit to check if your changes solve the problem."}}
{"actor":{"display_login":"EfronC"}}