{"action":"created","issue":{"url":"https://api.github.com/repos/mbdavid/LiteDB/issues/967","repository_url":"https://api.github.com/repos/mbdavid/LiteDB","labels_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/967/labels{/name}","comments_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/967/comments","events_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/967/events","html_url":"https://github.com/mbdavid/LiteDB/issues/967","id":319241791,"number":967,"title":"File write performance ","user":{"login":"christopherpross","id":26790901,"avatar_url":"https://avatars1.githubusercontent.com/u/26790901?v=4","gravatar_id":"","url":"https://api.github.com/users/christopherpross","html_url":"https://github.com/christopherpross","followers_url":"https://api.github.com/users/christopherpross/followers","following_url":"https://api.github.com/users/christopherpross/following{/other_user}","gists_url":"https://api.github.com/users/christopherpross/gists{/gist_id}","starred_url":"https://api.github.com/users/christopherpross/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/christopherpross/subscriptions","organizations_url":"https://api.github.com/users/christopherpross/orgs","repos_url":"https://api.github.com/users/christopherpross/repos","events_url":"https://api.github.com/users/christopherpross/events{/privacy}","received_events_url":"https://api.github.com/users/christopherpross/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":15,"created_at":"2018-05-01T16:02:26Z","updated_at":"2018-05-06T21:41:57Z","closed_at":null,"author_association":"NONE","body":"Hello,\r\nI have a short question, do you know a way how to optimice the performance of a file write on a database? At the moment I use openWrite to write a file, to display a progressbar etc. I work here with huge data and many files in total size of 1.5 gb, need in total more than one hour. Is there a way to do this quicker? Threads or push up the cache size? Because I know that my ssd can write quicker. :-) Maybe you have some hints for me.\r\n\r\nbest regards,\r\nChristopher Proß"},"comment":{"url":"https://api.github.com/repos/mbdavid/LiteDB/issues/comments/386917899","html_url":"https://github.com/mbdavid/LiteDB/issues/967#issuecomment-386917899","issue_url":"https://api.github.com/repos/mbdavid/LiteDB/issues/967","id":386917899,"user":{"login":"mbdavid","id":1137197,"avatar_url":"https://avatars1.githubusercontent.com/u/1137197?v=4","gravatar_id":"","url":"https://api.github.com/users/mbdavid","html_url":"https://github.com/mbdavid","followers_url":"https://api.github.com/users/mbdavid/followers","following_url":"https://api.github.com/users/mbdavid/following{/other_user}","gists_url":"https://api.github.com/users/mbdavid/gists{/gist_id}","starred_url":"https://api.github.com/users/mbdavid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbdavid/subscriptions","organizations_url":"https://api.github.com/users/mbdavid/orgs","repos_url":"https://api.github.com/users/mbdavid/repos","events_url":"https://api.github.com/users/mbdavid/events{/privacy}","received_events_url":"https://api.github.com/users/mbdavid/received_events","type":"User","site_admin":false},"created_at":"2018-05-06T21:41:57Z","updated_at":"2018-05-06T21:41:57Z","author_association":"OWNER","body":"@christopherpross, here my example:\r\n\r\n```C#\r\nusing System;\r\nusing System.Collections.Generic;\r\nusing System.Diagnostics;\r\nusing System.IO;\r\nusing System.Linq;\r\nusing System.Text;\r\nusing System.Threading.Tasks;\r\n\r\nnamespace SmallFilesLiteDB\r\n{\r\n    class Program\r\n    {\r\n        static int N = 618;\r\n        static string _path = @\"C:\\Temp\\SmallFiles\\\";\r\n\r\n        static void Main(string[] args)\r\n        {\r\n            Directory.CreateDirectory(_path);\r\n\r\n            var sw = new Stopwatch();\r\n\r\n            sw.Start();\r\n\r\n            CreateFiles();\r\n\r\n            Console.WriteLine($\"Create files: {sw.Elapsed.TotalSeconds} seconds.\");\r\n\r\n            sw.Restart();\r\n\r\n            CreateDatabase();\r\n\r\n            Console.WriteLine($\"Upload to LiteDB: {sw.Elapsed.TotalSeconds} seconds.\");\r\n\r\n            Console.ReadKey();\r\n\r\n        }\r\n\r\n        static void CreateFiles()\r\n        {\r\n            var rnd = new Random();\r\n\r\n            for(var i = 1; i <= N; i++)\r\n            {\r\n                File.WriteAllBytes(_path + i.ToString().PadLeft(3, '0') + \".bin\", new byte[rnd.Next(4 * 1024, 5 * 1024 * 1024)]);\r\n            }\r\n        }\r\n\r\n        static void CreateDatabase()\r\n        {\r\n            using (var db = new LiteDB.LiteDatabase(_path + \"test.db\"))\r\n            {\r\n                foreach(var file in Directory.GetFiles(_path, \"*.bin\"))\r\n                {\r\n                    db.FileStorage.Upload(Path.GetFileNameWithoutExtension(file), file);\r\n                }\r\n            }\r\n        }\r\n    }\r\n}\r\n```\r\n\r\nMy results was: 4 seconds to create files and 18 seconds to store inside LiteDB (my computer is a dell i7 + 16gb + ssd). Run this test to see if you get similar time.  Maybe your problem can be with how you get this files (are inside same machine?)"}}
{"actor":{"display_login":"mbdavid"}}