{"action":"opened","issue":{"url":"https://api.github.com/repos/cryfs/cryfs/issues/70","repository_url":"https://api.github.com/repos/cryfs/cryfs","labels_url":"https://api.github.com/repos/cryfs/cryfs/issues/70/labels{/name}","comments_url":"https://api.github.com/repos/cryfs/cryfs/issues/70/comments","events_url":"https://api.github.com/repos/cryfs/cryfs/issues/70/events","html_url":"https://github.com/cryfs/cryfs/issues/70","id":163991008,"number":70,"title":"Blocks of dynamic size","user":{"login":"franciscoluiz","id":506510,"avatar_url":"https://avatars.githubusercontent.com/u/506510?v=3","gravatar_id":"","url":"https://api.github.com/users/franciscoluiz","html_url":"https://github.com/franciscoluiz","followers_url":"https://api.github.com/users/franciscoluiz/followers","following_url":"https://api.github.com/users/franciscoluiz/following{/other_user}","gists_url":"https://api.github.com/users/franciscoluiz/gists{/gist_id}","starred_url":"https://api.github.com/users/franciscoluiz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/franciscoluiz/subscriptions","organizations_url":"https://api.github.com/users/franciscoluiz/orgs","repos_url":"https://api.github.com/users/franciscoluiz/repos","events_url":"https://api.github.com/users/franciscoluiz/events{/privacy}","received_events_url":"https://api.github.com/users/franciscoluiz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-07-06T03:23:43Z","updated_at":"2016-07-06T03:23:43Z","closed_at":null,"body":"I will insist on the implementation of the option to generate dynamic block sizes.\r\nAnd so I am opening this topic to discuss and argue deeply.\r\n\r\nIn issue #68 in the name of security, it was said that the best is to keep all blocks of the same size.\r\n\r\nbut I'm also thinking heavily in performance when reading many directories and files, and also the disk space that can be affected.\r\n\r\nsee these two situations can occur:\r\n\r\n**situation number 1:** I leave the default block size of 32k, and I use in my environment many files of large sizes, such as videos. this will generate an awful lot of folders / files. Do a simulation only 10 videos of 1GB each. 10GB divided into 32kb (10485760 kb / 32 kb), will generate 327680 files and folders just because of 10 videos. If a person has many terabytes of videos, this can be absurdly unfeasible.\r\n\r\nsee for example, the dropbox says that performance may decrease much when exceed more than 300000 files. Note that this is not difficult to exceed if the person has a few gigabytes of video, and use the cryfs with the block in standard size.\r\n\r\n**references:**\r\nhttps://www.dropbox.com/help/39 \"The performance of the Dropbox application can decline if you have more than 300,000 files.\"\r\nhttp://www.businessinsider.com/maximimum-number-of-files-dropbox-can-handle-2013-9\r\nhttps://medium.com/@dburrows/the-coming-dropbox-apocalypse-8682c686a795#.1p7ve4qna\r\n\r\nand the Amazon cloud drive can also suspect the attitude:\r\nhttp://www.zdnet.com/article/is-amazons-online-storage-really-unlimited-read-the-fine-print/\r\nhttp://www.amazon.com/gp/help/customer/display.html/?nodeId=201376540\r\n\"... We may terminate the Agreement or restrict, suspend or terminate your use of the Service at our discretion without notice at any time, including if we determine that your use violates the Agreement, is improper, substantially exceeds or differs from normal use by other users...\"\r\n\r\nthis part \"substantially exceeds or differs from normal use by other users\" this part would fit in the case of a person quickly have millions of files. I agree to ban and penalize those who engage in illegal activities, but I can simply want to encrypt all my personal files, and will it not generate this discrepancy? With a professional camera and a few hours of HD videos of my family, and I could already reach this.\r\n\r\n**situation number 2:** I set up so that the blocks have a larger size, such as 4MB. Let's do the simulation with 30gb of videos for example. (30720 mb / 4 mb), generating only 7680 files. The inconvenience is reduced. But see, that in the dropbox for example, allow just under 1.2 Tb (1200000 mb / mb 4), which will generate 300000 files. Of course, the situation can be alleviated, if I choose to larger blocks. But imagine that in the same environment, if I have together, many small files?\r\n\r\nthis second situation can also consume absurd disk space if I simply have many small files. if I have for example, only 300 text files or spreadsheets, a few tens or hundreds of kilobytes. A simulation of 300 files on average 100 kb each, which would occupy about 29 megabytes. Each file will be placed in a 4 megabytes block, then encrypted, will generate 300 files (300*4), and a total of 1200MB (approx. 1:17 Gb). a gross increase of 4137%.\r\n\r\nthese are the drawbacks to manage an environment with fixed block sizes.\r\n\r\nThere are two solutions to this. I do not know if I should call them solutions ... because I confess that I do not consider any of them attractive, but could be:\r\n\r\n**Solution 1:** Create different environments, each for its file type (or size). A folder / environment for small files and a folder or environment for large files.\r\n\r\nBut this creates the inconvenience that you have a password and spend time to create and mount your file and folder structure, separately. On the one hand it is safe, because if you're smart, you'll have a password for each environment. By another, it continues with the inconvenience of giving suggestion to the evildoer, about the type of file that may exist there.\r\n\r\n**Solution 2:** Allow the user voluntarily change the default setting, where he can choose and enable this option of creating dynamic blocks. Allow the system to use a table (predetermined or pre-set by the user), the block size configuration. Something like:\r\nExamples in megabytes:\r\n   range          |       block size\r\n 0.0~0.5         |      0.25\r\n 0.6~1.5         |      1.00\r\n 1.6~5.0         |       3.00\r\n 5.1~10.0       |       7.00\r\nmore then 10 |    10.00\r\nAnd so on\r\n\r\nBoth solutions may slightly hurt the safety criteria, and give to understand an intruder or someone malicious, a slight hint of what types of files are there. But it alleviates the problems described above, which in some circumstances may be critical not only affect performance, but can also make an account be excluded. And there is also the possibility to obfuscate this mechanism with the creation of dynamical blocks and also Random size. This make it harder the compression that would be such files. Here is an example / suggestion:\r\n\r\nExamples in megabytes:\r\n   range           |         block size\r\n 0.0~0.5          |  random(  0.25 ~  0.5 )\r\n 0.6~1.5          | random(  0.80 ~  1.5 )\r\n 1.6~5.0          | random(  2.00 ~  5.0 )\r\n 5.1~10.0        |  random(  6.10 ~ 10.0 )\r\nmore then 10  |     random( 12.00 ~ 15.0 )\r\n\r\n@smessmer, you wrote in issue #68:\r\n> I don't want to compromise any part of security for performance.\r\n\r\nI agree not to compromise the security, but if you have this suggested configuration, even if the intruder group the encrypted files by size, still have much difficulty abstracting about what could be. In this case, a block would not have any connection with a another block, if the person does not have your password discovered.\r\n\r\nAnd sure, this will not be the default setting, it could be, as suggested, an option in which the person is aware of this whole mechanism will opt for change in their environment.\r\n\r\nSee, in the original configuration, despite not compromising the security, could bring in consequences, such as being the victim of a suspicious attitude, and have all your content excluded.\r\n\r\nAnyway, this is my attempt to contribute to the improvement of Cryfs\r\n\r\nSorry for the long text. And thanks for the great job."}}
{"actor":{"display_login":"franciscoluiz"}}