{"action":"created","comment":{"url":"https://api.github.com/repos/ceph/ceph/pulls/comments/214065466","pull_request_review_id":151038174,"id":214065466,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxNDA2NTQ2Ng==","diff_hunk":"@@ -0,0 +1,82 @@\n+\n+Application best practices for distributed filesystems\n+======================================================\n+\n+CephFS is POSIX compatible, and therefore should work with any existing\n+applications that expect a POSIX filesystem.  However, because it is a\n+network filesystem (unlike e.g. XFS) and it is highly consistent (unlike\n+e.g. NFS), there are some consequences that application authors may\n+benefit from knowing about.\n+\n+The following sections describe some areas where distributed filesystems\n+may have noticeably different performance behaviours compared with\n+local filesystems.\n+\n+\n+ls -l (and colored ls)\n+----------------------\n+\n+When you run \"ls -l\" (or use ls with coloring enabled), the ``ls`` program\n+is first doing a directory listing, and then calling ``stat`` on every\n+file in the directory.\n+\n+This is usually far in excess of what an application really needs, and\n+it can be slow for large directories.  If you don't really need all\n+this metadata for each file, then use a plain ``ls`` with coloring\n+disabled.\n+\n+ls on files being extended\n+--------------------------\n+\n+If another client is currently extending files in the listed directory,\n+then an ``ls -l`` may take an exceptionally long time to complete, as\n+the lister must wait for the writer to flush data in order to do a valid\n+read of the every file's size.  So unless you *really* need to know the\n+exact size of every file in the directory, just don't do it!\n+\n+Very large directories\n+----------------------\n+\n+Do you really need that 10,000,000 file directory?  While directory\n+fragmentation enables CephFS to handle it, it is always going to be\n+less efficient than splitting your files into more modest-sized directories.\n+\n+Even standard userspace tools can become quite slow when operating on very\n+large directories. For example, the default behaviour of ``ls``\n+is to give an alphabetically ordered result, but ``readdir`` system\n+calls do not give an ordered result (this is true in general, not just\n+with CephFS).  So when you ``ls`` on a million file directory, it is\n+loading a list of a million names into memory, sorting the list, then writing\n+it out to the display.\n+\n+Hard links\n+----------\n+\n+Hard links have an intrinsic cost in terms of the internal housekeeping\n+that a filesystem has to do to keep two references to the same data.  In\n+CephFS there is a particular performance cost, because with normal files\n+the inode is embedded in the directory (i.e. there is no extra fetch of\n+the inode after looking up the path).\n+\n+Working set size\n+----------------\n+\n+The MDS acts as a cache for the metadata stored in RADOS.  Metadata\n+performance is very different for workloads whose metadata fits within\n+that cache.\n+\n+If your workload has more files than you ``mds_cache_size`` setting, then\n+make sure you test it appropriately: don't test your system with a small\n+number of files and then expect equivalent performance when you move\n+to a much larger number of files.\n+\n+Do you need a filesystem?\n+-------------------------\n+\n+Remember that Ceph also includes an object storage interface.  If your\n+application needs to store huge flat collections of files where you just\n+read and write whole files at once, then you might well be better off\n+using :doc:`/radosgw/`","path":"doc/cephfs/app-best-practices.rst","position":79,"original_position":79,"commit_id":"f893e552129b14cb4464c590f6bcbbe757fefc2e","original_commit_id":"f893e552129b14cb4464c590f6bcbbe757fefc2e","user":{"login":"LenzGr","id":263427,"node_id":"MDQ6VXNlcjI2MzQyNw==","avatar_url":"https://avatars3.githubusercontent.com/u/263427?v=4","gravatar_id":"","url":"https://api.github.com/users/LenzGr","html_url":"https://github.com/LenzGr","followers_url":"https://api.github.com/users/LenzGr/followers","following_url":"https://api.github.com/users/LenzGr/following{/other_user}","gists_url":"https://api.github.com/users/LenzGr/gists{/gist_id}","starred_url":"https://api.github.com/users/LenzGr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/LenzGr/subscriptions","organizations_url":"https://api.github.com/users/LenzGr/orgs","repos_url":"https://api.github.com/users/LenzGr/repos","events_url":"https://api.github.com/users/LenzGr/events{/privacy}","received_events_url":"https://api.github.com/users/LenzGr/received_events","type":"User","site_admin":false},"body":"The docs rendering failed with `/home/jenkins-build/build/workspace/ceph-pr-render-docs/doc/cephfs/app-best-practices.rst:76:unknown document: /radosgw/`  - that reference should probably be replaced with a reference to `object-gateway`?\r\n```\r\nusing :doc:`object_gateway`\r\n```","created_at":"2018-08-30T14:58:44Z","updated_at":"2018-08-30T14:58:44Z","html_url":"https://github.com/ceph/ceph/pull/23824#discussion_r214065466","pull_request_url":"https://api.github.com/repos/ceph/ceph/pulls/23824","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments/214065466"},"html":{"href":"https://github.com/ceph/ceph/pull/23824#discussion_r214065466"},"pull_request":{"href":"https://api.github.com/repos/ceph/ceph/pulls/23824"}}},"pull_request":{"url":"https://api.github.com/repos/ceph/ceph/pulls/23824","id":212078411,"node_id":"MDExOlB1bGxSZXF1ZXN0MjEyMDc4NDEx","html_url":"https://github.com/ceph/ceph/pull/23824","diff_url":"https://github.com/ceph/ceph/pull/23824.diff","patch_url":"https://github.com/ceph/ceph/pull/23824.patch","issue_url":"https://api.github.com/repos/ceph/ceph/issues/23824","number":23824,"state":"open","locked":false,"title":"doc/cephfs: add notes on application best practices","user":{"login":"jcsp","id":944640,"node_id":"MDQ6VXNlcjk0NDY0MA==","avatar_url":"https://avatars3.githubusercontent.com/u/944640?v=4","gravatar_id":"","url":"https://api.github.com/users/jcsp","html_url":"https://github.com/jcsp","followers_url":"https://api.github.com/users/jcsp/followers","following_url":"https://api.github.com/users/jcsp/following{/other_user}","gists_url":"https://api.github.com/users/jcsp/gists{/gist_id}","starred_url":"https://api.github.com/users/jcsp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcsp/subscriptions","organizations_url":"https://api.github.com/users/jcsp/orgs","repos_url":"https://api.github.com/users/jcsp/repos","events_url":"https://api.github.com/users/jcsp/events{/privacy}","received_events_url":"https://api.github.com/users/jcsp/received_events","type":"User","site_admin":false},"body":"I was tidying up my home directory, and found this orphan .rst file that I must have written on a train at some point!\r\n\r\nPutting it here to see if folks think it would be a useful addition.  It covers some FAQs, at the cost of perhaps scaring people a bit?","created_at":"2018-08-30T13:25:30Z","updated_at":"2018-08-30T14:58:44Z","closed_at":null,"merged_at":null,"merge_commit_sha":"47d3cad80f1748d5e18eb6218b525d3aed719cc9","assignee":null,"assignees":[],"requested_reviewers":[{"login":"batrick","id":548304,"node_id":"MDQ6VXNlcjU0ODMwNA==","avatar_url":"https://avatars3.githubusercontent.com/u/548304?v=4","gravatar_id":"","url":"https://api.github.com/users/batrick","html_url":"https://github.com/batrick","followers_url":"https://api.github.com/users/batrick/followers","following_url":"https://api.github.com/users/batrick/following{/other_user}","gists_url":"https://api.github.com/users/batrick/gists{/gist_id}","starred_url":"https://api.github.com/users/batrick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/batrick/subscriptions","organizations_url":"https://api.github.com/users/batrick/orgs","repos_url":"https://api.github.com/users/batrick/repos","events_url":"https://api.github.com/users/batrick/events{/privacy}","received_events_url":"https://api.github.com/users/batrick/received_events","type":"User","site_admin":false},{"login":"gregsfortytwo","id":908479,"node_id":"MDQ6VXNlcjkwODQ3OQ==","avatar_url":"https://avatars2.githubusercontent.com/u/908479?v=4","gravatar_id":"","url":"https://api.github.com/users/gregsfortytwo","html_url":"https://github.com/gregsfortytwo","followers_url":"https://api.github.com/users/gregsfortytwo/followers","following_url":"https://api.github.com/users/gregsfortytwo/following{/other_user}","gists_url":"https://api.github.com/users/gregsfortytwo/gists{/gist_id}","starred_url":"https://api.github.com/users/gregsfortytwo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gregsfortytwo/subscriptions","organizations_url":"https://api.github.com/users/gregsfortytwo/orgs","repos_url":"https://api.github.com/users/gregsfortytwo/repos","events_url":"https://api.github.com/users/gregsfortytwo/events{/privacy}","received_events_url":"https://api.github.com/users/gregsfortytwo/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[{"id":144329802,"node_id":"MDU6TGFiZWwxNDQzMjk4MDI=","url":"https://api.github.com/repos/ceph/ceph/labels/cephfs","name":"cephfs","color":"b192ff","default":false},{"id":141186344,"node_id":"MDU6TGFiZWwxNDExODYzNDQ=","url":"https://api.github.com/repos/ceph/ceph/labels/documentation","name":"documentation","color":"fef2c0","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/ceph/ceph/pulls/23824/commits","review_comments_url":"https://api.github.com/repos/ceph/ceph/pulls/23824/comments","review_comment_url":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ceph/ceph/issues/23824/comments","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/f893e552129b14cb4464c590f6bcbbe757fefc2e","head":{"label":"jcsp:wip-doc-cephfs-app-bp","ref":"wip-doc-cephfs-app-bp","sha":"f893e552129b14cb4464c590f6bcbbe757fefc2e","user":{"login":"jcsp","id":944640,"node_id":"MDQ6VXNlcjk0NDY0MA==","avatar_url":"https://avatars3.githubusercontent.com/u/944640?v=4","gravatar_id":"","url":"https://api.github.com/users/jcsp","html_url":"https://github.com/jcsp","followers_url":"https://api.github.com/users/jcsp/followers","following_url":"https://api.github.com/users/jcsp/following{/other_user}","gists_url":"https://api.github.com/users/jcsp/gists{/gist_id}","starred_url":"https://api.github.com/users/jcsp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcsp/subscriptions","organizations_url":"https://api.github.com/users/jcsp/orgs","repos_url":"https://api.github.com/users/jcsp/repos","events_url":"https://api.github.com/users/jcsp/events{/privacy}","received_events_url":"https://api.github.com/users/jcsp/received_events","type":"User","site_admin":false},"repo":{"id":13571473,"node_id":"MDEwOlJlcG9zaXRvcnkxMzU3MTQ3Mw==","name":"ceph","full_name":"jcsp/ceph","owner":{"login":"jcsp","id":944640,"node_id":"MDQ6VXNlcjk0NDY0MA==","avatar_url":"https://avatars3.githubusercontent.com/u/944640?v=4","gravatar_id":"","url":"https://api.github.com/users/jcsp","html_url":"https://github.com/jcsp","followers_url":"https://api.github.com/users/jcsp/followers","following_url":"https://api.github.com/users/jcsp/following{/other_user}","gists_url":"https://api.github.com/users/jcsp/gists{/gist_id}","starred_url":"https://api.github.com/users/jcsp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcsp/subscriptions","organizations_url":"https://api.github.com/users/jcsp/orgs","repos_url":"https://api.github.com/users/jcsp/repos","events_url":"https://api.github.com/users/jcsp/events{/privacy}","received_events_url":"https://api.github.com/users/jcsp/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jcsp/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":true,"url":"https://api.github.com/repos/jcsp/ceph","forks_url":"https://api.github.com/repos/jcsp/ceph/forks","keys_url":"https://api.github.com/repos/jcsp/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jcsp/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jcsp/ceph/teams","hooks_url":"https://api.github.com/repos/jcsp/ceph/hooks","issue_events_url":"https://api.github.com/repos/jcsp/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/jcsp/ceph/events","assignees_url":"https://api.github.com/repos/jcsp/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/jcsp/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/jcsp/ceph/tags","blobs_url":"https://api.github.com/repos/jcsp/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jcsp/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jcsp/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/jcsp/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jcsp/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/jcsp/ceph/languages","stargazers_url":"https://api.github.com/repos/jcsp/ceph/stargazers","contributors_url":"https://api.github.com/repos/jcsp/ceph/contributors","subscribers_url":"https://api.github.com/repos/jcsp/ceph/subscribers","subscription_url":"https://api.github.com/repos/jcsp/ceph/subscription","commits_url":"https://api.github.com/repos/jcsp/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/jcsp/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/jcsp/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/jcsp/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/jcsp/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/jcsp/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jcsp/ceph/merges","archive_url":"https://api.github.com/repos/jcsp/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jcsp/ceph/downloads","issues_url":"https://api.github.com/repos/jcsp/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/jcsp/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/jcsp/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/jcsp/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jcsp/ceph/labels{/name}","releases_url":"https://api.github.com/repos/jcsp/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/jcsp/ceph/deployments","created_at":"2013-10-14T19:52:34Z","updated_at":"2018-07-23T16:35:40Z","pushed_at":"2018-08-30T13:23:46Z","git_url":"git://github.com/jcsp/ceph.git","ssh_url":"git@github.com:jcsp/ceph.git","clone_url":"https://github.com/jcsp/ceph.git","svn_url":"https://github.com/jcsp/ceph","homepage":"http://ceph.com","size":291142,"stargazers_count":2,"watchers_count":2,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2,"mirror_url":null,"archived":false,"open_issues_count":1,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":2,"open_issues":1,"watchers":2,"default_branch":"master"}},"base":{"label":"ceph:master","ref":"master","sha":"f59a673cae36cc5c9d494d43f2b7a019b91b6dda","user":{"login":"ceph","id":1015767,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMTU3Njc=","avatar_url":"https://avatars3.githubusercontent.com/u/1015767?v=4","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"node_id":"MDEwOlJlcG9zaXRvcnkyMzEwNDk1","name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwMTU3Njc=","avatar_url":"https://avatars3.githubusercontent.com/u/1015767?v=4","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","deployments_url":"https://api.github.com/repos/ceph/ceph/deployments","created_at":"2011-09-01T21:41:26Z","updated_at":"2018-08-30T14:24:27Z","pushed_at":"2018-08-30T14:52:42Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":307291,"stargazers_count":5002,"watchers_count":5002,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2607,"mirror_url":null,"archived":false,"open_issues_count":735,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":2607,"open_issues":735,"watchers":5002,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/23824"},"html":{"href":"https://github.com/ceph/ceph/pull/23824"},"issue":{"href":"https://api.github.com/repos/ceph/ceph/issues/23824"},"comments":{"href":"https://api.github.com/repos/ceph/ceph/issues/23824/comments"},"review_comments":{"href":"https://api.github.com/repos/ceph/ceph/pulls/23824/comments"},"review_comment":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ceph/ceph/pulls/23824/commits"},"statuses":{"href":"https://api.github.com/repos/ceph/ceph/statuses/f893e552129b14cb4464c590f6bcbbe757fefc2e"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"LenzGr"}}