{"action":"created","comment":{"url":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments/233138621","pull_request_review_id":174483402,"id":233138621,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzMzEzODYyMQ==","diff_hunk":"@@ -0,0 +1,13 @@\n+Job product batch size\n+======================\n+\n+A product catalog is unique to one's need. It is configured with a custom number of attributes, families, locales, channels, etc. Those different combinations may affect the import/exports performances.\n+\n+Indeed the size of the products will define how much RAM they will take to be processed. During product import/exports, we regularly flush the processed products into the database to avoid too much memory consumption.\n+\n+But, if we flush them too often, the transaction time can slow the process: flushing is not free and doing it too often will be inefficient. On the other hand, if we flush too sporadically, the RAM consumption can increase to a point where the PHP's garbage collector is called too many times and slows down the import/exports.","path":"technical_architecture/performances_guide/batch_page_size.rst","position":8,"original_position":8,"commit_id":"e2c725597838677121007a1397834d3291bc9017","original_commit_id":"e2c725597838677121007a1397834d3291bc9017","user":{"login":"BitOne","id":2151567,"node_id":"MDQ6VXNlcjIxNTE1Njc=","avatar_url":"https://avatars3.githubusercontent.com/u/2151567?v=4","gravatar_id":"","url":"https://api.github.com/users/BitOne","html_url":"https://github.com/BitOne","followers_url":"https://api.github.com/users/BitOne/followers","following_url":"https://api.github.com/users/BitOne/following{/other_user}","gists_url":"https://api.github.com/users/BitOne/gists{/gist_id}","starred_url":"https://api.github.com/users/BitOne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BitOne/subscriptions","organizations_url":"https://api.github.com/users/BitOne/orgs","repos_url":"https://api.github.com/users/BitOne/repos","events_url":"https://api.github.com/users/BitOne/events{/privacy}","received_events_url":"https://api.github.com/users/BitOne/received_events","type":"User","site_admin":false},"body":"We can add that during at the end of the batch, we index the batch of product in Elasticsearch. If we reduce the batch size, we index more often. And the cost of communication between PHP and Elasticsearch begins to cumulate, and being counter productive.","created_at":"2018-11-13T17:09:58Z","updated_at":"2018-11-13T17:09:58Z","html_url":"https://github.com/akeneo/pim-docs/pull/1047#discussion_r233138621","pull_request_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments/233138621"},"html":{"href":"https://github.com/akeneo/pim-docs/pull/1047#discussion_r233138621"},"pull_request":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047"}},"in_reply_to_id":233096843},"pull_request":{"url":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047","id":230522515,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMwNTIyNTE1","html_url":"https://github.com/akeneo/pim-docs/pull/1047","diff_url":"https://github.com/akeneo/pim-docs/pull/1047.diff","patch_url":"https://github.com/akeneo/pim-docs/pull/1047.patch","issue_url":"https://api.github.com/repos/akeneo/pim-docs/issues/1047","number":1047,"state":"open","locked":false,"title":"PIM-7852: add documentation about pim_job_product_batch_size","user":{"login":"juliensnz","id":1117272,"node_id":"MDQ6VXNlcjExMTcyNzI=","avatar_url":"https://avatars2.githubusercontent.com/u/1117272?v=4","gravatar_id":"","url":"https://api.github.com/users/juliensnz","html_url":"https://github.com/juliensnz","followers_url":"https://api.github.com/users/juliensnz/followers","following_url":"https://api.github.com/users/juliensnz/following{/other_user}","gists_url":"https://api.github.com/users/juliensnz/gists{/gist_id}","starred_url":"https://api.github.com/users/juliensnz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/juliensnz/subscriptions","organizations_url":"https://api.github.com/users/juliensnz/orgs","repos_url":"https://api.github.com/users/juliensnz/repos","events_url":"https://api.github.com/users/juliensnz/events{/privacy}","received_events_url":"https://api.github.com/users/juliensnz/received_events","type":"User","site_admin":false},"body":"<!--- (<3 Thanks for taking the time to contribute! You're awesome! <3) --->\r\n\r\n<!--- (If you've never contributed to this repository before, please read https://github.com/akeneo/pim-docs/blob/master/CONTRIBUTING.md) --->\r\n\r\n**Description**\r\n\r\n<!--- (What does this Pull Request do? reference the related issue?) --->\r\n\r\n**Definition Of Done**\r\n\r\n| Q                                 | A\r\n| --------------------------------- | ---\r\n| Technical Review and 2 GTM        | Todo\r\n| English Review and 1 GTM          | Todo\r\n\r\n\r\n`Todo`: Pending / Work in progress\r\n`OK`: Done / Validated\r\n`-`: Not needed\r\n","created_at":"2018-11-13T15:38:28Z","updated_at":"2018-11-13T17:09:58Z","closed_at":null,"merged_at":null,"merge_commit_sha":"9a74b908dd75da9e3e34ecbffcaa61f6c2c63c72","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047/commits","review_comments_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047/comments","review_comment_url":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/akeneo/pim-docs/issues/1047/comments","statuses_url":"https://api.github.com/repos/akeneo/pim-docs/statuses/e2c725597838677121007a1397834d3291bc9017","head":{"label":"akeneo:PIM-7852","ref":"PIM-7852","sha":"e2c725597838677121007a1397834d3291bc9017","user":{"login":"akeneo","id":2510186,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTAxODY=","avatar_url":"https://avatars0.githubusercontent.com/u/2510186?v=4","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"repo":{"id":12193604,"node_id":"MDEwOlJlcG9zaXRvcnkxMjE5MzYwNA==","name":"pim-docs","full_name":"akeneo/pim-docs","private":false,"owner":{"login":"akeneo","id":2510186,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTAxODY=","avatar_url":"https://avatars0.githubusercontent.com/u/2510186?v=4","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/akeneo/pim-docs","description":"[Community Documentation Repository] The open source Product Information Management (PIM)","fork":false,"url":"https://api.github.com/repos/akeneo/pim-docs","forks_url":"https://api.github.com/repos/akeneo/pim-docs/forks","keys_url":"https://api.github.com/repos/akeneo/pim-docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/akeneo/pim-docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/akeneo/pim-docs/teams","hooks_url":"https://api.github.com/repos/akeneo/pim-docs/hooks","issue_events_url":"https://api.github.com/repos/akeneo/pim-docs/issues/events{/number}","events_url":"https://api.github.com/repos/akeneo/pim-docs/events","assignees_url":"https://api.github.com/repos/akeneo/pim-docs/assignees{/user}","branches_url":"https://api.github.com/repos/akeneo/pim-docs/branches{/branch}","tags_url":"https://api.github.com/repos/akeneo/pim-docs/tags","blobs_url":"https://api.github.com/repos/akeneo/pim-docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/akeneo/pim-docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/akeneo/pim-docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/akeneo/pim-docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/akeneo/pim-docs/statuses/{sha}","languages_url":"https://api.github.com/repos/akeneo/pim-docs/languages","stargazers_url":"https://api.github.com/repos/akeneo/pim-docs/stargazers","contributors_url":"https://api.github.com/repos/akeneo/pim-docs/contributors","subscribers_url":"https://api.github.com/repos/akeneo/pim-docs/subscribers","subscription_url":"https://api.github.com/repos/akeneo/pim-docs/subscription","commits_url":"https://api.github.com/repos/akeneo/pim-docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/akeneo/pim-docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/akeneo/pim-docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/akeneo/pim-docs/issues/comments{/number}","contents_url":"https://api.github.com/repos/akeneo/pim-docs/contents/{+path}","compare_url":"https://api.github.com/repos/akeneo/pim-docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/akeneo/pim-docs/merges","archive_url":"https://api.github.com/repos/akeneo/pim-docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/akeneo/pim-docs/downloads","issues_url":"https://api.github.com/repos/akeneo/pim-docs/issues{/number}","pulls_url":"https://api.github.com/repos/akeneo/pim-docs/pulls{/number}","milestones_url":"https://api.github.com/repos/akeneo/pim-docs/milestones{/number}","notifications_url":"https://api.github.com/repos/akeneo/pim-docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/akeneo/pim-docs/labels{/name}","releases_url":"https://api.github.com/repos/akeneo/pim-docs/releases{/id}","deployments_url":"https://api.github.com/repos/akeneo/pim-docs/deployments","created_at":"2013-08-18T10:17:30Z","updated_at":"2018-11-08T17:03:47Z","pushed_at":"2018-11-13T15:43:48Z","git_url":"git://github.com/akeneo/pim-docs.git","ssh_url":"git@github.com:akeneo/pim-docs.git","clone_url":"https://github.com/akeneo/pim-docs.git","svn_url":"https://github.com/akeneo/pim-docs","homepage":"http://docs.akeneo.com/latest/index.html","size":7144,"stargazers_count":33,"watchers_count":33,"language":"CSS","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":108,"mirror_url":null,"archived":false,"open_issues_count":36,"license":null,"forks":108,"open_issues":36,"watchers":33,"default_branch":"2.3"}},"base":{"label":"akeneo:2.3","ref":"2.3","sha":"bd8befb2c595556d8ead1796f707926b3202b208","user":{"login":"akeneo","id":2510186,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTAxODY=","avatar_url":"https://avatars0.githubusercontent.com/u/2510186?v=4","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"repo":{"id":12193604,"node_id":"MDEwOlJlcG9zaXRvcnkxMjE5MzYwNA==","name":"pim-docs","full_name":"akeneo/pim-docs","private":false,"owner":{"login":"akeneo","id":2510186,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTAxODY=","avatar_url":"https://avatars0.githubusercontent.com/u/2510186?v=4","gravatar_id":"","url":"https://api.github.com/users/akeneo","html_url":"https://github.com/akeneo","followers_url":"https://api.github.com/users/akeneo/followers","following_url":"https://api.github.com/users/akeneo/following{/other_user}","gists_url":"https://api.github.com/users/akeneo/gists{/gist_id}","starred_url":"https://api.github.com/users/akeneo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akeneo/subscriptions","organizations_url":"https://api.github.com/users/akeneo/orgs","repos_url":"https://api.github.com/users/akeneo/repos","events_url":"https://api.github.com/users/akeneo/events{/privacy}","received_events_url":"https://api.github.com/users/akeneo/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/akeneo/pim-docs","description":"[Community Documentation Repository] The open source Product Information Management (PIM)","fork":false,"url":"https://api.github.com/repos/akeneo/pim-docs","forks_url":"https://api.github.com/repos/akeneo/pim-docs/forks","keys_url":"https://api.github.com/repos/akeneo/pim-docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/akeneo/pim-docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/akeneo/pim-docs/teams","hooks_url":"https://api.github.com/repos/akeneo/pim-docs/hooks","issue_events_url":"https://api.github.com/repos/akeneo/pim-docs/issues/events{/number}","events_url":"https://api.github.com/repos/akeneo/pim-docs/events","assignees_url":"https://api.github.com/repos/akeneo/pim-docs/assignees{/user}","branches_url":"https://api.github.com/repos/akeneo/pim-docs/branches{/branch}","tags_url":"https://api.github.com/repos/akeneo/pim-docs/tags","blobs_url":"https://api.github.com/repos/akeneo/pim-docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/akeneo/pim-docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/akeneo/pim-docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/akeneo/pim-docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/akeneo/pim-docs/statuses/{sha}","languages_url":"https://api.github.com/repos/akeneo/pim-docs/languages","stargazers_url":"https://api.github.com/repos/akeneo/pim-docs/stargazers","contributors_url":"https://api.github.com/repos/akeneo/pim-docs/contributors","subscribers_url":"https://api.github.com/repos/akeneo/pim-docs/subscribers","subscription_url":"https://api.github.com/repos/akeneo/pim-docs/subscription","commits_url":"https://api.github.com/repos/akeneo/pim-docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/akeneo/pim-docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/akeneo/pim-docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/akeneo/pim-docs/issues/comments{/number}","contents_url":"https://api.github.com/repos/akeneo/pim-docs/contents/{+path}","compare_url":"https://api.github.com/repos/akeneo/pim-docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/akeneo/pim-docs/merges","archive_url":"https://api.github.com/repos/akeneo/pim-docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/akeneo/pim-docs/downloads","issues_url":"https://api.github.com/repos/akeneo/pim-docs/issues{/number}","pulls_url":"https://api.github.com/repos/akeneo/pim-docs/pulls{/number}","milestones_url":"https://api.github.com/repos/akeneo/pim-docs/milestones{/number}","notifications_url":"https://api.github.com/repos/akeneo/pim-docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/akeneo/pim-docs/labels{/name}","releases_url":"https://api.github.com/repos/akeneo/pim-docs/releases{/id}","deployments_url":"https://api.github.com/repos/akeneo/pim-docs/deployments","created_at":"2013-08-18T10:17:30Z","updated_at":"2018-11-08T17:03:47Z","pushed_at":"2018-11-13T15:43:48Z","git_url":"git://github.com/akeneo/pim-docs.git","ssh_url":"git@github.com:akeneo/pim-docs.git","clone_url":"https://github.com/akeneo/pim-docs.git","svn_url":"https://github.com/akeneo/pim-docs","homepage":"http://docs.akeneo.com/latest/index.html","size":7144,"stargazers_count":33,"watchers_count":33,"language":"CSS","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":108,"mirror_url":null,"archived":false,"open_issues_count":36,"license":null,"forks":108,"open_issues":36,"watchers":33,"default_branch":"2.3"}},"_links":{"self":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047"},"html":{"href":"https://github.com/akeneo/pim-docs/pull/1047"},"issue":{"href":"https://api.github.com/repos/akeneo/pim-docs/issues/1047"},"comments":{"href":"https://api.github.com/repos/akeneo/pim-docs/issues/1047/comments"},"review_comments":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047/comments"},"review_comment":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/akeneo/pim-docs/pulls/1047/commits"},"statuses":{"href":"https://api.github.com/repos/akeneo/pim-docs/statuses/e2c725597838677121007a1397834d3291bc9017"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"BitOne"}}