{"action":"created","comment":{"url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/comments/88699139","pull_request_review_id":9267191,"id":88699139,"diff_hunk":"@@ -0,0 +1,276 @@\n+<?php\n+\n+/**\n+ * File containing the MigrateFilesCommand class.\n+ *\n+ * @copyright Copyright (C) eZ Systems AS. All rights reserved.\n+ * @license For full copyright and license information view LICENSE file distributed with this source code.\n+ */\n+namespace eZ\\Bundle\\EzPublishIOBundle\\Command;\n+\n+use eZ\\Bundle\\EzPublishIOBundle\\Migration\\MigrationHandlerInterface;\n+use Symfony\\Component\\Console\\Command\\Command;\n+use Symfony\\Component\\Console\\Input\\InputInterface;\n+use Symfony\\Component\\Console\\Input\\InputOption;\n+use Symfony\\Component\\Console\\Helper\\ProgressBar;\n+use Symfony\\Component\\Console\\Output\\OutputInterface;\n+use Symfony\\Component\\Console\\Question\\ConfirmationQuestion;\n+\n+class MigrateFilesCommand extends Command\n+{\n+    private $configuredMetadataHandlers;","path":"eZ/Bundle/EzPublishIOBundle/Command/MigrateFilesCommand.php","position":21,"original_position":21,"commit_id":"341a98baf980992da0d26fdcc84c31c91fd2e032","original_commit_id":"341a98baf980992da0d26fdcc84c31c91fd2e032","user":{"login":"glye","id":289744,"avatar_url":"https://avatars.githubusercontent.com/u/289744?v=3","gravatar_id":"","url":"https://api.github.com/users/glye","html_url":"https://github.com/glye","followers_url":"https://api.github.com/users/glye/followers","following_url":"https://api.github.com/users/glye/following{/other_user}","gists_url":"https://api.github.com/users/glye/gists{/gist_id}","starred_url":"https://api.github.com/users/glye/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/glye/subscriptions","organizations_url":"https://api.github.com/users/glye/orgs","repos_url":"https://api.github.com/users/glye/repos","events_url":"https://api.github.com/users/glye/events{/privacy}","received_events_url":"https://api.github.com/users/glye/received_events","type":"User","site_admin":false},"body":"Added, thanks.","created_at":"2016-11-18T17:19:54Z","updated_at":"2016-11-18T17:19:54Z","html_url":"https://github.com/ezsystems/ezpublish-kernel/pull/1825#discussion_r88699139","pull_request_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825","_links":{"self":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/comments/88699139"},"html":{"href":"https://github.com/ezsystems/ezpublish-kernel/pull/1825#discussion_r88699139"},"pull_request":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825"}}},"pull_request":{"url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825","id":93175215,"html_url":"https://github.com/ezsystems/ezpublish-kernel/pull/1825","diff_url":"https://github.com/ezsystems/ezpublish-kernel/pull/1825.diff","patch_url":"https://github.com/ezsystems/ezpublish-kernel/pull/1825.patch","issue_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/1825","number":1825,"state":"open","locked":false,"title":"[WIP] EZP-25946: Procedure to migrate existing images/binaries to another IO adapter","user":{"login":"glye","id":289744,"avatar_url":"https://avatars.githubusercontent.com/u/289744?v=3","gravatar_id":"","url":"https://api.github.com/users/glye","html_url":"https://github.com/glye","followers_url":"https://api.github.com/users/glye/followers","following_url":"https://api.github.com/users/glye/following{/other_user}","gists_url":"https://api.github.com/users/glye/gists{/gist_id}","starred_url":"https://api.github.com/users/glye/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/glye/subscriptions","organizations_url":"https://api.github.com/users/glye/orgs","repos_url":"https://api.github.com/users/glye/repos","events_url":"https://api.github.com/users/glye/events{/privacy}","received_events_url":"https://api.github.com/users/glye/received_events","type":"User","site_admin":false},"body":"> Implements https://jira.ez.no/browse/EZP-25946\r\n> Work in progress\r\n\r\nEarly draft, currently migrates files from local storage to NFS/DFS cluster, and vice versa. (Theoretically, any Flysystem adapter should work.)\r\n\r\nAdds a Command which validates from/to input and runs the migration in batches. Input options partially based on [legacy clusterize.php](https://github.com/ezsystems/ezpublish-legacy/blob/d76ce9824dabaceeefa77530c8c611e447d1b109/bin/php/clusterize.php) but made more generic. Meaning it's not just for clusterize/unclusterize, but also for migrating between e.g. NFS and AWS/S3.\r\n\r\n`$ php app/console ezplatform:io:migrate-files --dry-run --from=default,default --to=dfs,nfs`\r\n\r\n- [x] ~~[We can't differentiate between media and binary files](https://github.com/ezsystems/ezpublish-kernel/blob/7170576929fefd6c1bb240f13553b14ca721ca43/eZ/Publish/Core/IO/IOMetadataHandler/LegacyDFSCluster.php#L179), as they have the same prefix (\"original\"). Problem?~~ Migrate all in one go.\r\n- [x] ~~Migration handlers (binary, media, image). Do we actually need 3 of them? Only purpose now is to allow skipping some of them, and even that is unreliable (see previous point)~~ Use only one.\r\n- [x] File handling is normally one-by-one. For migration we need to iterate over *all* files, and ideally count them first. Need support for that in handlers.\r\n- [x] ~~Consider using the [Replicate Adapter](http://flysystem.thephpleague.com/adapter/replicate/) for Flysystem-Flysystem migration~~ Rejected for now. Kind of a different use-case.\r\n- [x] Logging\r\n- [ ] Some adapters may not support counting - if so we have to detect that and work around it.\r\n- [ ] Since FlySystem doesn't support fetches with limit/offset we are currently loading metadata for *all* files - likely a memory/performance problem for big data.\r\n- [ ] Tests\r\n- [ ] Enable source files deletion (or remove the `--remove-files` option)\r\n- [ ] Doc `IOMetadataHandler`/`IOBinarydataHandler` additions","created_at":"2016-11-10T15:46:21Z","updated_at":"2016-11-18T17:19:54Z","closed_at":null,"merged_at":null,"merge_commit_sha":"36c5ae7ba50be7ebc4d9956a13b276d112561d6b","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825/commits","review_comments_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825/comments","review_comment_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/1825/comments","statuses_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/statuses/341a98baf980992da0d26fdcc84c31c91fd2e032","head":{"label":"ezsystems:ezp25946-migrate_files_to_other_io_adapter","ref":"ezp25946-migrate_files_to_other_io_adapter","sha":"341a98baf980992da0d26fdcc84c31c91fd2e032","user":{"login":"ezsystems","id":295311,"avatar_url":"https://avatars.githubusercontent.com/u/295311?v=3","gravatar_id":"","url":"https://api.github.com/users/ezsystems","html_url":"https://github.com/ezsystems","followers_url":"https://api.github.com/users/ezsystems/followers","following_url":"https://api.github.com/users/ezsystems/following{/other_user}","gists_url":"https://api.github.com/users/ezsystems/gists{/gist_id}","starred_url":"https://api.github.com/users/ezsystems/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ezsystems/subscriptions","organizations_url":"https://api.github.com/users/ezsystems/orgs","repos_url":"https://api.github.com/users/ezsystems/repos","events_url":"https://api.github.com/users/ezsystems/events{/privacy}","received_events_url":"https://api.github.com/users/ezsystems/received_events","type":"Organization","site_admin":false},"repo":{"id":1663890,"name":"ezpublish-kernel","full_name":"ezsystems/ezpublish-kernel","owner":{"login":"ezsystems","id":295311,"avatar_url":"https://avatars.githubusercontent.com/u/295311?v=3","gravatar_id":"","url":"https://api.github.com/users/ezsystems","html_url":"https://github.com/ezsystems","followers_url":"https://api.github.com/users/ezsystems/followers","following_url":"https://api.github.com/users/ezsystems/following{/other_user}","gists_url":"https://api.github.com/users/ezsystems/gists{/gist_id}","starred_url":"https://api.github.com/users/ezsystems/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ezsystems/subscriptions","organizations_url":"https://api.github.com/users/ezsystems/orgs","repos_url":"https://api.github.com/users/ezsystems/repos","events_url":"https://api.github.com/users/ezsystems/events{/privacy}","received_events_url":"https://api.github.com/users/ezsystems/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ezsystems/ezpublish-kernel","description":"Kernel (Repository, MVC layer, REST) for eZ Platform","fork":false,"url":"https://api.github.com/repos/ezsystems/ezpublish-kernel","forks_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/forks","keys_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/teams","hooks_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/hooks","issue_events_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/events{/number}","events_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/events","assignees_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/assignees{/user}","branches_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/branches{/branch}","tags_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/tags","blobs_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/refs{/sha}","trees_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/statuses/{sha}","languages_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/languages","stargazers_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/stargazers","contributors_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/contributors","subscribers_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/subscribers","subscription_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/subscription","commits_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/commits{/sha}","git_commits_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/commits{/sha}","comments_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/comments{/number}","issue_comment_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/comments{/number}","contents_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/contents/{+path}","compare_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/merges","archive_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/downloads","issues_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues{/number}","pulls_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls{/number}","milestones_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/milestones{/number}","notifications_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/labels{/name}","releases_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/releases{/id}","deployments_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/deployments","created_at":"2011-04-26T08:54:24Z","updated_at":"2016-11-06T09:58:27Z","pushed_at":"2016-11-18T12:28:27Z","git_url":"git://github.com/ezsystems/ezpublish-kernel.git","ssh_url":"git@github.com:ezsystems/ezpublish-kernel.git","clone_url":"https://github.com/ezsystems/ezpublish-kernel.git","svn_url":"https://github.com/ezsystems/ezpublish-kernel","homepage":"http://ezplatform.com/","size":52888,"stargazers_count":117,"watchers_count":117,"language":"PHP","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":163,"mirror_url":null,"open_issues_count":63,"forks":163,"open_issues":63,"watchers":117,"default_branch":"master"}},"base":{"label":"ezsystems:master","ref":"master","sha":"e1864c6a2283599394c73923db8f162fc3630cff","user":{"login":"ezsystems","id":295311,"avatar_url":"https://avatars.githubusercontent.com/u/295311?v=3","gravatar_id":"","url":"https://api.github.com/users/ezsystems","html_url":"https://github.com/ezsystems","followers_url":"https://api.github.com/users/ezsystems/followers","following_url":"https://api.github.com/users/ezsystems/following{/other_user}","gists_url":"https://api.github.com/users/ezsystems/gists{/gist_id}","starred_url":"https://api.github.com/users/ezsystems/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ezsystems/subscriptions","organizations_url":"https://api.github.com/users/ezsystems/orgs","repos_url":"https://api.github.com/users/ezsystems/repos","events_url":"https://api.github.com/users/ezsystems/events{/privacy}","received_events_url":"https://api.github.com/users/ezsystems/received_events","type":"Organization","site_admin":false},"repo":{"id":1663890,"name":"ezpublish-kernel","full_name":"ezsystems/ezpublish-kernel","owner":{"login":"ezsystems","id":295311,"avatar_url":"https://avatars.githubusercontent.com/u/295311?v=3","gravatar_id":"","url":"https://api.github.com/users/ezsystems","html_url":"https://github.com/ezsystems","followers_url":"https://api.github.com/users/ezsystems/followers","following_url":"https://api.github.com/users/ezsystems/following{/other_user}","gists_url":"https://api.github.com/users/ezsystems/gists{/gist_id}","starred_url":"https://api.github.com/users/ezsystems/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ezsystems/subscriptions","organizations_url":"https://api.github.com/users/ezsystems/orgs","repos_url":"https://api.github.com/users/ezsystems/repos","events_url":"https://api.github.com/users/ezsystems/events{/privacy}","received_events_url":"https://api.github.com/users/ezsystems/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ezsystems/ezpublish-kernel","description":"Kernel (Repository, MVC layer, REST) for eZ Platform","fork":false,"url":"https://api.github.com/repos/ezsystems/ezpublish-kernel","forks_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/forks","keys_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/teams","hooks_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/hooks","issue_events_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/events{/number}","events_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/events","assignees_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/assignees{/user}","branches_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/branches{/branch}","tags_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/tags","blobs_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/refs{/sha}","trees_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/statuses/{sha}","languages_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/languages","stargazers_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/stargazers","contributors_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/contributors","subscribers_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/subscribers","subscription_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/subscription","commits_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/commits{/sha}","git_commits_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/git/commits{/sha}","comments_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/comments{/number}","issue_comment_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/comments{/number}","contents_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/contents/{+path}","compare_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/merges","archive_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/downloads","issues_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues{/number}","pulls_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls{/number}","milestones_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/milestones{/number}","notifications_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/labels{/name}","releases_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/releases{/id}","deployments_url":"https://api.github.com/repos/ezsystems/ezpublish-kernel/deployments","created_at":"2011-04-26T08:54:24Z","updated_at":"2016-11-06T09:58:27Z","pushed_at":"2016-11-18T12:28:27Z","git_url":"git://github.com/ezsystems/ezpublish-kernel.git","ssh_url":"git@github.com:ezsystems/ezpublish-kernel.git","clone_url":"https://github.com/ezsystems/ezpublish-kernel.git","svn_url":"https://github.com/ezsystems/ezpublish-kernel","homepage":"http://ezplatform.com/","size":52888,"stargazers_count":117,"watchers_count":117,"language":"PHP","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":163,"mirror_url":null,"open_issues_count":63,"forks":163,"open_issues":63,"watchers":117,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825"},"html":{"href":"https://github.com/ezsystems/ezpublish-kernel/pull/1825"},"issue":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/1825"},"comments":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/issues/1825/comments"},"review_comments":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825/comments"},"review_comment":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/pulls/1825/commits"},"statuses":{"href":"https://api.github.com/repos/ezsystems/ezpublish-kernel/statuses/341a98baf980992da0d26fdcc84c31c91fd2e032"}}}}
{"actor":{"display_login":"glye"}}