{"action":"created","issue":{"url":"https://api.github.com/repos/mongodb-labs/mongo-connector/issues/460","repository_url":"https://api.github.com/repos/mongodb-labs/mongo-connector","labels_url":"https://api.github.com/repos/mongodb-labs/mongo-connector/issues/460/labels{/name}","comments_url":"https://api.github.com/repos/mongodb-labs/mongo-connector/issues/460/comments","events_url":"https://api.github.com/repos/mongodb-labs/mongo-connector/issues/460/events","html_url":"https://github.com/mongodb-labs/mongo-connector/issues/460","id":157007111,"number":460,"title":"mongo-connector works with oldest oplog entry","user":{"login":"EduVal","id":5978509,"avatar_url":"https://avatars.githubusercontent.com/u/5978509?v=3","gravatar_id":"","url":"https://api.github.com/users/EduVal","html_url":"https://github.com/EduVal","followers_url":"https://api.github.com/users/EduVal/followers","following_url":"https://api.github.com/users/EduVal/following{/other_user}","gists_url":"https://api.github.com/users/EduVal/gists{/gist_id}","starred_url":"https://api.github.com/users/EduVal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EduVal/subscriptions","organizations_url":"https://api.github.com/users/EduVal/orgs","repos_url":"https://api.github.com/users/EduVal/repos","events_url":"https://api.github.com/users/EduVal/events{/privacy}","received_events_url":"https://api.github.com/users/EduVal/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mongodb-labs/mongo-connector/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2016-05-26T15:10:24Z","updated_at":"2016-06-22T16:19:56Z","closed_at":null,"body":"Hello,\r\n\r\nit seems to me, that mongo-connector doesn't save the latest mongodb oplog entry in oplog.timestamp, but the oldest point. \r\n\r\nFor example:\r\nsudo mongo-connector -m <origin_mongodb_node>:27017 -t mongodb://<mongodb_user:mongodb_password>@<target_mongodb_node>:27017 -d mongo_doc_manager -v --stdout  -a <origin_mongodb_user> -p <origin_mongodb_password> --batch-size 100 --no-dump\r\n\r\nIt create a file oplog.timestamp\r\n[\"Collection(Database(MongoClient(['db-dev-xxx.lhotse.ov.otto.de:27017', u'db-dev-xxx.lhotse.ov.otto.de:27017\r\n', u'db-dev-xxx.lhotse.ov.otto.de:27017']), u'local'), u'oplog.rs')\", 6288710523760410669]\r\n\r\nIn my point of view the number 6288710523760410669 should be the current / latest oplog entry, but \r\nwhen you start mongo-connector again it  prints an unix timestamp =1464204519 , which is pointing to the oldest oplog entry of origin database.\r\n\r\n2016-05-26 16:59:45,165 [DEBUG] mongo_connector.oplog_manager:747 - OplogThread: reading last checkpoint as Timestamp(1464204519, 45) \r\n2016-05-26 16:59:47,910 [ERROR] mongo_connector.oplog_manager:203 - OplogThread: Last entry no longer in oplog cannot recover! Collection(Database(MongoClient(['db-dev-xxx.lhotse.ov.otto.de:27017', u'db-dev-xxx.lhotse.ov.otto.de:27017', u'db-dev-xxx.lhotse.ov.otto.de:27017']), u'local'), u'oplog.rs')\r\n\r\nother informations:\r\nsudo python --version => Python 2.7.6\r\nmongodb version => 3.0.10 \r\nmongodb storage engine => wiredtiger\r\n\r\n\r\nHow it ist possible to convert the last checkpoint (6288710523760410669) in oplog.timestamp file to unix timestamp (1464204519) format in python?\r\n\r\nThank you for helping\r\nEduard\r\n"},"comment":{"url":"https://api.github.com/repos/mongodb-labs/mongo-connector/issues/comments/227797230","html_url":"https://github.com/mongodb-labs/mongo-connector/issues/460#issuecomment-227797230","issue_url":"https://api.github.com/repos/mongodb-labs/mongo-connector/issues/460","id":227797230,"user":{"login":"llvtt","id":5168837,"avatar_url":"https://avatars.githubusercontent.com/u/5168837?v=3","gravatar_id":"","url":"https://api.github.com/users/llvtt","html_url":"https://github.com/llvtt","followers_url":"https://api.github.com/users/llvtt/followers","following_url":"https://api.github.com/users/llvtt/following{/other_user}","gists_url":"https://api.github.com/users/llvtt/gists{/gist_id}","starred_url":"https://api.github.com/users/llvtt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/llvtt/subscriptions","organizations_url":"https://api.github.com/users/llvtt/orgs","repos_url":"https://api.github.com/users/llvtt/repos","events_url":"https://api.github.com/users/llvtt/events{/privacy}","received_events_url":"https://api.github.com/users/llvtt/received_events","type":"User","site_admin":false},"created_at":"2016-06-22T16:19:55Z","updated_at":"2016-06-22T16:19:55Z","body":"The timestamp file is only periodically updated, so examining the file after mongo-connector has been running doesn't guarantee that the timestamp is the latest one: https://github.com/mongodb-labs/mongo-connector/wiki/FAQ#my-oplog-progress-file-always-seems-really-out-of-date-whats-going-on\r\n\r\nThe instructions in https://github.com/mongodb-labs/mongo-connector/wiki/Usage%20with%20MongoDB assume that the oplog timestamp file doesn't yet exist, and that you are running mongo-connector for the first time in order to transfer documents to another MongoDB cluster."}}
{"actor":{"display_login":"llvtt"}}