{"push_id":1442268364,"size":21,"distinct_size":18,"ref":"refs/heads/teardown-unregistered-framework","head":"7d6271ab66fa0adce8b45cd498f37026f045b2a4","before":"420476f9c000276becf4c719c6c394ba738ee3ce","commits":[{"sha":"861129913a698db59b77d867dda388053731cf89","author":{"name":"Jie Yu","email":"f7222f05d12e3ba12bb04fd30f9be9baa697b227@gmail.com"},"message":"Fixed a compiler warning on Ubuntu.","distinct":false,"url":"https://api.github.com/repos/neilconway/mesos/commits/861129913a698db59b77d867dda388053731cf89"},{"sha":"e8fe8bb6d11151940ea541a63ad1e1c4969ca7c2","author":{"name":"Jay Guo","email":"cfe742c07b3d037309d469262cfbda9a6617f89b@cn.ibm.com"},"message":"Updated master to add/remove active roles for multi-role frameworks.\n\nWhen a multi-role framework registers, the master should update the\n`activeRoles` accordingly.\n\nReview: https://reviews.apache.org/r/54360/","distinct":false,"url":"https://api.github.com/repos/neilconway/mesos/commits/e8fe8bb6d11151940ea541a63ad1e1c4969ca7c2"},{"sha":"9da19f5cc7a527a69a29a2d76c16cde8775c1b67","author":{"name":"Jie Yu","email":"f7222f05d12e3ba12bb04fd30f9be9baa697b227@gmail.com"},"message":"Waited for the domain socket file in IOSwitchboard::connect.\n\nDo not try to connect to the domain socket until the socket file has\nbeen created by the server.\n\nReview: https://reviews.apache.org/r/54472","distinct":false,"url":"https://api.github.com/repos/neilconway/mesos/commits/9da19f5cc7a527a69a29a2d76c16cde8775c1b67"},{"sha":"b9a32886d0d8cf390366f437834c967b21137c78","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Changed the allocator API to allow adding inactive frameworks.\n\nPreviously, a newly added framework was assumed to always be\nactive. Adding inactive frameworks will be used to support recovered\nframeworks that haven't yet re-registered.\n\nReview: https://reviews.apache.org/r/53895","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/b9a32886d0d8cf390366f437834c967b21137c78"},{"sha":"7f83ef1bfd1d3e7d1eddf3ba4cb923b9886cfce8","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Refactored representation of framework connectedness.\n\nPreviously, the master used a bool to track whether a given framework is\nconnected. This commit adjusts the master to use an enum instead. The\nenum currently only has two values, CONNECTED and DISCONNECTED, but an\nadditional value (RECOVERED) will be introduced shortly.\n\nReview: https://reviews.apache.org/r/53896","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/7f83ef1bfd1d3e7d1eddf3ba4cb923b9886cfce8"},{"sha":"8c2a209c09da1f140aa0e2b91064ecf632960a0f","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Changed how master represents \"recovered\" frameworks.\n\nAfter master failover, the new master doesn't know which frameworks were\nregistered with the previous master (because this information is not\ncurrently stored in the registry). In the period after the master fails\nover but before the framework scheduler has re-registered, the master\nlearns about the frameworks in the cluster when agents re-register (an\nagent reports the FrameworkInfo for all of the frameworks it is running\nwhen it re-registers).\n\nSuch frameworks were previously represented separately from the normal\nlist of frameworks in the master: the master kept a collection of\n`FrameworkInfo` for these \"recovered\" frameworks.\n\nThis commit removes this separate collection of \"recovered\" frameworks.\nInstead, the master now treats recovered frameworks very similarly to\nframeworks that are registered but currently disconnected. For example,\nrecovered frameworks will now have a `Framework` object which tracks the\ntasks/executors running under that framework; recovered frameworks will\nbe reported via the normal \"frameworks\" key when querying HTTP\nendpoints. Similarly, \"teardown\" operations on recovered frameworks will\nnow work correctly (MESOS-6419).\n\nThis means there is no longer a concept of \"orphan tasks\" [1]: if the\nmaster knows about a task, the task will be running under a framework\n(albeit the framework might be recovered or disconnected). A new\n\"recovered\" key has been added to various HTTP endpoints/APIs to\ndetermine if a framework hasn't yet re-registered after master failover.\n\nAnother behavioral change is that, previously, a framework\nre-registering after master failover was allowed to make arbitrary\nchanges to its `FrameworkInfo`. The master now only applies updates to\nFrameworkInfo fields that can be safely changed (as was already done\nwhen a framework fails over to a master that hasn't failed over). The\nprevious behavior qualifies as a bug.\n\n[1] The exception here is if the cluster contains Mesos agents older\nthan 1.0, because old Mesos agents don't report `FrameworkInfo`s when\nthey re-register.\n\nReview: https://reviews.apache.org/r/53897","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/8c2a209c09da1f140aa0e2b91064ecf632960a0f"},{"sha":"0ae1fbff3d6a0675404d039fdd1322c9beaaf639","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Fixed incorrect warning messages.\n\nThese warning messages printed the old value of the `FrameworkInfo`\nfield, not the new (ignored) value.\n\nReview: https://reviews.apache.org/r/54467","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/0ae1fbff3d6a0675404d039fdd1322c9beaaf639"},{"sha":"be4c25c1c6a5950957eacfccb54b96351632397b","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Added some additional CHECKs to the master.\n\nReview: https://reviews.apache.org/r/54468","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/be4c25c1c6a5950957eacfccb54b96351632397b"},{"sha":"32ee6d17fe866b97ae68a5eb4b6959dada137e80","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Don't send PIDs of disconnected frameworks to re-registering agents.\n\nReview: https://reviews.apache.org/r/54380","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/32ee6d17fe866b97ae68a5eb4b6959dada137e80"},{"sha":"478d66def0874ec9d4aba9f94091ee0c8e628ee5","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Removed an inaccurate comment.\n\nReview: https://reviews.apache.org/r/54387","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/478d66def0874ec9d4aba9f94091ee0c8e628ee5"},{"sha":"741c548089f549cdf2ea4e8f42442d5859e93c84","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Simplified some test code.\n\nReview: https://reviews.apache.org/r/54177","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/741c548089f549cdf2ea4e8f42442d5859e93c84"},{"sha":"680f29d458a94e5abda4c0795a5c486b279db499","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Added a fixed-capacity hashmap to stout.\n\nThis is similar to the LinkedHashMap and Cache constructs.\n\nUnlike LinkedHashMap, a BoundedHashMap has a fixed maximum capacity:\nwhen the BoundedHashMap is at capacity, inserting another element\nresults in removing the oldest element still in the map (i.e., FIFO\norder is used for eviction).\n\nUnlike Cache, reading an element from a BoundedHashMap does not change\nwhich entry is evicted next.\n\nReview: https://reviews.apache.org/r/54178","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/680f29d458a94e5abda4c0795a5c486b279db499"},{"sha":"fc99af4f0f680ee45a8c213d6e06a9ca55575700","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Updated master to use BoundedHashMap.\n\nThis replaces a few instances where boost::circular_buffer was\npreviously used.\n\nReview: https://reviews.apache.org/r/54179","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/fc99af4f0f680ee45a8c213d6e06a9ca55575700"},{"sha":"acc24935bba492732cb36acd432e750d7d95523b","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Updated agent to use BoundedHashMap.\n\nReview: https://reviews.apache.org/r/54180","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/acc24935bba492732cb36acd432e750d7d95523b"},{"sha":"3dae6a3f12ee88e9a55f01b8d7657644dd6fec65","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Synchronized v1 and unversioned master.proto.\n\nReview: https://reviews.apache.org/r/54181","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/3dae6a3f12ee88e9a55f01b8d7657644dd6fec65"},{"sha":"675ab25283115db2a2e3f526c751acf6e189b73c","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Improved various comments.\n\nReview: https://reviews.apache.org/r/54182","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/675ab25283115db2a2e3f526c751acf6e189b73c"},{"sha":"5121cbcc9093ed38404b0bec1470a23e1e3cb6af","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Improved management of unreachable and completed tasks in master.\n\nBefore partition-awareness, when an agent failed health checks, the\nmaster removed the agent from the registry, marked all of its tasks\nTASK_LOST, and moved them to the `completedTasks` list in the master's\nmemory. Although \"lost\" tasks might still be running, partitioned agents\nwould only be allowed to re-register if the master failed over, in which\ncase the `completedTasks` map would be emptied.\n\nWhen partition-awareness was introduced, we initially followed the same\nscheme, with the only difference that partition-aware tasks are marked\nTASK_UNREACHABLE, not TASK_LOST.\n\nThis scheme has a few shortcomings. First, partition-aware tasks might\nresume running when the partitioned agent re-registers. Second, we\nre-added non-partition aware tasks when the agent re-registered but then\nmarked them completed when the framework is shutdown, resulting in two\nentries in `completedTasks`.\n\nThis commit introduces a separate bounded map, `unreachableTasks`. These\ntasks are reported separately via the HTTP endpoints, because they have\ndifferent semantics (unlike completed tasks, unreachable tasks can\nresume running). The size of this map is limited by a new master flag,\n`--max_unreachable_tasks_per_framework`. This commit also changes the\nmaster to omit re-adding non-partition-aware tasks on re-registering\nagents (unless the master has failed over): those tasks will shortly be\nshutdown anyway.\n\nFinally, this commit fixes a minor bug in the previous code: the\nprevious coding neglected to shutdown non-partition-aware frameworks\nrunning on pre-1.0 Mesos agents that re-register with the master after\na network partition.\n\nReview: https://reviews.apache.org/r/54183","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/5121cbcc9093ed38404b0bec1470a23e1e3cb6af"},{"sha":"3877754af4fc1b018b30019e0ea91a7ceaaaf049","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Shutdown tasks of completed frameworks on agent re-registration.\n\nPreviously, if a framework completed (e.g., due to a teardown operation\nor framework shutdown), any framework tasks running on partitioned\nagents would not be shutdown when the agent re-registered. For tasks\nthat are not partition-aware, the task would be shutdown on agent\nre-registration anyway. But for partition-aware tasks, this could lead\nto orphan tasks.\n\nFix this by changing the master to shutdown such tasks when the agent\nreregisters.\n\nNote that if the master fails over between the time the framework\ncompletes and a partitioned agent re-registers, any framework tasks\nrunning on the agent will NOT be shutdown. This is a known bug; fixing\nit requires persisting the framework shutdown operation to the registry\n(MESOS-1719).\n\nReview: https://reviews.apache.org/r/54232","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/3877754af4fc1b018b30019e0ea91a7ceaaaf049"},{"sha":"6039ffce01cac32ab136458619334adb167c1d99","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Added TASK_UNREACHABLE to master's state-summary endpoint.\n\nReview: https://reviews.apache.org/r/54312","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/6039ffce01cac32ab136458619334adb167c1d99"},{"sha":"b109f79f04e20fd51fbf8bea42afdfd51393232b","author":{"name":"Neil Conway","email":"4f41ed19fc3accf446ec9111a93682686fd4d3d9@gmail.com"},"message":"Refactored Master::removeFramework to use Master::deactivate.","distinct":true,"url":"https://api.github.com/repos/neilconway/mesos/commits/b109f79f04e20fd51fbf8bea42afdfd51393232b"}]}
{"actor":{"display_login":"neilconway"}}