{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/3304","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/3304/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/3304/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/3304/events","html_url":"https://github.com/tensorflow/tensorflow/issues/3304","id":165456999,"number":3304,"title":"Ran out of GPU memory","user":{"login":"mbz","id":1122127,"avatar_url":"https://avatars.githubusercontent.com/u/1122127?v=3","gravatar_id":"","url":"https://api.github.com/users/mbz","html_url":"https://github.com/mbz","followers_url":"https://api.github.com/users/mbz/followers","following_url":"https://api.github.com/users/mbz/following{/other_user}","gists_url":"https://api.github.com/users/mbz/gists{/gist_id}","starred_url":"https://api.github.com/users/mbz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbz/subscriptions","organizations_url":"https://api.github.com/users/mbz/orgs","repos_url":"https://api.github.com/users/mbz/repos","events_url":"https://api.github.com/users/mbz/events{/privacy}","received_events_url":"https://api.github.com/users/mbz/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/awaiting%20googler","name":"awaiting googler","color":"b60205"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-07-14T01:28:11Z","updated_at":"2016-07-14T05:38:09Z","closed_at":null,"body":"I have a network similar to A3C network described [here](http://arxiv.org/abs/1602.01783), with lots of syncing and copying tensor values between different duplicates of the model. The code crashes with a `Ran out of memory` exception (see below). This might be a little bit hard to reproduce somewhere other than my machine (since the code is not open source ... yet) but I would be more than happy to provide any logs or help to track down the issue.\r\n\r\nWhat happens is that I receiving a lot of `pool_allocator` messages like below:\r\n\r\n> I tensorflow/core/common_runtime/gpu/pool_allocator.cc:244] PoolAllocator: After 40664481 get requests, put_count=40664484 evicted_count=39000 eviction_rate=0.000959068 and unsatisfied allocation rate=0.000959265\r\n\r\nand then finally it crashes like this:\r\n\r\n> I tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (256): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (512): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (1024): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (2048): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (4096): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (8192): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (16384): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (32768): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (65536): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (131072): \tTotal Chunks: 1, Chunks in use: 0 204.5KiB allocated for chunks. 1.0KiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (262144): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (524288): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (1048576): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (2097152): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (4194304): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (8388608): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (16777216): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (33554432): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (67108864): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (134217728): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (268435456): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:656] Bin for 220.5KiB was 128.0KiB, Chunk State: \r\nI tensorflow/core/common_runtime/bfc_allocator.cc:662]   Size: 204.5KiB | Requested Size: 1.0KiB | in_use: 0, prev:   Size: 16.0KiB | Requested Size: 16.0KiB | in_use: 1, next:   Size: 16.0KiB | Requested Size: 16.0KiB | in_use: 1\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060000 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060100 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060200 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060300 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060400 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060500 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060600 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060700 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060800 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060900 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a064a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a068a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a070a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a078a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a440a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a808a00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a808e00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a809200 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a809e00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a80aa00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a80ea00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a812a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a813600 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a81b600 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a823600 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0abeb600 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afb3600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afb3a00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afb3e00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afb4a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afb5600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afb5a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afbda00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afc1a00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afc1e00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afc9e00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afcde00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afcea00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afcee00 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afdd300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afddf00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afe1f00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afe5f00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afe6b00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afe6f00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0afeef00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0aff2f00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0aff3b00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0aff3f00 of size 39168\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0affd800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0affdc00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b005c00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b009c00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b00a800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b00ac00 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b019100 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b019d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b01dd00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b01e900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b01ed00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b022d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b023900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b023d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b02bd00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b02fd00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b030900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b030d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b038d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b03cd00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b03d900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b03dd00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b045d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b049d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b04a900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b04ad00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b052d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b056d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b057900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b057d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b05fd00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b063d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b064900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b064d00 of size 51968\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b071800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b079800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b07d800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b07e400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b07e800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b082800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b083400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b083800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b08b800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b08f800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b090400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b090800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b094800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b095400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b095800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b09d800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0a1800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0a2400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0a2800 of size 39168\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0ac100 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0b4100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0b8100 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0b8d00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0b9100 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0c1100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0c5100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0c5500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0cd500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0d1500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0d2100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0d2500 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0e0a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0e1600 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0e5600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0e5a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0e9a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0ea600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0eaa00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0f2a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0f6a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0f7600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b0f7a00 of size 39168\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b101300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b109300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b10d300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b10df00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b10e300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b116300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b11a300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b11af00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b11b300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b123300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b127300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b127f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b128300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b130300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b134300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b134f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b135300 of size 41984\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b13f700 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b147700 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b148300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b14c300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b14cf00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b14d300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b155300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b159300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b159f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b15a300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b162300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b166300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b166f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b167300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b16f300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b173300 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b173700 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b17b700 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b17f700 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b180300 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b180700 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b18ec00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b18f800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b193800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b194400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b194800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b19c800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1a0800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1a1400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1a1800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1a9800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1ad800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1adc00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1b5c00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1b9c00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1ba800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1bac00 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1c9100 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1c9d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1cdd00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1ce900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1ced00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1d2d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1d6d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1d7900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1d7d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1dfd00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1e3d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1e4900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1e4d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1ecd00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1f0d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1f1900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1f1d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1f9d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1fdd00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1fe900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b1fed00 of size 29184\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b205f00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b20df00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b20e300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b216300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b21a300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b21af00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b21b300 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b229800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b22a400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b22a800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b232800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b236800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b237400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b237800 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b245d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b246900 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b247500 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b247900 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b24f900 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b253900 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b254500 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b254900 of size 55552\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b262200 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b26a200 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b26e200 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b26ee00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b276e00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b27ae00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b27ba00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b27be00 of size 59648\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b28a700 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b28ab00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b28eb00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b28f700 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b28fb00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b297b00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b29bb00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b29c700 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b29cb00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2a4b00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2a8b00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2a9700 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2a9b00 of size 45568\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2b4d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2b8d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2bcd00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2bd900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2bdd00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2c5d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2c9d00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2ca100 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2d2100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2d6100 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2d6d00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2d7100 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2df100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2e3100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2e3500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2eb500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2ef500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2f0100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2f0500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2f8500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2fc500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2fd100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b2fd500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b305500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b309500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b30a100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b30a500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b312500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b316500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b317100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b317500 of size 21248\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b31c800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b31d400 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b31e000 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b326000 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b326c00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b327000 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b32b000 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b32bc00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b32c000 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b334000 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b338000 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b338c00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b339000 of size 39168\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b342900 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b34a900 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b34e900 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b34f500 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b34f900 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b357900 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b35b900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b35bd00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b363d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b367d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b368900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b368d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b370d00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b374d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b375900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b375d00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b37dd00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b381d00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b382900 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b382d00 of size 26880\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b389600 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b38a200 of size 33792\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b392600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b392a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b396a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0b75ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0bb26a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0beeea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0c2b6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0c67ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0ca46a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0ce0ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0d1d6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0d59ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0d966a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0dd2ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0e0f6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0e4bea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0e886a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0ec4ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0f016a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0f3dea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0f7a6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0fb6ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0ff36a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb102fea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb106c6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb10a8ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb10e56a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1121ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb115e6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb119aea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb11d76a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1213ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb12506a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb128cea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb12c96a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1305ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb13426a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb137eea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb13bb6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb13f7ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb14346a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1470ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb14ad6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb14e9ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb15266a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1562ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb159f6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb15dbea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb16186a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1654ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb16916a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb16cdea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb170a6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1746ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb17836a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb17bfea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb17fc6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1838ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb18756a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb18b1ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb18ee6a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb192aea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb19676a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb19a3ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb19e06a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1a1cea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1a596a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1a95ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ad26a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1b0eea00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1b0ef600 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1b4b7600 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1b87f600 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1bc47600 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1bc4f600 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c017600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c017a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c01fa00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c023a00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c023e00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c02be00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c02fe00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c033e00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c034a00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c034e00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c03ce00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c040e00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c041a00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c041e00 of size 51712\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c04e800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c04f400 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c417400 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c418000 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7e0000 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7e0400 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7e8400 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7ec400 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7ed000 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7ed400 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7fb900 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1c7fc500 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1cbc4500 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1cf8c500 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d354500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d358500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d359100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d359500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d361500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d365500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d366100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d366500 of size 56320\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d374100 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1d73c100 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1db04100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1db08100 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ded0100 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ded0d00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ded1100 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ded9100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1dedd100 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e2a5100 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e2a5d00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e2a6100 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e2ae100 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e2b2100 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e67a100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e67a500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e682500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e686500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e687100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e687500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e68f500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e693500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e694100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e694500 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e69c500 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e6a0500 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e6a1100 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e6a1500 of size 65024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e6b1300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1e6b1f00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ea79f00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1ee41f00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f209f00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f20df00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f5d5f00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f5d6b00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f5d6f00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f5def00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f5e2f00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9aaf00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9abb00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9abf00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9aff00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9b0b00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9b0f00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9b4f00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9b5b00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9b5f00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9bdf00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9c1f00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9c2b00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9c2f00 of size 39168\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9cc800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9d4800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9d8800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9d9400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9d9800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9e1800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9e5800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9e6400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9e6800 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9ee800 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9f2800 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9f3400 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9f3800 of size 45568\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1f9fea00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1fa06a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb1fdcea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb20196a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2055ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb20926a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb20ceea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb210b6a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb210baa00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21482a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21483600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21483a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2148ba00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2148fa00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21857a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21858600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21858a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21860a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21864a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21c2ca00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21c2d600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21c2da00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21c35a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb21c39a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22001a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22002600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22002a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2200aa00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2200ea00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb223d6a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb223d7600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb223d7a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb223dfa00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb223e3a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb227aba00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb227ac600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb227aca00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb227b4a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb227b8a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b80a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b81600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b81a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b85a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b86600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b86a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b8ea00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b92a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b93600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b93a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b9ba00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22b9fa00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22ba0600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22ba0a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22ba8a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22baca00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22bad600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22bada00 of size 45568\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22bb8c00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22bc0c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb22f88c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23350c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23718c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23ae0c00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23ae4c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23eacc00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23ead800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23eadc00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23eb5c00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb23eb9c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24281c00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24282800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24282c00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2428ac00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2428ec00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24656c00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24657800 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24657c00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2465fc00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24663c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a2bc00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a2c000 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a34000 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a38000 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a38c00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a39000 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a41000 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a45000 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a45c00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a46000 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a4e000 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a52000 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a52c00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a53000 of size 65024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a62e00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24a63a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb24e2ba00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb251f3a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb255bba00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb255bfa00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25987a00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25988600 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25988a00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25990a00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25994a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d5ca00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d5ce00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d60e00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d61200 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d69200 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d6d200 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d6de00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d6e200 of size 58624\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d7c700 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d7d300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d81300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d81f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d82300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d8a300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d8e300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d8ef00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d8f300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d97300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d9b300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d9bf00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25d9c300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25da4300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25da4f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25da5300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dad300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25db1300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25db1f00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25db2300 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dba300 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dbe300 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dbef00 of size 1024\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dbf300 of size 47872\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dcae00 of size 3072\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dcba00 of size 32768\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb25dd3a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2619ba00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb26563a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb2692ba00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb26cf3a00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb270bba00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb270f2c00 of size 16384\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb270f6c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb274bec00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb27886c00 of size 3964928\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb27c4ec00 of size 4219136\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:683] Free at 0xb270bfa00 of size 209408\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:689]      Summary of in-use Chunks by size: \r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 10 Chunks of size 256 totalling 2.5KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 125 Chunks of size 1024 totalling 125.0KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 125 Chunks of size 3072 totalling 375.0KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 122 Chunks of size 16384 totalling 1.91MiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 21248 totalling 20.8KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 26880 totalling 26.2KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 29184 totalling 28.5KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 98 Chunks of size 32768 totalling 3.06MiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 33792 totalling 33.0KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 5 Chunks of size 39168 totalling 191.2KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 41984 totalling 41.0KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 3 Chunks of size 45568 totalling 133.5KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 47872 totalling 46.8KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 51712 totalling 50.5KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 51968 totalling 50.8KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 55552 totalling 54.2KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 56320 totalling 55.0KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 9 Chunks of size 58624 totalling 515.2KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 59648 totalling 58.2KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 2 Chunks of size 65024 totalling 127.0KiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 124 Chunks of size 3964928 totalling 468.88MiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:692] 1 Chunks of size 4219136 totalling 4.02MiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:696] Sum Total of in-use chunks: 479.76MiB\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:698] Stats: \r\nLimit:                   503270604\r\nInUse:                   503061248\r\nMaxInUse:                503061248\r\nNumAllocs:                20717987\r\nMaxAllocSize:              4219136\r\n\r\n> W tensorflow/core/common_runtime/bfc_allocator.cc:270] ****************************************************************************************************\r\nW tensorflow/core/common_runtime/bfc_allocator.cc:271] Ran out of memory trying to allocate 220.5KiB.  See logs for memory state.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (256): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (512): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (1024): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (2048): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (4096): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (8192): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (16384): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (32768): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (65536): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (131072): \tTotal Chunks: 1, Chunks in use: 0 204.5KiB allocated for chunks. 1.0KiB client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (262144): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (524288): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (1048576): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (2097152): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (4194304): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (8388608): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (16777216): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (33554432): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (67108864): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (134217728): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:639] Bin (268435456): \tTotal Chunks: 0, Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 0B client-requested in use in bin.\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:656] Bin for 330.8KiB was 256.0KiB, Chunk State: \r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060000 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060100 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060200 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060300 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060400 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060500 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060600 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060700 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060800 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060900 of size 256\r\nI tensorflow/core/common_runtime/bfc_allocator.cc:674] Chunk at 0xb0a060a00 of size 16384\r\nException in thread Predictor3:\r\nTraceback (most recent call last):\r\n  File \"/home/mbz/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py\", line 715, in _do_call\r\n    return fn(*args)\r\n  File \"/home/mbz/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py\", line 697, in _run_fn\r\n    status, run_metadata)\r\n  File \"/home/mbz/anaconda3/lib/python3.5/contextlib.py\", line 66, in __exit__\r\n    next(self.gen)\r\n  File \"/home/mbz/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/errors.py\", line 450, in raise_exception_on_not_ok_status\r\n    pywrap_tensorflow.TF_GetCode(status))\r\ntensorflow.python.framework.errors.InternalError: Dst tensor is not initialized.\r\n\t [[Node: _recv_network1/X_0/_28 = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/gpu:1\", send_device=\"/job:localhost/replica:0/task:0/cpu:0\", send_device_incarnation=1, tensor_name=\"edge_9__recv_network1/X_0\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/gpu:1\"]()]]\r\n\t [[Node: network1/Squeeze/_30 = _Recv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/cpu:0\", send_device=\"/job:localhost/replica:0/task:0/gpu:1\", send_device_incarnation=1, tensor_name=\"edge_10_network1/Squeeze\", tensor_type=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"]()]]\r\n\r\n\r\n### Environment info\r\nOperating System: Ubuntu 14.04\r\n\r\nInstalled version of CUDA and cuDNN: \r\n-rw-r--r-- 1 root root   322936 Aug 15  2015 /usr/local/cuda/lib64//libcudadevrt.a\r\nlrwxrwxrwx 1 root root       16 Aug 15  2015 /usr/local/cuda/lib64//libcudart.so -> libcudart.so.7.5\r\nlrwxrwxrwx 1 root root       19 Aug 15  2015 /usr/local/cuda/lib64//libcudart.so.7.5 -> libcudart.so.7.5.18\r\n-rwxr-xr-x 1 root root   383336 Aug 15  2015 /usr/local/cuda/lib64//libcudart.so.7.5.18\r\n-rw-r--r-- 1 root root   720192 Aug 15  2015 /usr/local/cuda/lib64//libcudart_static.a\r\n-rwxr-xr-x 1 root root 61453024 May 19 18:24 /usr/local/cuda/lib64//libcudnn.so\r\n-rwxr-xr-x 1 root root 61453024 May 19 18:24 /usr/local/cuda/lib64//libcudnn.so.4\r\n-rwxr-xr-x 1 root root 61453024 May 19 18:24 /usr/local/cuda/lib64//libcudnn.so.4.0.7\r\n-rw-r--r-- 1 root root 62025862 May 19 18:24 /usr/local/cuda/lib64//libcudnn_static.a\r\n\r\nIf installed from binary pip package, provide:\r\nAnaconda3 (python3)\r\nTF v0.9 \r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/232561604","html_url":"https://github.com/tensorflow/tensorflow/issues/3304#issuecomment-232561604","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/3304","id":232561604,"user":{"login":"zheng-xq","id":15736910,"avatar_url":"https://avatars.githubusercontent.com/u/15736910?v=3","gravatar_id":"","url":"https://api.github.com/users/zheng-xq","html_url":"https://github.com/zheng-xq","followers_url":"https://api.github.com/users/zheng-xq/followers","following_url":"https://api.github.com/users/zheng-xq/following{/other_user}","gists_url":"https://api.github.com/users/zheng-xq/gists{/gist_id}","starred_url":"https://api.github.com/users/zheng-xq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zheng-xq/subscriptions","organizations_url":"https://api.github.com/users/zheng-xq/orgs","repos_url":"https://api.github.com/users/zheng-xq/repos","events_url":"https://api.github.com/users/zheng-xq/events{/privacy}","received_events_url":"https://api.github.com/users/zheng-xq/received_events","type":"User","site_admin":false},"created_at":"2016-07-14T05:38:09Z","updated_at":"2016-07-14T05:38:09Z","body":"@mbz, are you using the latest TensorFlow? Both both GPU and pinned memory, the latest TensorFlow uses BFC allocator, which has a lot less fragmentation. "}}
{"actor":{"display_login":"zheng-xq"}}