{"action":"created","issue":{"url":"https://api.github.com/repos/EOSIO/eos/issues/5107","repository_url":"https://api.github.com/repos/EOSIO/eos","labels_url":"https://api.github.com/repos/EOSIO/eos/issues/5107/labels{/name}","comments_url":"https://api.github.com/repos/EOSIO/eos/issues/5107/comments","events_url":"https://api.github.com/repos/EOSIO/eos/issues/5107/events","html_url":"https://github.com/EOSIO/eos/issues/5107","id":348614066,"node_id":"MDU6SXNzdWUzNDg2MTQwNjY=","number":5107,"title":"3060101 database_guard_exception: Database usage is at unsafe levels","user":{"login":"cppfuns","id":7980920,"node_id":"MDQ6VXNlcjc5ODA5MjA=","avatar_url":"https://avatars2.githubusercontent.com/u/7980920?v=4","gravatar_id":"","url":"https://api.github.com/users/cppfuns","html_url":"https://github.com/cppfuns","followers_url":"https://api.github.com/users/cppfuns/followers","following_url":"https://api.github.com/users/cppfuns/following{/other_user}","gists_url":"https://api.github.com/users/cppfuns/gists{/gist_id}","starred_url":"https://api.github.com/users/cppfuns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cppfuns/subscriptions","organizations_url":"https://api.github.com/users/cppfuns/orgs","repos_url":"https://api.github.com/users/cppfuns/repos","events_url":"https://api.github.com/users/cppfuns/events{/privacy}","received_events_url":"https://api.github.com/users/cppfuns/received_events","type":"User","site_admin":false},"labels":[{"id":936623288,"node_id":"MDU6TGFiZWw5MzY2MjMyODg=","url":"https://api.github.com/repos/EOSIO/eos/labels/Support","name":"Support","color":"7d7ad6","default":false}],"state":"open","locked":false,"assignee":{"login":"sergmetelin","id":2141500,"node_id":"MDQ6VXNlcjIxNDE1MDA=","avatar_url":"https://avatars1.githubusercontent.com/u/2141500?v=4","gravatar_id":"","url":"https://api.github.com/users/sergmetelin","html_url":"https://github.com/sergmetelin","followers_url":"https://api.github.com/users/sergmetelin/followers","following_url":"https://api.github.com/users/sergmetelin/following{/other_user}","gists_url":"https://api.github.com/users/sergmetelin/gists{/gist_id}","starred_url":"https://api.github.com/users/sergmetelin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sergmetelin/subscriptions","organizations_url":"https://api.github.com/users/sergmetelin/orgs","repos_url":"https://api.github.com/users/sergmetelin/repos","events_url":"https://api.github.com/users/sergmetelin/events{/privacy}","received_events_url":"https://api.github.com/users/sergmetelin/received_events","type":"User","site_admin":false},"assignees":[{"login":"sergmetelin","id":2141500,"node_id":"MDQ6VXNlcjIxNDE1MDA=","avatar_url":"https://avatars1.githubusercontent.com/u/2141500?v=4","gravatar_id":"","url":"https://api.github.com/users/sergmetelin","html_url":"https://github.com/sergmetelin","followers_url":"https://api.github.com/users/sergmetelin/followers","following_url":"https://api.github.com/users/sergmetelin/following{/other_user}","gists_url":"https://api.github.com/users/sergmetelin/gists{/gist_id}","starred_url":"https://api.github.com/users/sergmetelin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sergmetelin/subscriptions","organizations_url":"https://api.github.com/users/sergmetelin/orgs","repos_url":"https://api.github.com/users/sergmetelin/repos","events_url":"https://api.github.com/users/sergmetelin/events{/privacy}","received_events_url":"https://api.github.com/users/sergmetelin/received_events","type":"User","site_admin":false}],"milestone":null,"comments":12,"created_at":"2018-08-08T07:52:45Z","updated_at":"2018-09-25T13:45:21Z","closed_at":null,"author_association":"NONE","body":"root@iZj6c7toj4996n194z4h2pZ:~# nodeos --data-dir /mnt/data/data/ --hard-replay-blockchain\r\n2018-08-08T07:13:19.131 thread-0   chain_plugin.cpp:271          plugin_initialize    ] initializing chain plugin\r\n2018-08-08T07:13:19.132 thread-0   chain_plugin.cpp:414          plugin_initialize    ] Hard replay requested: deleting state database\r\n2018-08-08T07:13:19.520 thread-0   block_log.cpp:325             repair_log           ] Recovering Block Log...\r\n2018-08-08T07:13:19.520 thread-0   block_log.cpp:345             repair_log           ] Moved existing blocks directory to backup location: '/mnt/data/data/blocks-2018-08-08T07:13:19.520'\r\n2018-08-08T07:13:19.521 thread-0   block_log.cpp:350             repair_log           ] Reconstructing '/mnt/data/data/blocks/blocks.log' from backed up block log\r\n\r\n\r\n2018-08-08T07:16:03.886 thread-0   block_log.cpp:445             repair_log           ] Recovered only up to block number 9570628. The block 9570629 could not be deserialized from the block log due to error:\r\nAssert Exception\r\n2018-08-08T07:16:06.499 thread-0   block_log.cpp:455             repair_log           ] Data at tail end of block log which should contain the (incomplete) serialization of block 9570629 has been written out to '/mnt/data/data/blocks/blocks-bad-tail-2018-08-08T07:13:19.520.log'.\r\n2018-08-08T07:16:06.746 thread-0   chain_plugin.cpp:681          recover_reversible_b ] Reconstructing '/mnt/data/data/blocks/reversible' from backed up reversible directory\r\n2018-08-08T07:16:08.887 thread-0   chain_plugin.cpp:733          recover_reversible_b ] Recovered 343 blocks from reversible block database: blocks 10012920 to 10013262\r\n2018-08-08T07:16:12.010 thread-0   block_log.cpp:124             open                 ] Log is nonempty\r\n2018-08-08T07:16:12.010 thread-0   block_log.cpp:159             open                 ] Index is empty\r\n2018-08-08T07:16:12.010 thread-0   block_log.cpp:298             construct_index      ] Reconstructing Block Log Index...\r\n2018-08-08T07:17:23.863 thread-0   http_plugin.cpp:344           plugin_initialize    ] configured http to listen on 0.0.0.0:8888\r\n2018-08-08T07:17:23.865 thread-0   history_plugin.cpp:275        plugin_initialize    ] --filter-on * enabled. This can fill shared_mem, causing nodeos to stop.\r\n2018-08-08T07:17:23.867 thread-0   net_plugin.cpp:2919           plugin_initialize    ] Initialize net plugin\r\n2018-08-08T07:17:23.868 thread-0   net_plugin.cpp:2944           plugin_initialize    ] host: 0.0.0.0 port: 9876 \r\n2018-08-08T07:17:23.870 thread-0   net_plugin.cpp:3014           plugin_initialize    ] my node_id is 74f8379ab3548acd26575f44e35876213f02982fe42edcca6725e70c643a0e33\r\n2018-08-08T07:17:23.876 thread-0   http_plugin.cpp:299           operator()           ] configured http with Access-Control-Allow-Origin: *\r\n2018-08-08T07:17:23.877 thread-0   main.cpp:105                  main                 ] nodeos version 8777d8db\r\n2018-08-08T07:17:23.877 thread-0   main.cpp:106                  main                 ] eosio root is /root/.local/share\r\n2018-08-08T07:17:23.892 thread-0   controller.cpp:1256           startup              ] No head block in fork db, perhaps we need to replay\r\n2018-08-08T07:17:23.892 thread-0   controller.cpp:319            initialize_fork_db   ]  Initializing new blockchain with genesis state                  \r\n2018-08-08T07:17:23.929 thread-0   controller.cpp:215            init                 ] existing block log, attempting to replay 9570628 blocks\r\n2018-08-08T07:43:25.133 thread-0   wasm_interface.cpp:929        eosio_assert         ] message: refund is not available yet \r\n2018-08-08T07:47:21.970 thread-0   controller.cpp:612            push_scheduled_trans ] 3080004 tx_cpu_usage_exceeded: Transaction exceeded the current CPU usage limit imposed on the transaction\r\nbilled CPU time (9135 us) is greater than the maximum billable CPU time for the transaction (8751 us)\r\n    {\"billed\":9135,\"billable\":8751}\r\n    thread-0  transaction_context.cpp:361 validate_cpu_usage_to_bill\r\n2018-08-08T07:48:02.716 thread-0   controller.cpp:612            push_scheduled_trans ] 3080004 tx_cpu_usage_exceeded: Transaction exceeded the current CPU usage limit imposed on the transaction\r\nbilled CPU time (11027 us) is greater than the maximum billable CPU time for the transaction (8752 us)\r\n    {\"billed\":11027,\"billable\":8752}\r\n    thread-0  transaction_context.cpp:361 validate_cpu_usage_to_bill\r\n2018-08-08T07:48:21.075 thread-0   wasm_interface.cpp:929        eosio_assert         ] message: refund is not available yet \r\n2018-08-08T07:48:21.078 thread-0   controller.cpp:161            emit                 ] signal handler threw exception\r\n2018-08-08T07:48:27.088 thread-0   wasm_interface.cpp:929        eosio_assert         ] message: refund request not found \r\n2018-08-08T07:48:27.088 thread-0   controller.cpp:161            emit                 ] signal handler threw exception\r\n2018-08-08T07:48:27.340 thread-0   wasm_interface.cpp:929        eosio_assert         ] message: refund is not available yet \r\n2018-08-08T07:48:27.340 thread-0   controller.cpp:161            emit                 ] signal handler threw exception\r\n2018-08-08T07:48:41.244 thread-0   controller.cpp:612            push_scheduled_trans ] 3080004 tx_cpu_usage_exceeded: Transaction exceeded the current CPU usage limit imposed on the transaction\r\nbilled CPU time (19455 us) is greater than the maximum billable CPU time for the transaction (17510 us)\r\n    {\"billed\":19455,\"billable\":17510}\r\n    thread-0  transaction_context.cpp:361 validate_cpu_usage_to_bill\r\n2018-08-08T07:48:45.847 thread-0   wasm_interface.cpp:929        eosio_assert         ] message: refund request not found \r\n2018-08-08T07:48:45.847 thread-0   controller.cpp:161            emit                 ] signal handler threw exception\r\n2018-08-08T07:48:46.845 thread-0   wasm_interface.cpp:929        eosio_assert         ] message: refund request not found \r\n2018-08-08T07:48:46.846 thread-0   controller.cpp:161            emit                 ] signal handler threw exception\r\n2018-08-08T07:49:46.408 thread-0   controller.cpp:612            push_scheduled_trans ] 3080004 tx_cpu_usage_exceeded: Transaction exceeded the current CPU usage limit imposed on the transaction\r\nbilled CPU time (1380 us) is greater than the maximum billable CPU time for the transaction (817 us)\r\n    {\"billed\":1380,\"billable\":817}\r\n    thread-0  transaction_context.cpp:361 validate_cpu_usage_to_bill\r\n2018-08-08T07:50:14.561 thread-0   controller.cpp:612            push_scheduled_trans ] 3080004 tx_cpu_usage_exceeded: Transaction exceeded the current CPU usage limit imposed on the transaction\r\nbilled CPU time (1321 us) is greater than the maximum billable CPU time for the transaction (743 us)\r\n    {\"billed\":1321,\"billable\":743}\r\n    thread-0  transaction_context.cpp:361 validate_cpu_usage_to_bill\r\n2018-08-08T07:50:48.420 thread-0   chain_plugin.cpp:853          log_guard_exception  ] Database has reached an unsafe level of usage, shutting down to avoid corrupting the database.  Please increase the value set for \"chain-state-db-size-mb\" and restart the process!\r\n2018-08-08T07:50:48.420 thread-0   chain_plugin.cpp:859          log_guard_exception  ] Details: 3060101 database_guard_exception: Database usage is at unsafe levels\r\ndatabase free: 134217584, guard size: 134217728\r\n    {\"f\":134217584,\"g\":134217728}\r\n    thread-0  controller.cpp:1627 validate_db_available_size\r\n2018-08-08T07:50:49.091 thread-0   main.cpp:125                  main                 ] 3060101 database_guard_exception: Database usage is at unsafe levels\r\ndatabase free: 134217584, guard size: 134217728\r\n    {\"f\":134217584,\"g\":134217728}\r\n    thread-0  controller.cpp:1627 validate_db_available_size\r\n\r\n    {}\r\n    thread-0  chain_plugin.cpp:603 plugin_startup"},"comment":{"url":"https://api.github.com/repos/EOSIO/eos/issues/comments/424349197","html_url":"https://github.com/EOSIO/eos/issues/5107#issuecomment-424349197","issue_url":"https://api.github.com/repos/EOSIO/eos/issues/5107","id":424349197,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNDM0OTE5Nw==","user":{"login":"jacktang","id":44341,"node_id":"MDQ6VXNlcjQ0MzQx","avatar_url":"https://avatars3.githubusercontent.com/u/44341?v=4","gravatar_id":"","url":"https://api.github.com/users/jacktang","html_url":"https://github.com/jacktang","followers_url":"https://api.github.com/users/jacktang/followers","following_url":"https://api.github.com/users/jacktang/following{/other_user}","gists_url":"https://api.github.com/users/jacktang/gists{/gist_id}","starred_url":"https://api.github.com/users/jacktang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jacktang/subscriptions","organizations_url":"https://api.github.com/users/jacktang/orgs","repos_url":"https://api.github.com/users/jacktang/repos","events_url":"https://api.github.com/users/jacktang/events{/privacy}","received_events_url":"https://api.github.com/users/jacktang/received_events","type":"User","site_admin":false},"created_at":"2018-09-25T13:45:21Z","updated_at":"2018-09-25T13:45:21Z","author_association":"NONE","body":"same issue here, v1.2.5\r\n```\r\n2018-09-25T13:31:30.733 thread-0   chain_plugin.cpp:915          log_guard_exception  ] Database has reached an unsafe level of usage, shutting down to avoid corrupting the database.  Please increase the value set for \"chain-state-db-size-mb\" and restart the process!\r\n2018-09-25T13:31:30.733 thread-0   chain_plugin.cpp:921          log_guard_exception  ] Details: 3060101 database_guard_exception: Database usage is at unsafe levels\r\ndatabase free: 134215200, guard size: 134217728\r\n    {\"f\":134215200,\"g\":134217728}\r\n    thread-0  controller.cpp:1843 validate_db_available_size\r\n```\r\nand change the value to 102400 (MB), the error\r\n```\r\n2018-09-25T13:43:26.885 thread-0   chain_plugin.cpp:317          plugin_initialize    ] initializing chain plugin\r\n2018-09-25T13:43:26.941 thread-0   chain_plugin.cpp:632          plugin_initialize    ] 13 N5boost16exception_detail10clone_implINS0_19error_info_injectorISt13runtime_errorEEEE: could not grow database file to requested size.\r\nrethrow could not grow database file to requested size.:\r\n    {\"what\":\"could not grow database file to requested size.\"}\r\n    thread-0  chain_plugin.cpp:632 plugin_initialize\r\n2018-09-25T13:43:26.941 thread-0   main.cpp:125                  main                 ] 13 N5boost16exception_detail10clone_implINS0_19error_info_injectorISt13runtime_errorEEEE: could not grow database file to requested size.\r\nrethrow could not grow database file to requested size.:\r\n    {\"what\":\"could not grow database file to requested size.\"}\r\n    thread-0  chain_plugin.cpp:632 plugin_initialize\r\n```"}}
{"actor":{"display_login":"jacktang"}}