{"action":"created","issue":{"url":"https://api.github.com/repos/ethereum/aleth/issues/5300","repository_url":"https://api.github.com/repos/ethereum/aleth","labels_url":"https://api.github.com/repos/ethereum/aleth/issues/5300/labels{/name}","comments_url":"https://api.github.com/repos/ethereum/aleth/issues/5300/comments","events_url":"https://api.github.com/repos/ethereum/aleth/issues/5300/events","html_url":"https://github.com/ethereum/aleth/issues/5300","id":367419433,"node_id":"MDU6SXNzdWUzNjc0MTk0MzM=","number":5300,"title":"Create database performance tests","user":{"login":"halfalicious","id":15605026,"node_id":"MDQ6VXNlcjE1NjA1MDI2","avatar_url":"https://avatars1.githubusercontent.com/u/15605026?v=4","gravatar_id":"","url":"https://api.github.com/users/halfalicious","html_url":"https://github.com/halfalicious","followers_url":"https://api.github.com/users/halfalicious/followers","following_url":"https://api.github.com/users/halfalicious/following{/other_user}","gists_url":"https://api.github.com/users/halfalicious/gists{/gist_id}","starred_url":"https://api.github.com/users/halfalicious/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/halfalicious/subscriptions","organizations_url":"https://api.github.com/users/halfalicious/orgs","repos_url":"https://api.github.com/users/halfalicious/repos","events_url":"https://api.github.com/users/halfalicious/events{/privacy}","received_events_url":"https://api.github.com/users/halfalicious/received_events","type":"User","site_admin":false},"labels":[{"id":1081139122,"node_id":"MDU6TGFiZWwxMDgxMTM5MTIy","url":"https://api.github.com/repos/ethereum/aleth/labels/database","name":"database","color":"8e4bc1","default":false}],"state":"open","locked":false,"assignee":{"login":"halfalicious","id":15605026,"node_id":"MDQ6VXNlcjE1NjA1MDI2","avatar_url":"https://avatars1.githubusercontent.com/u/15605026?v=4","gravatar_id":"","url":"https://api.github.com/users/halfalicious","html_url":"https://github.com/halfalicious","followers_url":"https://api.github.com/users/halfalicious/followers","following_url":"https://api.github.com/users/halfalicious/following{/other_user}","gists_url":"https://api.github.com/users/halfalicious/gists{/gist_id}","starred_url":"https://api.github.com/users/halfalicious/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/halfalicious/subscriptions","organizations_url":"https://api.github.com/users/halfalicious/orgs","repos_url":"https://api.github.com/users/halfalicious/repos","events_url":"https://api.github.com/users/halfalicious/events{/privacy}","received_events_url":"https://api.github.com/users/halfalicious/received_events","type":"User","site_admin":false},"assignees":[{"login":"halfalicious","id":15605026,"node_id":"MDQ6VXNlcjE1NjA1MDI2","avatar_url":"https://avatars1.githubusercontent.com/u/15605026?v=4","gravatar_id":"","url":"https://api.github.com/users/halfalicious","html_url":"https://github.com/halfalicious","followers_url":"https://api.github.com/users/halfalicious/followers","following_url":"https://api.github.com/users/halfalicious/following{/other_user}","gists_url":"https://api.github.com/users/halfalicious/gists{/gist_id}","starred_url":"https://api.github.com/users/halfalicious/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/halfalicious/subscriptions","organizations_url":"https://api.github.com/users/halfalicious/orgs","repos_url":"https://api.github.com/users/halfalicious/repos","events_url":"https://api.github.com/users/halfalicious/events{/privacy}","received_events_url":"https://api.github.com/users/halfalicious/received_events","type":"User","site_admin":false}],"milestone":null,"comments":10,"created_at":"2018-10-06T03:35:21Z","updated_at":"2018-10-09T17:54:28Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Since we're working on making Aleth compatible with different databases (e.g. RocksDB i.e. #4037, potentially SQLite), it would be helpful to have database performance tests so we can have quantitative benchmarks which measure how Aleth performs with each database backend. Some initial thoughts:\r\n* The tests should not run as a part of the CircleCI test suite (i.e. the tests which run whenever new changes are pushed to a PR) because we're more interested in how the different databases compare relative to one another, not whether or not they meet certain absolute performance targets. \r\n* CircleCI tests would need to have pass/fail results...for perf tests this naturally means setting perf targets and basing pass/fail on meeting/not meeting the targets, but test performance would likely fluctuate if the tests weren't consistently run on the same hardware and in a clean/quieted OS environment (e.g. unnecessary services turned off, no other applications running so hardware would need to be dedicated to just running the tests i.e. no shared servers) which would result in intermittent failures. \r\n* The tests would need to run against each database implementation, which means that the test binary would need to compile/link in each database implementation which would increase build times. \r\n\r\nThe approach should rather be to have tests which one can compile and run locally. The tests should load blocks from disk and not from the network (so they aren't impacted by network latency) and should be run against each database implementation - this can be achieved easily enough by iterating through the `DatabaseKind` values defined in `db.h` and creating a new `DBFactory `instance for each one:\r\nhttps://github.com/ethereum/aleth/blob/f838dd4151a939ac1ac3b4cb720bb0283b24353e/libdevcore/DBFactory.h#L30-L34\r\n\r\nIf the tests use `TestEth ` they should be marked such that they can easily be excluded from CircleCI runs (e.g. via  a command-line switch - @chfast  mentioned that TestEth already has a performance test option which we can leverage to achieve this). \r\n\r\nFinally, the tests should output a table summarizing the results for each database type so one can easily compare the results among the different databases. "},"comment":{"url":"https://api.github.com/repos/ethereum/aleth/issues/comments/428288200","html_url":"https://github.com/ethereum/aleth/issues/5300#issuecomment-428288200","issue_url":"https://api.github.com/repos/ethereum/aleth/issues/5300","id":428288200,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODI4ODIwMA==","user":{"login":"halfalicious","id":15605026,"node_id":"MDQ6VXNlcjE1NjA1MDI2","avatar_url":"https://avatars1.githubusercontent.com/u/15605026?v=4","gravatar_id":"","url":"https://api.github.com/users/halfalicious","html_url":"https://github.com/halfalicious","followers_url":"https://api.github.com/users/halfalicious/followers","following_url":"https://api.github.com/users/halfalicious/following{/other_user}","gists_url":"https://api.github.com/users/halfalicious/gists{/gist_id}","starred_url":"https://api.github.com/users/halfalicious/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/halfalicious/subscriptions","organizations_url":"https://api.github.com/users/halfalicious/orgs","repos_url":"https://api.github.com/users/halfalicious/repos","events_url":"https://api.github.com/users/halfalicious/events{/privacy}","received_events_url":"https://api.github.com/users/halfalicious/received_events","type":"User","site_admin":false},"created_at":"2018-10-09T17:54:28Z","updated_at":"2018-10-09T17:54:28Z","author_association":"CONTRIBUTOR","body":"Thanks, I'll file a new issue to track this in case we want to look into it later. "}}
{"actor":{"display_login":"halfalicious"}}