{"action":"created","issue":{"url":"https://api.github.com/repos/openssl/openssl/issues/1682","repository_url":"https://api.github.com/repos/openssl/openssl","labels_url":"https://api.github.com/repos/openssl/openssl/issues/1682/labels{/name}","comments_url":"https://api.github.com/repos/openssl/openssl/issues/1682/comments","events_url":"https://api.github.com/repos/openssl/openssl/issues/1682/events","html_url":"https://github.com/openssl/openssl/pull/1682","id":181996218,"number":1682,"title":"WIP: Integrate boring tests","user":{"login":"mattcaswell","id":7422273,"avatar_url":"https://avatars.githubusercontent.com/u/7422273?v=3","gravatar_id":"","url":"https://api.github.com/users/mattcaswell","html_url":"https://github.com/mattcaswell","followers_url":"https://api.github.com/users/mattcaswell/followers","following_url":"https://api.github.com/users/mattcaswell/following{/other_user}","gists_url":"https://api.github.com/users/mattcaswell/gists{/gist_id}","starred_url":"https://api.github.com/users/mattcaswell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattcaswell/subscriptions","organizations_url":"https://api.github.com/users/mattcaswell/orgs","repos_url":"https://api.github.com/users/mattcaswell/repos","events_url":"https://api.github.com/users/mattcaswell/events{/privacy}","received_events_url":"https://api.github.com/users/mattcaswell/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":39,"created_at":"2016-10-10T11:52:03Z","updated_at":"2016-10-14T15:40:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/openssl/openssl/pulls/1682","html_url":"https://github.com/openssl/openssl/pull/1682","diff_url":"https://github.com/openssl/openssl/pull/1682.diff","patch_url":"https://github.com/openssl/openssl/pull/1682.patch"},"body":"This is a first (fairly crude) attempt at integrating the BoringSSL tests into OpenSSL. Still lots to do.\r\n\r\nI have taken the BoringSSL shim, as patched at https://github.com/google/openssl-tests and integrated it into the main OpenSSL tree. I have also added a \"test\" to the OpenSSL test suite that invokes the BoringSSL test suite using that shim.\r\n\r\nAt the moment building the shim is not integrated into the OpenSSL build system. Its got its own standalone Makefile (go into tests/ossl_shim and run make after building the rest of OpenSSL but before you run the tests). When running \"make test\" you need to define the location of an external BoringSSL runner directory using the BORING_RUNNER_DIR environment variable.\r\n\r\nTo just run the BoringSSL tests against OpenSSL use this:\r\n\r\n````\r\nVERBOSE=1 BORING_RUNNER_DIR=/path/to/boring/ssl/test/runner make TESTS=\"test_external\" test\r\n````\r\n\r\nI have *only* tried this against the version of BoringSSL as used at https://github.com/google/openssl-tests. That means you should checkout commit f277add6c from BoringSSL and run against that. Yes....its an ancient version!!\r\n\r\nThe tests *will* fail. That's for numerous reasons, e.g. some failures might be due to bugs in OpenSSL, some might be due to features that BoringSSL has that OpenSSL doesn't, some are due to assumptions about the way BoringSSL behaves that are not true for OpenSSL etc.\r\n\r\nBetter results may be obtained by configuring OpenSSL with (but it will still fail a lot):\r\n````\r\n    $ ./config enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers\r\n````\r\n\r\nStuff still to do:\r\n1) Use a more recent checkout of BoringSSL for both the shim, and the tests themselves....@davidben I tried pulling a more recent version of BoringSSL but got lots of merge failures that I don't know how to fix in the shim :-(\r\n2) Integrate building of the shim into the build system....@levitte...help!\r\n3) Switch off tests that we know to fail. The idea is to start with a set of tests that are known good. Then over time we can turn the other back on again (if appropriate), as we figure out how to fix them. @davidben - what is the best way to do this?\r\n\r\nPing @ekasper.\r\n\r\nLicence question...the shim patch was explicitly licensed under Apache v2...is that a problem?"},"comment":{"url":"https://api.github.com/repos/openssl/openssl/issues/comments/253837770","html_url":"https://github.com/openssl/openssl/pull/1682#issuecomment-253837770","issue_url":"https://api.github.com/repos/openssl/openssl/issues/1682","id":253837770,"user":{"login":"levitte","id":698918,"avatar_url":"https://avatars.githubusercontent.com/u/698918?v=3","gravatar_id":"","url":"https://api.github.com/users/levitte","html_url":"https://github.com/levitte","followers_url":"https://api.github.com/users/levitte/followers","following_url":"https://api.github.com/users/levitte/following{/other_user}","gists_url":"https://api.github.com/users/levitte/gists{/gist_id}","starred_url":"https://api.github.com/users/levitte/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/levitte/subscriptions","organizations_url":"https://api.github.com/users/levitte/orgs","repos_url":"https://api.github.com/users/levitte/repos","events_url":"https://api.github.com/users/levitte/events{/privacy}","received_events_url":"https://api.github.com/users/levitte/received_events","type":"User","site_admin":false},"created_at":"2016-10-14T15:40:55Z","updated_at":"2016-10-14T15:40:55Z","body":"Ah, I see now that `CXX` isn't considered...  Will fix."}}
{"actor":{"display_login":"levitte"}}