{"action":"created","issue":{"url":"https://api.github.com/repos/craffel/mir_eval/issues/170","repository_url":"https://api.github.com/repos/craffel/mir_eval","labels_url":"https://api.github.com/repos/craffel/mir_eval/issues/170/labels{/name}","comments_url":"https://api.github.com/repos/craffel/mir_eval/issues/170/comments","events_url":"https://api.github.com/repos/craffel/mir_eval/issues/170/events","html_url":"https://github.com/craffel/mir_eval/pull/170","id":132468787,"number":170,"title":"Transcription [wip]","user":{"login":"justinsalamon","id":3009670,"avatar_url":"https://avatars.githubusercontent.com/u/3009670?v=3","gravatar_id":"","url":"https://api.github.com/users/justinsalamon","html_url":"https://github.com/justinsalamon","followers_url":"https://api.github.com/users/justinsalamon/followers","following_url":"https://api.github.com/users/justinsalamon/following{/other_user}","gists_url":"https://api.github.com/users/justinsalamon/gists{/gist_id}","starred_url":"https://api.github.com/users/justinsalamon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinsalamon/subscriptions","organizations_url":"https://api.github.com/users/justinsalamon/orgs","repos_url":"https://api.github.com/users/justinsalamon/repos","events_url":"https://api.github.com/users/justinsalamon/events{/privacy}","received_events_url":"https://api.github.com/users/justinsalamon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/craffel/mir_eval/milestones/6","html_url":"https://github.com/craffel/mir_eval/milestones/0.3.0","labels_url":"https://api.github.com/repos/craffel/mir_eval/milestones/6/labels","id":1578395,"number":6,"title":"0.3.0","description":"","creator":{"login":"bmcfee","id":1190540,"avatar_url":"https://avatars.githubusercontent.com/u/1190540?v=3","gravatar_id":"","url":"https://api.github.com/users/bmcfee","html_url":"https://github.com/bmcfee","followers_url":"https://api.github.com/users/bmcfee/followers","following_url":"https://api.github.com/users/bmcfee/following{/other_user}","gists_url":"https://api.github.com/users/bmcfee/gists{/gist_id}","starred_url":"https://api.github.com/users/bmcfee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bmcfee/subscriptions","organizations_url":"https://api.github.com/users/bmcfee/orgs","repos_url":"https://api.github.com/users/bmcfee/repos","events_url":"https://api.github.com/users/bmcfee/events{/privacy}","received_events_url":"https://api.github.com/users/bmcfee/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":4,"state":"open","created_at":"2016-02-11T19:53:17Z","updated_at":"2016-02-16T15:14:45Z","due_on":null,"closed_at":null},"comments":35,"created_at":"2016-02-09T16:41:46Z","updated_at":"2016-02-19T23:00:46Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/craffel/mir_eval/pulls/170","html_url":"https://github.com/craffel/mir_eval/pull/170","diff_url":"https://github.com/craffel/mir_eval/pull/170.diff","patch_url":"https://github.com/craffel/mir_eval/pull/170.patch"},"body":"This will (eventually) contain all the code for transcription evaluation (and tests)"},"comment":{"url":"https://api.github.com/repos/craffel/mir_eval/issues/comments/186442172","html_url":"https://github.com/craffel/mir_eval/pull/170#issuecomment-186442172","issue_url":"https://api.github.com/repos/craffel/mir_eval/issues/170","id":186442172,"user":{"login":"justinsalamon","id":3009670,"avatar_url":"https://avatars.githubusercontent.com/u/3009670?v=3","gravatar_id":"","url":"https://api.github.com/users/justinsalamon","html_url":"https://github.com/justinsalamon","followers_url":"https://api.github.com/users/justinsalamon/followers","following_url":"https://api.github.com/users/justinsalamon/following{/other_user}","gists_url":"https://api.github.com/users/justinsalamon/gists{/gist_id}","starred_url":"https://api.github.com/users/justinsalamon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinsalamon/subscriptions","organizations_url":"https://api.github.com/users/justinsalamon/orgs","repos_url":"https://api.github.com/users/justinsalamon/repos","events_url":"https://api.github.com/users/justinsalamon/events{/privacy}","received_events_url":"https://api.github.com/users/justinsalamon/received_events","type":"User","site_admin":false},"created_at":"2016-02-19T23:00:46Z","updated_at":"2016-02-19T23:00:46Z","body":"Nope, it was indeed a bug in my comparison code, just realized I wasn't loading the mirex recall results correctly. Now I'm getting:\r\n```\r\nP\tR\tF\tP_nooff\tR_nooff\tF_nooff\r\n----------------------------------------------\r\n-0.010\t-0.009\t-0.009\t0.005\t0.004\t0.005\t\r\n-0.000\t0.000\t-0.000\t0.000\t-0.000\t-0.000\t\r\n0.000\t0.000\t-0.000\t0.000\t-0.000\t0.000\t\r\n0.000\t0.000\t-0.000\t-0.000\t0.000\t-0.000\t\r\n0.000\t-0.000\t0.000\t0.000\t-0.000\t-0.000\t\r\n0.000\t0.000\t0.000\t0.000\t-0.000\t-0.000\t\r\n0.000\t0.000\t0.000\t0.000\t-0.000\t-0.000\t\r\n0.000\t-0.000\t-0.000\t0.000\t0.000\t-0.000\t\r\n-0.000\t0.000\t-0.000\t0.000\t0.000\t-0.000\t\r\n-0.004\t-0.003\t-0.004\t-0.000\t0.000\t0.000\t\r\n----------------------------------------------\r\n-0.001\t-0.001\t-0.001\t0.000\t0.000\t0.000\t(mean diff)\r\n0.001\t0.001\t0.001\t0.000\t0.000\t0.000\t(mean abs diff)\r\nAll close 1e-3: False\r\nAll close 1e-2: True\r\nAll close 1e-1: True\r\n```\r\n\r\nWhich, if you ask me, is close enough! Woohoo.\r\n\r\nNext step: start replacing their stuff with my stuff an assess how much things change. I think the order should be:\r\n\r\n1. Replace their note matching with the my note matching\r\n2. Removing the rounding down of onsets and offsets\r\n\r\nAnd then I think we're done. There's still the question of how to calculate the offset tolerance, but that's not a problem in terms of reproducing the MIREX results."}}