{"action":"created","issue":{"url":"https://api.github.com/repos/mikeaboody/phishing-research/issues/50","repository_url":"https://api.github.com/repos/mikeaboody/phishing-research","labels_url":"https://api.github.com/repos/mikeaboody/phishing-research/issues/50/labels{/name}","comments_url":"https://api.github.com/repos/mikeaboody/phishing-research/issues/50/comments","events_url":"https://api.github.com/repos/mikeaboody/phishing-research/issues/50/events","html_url":"https://github.com/mikeaboody/phishing-research/pull/50","id":187267896,"number":50,"title":"Script to test various solvers for debugging issues with training logistic regression.","user":{"login":"MatthewChow","id":6220630,"avatar_url":"https://avatars.githubusercontent.com/u/6220630?v=3","gravatar_id":"","url":"https://api.github.com/users/MatthewChow","html_url":"https://github.com/MatthewChow","followers_url":"https://api.github.com/users/MatthewChow/followers","following_url":"https://api.github.com/users/MatthewChow/following{/other_user}","gists_url":"https://api.github.com/users/MatthewChow/gists{/gist_id}","starred_url":"https://api.github.com/users/MatthewChow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MatthewChow/subscriptions","organizations_url":"https://api.github.com/users/MatthewChow/orgs","repos_url":"https://api.github.com/users/MatthewChow/repos","events_url":"https://api.github.com/users/MatthewChow/events{/privacy}","received_events_url":"https://api.github.com/users/MatthewChow/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-11-04T07:15:59Z","updated_at":"2016-11-07T07:47:49Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/mikeaboody/phishing-research/pulls/50","html_url":"https://github.com/mikeaboody/phishing-research/pull/50","diff_url":"https://github.com/mikeaboody/phishing-research/pull/50.diff","patch_url":"https://github.com/mikeaboody/phishing-research/pull/50.patch"},"body":"### What is this?\r\nRight now, we don't know why training the logistic regression classifer doesn't terminate on Vern's dataset. I added a mode to the pipeline that allows us to run a separate script that tries each of the 4 solvers with different tolerances, allowing each combination to run for 10 minutes max before terminating. The hope is that at least one of these combinations will terminate within the 10 minute limit, and we can proceed with using that combination in the actual pipeline.\r\n\r\nTo run the debug mode, we can use the following command: \r\n`python spear_phishing_detector.py --debug_training`\r\n\r\n### How do we know this works?\r\nTested on some sample data to verify that the subprocesses terminate if they've been running for more than the fixed time interval. Verified that all of the information is being logged to the same location: `output/debug.log`. Ran the pipeline end to end with no issues.\r\n\r\nto: @nexusapoorvacus @mikeaboody \r\nWould be great if one of you could try pulling this branch and running the debug mode to make sure it works for you as well and you see the logs in the file I specified above. Also, if you have any other combinations of settings on the classifier that you think would be useful to test in the script (i.e. potentially limiting max iterations?), please let me know and I can add those in.\r\n\r\ncc: @davidwagner "},"comment":{"url":"https://api.github.com/repos/mikeaboody/phishing-research/issues/comments/258767294","html_url":"https://github.com/mikeaboody/phishing-research/pull/50#issuecomment-258767294","issue_url":"https://api.github.com/repos/mikeaboody/phishing-research/issues/50","id":258767294,"user":{"login":"nexusapoorvacus","id":3211268,"avatar_url":"https://avatars.githubusercontent.com/u/3211268?v=3","gravatar_id":"","url":"https://api.github.com/users/nexusapoorvacus","html_url":"https://github.com/nexusapoorvacus","followers_url":"https://api.github.com/users/nexusapoorvacus/followers","following_url":"https://api.github.com/users/nexusapoorvacus/following{/other_user}","gists_url":"https://api.github.com/users/nexusapoorvacus/gists{/gist_id}","starred_url":"https://api.github.com/users/nexusapoorvacus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nexusapoorvacus/subscriptions","organizations_url":"https://api.github.com/users/nexusapoorvacus/orgs","repos_url":"https://api.github.com/users/nexusapoorvacus/repos","events_url":"https://api.github.com/users/nexusapoorvacus/events{/privacy}","received_events_url":"https://api.github.com/users/nexusapoorvacus/received_events","type":"User","site_admin":false},"created_at":"2016-11-07T07:47:49Z","updated_at":"2016-11-07T07:47:49Z","body":"I tested this on the sample pcaps we generated as well as emails in my inbox and it LGTM. Running it on my inbox, which has around 6000 emails, still only took around 0.1 - 0.2 seconds for each solver."}}
{"actor":{"display_login":"nexusapoorvacus"}}