{"action":"created","issue":{"url":"https://api.github.com/repos/SeerLabs/CiteSeerX/issues/68","repository_url":"https://api.github.com/repos/SeerLabs/CiteSeerX","labels_url":"https://api.github.com/repos/SeerLabs/CiteSeerX/issues/68/labels{/name}","comments_url":"https://api.github.com/repos/SeerLabs/CiteSeerX/issues/68/comments","events_url":"https://api.github.com/repos/SeerLabs/CiteSeerX/issues/68/events","html_url":"https://github.com/SeerLabs/CiteSeerX/issues/68","id":157514846,"number":68,"title":"number of file descriptors increases beyond limit","user":{"login":"fanchyna","id":1759659,"avatar_url":"https://avatars.githubusercontent.com/u/1759659?v=3","gravatar_id":"","url":"https://api.github.com/users/fanchyna","html_url":"https://github.com/fanchyna","followers_url":"https://api.github.com/users/fanchyna/followers","following_url":"https://api.github.com/users/fanchyna/following{/other_user}","gists_url":"https://api.github.com/users/fanchyna/gists{/gist_id}","starred_url":"https://api.github.com/users/fanchyna/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fanchyna/subscriptions","organizations_url":"https://api.github.com/users/fanchyna/orgs","repos_url":"https://api.github.com/users/fanchyna/repos","events_url":"https://api.github.com/users/fanchyna/events{/privacy}","received_events_url":"https://api.github.com/users/fanchyna/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-05-30T14:37:33Z","updated_at":"2016-05-30T14:49:52Z","closed_at":null,"body":"This happens on one of the web servers (03). I first got a \"server down message\" from the monitor. I then looked at the localhost file and saw lots of errors like \"\"too many files opened\". Basically, the system cannot even open a sitemap file and respond to the client. I then checked \"ullimit -Hn\" and the number was 8192. Using \"lsof\" I see more than 10,000 rows and most of them are like \"web server -> repo server [CLOSE_WAIT]\". This CLOSE_WAIT status is very annoying and has been directly causing web server down issues. However, as far as I know, there's no way to clean the socket except to restart the service or reboot. In this situation, I had to restart Tomcat, but I wonder if this could be a software or hardware issue. The numbers of file descriptors  are below the limit on the other web servers (also 8192). "},"comment":{"url":"https://api.github.com/repos/SeerLabs/CiteSeerX/issues/comments/222508157","html_url":"https://github.com/SeerLabs/CiteSeerX/issues/68#issuecomment-222508157","issue_url":"https://api.github.com/repos/SeerLabs/CiteSeerX/issues/68","id":222508157,"user":{"login":"dwj300","id":1051774,"avatar_url":"https://avatars.githubusercontent.com/u/1051774?v=3","gravatar_id":"","url":"https://api.github.com/users/dwj300","html_url":"https://github.com/dwj300","followers_url":"https://api.github.com/users/dwj300/followers","following_url":"https://api.github.com/users/dwj300/following{/other_user}","gists_url":"https://api.github.com/users/dwj300/gists{/gist_id}","starred_url":"https://api.github.com/users/dwj300/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dwj300/subscriptions","organizations_url":"https://api.github.com/users/dwj300/orgs","repos_url":"https://api.github.com/users/dwj300/repos","events_url":"https://api.github.com/users/dwj300/events{/privacy}","received_events_url":"https://api.github.com/users/dwj300/received_events","type":"User","site_admin":false},"created_at":"2016-05-30T14:49:52Z","updated_at":"2016-05-30T14:49:52Z","body":"I think we can add a timeout so it automatically closes the socket:\nhttp://stackoverflow.com/questions/3000214/java-http-client-request-with-defined-timeout\nOn Mon, May 30, 2016 at 07:45 Kyle Williams <notifications@github.com>\nwrote:\n\n> Does this only happen on one of the web servers or all of them?\n>\n> On Mon, May 30, 2016 at 10:37 AM, Jian Wu <notifications@github.com>\n> wrote:\n>\n> > This happens on one of the web servers (03). I first got a \"server down\n> > message\" from the monitor. I then looked at the localhost file and saw\n> lots\n> > of errors like \"\"too many files opened\". Basically, the system cannot\n> even\n> > open a sitemap file and respond to the client. I then checked \"ullimit\n> -Hn\"\n> > and the number was 8192. Using \"lsof\" I see more than 10,000 rows and\n> most\n> > of them are like \"web server -> repo server [CLOSE_WAIT]\". This\n> CLOSE_WAIT\n> > status is very annoying and has been directly causing web server down\n> > issues. However, as far as I know, there's no way to clean the socket\n> > except to restart the service or reboot. In this situation, I had to\n> > restart Tomcat, but I wonder if this could be a software or hardware\n> issue.\n> > The numbers of file descriptors are below the limit on the other web\n> > servers (also 8192).\n> >\n> > —\n> > You are receiving this because you are subscribed to this thread.\n> > Reply to this email directly, view it on GitHub\n> > <https://github.com/SeerLabs/CiteSeerX/issues/68>, or mute the thread\n> > <\n> https://github.com/notifications/unsubscribe/ABLPxEkVXFVvUA8Y7lrANcuGQYvm4_6Bks5qGvYugaJpZM4Ip2Wf\n> >\n> > .\n> >\n>\n>\n>\n> --\n> http://www.personal.psu.edu/kiw5209/\n>\n> —\n> You are receiving this because you are subscribed to this thread.\n> Reply to this email directly, view it on GitHub\n> <https://github.com/SeerLabs/CiteSeerX/issues/68#issuecomment-222507385>,\n> or mute the thread\n> <https://github.com/notifications/unsubscribe/ABAMfvg_fKEhpg6SftOR-SY7ay5fmbvXks5qGvgRgaJpZM4Ip2Wf>\n> .\n>\n"}}
{"actor":{"display_login":"dwj300"}}