{"action":"created","issue":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1379","repository_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware","labels_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1379/labels{/name}","comments_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1379/comments","events_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1379/events","html_url":"https://github.com/nodemcu/nodemcu-firmware/pull/1379","id":163565784,"number":1379,"title":"Net functionality rewrite on top of LWIP","user":{"login":"djphoenix","id":1833493,"avatar_url":"https://avatars.githubusercontent.com/u/1833493?v=3","gravatar_id":"","url":"https://api.github.com/users/djphoenix","html_url":"https://github.com/djphoenix","followers_url":"https://api.github.com/users/djphoenix/followers","following_url":"https://api.github.com/users/djphoenix/following{/other_user}","gists_url":"https://api.github.com/users/djphoenix/gists{/gist_id}","starred_url":"https://api.github.com/users/djphoenix/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/djphoenix/subscriptions","organizations_url":"https://api.github.com/users/djphoenix/orgs","repos_url":"https://api.github.com/users/djphoenix/repos","events_url":"https://api.github.com/users/djphoenix/events{/privacy}","received_events_url":"https://api.github.com/users/djphoenix/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":46,"created_at":"2016-07-03T15:01:55Z","updated_at":"2016-12-28T16:04:33Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/pulls/1379","html_url":"https://github.com/nodemcu/nodemcu-firmware/pull/1379","diff_url":"https://github.com/nodemcu/nodemcu-firmware/pull/1379.diff","patch_url":"https://github.com/nodemcu/nodemcu-firmware/pull/1379.patch"},"body":"Fixes #1080"},"comment":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/comments/269497835","html_url":"https://github.com/nodemcu/nodemcu-firmware/pull/1379#issuecomment-269497835","issue_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1379","id":269497835,"user":{"login":"luismfonseca","id":2430467,"avatar_url":"https://avatars.githubusercontent.com/u/2430467?v=3","gravatar_id":"","url":"https://api.github.com/users/luismfonseca","html_url":"https://github.com/luismfonseca","followers_url":"https://api.github.com/users/luismfonseca/followers","following_url":"https://api.github.com/users/luismfonseca/following{/other_user}","gists_url":"https://api.github.com/users/luismfonseca/gists{/gist_id}","starred_url":"https://api.github.com/users/luismfonseca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/luismfonseca/subscriptions","organizations_url":"https://api.github.com/users/luismfonseca/orgs","repos_url":"https://api.github.com/users/luismfonseca/repos","events_url":"https://api.github.com/users/luismfonseca/events{/privacy}","received_events_url":"https://api.github.com/users/luismfonseca/received_events","type":"User","site_admin":false},"created_at":"2016-12-28T16:04:33Z","updated_at":"2016-12-28T16:04:33Z","body":"Following up on my previous comment, I've added `printf`s to basically every call that `espconn_tcp` layer does to `lwip` [[raw output]](http://pastebin.com/raw/wBSDQ64z).\r\n\r\nIt seems that it first will do a simple `tcp_close` call. But it keeps track of the connections in its `tcp_pcb_lists[]`.\r\n\r\nAfter a few connections it will call `espconn_kill_oldest_pcb` and it will do:\r\n\r\n```\r\ntcp_abort(3fff0e68):espconn_kill_oldest\r\ntcp_pcb_remove(3ffef73c, 3fff2928):espconn_kill_oldest\r\n```\r\n\r\nSo it seems that it will wait (per lwip standards/timeouts for the connection to be closed) up until the point where it has multiple existing connections (?) or connections with an inactive state for a long period of time (judging by the `espconn_kill_oldest` code). Then, it will trigger the `tcp_abort` and `tcp_pcb_remove` - which oddly enough, is for a different `pcb`.\r\n\r\nThe question is whether we want this optimization in our `net` module. And if we do, we would benefit from something like the `espconn_tcp` - an intermediate layer that interacts with lwip and does its optimizations. The benefit being that other modules like `http` or `websocket` wouldn't need to reimplement them. As is, this merge request mixes `lua` directly with `lwip`, so it's not possible to reuse any code in `http` or `websocket`.\r\n\r\nI've tried changing `TCP_FIN_WAIT_TIMEOUT` from `20000` to `2000` but it didn't make a difference in this case.\r\n\r\nI've also tried changing `TCP_MSL` from `60000UL` to `30000UL` and that did seem to make an impact. So `lwip` is internally now freeing the connections earlier (this is still a bit different than the `espconn` behaviour). So with the same test, the heap go to to a lowest `35816` (which is much more acceptable than the previous `29616`).\r\n\r\nLooking at LWIP code, it seems that if it is failing to allocate a new `pcb`, one thing it does is to kill off old connections that are in the `TIME_WAIT` state. It would be interesting to add a hook to lua GC somehow so that when memory was really running low, kill off all `TIME_WAIT` connections, regardless of their age."}}
{"actor":{"display_login":"luismfonseca"}}