{"action":"created","issue":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1319","repository_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware","labels_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1319/labels{/name}","comments_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1319/comments","events_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1319/events","html_url":"https://github.com/nodemcu/nodemcu-firmware/issues/1319","id":157121391,"number":1319,"title":"RTOS-SDK, ESP32 and the way forward","user":{"login":"jmattsson","id":869607,"avatar_url":"https://avatars.githubusercontent.com/u/869607?v=3","gravatar_id":"","url":"https://api.github.com/users/jmattsson","html_url":"https://github.com/jmattsson","followers_url":"https://api.github.com/users/jmattsson/followers","following_url":"https://api.github.com/users/jmattsson/following{/other_user}","gists_url":"https://api.github.com/users/jmattsson/gists{/gist_id}","starred_url":"https://api.github.com/users/jmattsson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jmattsson/subscriptions","organizations_url":"https://api.github.com/users/jmattsson/orgs","repos_url":"https://api.github.com/users/jmattsson/repos","events_url":"https://api.github.com/users/jmattsson/events{/privacy}","received_events_url":"https://api.github.com/users/jmattsson/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/labels/governance","name":"governance","color":"207de5"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-05-27T03:12:26Z","updated_at":"2016-05-30T08:07:02Z","closed_at":null,"body":"With the ESP32 release coming up in a few months time, it's time to seriously start thinking about the way forward. I think it's a given that we'd all like to see NodeMCU run on the ESP32 as well. With the ESP32 there is only the RTOS SDK however, which means we really need to consider how to get ourselves switched from the non-OS SDK over to RTOS.\r\n\r\nSince $work is rather interested in shifting some of our products over to the ESP32 I've had a bit of time to investigate the effort that will be required in terms of NodeMCU. I've been \"spiking\" over on the [DiUS dev-rtos](https://github.com/DiUS/nodemcu-firmware/tree/dev-rtos) branch to see what I can get going. Here's the overview so far:\r\n\r\n- [x] **Make NodeMCU *compile* with RTOS-SDK headers rather than non-OS SDK headers.** There's a bunch of glue in the `sdk-overrides/`directory which would need cleaning up, but overall this step wasn't too bad - the SDK functions are largely the same.\r\n\r\n- [x] **Make NodeMCU *link* with RTOS SDK**. This took a bunch of changes, and a couple of functions needed to be stubbed for now.\r\n\r\n- [x] **Reimplement NodeMCU task interface on top of RTOS tasks.** Thanks to Terry's earlier work, this seems to be relatively straight forward. Once it gets run-tested some issues may surface though (cue Jaws music...)\r\n\r\n- [x] **Complete reimplementation of our exception handler to allow constants in flash**. Turns out the RTOS-SDK doesn't use any of the ROM functions for hooking exceptions (probably performance reasons), and the documented method of installing user hooks simply does not exist. Took a fair chunk of work to find a good way to hi-jack the UserExceptionVector, but on the upside it's now also a whole lot faster than the previous one.\r\n\r\n- [x] **Remove NodeMCU's partial libc implementation.** This was conflicting with the SDK's libc and causing complete hangs. On the upside, there now is a real libc available. Almost all the various `c_` prefixed functions (and a bunch of `os_` prefixed ones) have been consolidated back to standard C library names.\r\n\r\n- [x] **Fixed SPI flash reading functions.** The RTOS-SDK changed the `flashchip` variable from a pointer to a struct, so our use of it bombed completely...\r\n\r\n- [ ] **Understand why `printf()` now doesn't work, but `ets_printf()` does.** This is where things are currently falling over.\r\n\r\n- [ ] **???** (no profit guaranteed)\r\n\r\n- [ ] **Get to a Lua prompt.** This would be a major milestone, and hopefully be the starting point for others to join in the effort.\r\n\r\n- [ ] **Deal with timer callbacks executing from a different task context.** Considering that pretty much all NodeMCU code is written expecting run-to-completion semantics, switching to a preempting OS framework has huge potential for random lockups and crashes. My current approach is to dedicate a single RTOS task to running all of NodeMCU in, which should hopefully mitigate most issues. To make this happen I think I'll probably need to wrap the timer API to have the actual callbacks posted back to the NodeMCU task for execution.\r\n\r\n- [ ] **Understand which tasks the SDK callbacks execute from, and develop a strategy for dealing with that.** Similar to the timers, but a bit more challenging, possibly. Our earlier work in not allowing Lua callbacks to run from SDK callbacks should help here and limit the amount of rework needed. I hope.\r\n\r\n- [ ] **Reimplement the `net` module (and others) on top of lwIP API**, since `espconn` is only partially supported on the ESP8266 RTOS, and not at all on the ESP32 RTOS. Probably look at including mbedTLS for TLS support.\r\n\r\n- [ ] **Fix whatever other issues and races we encounter.** There will be races, I'm sure. There will also be regressions\r\n\r\n- [ ] **Look at upgrading various components and drivers to take better advantage of the RTOS aspects.**\r\n\r\n\r\nIf we can get our current NodeMCU to run stable on the ESP8266 with the RTOS SDK, it should be quite easy to get ESP32 support in I believe. If/when I get my hands on ESP32 hardware, I'll have an even better idea.\r\n\r\nOh, and the dev-rtos branch is subject to force-pushing and other unpleasant things, and it is most assuredly not ready for public consumption, but if you want to track my progress you'll see it there."},"comment":{"url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/comments/222436515","html_url":"https://github.com/nodemcu/nodemcu-firmware/issues/1319#issuecomment-222436515","issue_url":"https://api.github.com/repos/nodemcu/nodemcu-firmware/issues/1319","id":222436515,"user":{"login":"jmattsson","id":869607,"avatar_url":"https://avatars.githubusercontent.com/u/869607?v=3","gravatar_id":"","url":"https://api.github.com/users/jmattsson","html_url":"https://github.com/jmattsson","followers_url":"https://api.github.com/users/jmattsson/followers","following_url":"https://api.github.com/users/jmattsson/following{/other_user}","gists_url":"https://api.github.com/users/jmattsson/gists{/gist_id}","starred_url":"https://api.github.com/users/jmattsson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jmattsson/subscriptions","organizations_url":"https://api.github.com/users/jmattsson/orgs","repos_url":"https://api.github.com/users/jmattsson/repos","events_url":"https://api.github.com/users/jmattsson/events{/privacy}","received_events_url":"https://api.github.com/users/jmattsson/received_events","type":"User","site_admin":false},"created_at":"2016-05-30T08:07:01Z","updated_at":"2016-05-30T08:07:01Z","body":"It could probably be moved over to the regular SDK with some care, but I haven't looked at that."}}
{"actor":{"display_login":"jmattsson"}}