{"action":"created","issue":{"url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/issues/38","repository_url":"https://api.github.com/repos/Kotlin/kotlin-coroutines","labels_url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/issues/38/comments","events_url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/issues/38/events","html_url":"https://github.com/Kotlin/kotlin-coroutines/issues/38","id":149745744,"number":38,"title":"Feedback, use cases, and request","user":{"login":"soywiz","id":570848,"avatar_url":"https://avatars.githubusercontent.com/u/570848?v=3","gravatar_id":"","url":"https://api.github.com/users/soywiz","html_url":"https://github.com/soywiz","followers_url":"https://api.github.com/users/soywiz/followers","following_url":"https://api.github.com/users/soywiz/following{/other_user}","gists_url":"https://api.github.com/users/soywiz/gists{/gist_id}","starred_url":"https://api.github.com/users/soywiz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soywiz/subscriptions","organizations_url":"https://api.github.com/users/soywiz/orgs","repos_url":"https://api.github.com/users/soywiz/repos","events_url":"https://api.github.com/users/soywiz/events{/privacy}","received_events_url":"https://api.github.com/users/soywiz/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/labels/use%20case","name":"use case","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-20T11:49:49Z","updated_at":"2016-04-20T16:46:10Z","closed_at":null,"body":"First of all: thank you very much for all your awesome work.\r\nIt was a very nice surprise that you were working on suspendable functions already,\r\nand even trying for Kotlin 1.1.\r\n\r\nI just wanted to provide feedback, about my experience with await/async. I did sometime ago on the forums.\r\nBut maybe now I can provide more information, including our current use cases, and my own experience with this kind of stuff.\r\n\r\n**Introduction:**\r\n\r\nI first used await/async when C# came with it, after some time considering that fibers/green threads were the way to go.\r\n\r\nLots of years before I used DIV and DIV like programming languages (process oriented for games) that used suspendable fibers in order to achive concurrency: each process/entity executed some code until a FRAME statement reached, and the function was suspended. That was great because the machine state is implicit and it is really a pleasure to read and to understand what's going on.\r\n\r\n**Fibers:**\r\n\r\nFibers/Green thread require support for the operating system / high level language or at least having raw access to assembly.\r\nYou have to have a stack per green thread (and handle growing of the stack or having a trade off between memory and execution capabilities), and also you have to backup all your registers and restore them, and of course handling exception handling/traps whose information is usually stored in the stack and some information before the forking.\r\n\r\nSo even when stacks of green threads tend to be smaller than full thread stacks, it's not completely efficient: http://www.kegel.com/c10k.html\r\n\r\nSo even when your execution process is very small you have to have a stack.\r\n\r\n**Await/Async:**\r\n\r\nAwait/async/yield works different. Instead of using the stack and requiring support for it, it can work everywhere. A simplification of the process:\r\n\r\n* Function is transformed into a generator + a struct/class with the state + a function that can be called several times specifying the state\r\n* So usually a function with a reductible CFG is transformed into a function with a switch matching the current state position at the very beggining with gotos and local variables are transformed into access to fields in the state structure.\r\n* And each await, changes that state position to the next one, and registers the specified promise/task/future to call itself, and just after returns the function. So when the awaitable is completed, it will execute itself but will jump to the next state position.\r\n\r\nI created a project that does this process here with bytecode instrumentation:\r\nhttps://github.com/soywiz/jawaitasync\r\n\r\nMy experience with it:\r\nI had several issues. For example jvm has an opcode to create an instance of an object, but that object is uninitializated before invokespecial <init> and in some cases I had suspension points having the stack with uninitialized objects (that would have been easier if they called the construction in just one opcode or the object/arguments had the reverse order to avoid that nasty DUP :)).\r\nBefore the suspension I had to store the stack into synthetic locals that were actually fields in the state structure, so I could restore it later.\r\nI stopped it because I didn't like Java and Scala was not what we were looking for and experimented with other stuff (including typescript and haxe), until I found Kotlin! Then expected this moment to happen.\r\n\r\nSince in kotlin you have a proper AST, it should be easier to avoid the problems I had with bytecode intrumentation.\r\n\r\nBefore typescript had await/async I implemented it using generators:\r\nhttps://github.com/soywiz/tspromise/blob/master/Promise.ts#L157\r\n\r\nSo in the case you have bidirectional generators (python or javascript ones), you can separate the await/async logic from that. C# for example had unidirectional generators (it can emit values, but not retrieve values after resuming the function), so it was not easy to emulate await/async with generators.\r\n\r\n**Our use cases**\r\n\r\nWe are using right now kotlin jvm in both backend and frontend.\r\n\r\nWe have a common \"promises/thenable\" implementation that is used in both, frontend and backend. In fact, futhermore to thenable, it has pipeing and other useful stuff, that wouldn't be required after async/await is implemented.\r\nIn the backend we are using vert.x as single threaded eventloop, and we have all the IO asynchronous; lock free. And we are scaling adding more nodes, just like nodejs works.\r\n\r\nIn the frontend we are using: https://github.com/jtransc/jtransc/ (which is my current project that I have been working on in the last months). It works already and we have games that are compiled into javascript, flash and C++ for both android and ios. Since it uses bytecode we can have mixed code: kotlin + java libraries.\r\nAnd because jtransc aims to target several stuff including javascript and flash, it doesn't implement threads at all.\r\nIn the frontend part we have an eventloop too, js and flash use their own and in C++ it is emulated.\r\n\r\n\r\n**Futures/Tasks/Threads/Promises/Thenables : Proposals/requests**\r\n\r\nI have seen that the current implementation uses CompletableFuture.\r\n\r\nThe main problem of java Futures is that they can block the thread waiting for a value and its API is pretty complex. And thenable are simle and hell and promises makes that impossible, they are completely reactive and don't even allow to determine wheter a promise is completed or not which is great to avoid bad code.\r\n\r\nSo it would be great if the implementation works with our custom promise implementation.\r\n\r\nIf Kotlin would have some kind interfaces (like typescript or typedefs in haxe or scala) that wouldn't require to be implemented explicitly but enforced at compile time, it would be pretty easy to add just a thenable/continuable contract that would work. I know that it is costly at runtime because of the reflection or would require invokedynamic stuff.\r\n\r\nOther possible stuff would be if Kotlin provides a proper interface that we could implement (that would be enough for use). But it wouldn't work with java Futures. Though you could check if it is a Future to call a method or if it implements the interface to call the other.\r\n\r\nhttps://github.com/Kotlin/kotlin-coroutines/blob/master/examples/async.kt#L43\r\nOther thing that could work would be instead separating the suspension from the await/async executor (the one that knows about Promises/Futures and stuff), to embed calling in the suspendable function. That way you can determine at compile time which method to call (.then, .whenComplete or other) without any kind of overhead. But I know that this approach would couple stuff and make stuff more complicated, and being able to separate is a design goal as far as I read.\r\n\r\nI think that a small overhead (instanceof) is not a problem, since most asynchronous programming is pretty simple. Most of the time the process is idle waiting for IO / or tweening / waiting other kind of events.\r\n\r\nSo my request here is to support some kind of thenable without involving java's futures at all.\r\n"},"comment":{"url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/issues/comments/212508834","html_url":"https://github.com/Kotlin/kotlin-coroutines/issues/38#issuecomment-212508834","issue_url":"https://api.github.com/repos/Kotlin/kotlin-coroutines/issues/38","id":212508834,"user":{"login":"soywiz","id":570848,"avatar_url":"https://avatars.githubusercontent.com/u/570848?v=3","gravatar_id":"","url":"https://api.github.com/users/soywiz","html_url":"https://github.com/soywiz","followers_url":"https://api.github.com/users/soywiz/followers","following_url":"https://api.github.com/users/soywiz/following{/other_user}","gists_url":"https://api.github.com/users/soywiz/gists{/gist_id}","starred_url":"https://api.github.com/users/soywiz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soywiz/subscriptions","organizations_url":"https://api.github.com/users/soywiz/orgs","repos_url":"https://api.github.com/users/soywiz/repos","events_url":"https://api.github.com/users/soywiz/events{/privacy}","received_events_url":"https://api.github.com/users/soywiz/received_events","type":"User","site_admin":false},"created_at":"2016-04-20T16:46:09Z","updated_at":"2016-04-20T16:46:09Z","body":"With the AST I meant that while generating the bytecode from the AST, you can handle how to generate some stuff to avoid some kind of problems. Indeed with tons of high level structures I understand that it would be incredibly complicated as a AST transformation. And probably the only sane way would be simulating labels through switch/cases which would be the same as generating the bytecode.\r\n\r\n----\r\n\r\nThe typescript micro-library was \"type-safe\", all the type-safe typescript can be (promises were handling and returning typed values and enforced at compile-time). But you can always do strange stuff with \"any\" (the equivalent to dynamic in kotlin.js), and you can cast stuff wrong.\r\n\r\nBut allowed some signatures (supported by plain ES6 promises) that can't be supported on java because collisions:\r\n```\r\nthen<TR>(onFulfilled: (value: T) => Promise<TR>, onRejected?: (error: Error) => TR): Promise<TR>;\r\nthen<TR>(onFulfilled: (value: T) => TR, onRejected?: (error: Error) => void): Promise<TR>;\r\n```\r\nIn java/kotlin we have two different methods, the second one called `then` and other called `pipe`.\r\n\r\nThough when using with await/async `pipe` is not necessary anymore, just a plain `then`.\r\n\r\nI wrote that micro-library a couple of years ago, and it has been superseded. Now promises are mainstream, ES6 is almost on all browsers including generators, and typescript supports await/async already using generators when targeting to ES6 and ES7 is going to include it.\r\n\r\n----\r\n\r\nShame on me! I missed the cool document :) I will read it. I'm really eager to understand how it is going to be designed to work with all the stuff and handled by the library (potentially other libraries?) and being async a keyword over there already!\r\n\r\nI will write again after I read the document if I have some related feedback about it.\r\n"}}