{"action":"created","issue":{"url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/196","repository_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver","labels_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/196/labels{/name}","comments_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/196/comments","events_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/196/events","html_url":"https://github.com/spark-jobserver/spark-jobserver/issues/196","id":98096266,"number":196,"title":"Jobs in Python","user":{"login":"ssoto","id":592484,"avatar_url":"https://avatars.githubusercontent.com/u/592484?v=3","gravatar_id":"","url":"https://api.github.com/users/ssoto","html_url":"https://github.com/ssoto","followers_url":"https://api.github.com/users/ssoto/followers","following_url":"https://api.github.com/users/ssoto/following{/other_user}","gists_url":"https://api.github.com/users/ssoto/gists{/gist_id}","starred_url":"https://api.github.com/users/ssoto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ssoto/subscriptions","organizations_url":"https://api.github.com/users/ssoto/orgs","repos_url":"https://api.github.com/users/ssoto/repos","events_url":"https://api.github.com/users/ssoto/events{/privacy}","received_events_url":"https://api.github.com/users/ssoto/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/labels/feature%20request","name":"feature request","color":"85144B"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":23,"created_at":"2015-07-30T06:31:15Z","updated_at":"2016-07-14T09:32:10Z","closed_at":null,"body":"We have many jobs writed in python and we would like to use spark-jobserver.\r\n\r\nIs there anyone that achieve it? Is there any impediment in spark-jobserver to use jobs compilation from python to Jython?\r\n\r\nThank you!"},"comment":{"url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/comments/232615288","html_url":"https://github.com/spark-jobserver/spark-jobserver/issues/196#issuecomment-232615288","issue_url":"https://api.github.com/repos/spark-jobserver/spark-jobserver/issues/196","id":232615288,"user":{"login":"mattinbits","id":3765307,"avatar_url":"https://avatars.githubusercontent.com/u/3765307?v=3","gravatar_id":"","url":"https://api.github.com/users/mattinbits","html_url":"https://github.com/mattinbits","followers_url":"https://api.github.com/users/mattinbits/followers","following_url":"https://api.github.com/users/mattinbits/following{/other_user}","gists_url":"https://api.github.com/users/mattinbits/gists{/gist_id}","starred_url":"https://api.github.com/users/mattinbits/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattinbits/subscriptions","organizations_url":"https://api.github.com/users/mattinbits/orgs","repos_url":"https://api.github.com/users/mattinbits/repos","events_url":"https://api.github.com/users/mattinbits/events{/privacy}","received_events_url":"https://api.github.com/users/mattinbits/received_events","type":"User","site_admin":false},"created_at":"2016-07-14T09:32:10Z","updated_at":"2016-07-14T09:32:10Z","body":"I'm planning to start working on this day, with the following decisions/assumptions on the Qs above:\r\n\r\n- Q1: Assume the workflow stated is basically valid, as shown in the simple prototype\r\n- Q2: To start of with, planning to use a Python class which is to be overridden since it is the closest equivalent to implementing a trait.\r\n- Q3: Allow any return type which meets the following rules (don't plan to explicitly check these, but errors will be thrown if the rules are broken, which can be returned to the client):\r\n  - Can be successfully sent from Python to Java via Py4J when 'autoconvert=true' is set on the JVM side,\r\nsee https://www.py4j.org/advanced_topics.html#converting-python-collections-to-java-collections\r\n  - Can be successfully serialized by Spray JSON. As I understand it this is implicitly the rule already for returned data from Scala/Java jobs.\r\n- Q4: Planning to implement SparkContext, SQLContext, HiveContext versions. Given Python is not statically typed, I think this distinction is only necessary on the Scala side in the context factory. The abstract Python class can assume the passed context is any type of context.\r\n- Q5: Implement `/binaries` using content-type header to differentiate, and `/jars` will end up as an alias for `/binaries` with `application/java-archive` content type header.\r\n"}}
{"actor":{"display_login":"mattinbits"}}