{"action":"created","issue":{"url":"https://api.github.com/repos/pluralsight/spavro/issues/3","repository_url":"https://api.github.com/repos/pluralsight/spavro","labels_url":"https://api.github.com/repos/pluralsight/spavro/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/pluralsight/spavro/issues/3/comments","events_url":"https://api.github.com/repos/pluralsight/spavro/issues/3/events","html_url":"https://github.com/pluralsight/spavro/issues/3","id":318841413,"number":3,"title":"Issue with DataFileWriter and union of records.","user":{"login":"shawnsarwar","id":4309669,"avatar_url":"https://avatars3.githubusercontent.com/u/4309669?v=4","gravatar_id":"","url":"https://api.github.com/users/shawnsarwar","html_url":"https://github.com/shawnsarwar","followers_url":"https://api.github.com/users/shawnsarwar/followers","following_url":"https://api.github.com/users/shawnsarwar/following{/other_user}","gists_url":"https://api.github.com/users/shawnsarwar/gists{/gist_id}","starred_url":"https://api.github.com/users/shawnsarwar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shawnsarwar/subscriptions","organizations_url":"https://api.github.com/users/shawnsarwar/orgs","repos_url":"https://api.github.com/users/shawnsarwar/repos","events_url":"https://api.github.com/users/shawnsarwar/events{/privacy}","received_events_url":"https://api.github.com/users/shawnsarwar/received_events","type":"User","site_admin":false},"labels":[{"id":620158523,"url":"https://api.github.com/repos/pluralsight/spavro/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-04-30T10:08:30Z","updated_at":"2018-05-04T08:14:16Z","closed_at":"2018-05-04T02:36:12Z","author_association":"NONE","body":"I'm trying to serialize a message with its schema, but I'm getting the following error from spavro:\r\n```\r\nTraceback (most recent call last):\r\n  File \"src/spavro/fast_binary.pyx\", line 702, in spavro.fast_binary.get_writer\r\n  File \"src/spavro/fast_binary.pyx\", line 496, in spavro.fast_binary.make_union_writer\r\n  File \"src/spavro/fast_binary.pyx\", line 360, in spavro.fast_binary.get_check\r\n  File \"src/spavro/fast_binary.pyx\", line 364, in spavro.fast_binary.make_record_check\r\n  File \"src/spavro/fast_binary.pyx\", line 360, in spavro.fast_binary.get_check\r\n  File \"src/spavro/fast_binary.pyx\", line 408, in spavro.fast_binary.make_union_check\r\n  File \"src/spavro/fast_binary.pyx\", line 360, in spavro.fast_binary.get_check\r\nKeyError: 'org.eha.demo.parents'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"./manage.py\", line 19, in <module>\r\n    aether_producer.main_loop()\r\n  File \"/code/producer/aether_producer.py\", line 277, in main_loop\r\n    manager.send(entities)\r\n  File \"/code/producer/aether_producer.py\", line 250, in send\r\n    self.streams[topic].send(row)\r\n  File \"/code/producer/aether_producer.py\", line 177, in send\r\n    writer = DataFileWriter(bytes_writer, DatumWriter(), self.schema, codec='deflate')\r\n  File \"/usr/local/lib/python3.6/site-packages/spavro/datafile.py\", line 99, in __init__\r\n    self.datum_writer.writers_schema = writers_schema\r\n  File \"/usr/local/lib/python3.6/site-packages/spavro/io.py\", line 820, in writers_schema\r\n    self.write_datum = get_writer(parsed_writer_schema.to_json())\r\n  File \"src/spavro/fast_binary.pyx\", line 706, in spavro.fast_binary.get_writer\r\nKeyError: 'union'\r\n```\r\n\r\nI can perform the operation in both avro and avro-python3.\r\nAlso, the message validates against the schema in avro, avro-python3 and spavro.\r\n\r\nHere is the function in question:\r\n\r\n```\r\nimport json\r\nobj = None\r\nschema = None\r\nwith open(\"./payload.json\") as f:\r\n    obj = json.load(f)\r\n\r\nwith open(\"./schema.json\") as f:\r\n    schema = json.load(f)\r\n#spavro (current)\r\ndef t2(obj, schema):\r\n\r\n    from io import BytesIO\r\n    from spavro.schema import parse\r\n    from spavro.io import validate\r\n    from spavro.io import DatumWriter\r\n    from spavro.datafile import DataFileWriter\r\n    bytes_writer = BytesIO()\r\n    avsc = parse(json.dumps(schema, indent=2))\r\n    res = validate(avsc, obj)\r\n    print(res)\r\n\r\n    with DataFileWriter(bytes_writer, DatumWriter(), avsc, codec='deflate') as writer:\r\n        writer.append(obj)\r\n        writer.flush()\r\n        raw_bytes = bytes_writer.getvalue()\r\n```\r\n\r\nThe schema is:\r\n\r\n```\r\n[\r\n        {\r\n            \"name\": \"org.eha.demo.children\",\r\n            \"type\": \"record\",\r\n            \"label\": \"children\",\r\n            \"fields\": [\r\n                {\r\n                    \"name\": \"Person-id\",\r\n                    \"type\": [\r\n                        \"string\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        }\r\n                    ],\r\n                    \"jsonldPredicate\": {\r\n                        \"_id\": \"org.eha.demo.Person\",\r\n                        \"_type\": \"@id\"\r\n                    }\r\n                }\r\n            ],\r\n            \"namespace\": \"org.eha.demo\"\r\n        },\r\n        {\r\n            \"name\": \"org.eha.demo.home\",\r\n            \"type\": \"record\",\r\n            \"label\": \"home\",\r\n            \"fields\": [\r\n                {\r\n                    \"name\": \"Text\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        \"string\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        }\r\n                    ],\r\n                    \"jsonldPredicate\": \"xsd:string\"\r\n                },\r\n                {\r\n                    \"name\": \"Place-id\",\r\n                    \"type\": [\r\n                        \"string\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        }\r\n                    ],\r\n                    \"jsonldPredicate\": {\r\n                        \"_id\": \"org.eha.demo.Place\",\r\n                        \"_type\": \"@id\"\r\n                    }\r\n                }\r\n            ],\r\n            \"namespace\": \"org.eha.demo\"\r\n        },\r\n        {\r\n            \"name\": \"org.eha.demo.parents\",\r\n            \"type\": \"record\",\r\n            \"label\": \"parents\",\r\n            \"fields\": [\r\n                {\r\n                    \"name\": \"Person-id\",\r\n                    \"type\": [\r\n                        \"string\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        }\r\n                    ],\r\n                    \"jsonldPredicate\": {\r\n                        \"_id\": \"org.eha.demo.Person\",\r\n                        \"_type\": \"@id\"\r\n                    }\r\n                }\r\n            ],\r\n            \"namespace\": \"org.eha.demo\"\r\n        },\r\n        {\r\n            \"name\": \"org.eha.demo.Person\",\r\n            \"type\": \"record\",\r\n            \"label\": \"Person\",\r\n            \"fields\": [\r\n                {\r\n                    \"name\": \"id\",\r\n                    \"type\": \"string\",\r\n                    \"inherited_from\": \"org.eha.demo.BaseModel\",\r\n                    \"jsonldPredicate\": \"@id\"\r\n                },\r\n                {\r\n                    \"name\": \"rev\",\r\n                    \"type\": \"string\",\r\n                    \"inherited_from\": \"org.eha.demo.BaseModel\"\r\n                },\r\n                {\r\n                    \"name\": \"parents\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        \"org.eha.demo.parents\"\r\n                    ],\r\n                    \"jsonldPredicate\": \"org.eha.demo.parents\"\r\n                },\r\n                {\r\n                    \"doc\": \"A Name\",\r\n                    \"name\": \"name\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        },\r\n                        \"string\"\r\n                    ],\r\n                    \"jsonldPredicate\": \"org.eha.demo.name\"\r\n                },\r\n                {\r\n                    \"doc\": \"ISO date as a string\",\r\n                    \"name\": \"dateOfBirth\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        },\r\n                        \"string\"\r\n                    ],\r\n                    \"jsonldPredicate\": \"org.eha.demo.dateOfBirth\"\r\n                },\r\n                {\r\n                    \"name\": \"children\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        \"org.eha.demo.children\"\r\n                    ],\r\n                    \"jsonldPredicate\": \"org.eha.demo.children\"\r\n                },\r\n                {\r\n                    \"doc\": \"A description of the thing.\",\r\n                    \"name\": \"description\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        {\r\n                            \"type\": \"array\",\r\n                            \"items\": \"string\"\r\n                        },\r\n                        \"string\"\r\n                    ],\r\n                    \"jsonldPredicate\": \"org.eha.demo.description\"\r\n                },\r\n                {\r\n                    \"name\": \"home\",\r\n                    \"type\": [\r\n                        \"null\",\r\n                        \"org.eha.demo.home\"\r\n                    ],\r\n                    \"jsonldPredicate\": \"org.eha.demo.home\"\r\n                }\r\n            ],\r\n            \"namespace\": \"org.eha.demo\",\r\n            \"aetherBaseSchema\": true\r\n        }\r\n    ]\r\n```\r\n\r\nThe message body is:\r\n\r\n```\r\n{\r\n\t\"id\": \"c55fa41c-f2cb-4378-9a58-631d417fc085\",\r\n\t\"description\": [\"wlpqz\", \"bfg\"],\r\n\t\"home\": {\r\n\t\t\"Place-id\": \"38de9d89-f2e4-4639-884b-0f6e2246b4df\",\r\n\t\t\"Text\": \"srjygtwcd\"\r\n\t},\r\n\t\"dateOfBirth\": [\"unzmgdpweh\", \"lqfjvtigdkn\"],\r\n\t\"children\": null,\r\n\t\"name\": \"mgzn\",\r\n\t\"rev\": \"tqyams\",\r\n\t\"parents\": {\r\n\t\t\"Person-id\": [\"2e08b855-345c-4573-bd45-cdf1a25624f6\", \"503d0b26-989b-4967-871d-2c3194616013\", \"517bb3a0-26c3-4708-85c3-4ed4c6fb95af\"]\r\n\t}\r\n}\r\n```"},"comment":{"url":"https://api.github.com/repos/pluralsight/spavro/issues/comments/386533379","html_url":"https://github.com/pluralsight/spavro/issues/3#issuecomment-386533379","issue_url":"https://api.github.com/repos/pluralsight/spavro/issues/3","id":386533379,"user":{"login":"shawnsarwar","id":4309669,"avatar_url":"https://avatars3.githubusercontent.com/u/4309669?v=4","gravatar_id":"","url":"https://api.github.com/users/shawnsarwar","html_url":"https://github.com/shawnsarwar","followers_url":"https://api.github.com/users/shawnsarwar/followers","following_url":"https://api.github.com/users/shawnsarwar/following{/other_user}","gists_url":"https://api.github.com/users/shawnsarwar/gists{/gist_id}","starred_url":"https://api.github.com/users/shawnsarwar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shawnsarwar/subscriptions","organizations_url":"https://api.github.com/users/shawnsarwar/orgs","repos_url":"https://api.github.com/users/shawnsarwar/repos","events_url":"https://api.github.com/users/shawnsarwar/events{/privacy}","received_events_url":"https://api.github.com/users/shawnsarwar/received_events","type":"User","site_admin":false},"created_at":"2018-05-04T08:14:16Z","updated_at":"2018-05-04T08:14:16Z","author_association":"NONE","body":"I need to turn on notifications. I didn't see this until it had been closed. I'll test and if it's not resolved I'll open a new ticket referencing this one."}}
{"actor":{"display_login":"shawnsarwar"}}