{"action":"created","issue":{"url":"https://api.github.com/repos/tebeka/fastavro/issues/159","repository_url":"https://api.github.com/repos/tebeka/fastavro","labels_url":"https://api.github.com/repos/tebeka/fastavro/issues/159/labels{/name}","comments_url":"https://api.github.com/repos/tebeka/fastavro/issues/159/comments","events_url":"https://api.github.com/repos/tebeka/fastavro/issues/159/events","html_url":"https://github.com/tebeka/fastavro/issues/159","id":296200804,"number":159,"title":"Writing with schemas defined in different files gives UnknownType errors","user":{"login":"hajapy","id":21218230,"avatar_url":"https://avatars3.githubusercontent.com/u/21218230?v=4","gravatar_id":"","url":"https://api.github.com/users/hajapy","html_url":"https://github.com/hajapy","followers_url":"https://api.github.com/users/hajapy/followers","following_url":"https://api.github.com/users/hajapy/following{/other_user}","gists_url":"https://api.github.com/users/hajapy/gists{/gist_id}","starred_url":"https://api.github.com/users/hajapy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hajapy/subscriptions","organizations_url":"https://api.github.com/users/hajapy/orgs","repos_url":"https://api.github.com/users/hajapy/repos","events_url":"https://api.github.com/users/hajapy/events{/privacy}","received_events_url":"https://api.github.com/users/hajapy/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-02-11T16:01:15Z","updated_at":"2018-02-12T16:56:37Z","closed_at":null,"author_association":"NONE","body":"I've been trying to make use of avro schemas in separate files, which is supposed to be possible: https://github.com/tebeka/fastavro/blob/b53954df0c6e5f466316792671c28839510d73e9/fastavro/_schema.pyx#L110-L116\r\n\r\nEven using the checked in examples https://github.com/tebeka/fastavro/blob/master/tests/avro-files/Child.avsc and https://github.com/tebeka/fastavro/blob/master/tests/avro-files/Parent.avsc, I found that this is not working.\r\n\r\nIf one loads the Parent.avsc schema, you get:\r\n```\r\n# inside fastavro/tests/avro-files\r\nschema = fastavro.schema.load_schema('Parent.avsc')\r\n{'fields': [{'name': 'child', 'type': 'Child'}],\r\n 'name': 'Parent',\r\n 'type': 'record'}\r\n```\r\n\r\nNow, if you try to serialize records with this schema, you get `UnknownType` error\r\n```\r\nIn [6]: with open('parent.avro', 'wb') as out:\r\n   ...:     fastavro.writer(out, schema, [{'child': []}])\r\n   ...:\r\n---------------------------------------------------------------------------\r\nUnknownType                               Traceback (most recent call last)\r\n<ipython-input-6-071f2d2cfde3> in <module>()\r\n      1 with open('parent.avro', 'wb') as out:\r\n----> 2     fastavro.writer(out, schema, [{'child': []}])\r\n      3\r\nfastavro/_write.pyx in fastavro._write.writer()\r\nfastavro/_write.pyx in fastavro._write.Writer.__init__()\r\nfastavro/_write.pyx in fastavro._write.acquaint_schema()\r\nfastavro/_schema.pyx in fastavro._schema.extract_named_schemas_into_repo()\r\nfastavro/_schema.pyx in fastavro._schema.extract_named_schemas_into_repo()\r\nUnknownType: Child\r\n```\r\n\r\nThat seems strange, seeing as the \"Child\" reader is indeed properly \"acquainted\" \r\n```\r\nfastavro.read.READERS\r\n{'Child': <cyfunction acquaint_schema.<locals>.<lambda>.<locals>.<lambda> at 0x103659830>,\r\n 'Parent': <cyfunction acquaint_schema.<locals>.<lambda>.<locals>.<lambda> at 0x1036598e8>, ...}\r\n```\r\n\r\nDoes the writer need to prepare the schema in a fully dereferenced form before being able to write? My understanding was that schemas embedded in avro files were fully self-contained, so perhaps this step is missing. I don't think serializing with such schemas is covered in the tests  https://github.com/tebeka/fastavro/blob/23b52f3376bb1777d55d9d239ed021ff558a191a/tests/test_fastavro.py#L219-L229"},"comment":{"url":"https://api.github.com/repos/tebeka/fastavro/issues/comments/364987900","html_url":"https://github.com/tebeka/fastavro/issues/159#issuecomment-364987900","issue_url":"https://api.github.com/repos/tebeka/fastavro/issues/159","id":364987900,"user":{"login":"hajapy","id":21218230,"avatar_url":"https://avatars3.githubusercontent.com/u/21218230?v=4","gravatar_id":"","url":"https://api.github.com/users/hajapy","html_url":"https://github.com/hajapy","followers_url":"https://api.github.com/users/hajapy/followers","following_url":"https://api.github.com/users/hajapy/following{/other_user}","gists_url":"https://api.github.com/users/hajapy/gists{/gist_id}","starred_url":"https://api.github.com/users/hajapy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hajapy/subscriptions","organizations_url":"https://api.github.com/users/hajapy/orgs","repos_url":"https://api.github.com/users/hajapy/repos","events_url":"https://api.github.com/users/hajapy/events{/privacy}","received_events_url":"https://api.github.com/users/hajapy/received_events","type":"User","site_admin":false},"created_at":"2018-02-12T16:56:37Z","updated_at":"2018-02-12T16:56:37Z","author_association":"NONE","body":"This may be helpful. I realized that if I put Child.avsc and Parent.avsc into a single schema file, I get get everything working:\r\n\r\n`Combo.avsc`\r\n```\r\n[\r\n  {\r\n    \"type\": \"record\",\r\n    \"name\": \"Child\",\r\n    \"fields\": []\r\n  },\r\n  {\r\n    \"type\": \"record\",\r\n    \"name\": \"Parent\",\r\n    \"fields\": [\r\n      {\r\n        \"name\": \"child\",\r\n        \"type\": \"Child\"\r\n      }\r\n    ]\r\n  }\r\n]\r\n```\r\n\r\n```\r\nIn [1]: import fastavro\r\nIn [2]: schema = fastavro.schema.load_schema('Combo.avsc')\r\nIn [3]: records = [{\"child\": {}}, {\"child\": {}}]\r\nIn [4]: with open('parent.avro', 'wb') as out:\r\n   ...:     fastavro.writer(out, schema, records)\r\n   ...:\r\n```\r\n\r\nNow, I see this:\r\n\r\n```\r\n(fastavro-qZB9hzSx) ➜  avro-files git:(6b9e222) ✗ fastavro --schema parent.avro\r\n[\r\n {\r\n  \"type\": \"record\",\r\n  \"name\": \"Child\",\r\n  \"fields\": []\r\n },\r\n {\r\n  \"type\": \"record\",\r\n  \"name\": \"Parent\",\r\n  \"fields\": [\r\n   {\r\n    \"name\": \"child\",\r\n    \"type\": \"Child\"\r\n   }\r\n  ]\r\n }\r\n]\r\n(fastavro-qZB9hzSx) ➜  avro-files git:(6b9e222) ✗ fastavro parent.avro\r\n{\"child\": {}}\r\n{\"child\": {}}\r\n```"}}
{"actor":{"display_login":"hajapy"}}