Metadata-Version: 2.1 Name: dataclasses-json Version: 0.5.9 Summary: Easily serialize dataclasses to and from JSON Home-page: https://github.com/lidatong/dataclasses-json Author: lidatong Author-email: charles.dt.li@gmail.com License: MIT Keywords: dataclasses json Requires-Python: >=3.6 Description-Content-Type: text/markdown License-File: LICENSE Requires-Dist: marshmallow (<4.0.0,>=3.3.0) Requires-Dist: marshmallow-enum (<2.0.0,>=1.5.1) Requires-Dist: typing-inspect (>=0.4.0) Requires-Dist: dataclasses ; python_version == "3.6" Provides-Extra: dev Requires-Dist: pytest (>=7.2.0) ; extra == 'dev' Requires-Dist: ipython ; extra == 'dev' Requires-Dist: mypy (>=0.710) ; extra == 'dev' Requires-Dist: hypothesis ; extra == 'dev' Requires-Dist: portray ; extra == 'dev' Requires-Dist: flake8 ; extra == 'dev' Requires-Dist: simplejson ; extra == 'dev' Requires-Dist: setuptools ; extra == 'dev' Requires-Dist: wheel ; extra == 'dev' Requires-Dist: twine ; extra == 'dev' Requires-Dist: types-dataclasses ; (python_version == "3.6") and extra == 'dev' # Dataclasses JSON ![](https://github.com/lidatong/dataclasses-json/workflows/dataclasses-json/badge.svg) This library provides a simple API for encoding and decoding [dataclasses](https://docs.python.org/3/library/dataclasses.html) to and from JSON. It's very easy to get started. [README / Documentation website](https://lidatong.github.io/dataclasses-json). Features a navigation bar and search functionality, and should mirror this README exactly -- take a look! ## Quickstart `pip install dataclasses-json` ```python from dataclasses import dataclass from dataclasses_json import dataclass_json @dataclass_json @dataclass class Person: name: str person = Person(name='lidatong') person.to_json() # '{"name": "lidatong"}' <- this is a string person.to_dict() # {'name': 'lidatong'} <- this is a dict Person.from_json('{"name": "lidatong"}') # Person(1) Person.from_dict({'name': 'lidatong'}) # Person(1) # You can also apply _schema validation_ using an alternative API # This can be useful for "typed" Python code Person.from_json('{"name": 42}') # This is ok. 42 is not a `str`, but # dataclass creation does not validate types Person.schema().loads('{"name": 42}') # Error! Raises `ValidationError` ``` **What if you want to work with camelCase JSON?** ```python # same imports as above, with the additional `LetterCase` import from dataclasses import dataclass from dataclasses_json import dataclass_json, LetterCase @dataclass_json(letter_case=LetterCase.CAMEL) # now all fields are encoded/decoded from camelCase @dataclass class ConfiguredSimpleExample: int_field: int ConfiguredSimpleExample(1).to_json() # {"intField": 1} ConfiguredSimpleExample.from_json('{"intField": 1}') # ConfiguredSimpleExample(1) ``` ## Supported types It's recursive (see caveats below), so you can easily work with nested dataclasses. In addition to the supported types in the [py to JSON table](https://docs.python.org/3/library/json.html#py-to-json-table), this library supports the following: - any arbitrary [Collection](https://docs.python.org/3/library/collections.abc.html#collections.abc.Collection) type is supported. [Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) types are encoded as JSON objects and `str` types as JSON strings. Any other Collection types are encoded into JSON arrays, but decoded into the original collection types. - [datetime](https://docs.python.org/3/library/datetime.html#available-types) objects. `datetime` objects are encoded to `float` (JSON number) using [timestamp](https://docs.python.org/3/library/datetime.html#datetime.datetime.timestamp). As specified in the `datetime` docs, if your `datetime` object is naive, it will assume your system local timezone when calling `.timestamp()`. JSON numbers corresponding to a `datetime` field in your dataclass are decoded into a datetime-aware object, with `tzinfo` set to your system local timezone. Thus, if you encode a datetime-naive object, you will decode into a datetime-aware object. This is important, because encoding and decoding won't strictly be inverses. See [this section](#Overriding) if you want to override this default behavior (for example, if you want to use ISO). - [UUID](https://docs.python.org/3/library/uuid.html#uuid.UUID) objects. They are encoded as `str` (JSON string). - [Decimal](https://docs.python.org/3/library/decimal.html) objects. They are also encoded as `str`. **The [latest release](https://github.com/lidatong/dataclasses-json/releases/latest) is compatible with both Python 3.7 and Python 3.6 (with the dataclasses backport).** ## Usage #### Approach 1: Class decorator ```python from dataclasses import dataclass from dataclasses_json import dataclass_json @dataclass_json @dataclass class Person: name: str lidatong = Person('lidatong') # Encoding to JSON lidatong.to_json() # '{"name": "lidatong"}' # Decoding from JSON Person.from_json('{"name": "lidatong"}') # Person(name='lidatong') ``` Note that the `@dataclass_json` decorator must be stacked above the `@dataclass` decorator (order matters!) #### Approach 2: Inherit from a mixin ```python from dataclasses import dataclass from dataclasses_json import DataClassJsonMixin @dataclass class Person(DataClassJsonMixin): name: str lidatong = Person('lidatong') # A different example from Approach 1 above, but usage is the exact same assert Person.from_json(lidatong.to_json()) == lidatong ``` Pick whichever approach suits your taste. Note that there is better support for the mixin approach when using _static analysis_ tools (e.g. linting, typing), but the differences in implementation will be invisible in _runtime_ usage. ## How do I... ### Use my dataclass with JSON arrays or objects? ```python from dataclasses import dataclass from dataclasses_json import dataclass_json @dataclass_json @dataclass class Person: name: str ``` **Encode into a JSON array containing instances of my Data Class** ```python people_json = [Person('lidatong')] Person.schema().dumps(people_json, many=True) # '[{"name": "lidatong"}]' ``` **Decode a JSON array containing instances of my Data Class** ```python people_json = '[{"name": "lidatong"}]' Person.schema().loads(people_json, many=True) # [Person(name='lidatong')] ``` **Encode as part of a larger JSON object containing my Data Class (e.g. an HTTP request/response)** ```python import json response_dict = { 'response': { 'person': Person('lidatong').to_dict() } } response_json = json.dumps(response_dict) ``` In this case, we do two steps. First, we encode the dataclass into a **python dictionary** rather than a JSON string, using `.to_dict`. Second, we leverage the built-in `json.dumps` to serialize our `dataclass` into a JSON string. **Decode as part of a larger JSON object containing my Data Class (e.g. an HTTP response)** ```python import json response_dict = json.loads('{"response": {"person": {"name": "lidatong"}}}') person_dict = response_dict['response'] person = Person.from_dict(person_dict) ``` In a similar vein to encoding above, we leverage the built-in `json` module. First, call `json.loads` to read the entire JSON object into a dictionary. We then access the key of the value containing the encoded dict of our `Person` that we want to decode (`response_dict['response']`). Second, we load in the dictionary using `Person.from_dict`. ### Encode or decode into Python lists/dictionaries rather than JSON? This can be by calling `.schema()` and then using the corresponding encoder/decoder methods, ie. `.load(...)`/`.dump(...)`. **Encode into a single Python dictionary** ```python person = Person('lidatong') person.to_dict() # {'name': 'lidatong'} ``` **Encode into a list of Python dictionaries** ```python people = [Person('lidatong')] Person.schema().dump(people, many=True) # [{'name': 'lidatong'}] ``` **Decode a dictionary into a single dataclass instance** ```python person_dict = {'name': 'lidatong'} Person.from_dict(person_dict) # Person(name='lidatong') ``` **Decode a list of dictionaries into a list of dataclass instances** ```python people_dicts = [{"name": "lidatong"}] Person.schema().load(people_dicts, many=True) # [Person(name='lidatong')] ``` ### Encode or decode from camelCase (or kebab-case)? JSON letter case by convention is camelCase, in Python members are by convention snake_case. You can configure it to encode/decode from other casing schemes at both the class level and the field level. ```python from dataclasses import dataclass, field from dataclasses_json import LetterCase, config, dataclass_json # changing casing at the class level @dataclass_json(letter_case=LetterCase.CAMEL) @dataclass class Person: given_name: str family_name: str Person('Alice', 'Liddell').to_json() # '{"givenName": "Alice"}' Person.from_json('{"givenName": "Alice", "familyName": "Liddell"}') # Person('Alice', 'Liddell') # at the field level @dataclass_json @dataclass class Person: given_name: str = field(metadata=config(letter_case=LetterCase.CAMEL)) family_name: str Person('Alice', 'Liddell').to_json() # '{"givenName": "Alice"}' # notice how the `family_name` field is still snake_case, because it wasn't configured above Person.from_json('{"givenName": "Alice", "family_name": "Liddell"}') # Person('Alice', 'Liddell') ``` **This library assumes your field follows the Python convention of snake_case naming.** If your field is not `snake_case` to begin with and you attempt to parameterize `LetterCase`, the behavior of encoding/decoding is undefined (most likely it will result in subtle bugs). ### Encode or decode using a different name ```python from dataclasses import dataclass, field from dataclasses_json import config, dataclass_json @dataclass_json @dataclass class Person: given_name: str = field(metadata=config(field_name="overriddenGivenName")) Person(given_name="Alice") # Person('Alice') Person.from_json('{"overriddenGivenName": "Alice"}') # Person('Alice') Person('Alice').to_json() # {"overriddenGivenName": "Alice"} ``` ### Handle missing or optional field values when decoding? By default, any fields in your dataclass that use `default` or `default_factory` will have the values filled with the provided default, if the corresponding field is missing from the JSON you're decoding. **Decode JSON with missing field** ```python @dataclass_json @dataclass class Student: id: int name: str = 'student' Student.from_json('{"id": 1}') # Student(id=1, name='student') ``` Notice `from_json` filled the field `name` with the specified default 'student' when it was missing from the JSON. Sometimes you have fields that are typed as `Optional`, but you don't necessarily want to assign a default. In that case, you can use the `infer_missing` kwarg to make `from_json` infer the missing field value as `None`. **Decode optional field without default** ```python @dataclass_json @dataclass class Tutor: id: int student: Optional[Student] = None Tutor.from_json('{"id": 1}') # Tutor(id=1, student=None) ``` Personally I recommend you leverage dataclass defaults rather than using `infer_missing`, but if for some reason you need to decouple the behavior of JSON decoding from the field's default value, this will allow you to do so. ### Handle unknown / extraneous fields in JSON? By default, it is up to the implementation what happens when a `json_dataclass` receives input parameters that are not defined. (the `from_dict` method ignores them, when loading using `schema()` a ValidationError is raised.) There are three ways to customize this behavior. Assume you want to instantiate a dataclass with the following dictionary: ```python dump_dict = {"endpoint": "some_api_endpoint", "data": {"foo": 1, "bar": "2"}, "undefined_field_name": [1, 2, 3]} ``` 1. You can enforce to always raise an error by setting the `undefined` keyword to `Undefined.RAISE` (`'RAISE'` as a case-insensitive string works as well). Of course it works normally if you don't pass any undefined parameters. ```python from dataclasses_json import Undefined @dataclass_json(undefined=Undefined.RAISE) @dataclass() class ExactAPIDump: endpoint: str data: Dict[str, Any] dump = ExactAPIDump.from_dict(dump_dict) # raises UndefinedParameterError ``` 2. You can simply ignore any undefined parameters by setting the `undefined` keyword to `Undefined.EXCLUDE` (`'EXCLUDE'` as a case-insensitive string works as well). Note that you will not be able to retrieve them using `to_dict`: ```python from dataclasses_json import Undefined @dataclass_json(undefined=Undefined.EXCLUDE) @dataclass() class DontCareAPIDump: endpoint: str data: Dict[str, Any] dump = DontCareAPIDump.from_dict(dump_dict) # DontCareAPIDump(endpoint='some_api_endpoint', data={'foo': 1, 'bar': '2'}) dump.to_dict() # {"endpoint": "some_api_endpoint", "data": {"foo": 1, "bar": "2"}} ``` 3. You can save them in a catch-all field and do whatever needs to be done later. Simply set the `undefined` keyword to `Undefined.INCLUDE` (`'INCLUDE'` as a case-insensitive string works as well) and define a field of type `CatchAll` where all unknown values will end up. This simply represents a dictionary that can hold anything. If there are no undefined parameters, this will be an empty dictionary. ```python from dataclasses_json import Undefined, CatchAll @dataclass_json(undefined=Undefined.INCLUDE) @dataclass() class UnknownAPIDump: endpoint: str data: Dict[str, Any] unknown_things: CatchAll dump = UnknownAPIDump.from_dict(dump_dict) # UnknownAPIDump(endpoint='some_api_endpoint', data={'foo': 1, 'bar': '2'}, unknown_things={'undefined_field_name': [1, 2, 3]}) dump.to_dict() # {'endpoint': 'some_api_endpoint', 'data': {'foo': 1, 'bar': '2'}, 'undefined_field_name': [1, 2, 3]} ``` Notes: - When using `Undefined.INCLUDE`, an `UndefinedParameterError` will be raised if you don't specify exactly one field of type `CatchAll`. - Note that `LetterCase` does not affect values written into the `CatchAll` field, they will be as they are given. - When specifying a default (or a default factory) for the the `CatchAll`-field, e.g. `unknown_things: CatchAll = None`, the default value will be used instead of an empty dict if there are no undefined parameters. - Calling __init__ with non-keyword arguments resolves the arguments to the defined fields and writes everything else into the catch-all field. 4. All 3 options work as well using `schema().loads` and `schema().dumps`, as long as you don't overwrite it by specifying `schema(unknown=)`. marshmallow uses the same 3 keywords ['include', 'exclude', 'raise'](https://marshmallow.readthedocs.io/en/stable/quickstart.html#handling-unknown-fields). 5. All 3 operations work as well using `__init__`, e.g. `UnknownAPIDump(**dump_dict)` will **not** raise a `TypeError`, but write all unknown values to the field tagged as `CatchAll`. Classes tagged with `EXCLUDE` will also simply ignore unknown parameters. Note that classes tagged as `RAISE` still raise a `TypeError`, and **not** a `UndefinedParameterError` if supplied with unknown keywords. ### Override the default encode / decode / marshmallow field of a specific field? See [Overriding](#Overriding) ### Handle recursive dataclasses? Object hierarchies where fields are of the type that they are declared within require a small type hinting trick to declare the forward reference. ```python from typing import Optional from dataclasses import dataclass from dataclasses_json import dataclass_json @dataclass_json @dataclass class Tree(): value: str left: Optional['Tree'] right: Optional['Tree'] ``` Avoid using ```python from __future__ import annotations ``` as it will cause problems with the way dataclasses_json accesses the type annotations. ## Marshmallow interop Using the `dataclass_json` decorator or mixing in `DataClassJsonMixin` will provide you with an additional method `.schema()`. `.schema()` generates a schema exactly equivalent to manually creating a marshmallow schema for your dataclass. You can reference the [marshmallow API docs](https://marshmallow.readthedocs.io/en/3.0/api_reference.html#schema) to learn other ways you can use the schema returned by `.schema()`. You can pass in the exact same arguments to `.schema()` that you would when constructing a `PersonSchema` instance, e.g. `.schema(many=True)`, and they will get passed through to the marshmallow schema. ```python from dataclasses import dataclass from dataclasses_json import dataclass_json @dataclass_json @dataclass class Person: name: str # You don't need to do this - it's generated for you by `.schema()`! from marshmallow import Schema, fields class PersonSchema(Schema): name = fields.Str() ``` Briefly, on what's going on under the hood in the above examples: calling `.schema()` will have this library generate a [marshmallow schema]('https://marshmallow.readthedocs.io/en/3.0/api_reference.html#schema) for you. It also fills in the corresponding object hook, so that marshmallow will create an instance of your Data Class on `load` (e.g. `Person.schema().load` returns a `Person`) rather than a `dict`, which it does by default in marshmallow. **Performance note** `.schema()` is not cached (it generates the schema on every call), so if you have a nested Data Class you may want to save the result to a variable to avoid re-generation of the schema on every usage. ```python person_schema = Person.schema() person_schema.dump(people, many=True) # later in the code... person_schema.dump(person) ``` ## Overriding / Extending #### Overriding For example, you might want to encode/decode `datetime` objects using ISO format rather than the default `timestamp`. ```python from dataclasses import dataclass, field from dataclasses_json import dataclass_json, config from datetime import datetime from marshmallow import fields @dataclass_json @dataclass class DataClassWithIsoDatetime: created_at: datetime = field( metadata=config( encoder=datetime.isoformat, decoder=datetime.fromisoformat, mm_field=fields.DateTime(format='iso') ) ) ``` #### Extending Similarly, you might want to extend `dataclasses_json` to encode `date` objects. ```python from dataclasses import dataclass, field from dataclasses_json import dataclass_json, config from datetime import date from marshmallow import fields dataclasses_json.cfg.global_config.encoders[date] = date.isoformat dataclasses_json.cfg.global_config.decoders[date] = date.fromisoformat @dataclass_json @dataclass class DataClassWithIsoDatetime: created_at: date modified_at: date accessed_at: date ``` As you can see, you can **override** or **extend** the default codecs by providing a "hook" via a callable: - `encoder`: a callable, which will be invoked to convert the field value when encoding to JSON - `decoder`: a callable, which will be invoked to convert the JSON value when decoding from JSON - `mm_field`: a marshmallow field, which will affect the behavior of any operations involving `.schema()` Note that these hooks will be invoked regardless if you're using `.to_json`/`dump`/`dumps` and `.from_json`/`load`/`loads`. So apply overrides / extensions judiciously, making sure to carefully consider whether the interaction of the encode/decode/mm_field is consistent with what you expect! #### What if I have other dataclass field extensions that rely on `metadata` All the `dataclasses_json.config` does is return a mapping, namespaced under the key `'dataclasses_json'`. Say there's another module, `other_dataclass_package` that uses metadata. Here's how you solve your problem: ```python metadata = {'other_dataclass_package': 'some metadata...'} # pre-existing metadata for another dataclass package dataclass_json_config = config( encoder=datetime.isoformat, decoder=datetime.fromisoformat, mm_field=fields.DateTime(format='iso') ) metadata.update(dataclass_json_config) @dataclass_json @dataclass class DataClassWithIsoDatetime: created_at: datetime = field(metadata=metadata) ``` You can also manually specify the dataclass_json configuration mapping. ```python @dataclass_json @dataclass class DataClassWithIsoDatetime: created_at: date = field( metadata={'dataclasses_json': { 'encoder': date.isoformat, 'decoder': date.fromisoformat, 'mm_field': fields.DateTime(format='iso') }} ) ``` ## A larger example ```python from dataclasses import dataclass from dataclasses_json import dataclass_json from typing import List @dataclass_json @dataclass(frozen=True) class Minion: name: str @dataclass_json @dataclass(frozen=True) class Boss: minions: List[Minion] boss = Boss([Minion('evil minion'), Minion('very evil minion')]) boss_json = """ { "minions": [ { "name": "evil minion" }, { "name": "very evil minion" } ] } """.strip() assert boss.to_json(indent=4) == boss_json assert Boss.from_json(boss_json) == boss ``` ## Performance Take a look at [this issue](https://github.com/lidatong/dataclasses-json/issues/228) ## Versioning Note this library is still pre-1.0.0 (SEMVER). The current convention is: - **PATCH** version upgrades for bug fixes and minor feature additions. - **MINOR** version upgrades for big API features and breaking changes. Once this library is 1.0.0, it will follow standard SEMVER conventions. ## Roadmap Currently the focus is on investigating and fixing bugs in this library, working on performance, and finishing [this issue](https://github.com/lidatong/dataclasses-json/issues/31). That said, if you think there's a feature missing / something new needed in the library, please see the contributing section below. ## Contributing First of all, thank you for being interested in contributing to this library. I really appreciate you taking the time to work on this project. - If you're just interested in getting into the code, a good place to start are issues tagged as bugs. - If introducing a new feature, especially one that modifies the public API, consider submitting an issue for discussion before a PR. Please also take a look at existing issues / PRs to see what you're proposing has already been covered before / exists. - I like to follow the commit conventions documented [here](https://www.conventionalcommits.org/en/v1.0.0/#summary)