Take the 2-minute tour ×

I am creating an application that needs to allow users to create custom fields, which I think would be best stored in a document-based (basically a serialized dictionary) model field.

I am concerned that I would run into performance issues storing these potentially very large documents in a SQL database, so I thought instead of storing the documents in the SQL database, I would just store pointers in the SQL database to the documents. The documents themselves would then be stored in a separate NoSQL database.

Assuming this structure makes sense, what is the best way to go about constructing a field that stores custom data fields in this manner? Optimally, these custom fields would be accessible on the object as attributes and would be denoted as custom with a "_c" appended to the name. E.g. created_date would become created_date_c on the django model object. I'm thinking custom managers would be best to tackle this: https://docs.djangoproject.com/en/dev/topics/db/managers/

EDIT: My SQL database is MySQL. Also, documents could make more sense as columns (as e.g. in Cassandra). Thoughts on this would be helpful.

share|improve this question
what is your sql db? –  alko Nov 5 '13 at 8:47
MySQL - thanks. –  Andrew Nov 5 '13 at 8:49
add comment

1 Answer

As far as I know, there is no best approach to this task, but I advise you to look into two other options, that has its own pros and contras:

  • Store your user defined data in a JSONfield or pickled field. This will save you a lot of efforts in writing custom managers and NoSQL. If you are worried about storing it along with your fixed structure data, store it in a separate model with one-to-one relationship in a separate InnoDB file for example.

  • Store yours user data in a generalized (field_id, field_name, content_type) and (object_id, field_id, field_value) dictionaries. They can be split by field types (i.e. int, string, float etc.). This approach won't give you well performing data model from scratch, but smart indexing and partitioning can make it worth noting. And your data query, model structure enforcement will be alot easier from other approaches.

  • If you consider using NoSQL or other db for your mutable content, be sure to choose one that has means for your data efficient querying, see this discussion and wiki.

share|improve this answer
add comment

Your Answer


