Sign up ×

I'm considering how to implement a component-based game, as that seems to be the hot thing and I like the idea of such a flexible design. One of the features of such a design is that adding new things to the game can be done through data, often presented as loading content through text files such as XML. This has the advantage of being human readable and easily editable in any text editor. On the downside, text can be slower to deal with, and you have to manage a large collection of data files. Similar text-based formats like JSON or config files would have similar benefits.

On the other side, there are small, portable databases like SQLite or Tokyo Cabinet. While not directly human readable, these files are easy to interface with, and I'd imagine some kind of editing tool would be preferable for game content design anyway. Using a DB allows for a consistent storage of config information and easy retrieval. You could serialize data into a DB for save games as well.

Performance wise, I think generally XML is faster for small files but a database scales better to large amounts of data. I'd imagine any real game is going to have a whole lot of game objects.

So the question: Which is the preferable approach? I'm leaning towards the DB, but I want to know if there are hidden pitfalls or real strong advantages to text files. Or if there are other alternatives besides these (serialize to binary format I guess?)

share|improve this question

6 Answers 6

up vote 13 down vote accepted

It might not come up so much for a small personal game, but one hard problem when it comes to game data is multi-user editing/versioning. We use a lot of small text files that get baked down to a small number of binary blobs by a build process. This makes life easier for designers since they have a lot of flexibility in their workflow. CCP, as a counter example, uses a central editing database that all designers connect to. This makes the build step unneeded (or at least a lot simpler) but it means you need to implement all the versioning and workflow features yourself, so they are bound to be simpler than other tools out there. You can deal with performance in either case, so the real question is what do you want for a designer workflow and how can you get there?

share|improve this answer
That's an excellent point. I hadn't considered the multi-person workflow too closely nor the version control. Those are both very good arguments in favor of text files, at least as a source document. Easier tool support as well. Thanks for that perspective! –  CodexArcanum Oct 21 '10 at 2:26
It's fairly easy to export a database into XML too. With only a small bit of forward planning, you can write your component loader as a interface - then just plug in either the database reader or xml file reader. While building out components, a database might be easier as there are ready-made GUI's for all data manipulations vs editing multiple files. Then, in the final build process, just save out the database to xml, bake into binary, etc... and the production version of the game just uses the appropriate loader. –  Leniency Jan 3 '11 at 3:59
How would that help? Using a custom editor and then outputting to text files actually gives you the worst parts of each option :-P –  coderanger Jan 3 '11 at 7:15

JSON is very lightweight and easy to understand. I think it's better suited for a game. cJSON is really nice. it will also incorporate into your source in the same manner as SQLlite does.

XML files are harder for users to edit than you may think. if you go that route you may want to make an editor for folks to use to help them avoid common pitfalls.

share|improve this answer
I really have never looked at JSON that much, as I was happy with XML (How much easier could it really be).. turns out JSON is pretty amazing... but like XML not sure how will it would work if it was very data heavy –  Spooks Nov 9 '10 at 0:27

The answer to this depends on the what language you are using for the game.

If you are using C++, you would be well advised to use one of the existing XML libraries (such as TinyXml or eXpat).

On the other hand, if you were using PHP, you could very easily use a database server such as MySQL or SQLite. (Keep in mind that if you go this route, you will need more resources because the database server will be running as a separate process and larger database applications like MySQL can consume a lot of RAM when running.)

JSON is gaining a lot of momentum and is certainly the best choice if your application is written using more than one language.

In short, it depends on what is easily usable in your language.

share|improve this answer
What's a problem to use any DB from C++? –  Budda Apr 13 '13 at 22:09

If you wanted to stay in the XML realm of things you can use Binary XML to help increase load performance.

For example Fast Infoset is an implementation of binary xml

One can think of Fast Infoset as gzip for XML, though FI aims to optimize both document size and processing performance, whereas gzip optimizes only the size. While the original formatting is lost, no information is lost in the conversion from XML to FI and back to XML.

share|improve this answer
Though probably not something I would use (I'm leaning towards JSON over XML) I do appreciate the links. –  CodexArcanum Oct 21 '10 at 2:27

I've been designing databases and played around with many game ideas for years now. My personal favorite at the moment, is some kinda textbased, human-readable format for configurations, but then parsing these "slow scripts" into something more runable. Much like JAVA and .NET is compiled into Run-time bytecode.

Same idea goes here. I have a "source" version of components and then a pre-compiler/parser runs through these. If they take up too much memory, you can put them in a fast SQL-datbase of some kinda or save them as binary files. But my point is, use the memory or SQL-database as your workspace/cache so that you dont have to parse the scripts over and over again. You could make the compiler, move the parsed files into a "source-lib" and this way let the precompiler do the versioning too (keeping the previous files for rollback purpose).

If its about "unlimited harddrive space", then sign up to something like Dropbox or Amazon S3 and sync with them.

So the plan for me is currently:

  1. Config/scripts/xml (some human readable format) as textfiles (just like sourcecode)
  2. Parser/validator that runs through new scripts and checkes they are following rules
  3. Compiler that makes binary or SQL-row out of the original files, so that these are fast to retreive + fast to run.

Regarding stability, I'm currently building the database to be "co-location hosted" and autosync between multiple locations too, so that if there is a network/hardware failure one location, the other sites should be able to handle the same traffic/gamestates.

share|improve this answer

If you use couchdb you will essentially be saving everything as a JSON structure. Couchdb will take care of replicating your (multiple) user's data more or less painlessly.

share|improve this answer

Your Answer


