
\section{Evaluation}
\label{sec:evaluation}
% 10. Describe how you have (or would have if given more time) evaluated your
%    language.

We evaluated the effectiveness of DeckBuild language as a game design tool for
both programmers and non-programmers by considering game designer productivity.

For this class project, we used ourselves as test subjects and performed an
informal evaluation of how easy it is to design cards using DeckBuild. We used
our DeckBuild language to describe all the existing cards from the base set of
Dominion \cite{DomCardList}. We found that specifying cards in our language was
easy and straightforward.

Given the limited scope of the semester, we could not attempt a more formal
evaluation of our language. If we had the time, there are a couple of possible
paths we could take. First, we could release the language, tools and runtime
system to game designers for them to use. We could gather an evaluation of our
current design and implementation by soliciting bug reports and feature
requests. This method of evaluation allows us to incrementally improve the
design that using the target users' wishes. In this scenario, we would measure
the success of our design in terms of the rate of adoption by our target users. 

Another possible evaluation method would be to conduct more formal user studies.
We would identify one or more competing technologies and use this as a basis for
comparison. While this is more principled in theory, in practice we think that
this would be little benefit for a lot of effort. Finding suitable subjects
would be difficult as DeckBuild has a very niche target market.
