# SmoothNets: Optimizing CNN architecture design for differentially private
deep learning

Nicolas W. Remerscheid  Alexander Ziller  Daniel Rueckert  Georgios Kaissis

###### Abstract

The arguably most widely employed algorithm to train deep neural networks with
Differential Privacy is Differentially Private Stochastic Gradient Descent
(DP-SGD), which requires clipping and noising of per-sample gradients. This
introduces a reduction in model utility compared to non-private training.
Empirically, it can be observed that this accuracy degradation is strongly
dependent on the model architecture. We investigated this phenomenon and, by
combining components which exhibit good individual performance, distilled a
new model architecture termed SmoothNet, which is characterised by increased
robustness to the challenges of DP-SGD training. Experimentally, we benchmark
SmoothNet against standard architectures on two benchmark datasets and observe
that our architecture outperforms others, reaching an accuracy of 73.5% on
CIFAR-10 at $\varepsilon=7.0$ and 69.2% at $\varepsilon=7.0$ on ImageNette, a
state-of-the-art result compared to prior architectural modifications for
Differential Privacy (DP).

Differential Privacy, Deep Learning, Architecture Design

##  1 Introduction

DP-SGD, introduced by Abadi et al., enables training deep learning models with
Differential Privacy (DP). It offers a guarantee to individuals whose data is
used to train the model that the information which can be learned about them
is limited. Often, DP-SGD comes at the cost of impaired model utility. This
trade-off is especially undesirable in critical domains as medicine, where the
unfortunate requirement of choosing between strong privacy and high-quality
predictions is introduced. Prior work suggests that various model
architectures provide very different robustness to such a utility drop
(Papernot et al., 2019; Morsbach et al., 2021; Ziller et al., 2021b). These
findings lead to the hypothesis that the presence of specific architectural
components in deep learning architectures could influence their overall
robustness against the specific challenges of DP-SGD, namely a geometric bias
induced by per-sample gradient clipping (Chen et al., 2020) and an overall
noisy optimisation trajectory (Dörmann et al., 2021). To investigate this
conjecture, we systematically evaluated specific components of standard
architectures and their influence on DP-robustness, concretely residual
connections (He et al., 2016), dense connections (Huang et al., 2017) and,
inspired by Tan & Le (2019), the depth and width of the architecture’s
convolutional layers. From the conclusions gained, we distilled a model
architecture for computer vision tasks, specifically designed to withstand the
challenges of DP-Training, which we term SmoothNet. Our key contributions are:

  * •

We systematically evaluated individual model components of popular deep
learning architectures in regards to their influence on the performance of DP-
SGD training.

  * •

From this we distilled optimal components and assembled a new model
architecture, termed SmoothNet, which yields SOTA results in terms of
differentially privately trained models on benchmark datasets CIFAR-10 and
ImageNette.

###  1.1 Prior Work

Several lines of work have attempted to tackle the utility ramifications of
deep network training with Differential Privacy guarantees. They can be
largely categorised into two types of approaches: (1) Architectural
modifications, where model architectures are adapted to be robust against the
challenges in DP-SGD-based training and (2) training methods, which propose
algorithmic modifications to the model training process. We provide an
overview of these results categorised by approach in Table 1.

Reference | Dataset | $\varepsilon$ | Accuracy | Type of work
---|---|---|---|---
Papernot et al. 2020 | CIFAR-10 | $7.53$ | $66.2\%$ | Architecture
Klause et al. 2022 | $7.5$ | $71.7\%$ | Architecture
Dörmann et al. 2021 | $7.42$ | $70.1\%$ | Training method
De et al. 2022 | $8.0$ | $81.4\%$ | Training method
Klause et al. 2022 | ImageNette | $7.5$ | $64.8\%$ | Architecture
Table 1: Results of other works on the same datasets utilised in our study.
Type of work refers to the categories specified in Section 1.1.

#### Architectural modifications

Papernot et al. proposed Tempered Sigmoids as activation functions for DP-SGD
training, which they applied in small feed-forward convolutional neural
networks on the MNIST, FashionMNIST and CIFAR-10 datasets. Klause et al.
adapted the ResNet architecture (He et al., 2016) and introduced an additional
normalization layer after each residual block. Authors argue that the scale
mismatch between the activations output by the convolutional and residual
paths of the architecture inhibit convergence when trained with DP, especially
since –contrary to non-private training– Batch Normalisation layers (Ioffe &
Szegedy, 2015) are not available.

#### Training methods

Dörmann et al. showed that the utility of DP-SGD training crucially depends on
the batch size used during training (Dörmann et al., 2021). They empirically
demonstrated that higher batch sizes allow for better model utility at the
same privacy budget levels, as the sub-sampling amplification guarantees of
DP-SGD get outweighed by the stabilisation of the optimisation trajectory due
to a decrease in gradient bias. This finding was underscored by the very
recent work by De et al., which was released during the preparation of this
manuscript. This work also showed the positive impact of weight
standardization (Qiao et al., 2019), parameter averaging (Polyak & Juditsky,
1992; Tan & Le, 2019) and lastly augmentation multiplicity (Fort et al., 2021;
Hoffer et al., 2019; Touvron et al., 2021), where each sample is augmented
multiple times and an average gradient of all augmentations is treated as the
gradient of this sample. These training modifications led to a new SOTA
performance on the CIFAR-10 benchmark dataset (Krizhevsky et al., 2009) and
(with pre-training on a proprietary dataset) on ImageNet (Deng et al., 2009).

##  2 SmoothNets

(a) Main Building Block of SmoothNet. Similar to a DenseBlock, a residual
connection concatenates the input and the processed features. However, we use
wider convolutional layers, Group Normalization with 8 groups for DP
compatibility, and a SELU activation function.

(b) Architecture of the SmoothNet employed in our experiments consisting of
SmoothBlocks, wide convolutional layers, average pooling layers, linear layers
and SELU activation functions. $5\times$ is used as shorthand to denote 5
consecutive SmoothBlocks.

Figure 1: Overview of the building blocks for our SmoothNet architecture.

So far, only few works have considered which concrete model design choices
provide robustness against utility drops for DP-SGD training. The specific
challenges in this scenario originate from the requirements of ascertaining a
bounded gradient norm and of adding noise to the gradients during training in
order to realise DP guarantees. To shed more light on the correlation between
model architecture and robustness/utility, we conducted preliminary
experiments in which architectural design choices were isolated and
systematically evaluated regarding their influence on the performance of DP
training.

#### Empirical study of popular model components

In this work, we focused on determining the optimal ratio of depth to width
(in terms of convolutional filters), a line of research initiated by Tan & Le
(2019). Moreover, we investigated residual connections proposed by He et al.
(2016), and dense connections proposed by (Huang et al., 2017). In addition,
we evaluated the effect of different normalization layers, namely instance
normalization (Ulyanov et al., 2016), group normalization (Wu & He, 2018) and
layer normalization (Ba et al., 2016) as optimal normalization components for
DP-SGD training. Furthermore, we evaluated pooling layers (maximum pooling and
average pooling) within the SmoothBlocks (compare Figure 1(a)) as well as
activation functions (ReLU (Nair & Hinton, 2010), SELU (Klambauer et al.,
2017)). In total, we evaluated over 100 different model architectures in more
than 6000 experiments on CIFAR-10 based on (a) handcrafted models, intended to
inspect the effect of isolated model components and (b) on established (SOTA)
models to guarantee representative results. We use the SGD optimizer with a
learning rate scheduler decaying the initial learning rate of $0.1$ by a
factor of $0.7$ every epoch. Furthermore, we use a batch-size of 128, an early
stopping patience of 15 epochs on the validation loss, a divergence threshold
of 1000, and conducted a Bayesian optimization over the $L_{2}$ clipping norm
in the range of $[0.1,10.0]$ and over the number of epochs resulting in a
binary decision of either 15 or 30 epochs. We note that we did not count the
privacy budget of these hyperparameter searches against the overall allotted
privacy budget.We conducted 15 runs per sweep and considered the 10 best
models ordered by validation accuracy (to suppress artifacts from poor
initialization). For DP-SGD training we employed the Opacus (Yousefpour et
al., 2021) and deepee (Ziller et al., 2021a) packages.

#### Observations

From our experiments, we deduced two main lines of observations: (1) The width
to depth ratio is strongly correlated with model performance. Concretely, the
optimal width-depth ratio is higher for private training compared to non-
private training and optimally $>1$, corresponding to a wide model. (2) Both
residual and dense connections are beneficial for robust models, with dense
connections being especially useful for training with DP-guarantees.
Furthermore, Group Normalization proved to outperform other normalization
layers, which corroborates findings from prior works (Klause et al., 2022; De
et al., 2022). The best-performing activation function in our experiments was
SELU. Max Pooling as a SmoothBlock component led to superior results compared
to other pooling functions.

#### SmoothNet Architecture

Based on the aforementioned conclusions, we synthesized the SmoothNet
architecture. The main components are building blocks we term SmoothBlocks.
These are conceptually similar to DenseBlocks (Huang et al., 2017), however
with several modifications: (1) The $3\times 3$ convolutional layers rapidly
increase in width such that, at the widest stage, more than 800 channels are
present. (2) We use Group Normalisation layers with 8 groups, which replace
Batch Normalisation in the original architecture. (3) SELU layers (Klambauer
et al., 2017) are used as activation functions. We restrict ourselves to a
limited depth, of 10 SmoothBlocks. As in DenseNets (Huang et al., 2017), we
use average pooling between SmoothBlocks and to compress the extracted
features to 2048 features, which are input to the classifying part. As a
classifier block, we use three linear layers, separated by SELU activation
functions. An overview of the SmoothBlock and the SmoothNet architecture can
be found in Figure 1.

##  3 Results

Dataset | $\varepsilon$ | Architecture | N Params | Validation Acc.
---|---|---|---|---
CIFAR-10 | $7.0$ | ResNet-18 | $11.2M$ | $59.8\%$
ResNet-34 | $21.2M$ | $57.7\%$
DenseNet-121 | $7.0M$ | $65.4\%$
EfficientNet-B0 | $4.0M$ | $54.1\%$
SmoothNet (ours) | $3.4M$ | $\mathbf{73.5\%}$
ImageNette | $7.0$ | ResNet-18 | $11.2M$ | $59.9\%$
ResNet-34 | $21.2M$ | $58.7\%$
DenseNet-121 | $7.0M$ | $65.1\%$
EfficientNet-B0 | $4.0M$ | $60.5\%$
SmoothNet (ours) | $3.4M$ | $\mathbf{69.7}\%$
Table 2: Benchmarking SmoothNet against standard architectures on CIFAR-10 and
ImageNette with Differential Privacy. N Params refers to the number of
trainable parameters in the network in Millions. Validation Acc. refers to the
corresponding row’s best model’s accuracy on the validation set.

To validate our findings, we evaluated our novel architecture on two benchmark
datasets. Firstly, CIFAR-10 served as a basis for evaluations with a long line
of previous works. Moreover, to assess the performance of our architecture on
a larger-image dataset, we also used ImageNette (Howard, 2019), a subset of
ImageNet, which –like the original dataset– has a resolution of $224\times
224$. We compared our architecture to several standard architectures, more
specifically a small and medium sized ResNet (18, 34), a DenseNet-121. These
architectures are the original architectures employing residual and dense
connections respectively. Lastly, we compared against EfficientNet-B0 (Tan &
Le, 2019), a high performance model with high parameter-efficiency. Similarly
to the evaluation of specific model components, we trained each competing
model 15 times with varying $L_{2}$-clipping norms in the range of
$[0.1,20.0]$ and for either 90, 120, or 180 epochs. We use Xavier-Glorot
weight initialization (Glorot & Bengio, 2010), a learning-rate decay of $0.9$
every five epochs, an initial-learning rate of $0.002$, an SGD momentum of
$0.9$, a weight-decay of $0.0002$, and a batch-size of 256. On both datasets,
we found SmoothNets to achieve the highest performance in terms of validation
accuracy (Table 2).

#### Comparison to other works

To contextualise our findings, we curated an overview of the results of
related works in Table 1. Furthermore, we investigate the Pareto front of our
model over different values of $\varepsilon$ on CIFAR-10, which is visualised
in Figure 2, alongside results of other works. Our results outperform all
previous works except De et al. 2022, which was released during the
preparation of our manuscript. However, the authors of this work found that
the greatest increase in accuracy resulted from an adaptation of the model’s
training instead of the architecture, hence we anticipate that our
architectural modifications would act synergistically with the work of De et
al. 2022 to yield even higher performance, which we intend to evaluate in
upcoming work. Interestingly, the authors of this work also employ WideResNets
as an architecture, empirically corroborating our findings that wide models
with residual connections improve training stability under DP conditions. Of
note, these results also motivate a more differentiated view on the claim that
larger models are expected to have lower accuracy by default because the DP-
SGD noise scales proportionally to the number of parameters. Instead, model
depth and width seem to interact in a more nuanced way with each other and
with other architectural components.

Figure 2: Pareto front contextualising our results compared to other work on
the CIFAR-10 dataset.

##  4 Discussion and Conclusion

In this paper, we continue the investigation into optimal architectural
choices for high-utility training of neural networks with DP guarantees. Prior
works have demonstrated that specialised training methods and model
adaptations can allow for high model utility under DP guarantees. We naturally
augment these previous results by introducing SmoothNet, an architecture that
outperforms previous works focused on architectural modifications.

One limitation of our work is that, so far, we have no conclusive explanation
for all of the contributing factors behind the high performance of our
architecture in DP-SGD-training. We conjecture that the smoothness of the
optimisation landscape is a strong influence on the performance. We observed
that both dense connections and normalisation layers, which have been shown to
improve this attribute (Li et al., 2018; Santurkar et al., 2019) to also
improve model accuracy. This hypothesis also motivates the choice of name
SmoothNet for our architecture. Moreover, the utilisation of SELU activation
functions, stemming from previous work on normalisation-free architectures
(Klambauer et al., 2017), seems to indicate that activation scale matters
especially in DP-SGD. Similar findings are discussed in previous works
(Papernot et al., 2020; Klause et al., 2022). We intend to empirically and
theoretically investigate the training dynamics of SmoothNets in future work.
Moreover, as our architectures allow for higher utility while preserving
privacy budget, we intend to investigate their behaviour in the very-low
$\varepsilon$ regime, which is –so far– under-investigated in the realm of
private deep learning. In conclusion, our work is an empirical investigation
into the factors enabling neural network training on sensitive datasets with
high utility and strong privacy guarantees. We hope that our work will
stimulate further research in this direction and elucidate the underlying
reasons why specific model components are better suited for DP training, which
may lead to findings of independent theoretical interest.

## References

  * Abadi et al. (2016) Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L.  Deep learning with differential privacy.  In _Proceedings of the 2016 ACM SIGSAC conference on computer and communications security_ , pp. 308–318, 2016.
  * Ba et al. (2016) Ba, J. L., Kiros, J. R., and Hinton, G. E.  Layer normalization.  _arXiv preprint arXiv:1607.06450_ , 2016.
  * Chen et al. (2020) Chen, X., Wu, S. Z., and Hong, M.  Understanding gradient clipping in private sgd: A geometric perspective.  _Advances in Neural Information Processing Systems_ , 33:13773–13782, 2020.
  * De et al. (2022) De, S., Berrada, L., Hayes, J., Smith, S. L., and Balle, B.  Unlocking high-accuracy differentially private image classification through scale, 2022.  URL https://arxiv.org/abs/2204.13650.
  * Deng et al. (2009) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pp. 248–255. Ieee, 2009.
  * Dörmann et al. (2021) Dörmann, F., Frisk, O., Andersen, L. N., and Pedersen, C. F.  Not all noise is accounted equally: How differentially private learning benefits from large sampling rates.  In _2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP)_ , pp. 1–6. IEEE, 2021.
  * Fort et al. (2021) Fort, S., Brock, A., Pascanu, R., De, S., and Smith, S. L.  Drawing multiple augmentation samples per image during training efficiently decreases test error.  _arXiv preprint arXiv:2105.13343_ , 2021.
  * Glorot & Bengio (2010) Glorot, X. and Bengio, Y.  Understanding the difficulty of training deep feedforward neural networks.  In _AISTATS_ , 2010.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * Hoffer et al. (2019) Hoffer, E., Ben-Nun, T., Hubara, I., Giladi, N., Hoefler, T., and Soudry, D.  Augment your batch: better training with larger batches.  _arXiv preprint arXiv:1901.09335_ , 2019.
  * Howard (2019) Howard, J.  Imagenette, 2019.  URL https://github.com/fastai/imagenette.
  * Huang et al. (2017) Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q.  Densely connected convolutional networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 4700–4708, 2017.
  * Ioffe & Szegedy (2015) Ioffe, S. and Szegedy, C.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In _International conference on machine learning_ , pp. 448–456. PMLR, 2015.
  * Klambauer et al. (2017) Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S.  Self-normalizing neural networks.  _Advances in neural information processing systems_ , 30, 2017.
  * Klause et al. (2022) Klause, H., Ziller, A., Rueckert, D., Hammernik, K., and Kaissis, G.  Differentially private training of residual networks with scale normalisation.  _arXiv preprint arXiv:2203.00324_ , 2022.
  * Krizhevsky et al. (2009) Krizhevsky, A., Hinton, G., et al.  Learning multiple layers of features from tiny images.  2009\.
  * Li et al. (2018) Li, H., Xu, Z., Taylor, G., Studer, C., and Goldstein, T.  Visualizing the loss landscape of neural nets, 2018.
  * Morsbach et al. (2021) Morsbach, F., Dehling, T., and Sunyaev, A.  Architecture matters: Investigating the influence of differential privacy on neural network design.  _arXiv preprint arXiv:2111.14924_ , 2021.
  * Nair & Hinton (2010) Nair, V. and Hinton, G. E.  Rectified linear units improve restricted boltzmann machines.  In _Icml_ , 2010.
  * Papernot et al. (2019) Papernot, N., Chien, S., Song, S., Thakurta, A., and Erlingsson, U.  Making the shoe fit: Architectures, initializations, and tuning for learning with privacy.  2019\.
  * Papernot et al. (2020) Papernot, N., Thakurta, A., Song, S., Chien, S., and Erlingsson, U.  Tempered sigmoid activations for deep learning with differential privacy.  _arXiv preprint arXiv:2007.14191_ , pp. 10, 2020.
  * Polyak & Juditsky (1992) Polyak, B. T. and Juditsky, A. B.  Acceleration of stochastic approximation by averaging.  _SIAM journal on control and optimization_ , 30(4):838–855, 1992.
  * Qiao et al. (2019) Qiao, S., Wang, H., Liu, C., Shen, W., and Yuille, A.  Micro-batch training with batch-channel normalization and weight standardization.  _arXiv preprint arXiv:1903.10520_ , 2019.
  * Santurkar et al. (2019) Santurkar, S., Tsipras, D., Ilyas, A., and Madry, A.  How does batch normalization help optimization?, 2019.
  * Tan & Le (2019) Tan, M. and Le, Q.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In _International conference on machine learning_ , pp. 6105–6114. PMLR, 2019.
  * Touvron et al. (2021) Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H.  Training data-efficient image transformers & distillation through attention.  In _International Conference on Machine Learning_ , pp. 10347–10357. PMLR, 2021.
  * Ulyanov et al. (2016) Ulyanov, D., Vedaldi, A., and Lempitsky, V.  Instance normalization: The missing ingredient for fast stylization.  _arXiv preprint arXiv:1607.08022_ , 2016.
  * Wu & He (2018) Wu, Y. and He, K.  Group normalization.  In _Proceedings of the European conference on computer vision (ECCV)_ , pp. 3–19, 2018.
  * Yousefpour et al. (2021) Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj, A., Zhao, J., Cormode, G., and Mironov, I.  Opacus: User-friendly differential privacy library in pytorch, 2021.  URL https://arxiv.org/abs/2109.12298.
  * Ziller et al. (2021a) Ziller, A., Usynin, D., Braren, R., Makowski, M., Rueckert, D., and Kaissis, G.  Medical imaging deep learning with differential privacy.  _Scientific Reports_ , 11(1):1–8, 2021a.
  * Ziller et al. (2021b) Ziller, A., Usynin, D., Remerscheid, N., Knolle, M., Makowski, M., Braren, R., Rueckert, D., and Kaissis, G.  Differentially private federated deep learning for multi-site medical image segmentation, 2021b.  URL https://arxiv.org/abs/2107.02586.


# Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence

Chris Callison-Burch
University of Pennsylvania
<EMAIL_ADDRESS>
Gaurav Singh Tomar∗
Google Research
<EMAIL_ADDRESS>
Lara J. Martin
University of Pennsylvania
Daphne Ippolito
University of Pennsylvania
Google Research
Suma Bailis
Google Research
David Reitter
Google Research *Denotes equal contribution

###### Abstract

AI researchers have posited Dungeons and Dragons (D&D) as a challenge problem
to test systems on various language-related capabilities. In this paper, we
frame D&D specifically as a dialogue system challenge, where the tasks are to
both generate the next conversational turn in the game and predict the state
of the game given the dialogue history. We create a gameplay dataset
consisting of nearly 900 games, with a total of 7,000 players, 800,000
dialogue turns, 500,000 dice rolls, and 58 million words. We automatically
annotate the data with partial state information about the game play. We train
a large language model (LM) to generate the next game turn, conditioning it on
different information. The LM can respond as a particular character or as the
player who runs the game—i.e., the Dungeon Master (DM). It is trained to
produce dialogue that is either in-character (roleplaying in the fictional
world) or out-of-character (discussing rules or strategy). We perform a human
evaluation to determine what factors make the generated output plausible and
interesting. We further perform an automatic evaluation to determine how well
the model can predict the game state given the history and examine how well
tracking the game state improves its ability to produce plausible
conversational output.

##  1 Introduction

Artificial Intelligence has a long and rich history of using games as
challenge problems that lead to advances in the field. In many cases, AI game-
playing systems have gone on to rival human champions of the game. Dungeons
and Dragons has been identified as an appropriate challenge for the next stage
of artificial intelligence Ellis and Hendler (2017); Louis and Sutton (2018);
Martin et al. (2018b). Ellis and Hendler (2017) proposed open-ended creative
games like D&D as the next challenge for AI after the human-level successes of
AI at Chess and Go, which are zero-sum, deterministic, sequential two-player
games with perfect information. Louis and Sutton (2018) understood the
importance of narrative in natural language processing (NLP) and generation
(NLG). In particular, they saw how cooperative story generation between humans
already exists in these games and can be used for automated generation. Martin
et al. (2018b) outlined some of the specific challenges D&D presents to the
NLP community; such as a state of the game world distributed across the
Dungeon Master (DM) and other players or dealing with the intrinsic rewards
players get from taking certain actions that would not necessarily provide
them with points in the game.

D&D involves multiple players who roleplay characters in a fantasy setting,
guided by a Dungeon Master who sets obstacles and adventures and plays as
monsters. In roleplaying games like Dungeons and Dragons, the gameplay happens
through language rather than moves on a game board, making it an interesting
domain for NLP research. To have an AI successfully play D&D , it would
require abilities like

  * •

Language generation (multi-party dialog, generating descriptions of the
world/actions, storytelling)

  * •

Language understanding (knowledge acquisition and representation, state
tracking, automated reasoning)

  * •

Planning / strategic play during battles (similar to chess or go)

Appendix A gives an example of D&D gameplay and the AI challenges presented by
it.

Is it possible to design an AI system that is capable of playing a game of D&D
either as a character in the game or as the Dungeon Master using current AI
technology? We argue that now is the perfect time for this challenge, since
large scale neural language models like GPT have shown impressive generation
results (Brown et al., 2020), and since incorporating neural LMs into a game
setting both exercises their strengths and exposes their weaknesses.

In this paper, we introduce a new dataset of “actual play” game transcripts.
Each turn is labeled with game state variables like character information and
whether the conversational turn was in-character or out-of-character. Our data
is a novel, large scale, real-world conversational dataset. It is unique in
that the dialog turns are generated entirely through player collaboration and
written interaction in a multi-player game. We propose our dataset as a
challenge for dialogue systems for the following reasons:

  * •

It is naturally occurring conversational dialog that covers a spectrum of task
oriented and non-task oriented (e.g. chit chat) dialog.

  * •

It is strongly history dependent – a substantive criticism of recent dialog
datasets is their history independence (Mosig et al., 2020).

  * •

It has many participants in the conversation, since there are several players
in the game.

  * •

It conveys narrative elements including descriptions of events that denote
changes in the state of the game.

Unlike existing dialog datasets, our data reflects the challenging nature of
the D&D game as a multi-party dialogue with creative roleplaying and
underlying game states.

##  2 Tasks

We trained a large language model (LLM) to perform two tasks: Next Utterance
Prediction and Game State Tracking.

#### Next Utterance Prediction.

We trained our language model on a corpus of human conversations (see Section
3) to predict the next utterance. We varied the conditioning information to
examine the effects on the quality of predicted next utterance. In all
variations, we included the conversational history as input. Given the
conversational input (and other input in the variant models), the LLM must
generate the next utterance, such that it is both interesting and a plausible
next turn in the D&D game.

#### Game State Tracking.

In this task, rather than producing the next utterance, we had the model
predict the game state for a given dialogue turn in the conversation. We have
kept the state definition similar to task-oriented dialogue state tracking
(DST). In DST, the dialogue state is a collection of slot-value pairs. In our
case, each slot is a state variable feature related to D&D games. Our target
slot values do not need to appear as a word in the dialogue context. We track
several game states aspects including some that remain relatively static
throughout the game (character attributes like their pronouns, class, fantasy
race, and their inventory), some that change periodically (like being in
combat or out of combat), and some that change from turn to turn (like what
action the player is taking).

##  3 Dataset

For this paper, we have created a novel dataset for our dialogue-oriented test
of AI’s ability to play Dungeons & Dragons. We scraped Play-By-Post data from
a web forum111https://www.dndbeyond.com/forums/d-d-beyond-general/play-by-post
where people play by taking turns posting on the forum to describe their move.

Figure 1: Example of 3 turns in the D&D Beyond play-by-post forum

Figure 1 shows an example of part of the gameplay from the play-by-post forums
from D&D Beyond. D&D Beyond provides a mechanism in its forum to roll dice
using a “roll” tag. Their dice roller allows players to conduct the rolls that
are used for D&D ability checks and in combat.

Table 1 summarizes the amount of play-by-post data that we collected from the
D&D Beyond website (with permission from the company).

Play-By-Post Corpus
---
Number of campaigns | 896
Average players per campaign | 8
Average turns per campaign | 910
Average words per campaign | 64,941
Total turns | 815,106
Total words | 58,187,526
Average dice rolls per campaign | 594
Total dice rolls | 532,270
Table 1: Statistics for our play-by-post corpus

###  3.1 Heuristic annotation of game states

We designed a set of rule-based heuristics to extract game state information
from the play by post. These were implemented using regular expressions and
NLP tools like named entity recognizers Gardner et al. (2018). Although this
heuristically extracted information is not perfect, it provides a reasonable
approximation of the game state. It is useful for testing whether large
language models can benefit from inclusion of complex state information for
next utterance prediction and whether LLMs can be used for state tracking. We
designed rules to extract state information relating to character properties,
combat and player actions.

Control Feature |  Description |  Expected Impact on Model’s Output
---|---|---
Player ID |  Player writing a given dialog turn |  Connects the current turn to the player’s previous turns, which is important in multi-party conversations.
IC versus OOC |  Whether a player is in-character or out-of-character for a given dialog turn |  Changes whether the generated text is more like descriptive text found in a novel, or more like a discussion of rules and strategies.
Character Name |  Name of the character being played by the player of a given dialog turn |  IC descriptions use the character’s name.
Character Class |  D&D classes |  Character classes perform different actions (e.g. wizards cast spells, thieves pick locks)
Character Race |  D&D fantasy races |  Different physical characteristics (e.g. halflings are small, dragonborn have scales).
Character Pronouns |  The character’s pronouns |  Uses the correct pronouns when describing the character.
Character Actions |  List of actions taken by the character in the current turn |  Allows a description to be generated for a given action. The action can be thought of as a goal for the description.
Combat |  Whether the players are currently engaged in combat or not during a given dialog turn |  Affects the likelihood of actions (e.g. attacks are more likely during combat and investigations checks are more likely outside of combat)
Table 2: Our LLMs are conditioned on a variety of control features that allow
the models to better learn what kind of text to generate for the next
utterance prediction task

#### Character properties

  * •

Name: Perform NER on all the player’s turns in a campaign. The character’s
name is assigned to be the player’s most frequently mentioned name, on the
assumption that they tend to describe their own character’s actions.

  * •

Class: Count how many times each D&D class222https://www.dndbeyond.com/classes
is mentioned by each player. Most frequently mentioned class is their
character’s class.

  * •

Race: On a player’s first turn, check whether any of the D&D fantasy
races333https://www.dndbeyond.com/races are mentioned. Assign it to character.
If not, guess based on the most frequently mentioned race.

  * •

Pronouns: Count pronouns mentioned by a player. Assign their character’s
pronouns to be the most frequent pronouns used by the player.

  * •

Inventory: Use regex to match items occurring after character’s personal
pronouns (e.g. her sword).

  * •

Spells known: Regex that matches cast followed by a spell name

The DM is assumed to be the player who has the first post in the game. The
DM’s entries in the dataset are scrubbed of other character properties, since
they play multiple NPCs (non-player characters) and monsters.

#### Combat

  * •

We detect the start of combat when there is a roll for initiative, or when
there are attack rolls before initiative (from surprise attacks).

  * •

Combat continues while there are attack rolls happening.

  * •

Combat concludes after there are no rolls for a number of turns.

  * •

In a combat span, we extract a list of monsters mentioned, and heuristically
guess the number of each kind of monster.

#### Actions

  * •

Dice rolls are marked in D&D Beyond posts. We detect the associated actions
based on the kind of die used (D20 = a check, other dice are used for
calculating damage if an attack check is successful)

  * •

We use a regex to match the nearest pattern, which includes attack or a list
of abilities like acrobatics, animal handling, arcana, athletics, etc.

  * •

Damage rolls are matched with damage, dmg, cure, heal, healing, points.

Our heuristics resulted in features for around 60% of all conversational
turns. We train a convolutional neural network classifier using these
conversational turns to predict all of the above control features for each
conversational turn in training data. Appendix C estimates the accuracy of the
model’s prediction on these state features.

###  3.2 In-Character Versus Out-Of-Character Text

In addition to labeling the game states in our Play by Post data, we also
labeled the text of each turn as being either spoken in-character (IC) or out-
of-character (OOC). To do so, we crawled another Play by Post forum hosted at
Giant in the Playground444https://forums.giantitp.com/forumdisplay.php?3-Play-
by-Post-Games, where play happens on two discussion boards – one in-character
and one out-of-characters. For example, here is an IC post:

> Kuros pulls the feathered shaft of the arrow back to his cheek winning
> easily against the resistance of the bowstring. He pulls a lungful of air to
> keep himself steady, takes aim at the Bandit with the deer, and lets fly.

And here is its corresponding OOC post:

> Surprise round so only 1 standard or move action. Shoot the bow:
> (1d20+6)[20] vs Flat Footed AC at Bandit 1. Damage: (1d8+2)[10]

We train a classifier to predict IC versus OOC text, and then apply it to each
paragraph in our D&D Beyond forum data.

##  4 Models

For our large language model, we use a 64B parameter version of Google’s LaMDA
language model Thoppilan et al. (2022), which was trained on conversations.
LaMDA is similar to other Transformer-based pre-trained language models like
GPT-3. As with other pre-trained language models Howard and Ruder (2018),
LaMDA can be finetuned to different tasks. The two tasks that we finetune
LaMDA to perform are game state tracking and response generation. In both
cases, the LLM can be thought of as a function that maps inputs onto an
output. For instance, game state tracking is a language understanding task
where the function takes in inputs like $f($current utterance, previous state,
history$)\rightarrow$ new state, and response generation is a language
generation task where $f($current state, history$)\rightarrow$ next utterance.
The LLM functions are trained via the fine tuning process.

In our experiments we try a variety of different inputs to our LLM functions
to see how they enable better learning of the tasks. We train our LLMs on the
conversation history (which is typical in dialog modeling) and we also augment
the conversations by conditioning other explicit signals. These conditioning
signals can be thought of as sophisticated “control features”, inspired by the
CTRL language model Keskar et al. (2019). During training, the model learns a
relationship between the control features and appropriate responses. In turn,
during inference, one can explicitly influence dimensions of the conversation
– enabling more compelling dialogue – by setting the values of control
features. These control features can be set dynamically, without necessitating
finetuning or additional post-processing. Table 2 describes the control
features we have proposed and describes how they could steer generation. Note
that we use the terms ‘control features’ and ‘state variables’ interchangeably
when referring to our next utterance prediction models.

###  4.1 Baseline Pre-Training Data

LaMDA is trained on turn-based conversational data. For a conversation of
length $n$, LaMDA takes as input the first $n-1$ turns, and the $n$th turn as
the target. For all models, we used the 7 most recent conversational turns as
input, and predict turn 8.

###  4.2 D&D FineTuning Data

Here is an example of the data used in our versions of LaMDA that are
finetuned to on our D&D data.

TURN 1: Text You attack. You launch some fire onto the goblin closest to the
wagon. And with that, he looks like he is on death’s door. And the other
goblin that you can see, the one that’s not in the brush somewhere, just sort
of stops in his tracks. What do you do next? Player ID 0 Character Dungeon
Master Race N/A Class Dungeon Master Pronouns N/A Inventory N/A In combat? Yes
In character? Yes Action Attack

TURN 2: |
---|---
Text |  I grab my axe and bring it down on the wounded goblin.
Player ID |  1
Character |  Magnus
Race |  Human
Class |  Fighter
Pronouns |  he/him
Inventory |  Axe
In combat? |  Yes
In character? |  Yes
Action |  Attack

###  4.3 Next Utterance Prediction Models

#### LLM-Dialog:

We call our baseline model LLM-Dialog. It is a LaMDA dialogue model that does
not use not use any D&D data.

#### LLM-DND:

LLM-Dialog that has been finetuned on Play-by-post D&D gameplay dataset using
no control features

#### LLM-DND-ALL-CTRL:

LLM-Dialog that has been finetuned on Play-by-post D&D gameplay dataset using
control features (state variables) for all dialog turns including the state
variables for the current turn the utterance is being predicted for.

#### LLM-DND-PREV-CTRL:

LLM-Dialog that has been finetuned on Play-by-post D&D gameplay dataset using
control features for all previous dialog turns, not including the current
turn.

#### LLM-DND-CURRENT-CTRL:

LLM-Dialog that has been finetuned on Play-by-post D&D gameplay dataset using
control features (state variables) for only the current turn the utterance is
being predicted for.

###  4.4 Dev Set Perplexity During Training

Each of our models starts from a pretrained LaMDA model trained for 600K steps
and then is finetuned for a further 60K steps. Figure 2 plots the Negative log
perplexity on our development set, and Table 3 shows the final perpexity and
token accuracies on the dev set. At the end of finetuning, the models with the
best perplexity scores and the best token accuracy scores were LLM-DND-
CURRENT-CTRL and LLM-DND-ALL-CTRL, which used our control features.

Figure 2: Negative Log Perplexity of our models after pretraining on generic dialogue data for 600k steps, and then finetuning to our data for a further 60k steps. Colors correspond to the models in Table 3.  Model | Perplexity | Token Accuracy
---|---|---
LLM-Dialog | 2.65 | 44.61
LLM-DND | 2.50 | 46.92
LLM-DND-PREV-CTRL | 2.51 | 46.84
LLM-DND-CURR-CTRL | 2.34 | 49.67
LLM-DND-ALL-CTRL | 2.37 | 49.02
Table 3: Perplexity and token accuracy of our models after finetuning to our
data.

##  5 Manual Evaluation

To evaluate the quality of our models for the task of next utterance
prediction in D&D , we perform a human evaluation. We recruited professional
raters to perform a manual evaluation. They read a version of the content that
was provided to the models – the seven turns of conversational history plus a
list of players and the names/classes of the characters that they played. Then
they were shown several model outputs for the context (or the “gold”, which
was the actual next turn in the game), The annotators asked to rate each
output along the three dimensions, following the evaluation procedure used for
the Meena LM Adiwardana et al. (2020):

  * •

Does the response make sense? (yes/no)

  * •

Is the response specific? (yes/no)

  * •

How interesting is the response? (10 point scale)

The full annotator instructions and the annotation interface are given in
Appendix D.

###  5.1 Raters

Because of the specialized nature of the D&D domain, we recruited 6
professional raters rather than crowd workers to perform the task. The raters
were selected based on their professed interest in the fantasy genre, and on
their background with D&D. All raters were fantasy fans, and 5 of the 6 had
played D&D. 3 raters had been the DM in a game before.

###  5.2 Inter-Rater Agreement

Our raters annotated 500 system outputs with 3-way redundancy on each output.
For the binary sense and specific scores, pairwise annotator agreement was
0.8, with a chance-adjusted Randolph Kappa score of 0.6. For the scalar
interestingness scores, the Kendall’s Tau correlation was 0.46.

Model | Sense | Specific | Interest
---|---|---|---
LLM-Dialog | 0.81 | 0.85 | 3.57
LLM-DND | 0.9 | 0.9 | 3.91
LLM-DND-PREV-CTRL | 0.86 | 0.88 | 3.96
LLM-DND-CURR-CTRL | 0.88 | 0.9 | 3.96
LLM-DND-ALL-CTRL | 0.87 | 0.88 | 3.92
GOLD | 0.92 | 0.92 | 4.17
Table 4: Average sense, specific and interestingness scores for our systems,
and for human-written gold responses

###  5.3 Analysis

#### Model Comparison.

Table 4 shows the average sense, specific and interestingness scores for the
systems, and for the human-written gold response. All of the D&D adapted
systems outperform the vanilla dialogue system. On average, the adapted
systems make sense 6.75% more often than the baseline, are specific 4% more
often, and are 0.37 points more interesting. However, the added control
features do not seem to differ substantially from the LLM that is adapted to
the D&D data without any control features. Why then use the control features
at all?

Model |  Output
---|---
LLM-Dialog |  moon$\\_$monk you can just join our discord conversation. it’s free and easier to deal with
LLM-DND |  I cast Ice Knife (I’m assuming this is not one of the ones I can’t do?)
LLM-DND-CURRENT-CTRL
Responding as Magnus (human fighter, inventory: axe) In combat |  I turn from the goblin and go to help Merle and Taako with getting this wagon moving again. Magnus can’t see them anymore with such a dense forest. They’re likely in a battle right now… I have to find them as fast as I can!
LLM-DND-CURRENT-CTRL
Responding as Taako (elf wizard, inventory: spellbook) In combat |  If the goblins still don’t look like they’re getting closer, Taako will yell, “Alright you little monsters! Let’s see what ya got!” And he’ll cast Chill Touch on one of the goblins that isn’t dead/on its doorstep. Attack: 22 Damage: 6
LLM-DND-CURRENT-CTRL
Responding as Mearle (dwarf cleric, inventory: holy symbol) In combat |  Merle steps away from the wagon. He draws his sword and takes aim at the hidden goblin! Attack: 27 Damage: 8 He puts his free hand on the pommel of his sword and shouts his battle cry, "For Clanggeddin!" His eyes glow brightly red as he strikes!
Table 5: Example model outputs for the next utterance prediction task, given
the sample gameplay from Table 8 as input. The 3 LLM-DND-CURRENT-CTRL examples
use the same model with different control features (in red).

#### Control Features.

One reason is that the control features are useful to allow the model to role-
play. For example, we can ask the model to play an elf wizard via the control
features. Table 5 gives a qualitative example of this. The Table shows outputs
from two LLMs without control features, plus 3 outputs from our LLM-DND-
CURRENT-CTRL model roleplaying different characters. As input, we gave the
models the final seven turns in the game play example from Appendix A –from
“With his eagle eyes, Magnus spots two dead horses…” to “… What do you do
next?”, and then generate what each model thinks the next utterance ought to
be. The vanilla dialogue system with no D&D data generates an irrelevant reply
about Discord. The dialogue system fine-tuned on D&D data without control
feature generates a relevant response. The dialgoue system that adds
conditioning control features allows the model to generate specific replies
for each of the different characters. These replies are relevant to the
character classes – the wizard casts a spell, and the Dwarf cleric shouts a
battle cry by invoking the name of a Dwarf god.

Model | Sense | Specific | Interest
---|---|---|---
LLM-Dialog | -0.01 | -0.01 | +0.06
LLM-DND | -0.02 | +0.03 | +0.4
LLM-DND-PREV-CTRL | +0.02 | +0.02 | +0.6
LLM-DND-CURR-CTRL | +0.06 | +0.06 | +0.93
LLM-DND-ALL-CTRL | +0.07 | +0.06 | +0.81
GOLD | +0.07 | +0.05 | +1.02
Table 6: Improvements by generating in-character (IC) text rather than out-of-character (OOC) text. Numbers are IC scores minus OOC scores.  State variable | Majority | LLM-DND-GST
---|---|---
All | .73 | 82
Combat | .89 | .82
Character Class | .58 | .76
Character Name | .58 | .78
Character Race | .75 | .79
Character Pronouns | .58 | .89
Character Actions | .80 | .85
Table 7: Average accuracy for our Game State Tracking LLM on the slot-filling
our state variables, compared to a majority class baseline.

#### In-Character Turns Are More Interesting.

Among our most impactful control features was the one that allowed systems to
generate in-character (IC) versus out-of-character (OOC) turns. Table 6 shows
that control models’ scores substantially increased on IC turns compared to
when their output was generated OOC. The pronounced increase in
intersestingness makes sense because IC turns are ones where the players
describe their characters in the fictional world often with evocative
language, whereas OOC turns usually discuss rules or mechanics. Our control
features allowed the system to intentionally generate IC responses, resulting
in substantially improved interestingness scores for those in-character turns.

##  6 Game State Tracking Model

We conducted an experiment to evaluate whether a LLM could be finetuned to
perform game state tracking for D&D using our heuristically annotated game
state features. We trained a new model LLM-DND-GST (Game State Tracking). It
is a LLM-Dialog that has been finetuned on our Play-by-post D&D gameplay
dataset. As input, it takes all previous dialog turns and their state
variables, plus the text of the current turn, and then it outputs the
corresponding state variables for the current turn.

We analyzed the accuracy of the LLM-DND-GST model its ability to do slot-
filling for each of the individual game states, and compared its performance
to a simple baseline that always output the the majority class. The results
are shown in Table 7. The average accuracy of the dialogue state tracker is
better than the majority class baseline, but likely falls short of being
useful when it comes to joint accuracy. The joint accuracy for LLM-DND-GST is
58%. This suggests that accurately tracking the full game state may require
additional machinery beyond a finetuned LLM.

##  7 Related Work

Previous work has examined AI to play text adventures games Haroush et al.
(2018); Yao et al. (2020); Dambekodi et al. (2020). These games are simpler
than D&D because they have a limited vocabulary and more straightforward game
states. Creating text adventure games Ammanabrolu et al. (2020a); Fan et al.
(2020) is more challenging than playing them, and is similar to the world-
building job of the DM in D&D . There has also been work on persona/character
generation in stories Prabhumoye et al. (2019), and within D&D itself Louis
and Sutton (2018). Others Urbanek et al. (2019); Ammanabrolu et al. (2020b)
have realized that NPCs are lacking in their abilities to speak and act in
text games.

Findings of the automated story generation community are relevant for D&D AI
systems. Neural language models have become increasingly more popular for
story generation Roemmele (2018); Martin et al. (2018a); Mathewson et al.
(2019); Hou et al. (2019). We have also started to see storytelling with
transformers See et al. (2019); Peng et al. (2021); Branch et al. (2021).
Transformer-based storytelling systems have even been introduced to the
general public thanks to the popularity of AI Dungeon Walton (2019). Although
neural networks possess a lot of power in terms of what text they generate,
they are still limited in their ability to produce longer spans of coherent
text. Many Fan et al. (2018); Yao et al. (2019); Ippolito et al. (2019);
Tambwekar et al. (2019); Ammanabrolu et al. (2020b); Rashkin et al. (2020)
have improved the coherence of neural storytellers by splitting the generation
into two steps: ideation of the story plot, followed by the realization of
sentences. This controllable story generation is the focus of a lot of current
work in neural automated story generation.

Due to the conversational nature of D&D , we decided to use a dialog-based
system. Deep neural networks have been used for dialog agents for a while
Serban et al. (2016), with a shift toward using transformers in recent years
Zhang et al. (2019); Ghazarian et al. (2021). Like in automated story
generation and other neural text generation tasks, we are also seeing
controllability being an important factor being integrated into systems. This
includes using deep reinforcement learning techniques to guide the dialog
toward a goal Li et al. (2016); Saleh et al. (2020) or controlling for style
Zhang et al. (2018); Smith et al. (2020).

In this paper, we use LaMDA, a transformer-based open-domain dialogue system
that builds on the Meena model Adiwardana et al. (2020). The original Meena
model was an end-to-end model trained on public conversations found on social
media. Controllable text generation with transformers has been seen before
with CTRL Keskar et al. (2019), a language model that is conditioned on a
given “control code” in addition to the textual history. This work takes a
similar approach. We integrate contextual information such as character
descriptions, actions, and in- and out-of-character classifications.

We have finetuned our LaMDA models on data crawled from D&D Beyond. This data
contains both in-character and out-of-character dialog and can be used in
conjunction with Rameshkumar and Bailey (2020)’s dataset from Critical Role (a
D&D podcast), Louis and Sutton (2018)’s dataset from roleplayerguild.com (a
D&D forum), Urbanek et al. (2019)’s crowdsourced LIGHT dataset, and/or Akoury
et al. (2020)’s STORIUM dataset for human+AI collaborative story generation.

##  8 Discussion and Conclusions

We have demonstrated that training on D&D data results in much higher quality
outputs than a vanilla dialogue system (as expected), that controlling the
model to generate in-character responses results in substantially more
interesting output, and that conditioning on game state information
qualitatively results in responses that are appropriate to the character
class. Our preliminary experiments with using the large language models to
perform game state tracking show low performance even after finetuning,
suggesting that other models may be required for an AI to play D&D track the
full state of the game.

Although our models are unable to play D&D fully autonomously by acting as the
Dungeon Master, they could act as an aid for novice DMs. Since our models can
generated evocative, in-character text that is appropriate for the context and
the game state, DMs could use it as inspiration as they narrate the adventure
to the other players.

Here is some model output to inspire your next adventure:

> You get a much closer look than the other two… the sarcophagi have the
> inscriptions of some sort of magic, probably to keep the dead inside, but
> you can not read them to save your life.

What will you do next? Download our
dataset555https://www.cis.upenn.edu/~ccb/dnd-data.html to start your new
adventure!

##  9 Limitations

One limitation of our human evaluation is that it is a static evaluation. The
raters are simply reading the outputs of the model, and there is no
interactive evaluation wherein they engage in gameplay with the system. An
interactive user-study would be required before any claims could be made about
how well AI is able to play D&D alongside human players.

Because our state information was created heuristically, it therefore
potentially contains errors. It is also incomplete. There are several kinds of
state tracking variables that would be useful to include, but were not
possible to heuristically extract from our data. To address this problem in
the future, we have begun a collaboration with the developer of Avrae, which
is a Discord bot for playing D&D online. Avrae contains many state variables
that are missing from our current annotations, such as HIT points and slot-
filler values for attacks.

##  10 Acknowledgments

We would like to thank Antony Pegg of Fandom for granting us permission to use
D&D Beyond’s forum data for research purposes.

Thank you to Rich Burlew and forum moderator truemane for granting us
permission to crawl the Giant in the Playground forum and to build models
using the forum posts.

We would like to thank the many Google Researchers who gave valuable input on
this project, especially Dipanjan Das and Suma Bailis.

Chris Callison-Burch is grateful for having the opportunity to be a part-time
visiting researcher at Google for two years. It was amazing to spend time
among such incredibly smart people, and it was eye opening to see large LMs
before they became widely available. Thanks!

## References

  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu Quoc, and V Le. 2020.  Towards a Human-like Open-Domain Chatbot.  _arXiv preprint arXiv:2001.09977_.
  * Akoury et al. (2020) Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020.  STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6470–6484, Online. Association for Computational Linguistics.
  * Ammanabrolu et al. (2020a) Prithviraj Ammanabrolu, William Broniec, Alex Mueller, Jeremy Paul, and Mark O. Riedl. 2020a.  Toward Automated Quest Generation in Text-Adventure Games.  In _International Conference on Computational Creativity (ICCC)_.
  * Ammanabrolu et al. (2020b) Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, and Jason Weston. 2020b.  How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds.  In _The North American Chapter of the Association for Computational Linguistics (NAACL-HLT)_.
  * Branch et al. (2021) Boyd Branch, Piotr Mirowski, and Kory Mathewson. 2021.  Collaborative Storytelling with Human Actors and AI Narrators.  In _International Conference on Computational Creativity (ICCC)_ , pages 97–101.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Dambekodi et al. (2020) Sahith Dambekodi, Spencer Frazier, Prithviraj Ammanabrolu, and Mark O Riedl. 2020\.  Playing Text-Based Games with Common Sense.  In _Wordplay: When Language Meets Games Workshop (NeurIPS 2020)_.
  * Ellis and Hendler (2017) Simon Ellis and James Hendler. 2017.  Computers Play Chess, Computers Play Go…Humans Play Dungeons & Dragons.  _IEEE Intelligent Systems_ , 32(4):31–34.
  * Fan et al. (2018) Angela Fan, Mike Lewis, and Yann Dauphin. 2018.  Hierarchical Neural Story Generation.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics_ , pages 889–898.
  * Fan et al. (2020) Angela Fan, Jack Urbanek, Pratik Ringshia, Emily Dinan, Emma Qian, Siddharth Karamcheti, Shrimai Prabhumoye, Douwe Kiela, Tim Rocktäschel, Arthur Szlam, and Jason Weston. 2020.  Generating Interactive Worlds with Text.  In _AAAI Conference on Artificial Intelligence (AAAI)_ , pages 1693–1700.
  * Gardner et al. (2018) Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018.  AllenNLP: A deep semantic natural language processing platform.  _arXiv preprint arXiv:1803.07640_.
  * Ghazarian et al. (2021) Sarik Ghazarian, Zixi Liu, Tuhin Chakrabarty, Xuezhe Ma, Aram Galstyan, and Nanyun Peng. 2021.  DiSCoL: Toward Engaging Dialogue Systems through Conversational Line Guided Response Generation.  _arXiv preprint arXiv:2102.02191_.
  * Haroush et al. (2018) Matan Haroush, Tom Zahavy, Daniel J. Mankowitz, and Shie Mannor. 2018.  Learning How Not to Act in Text-Based Games.  In _Workshop Track at ICLR 2018_ , pages 1–4.
  * Hou et al. (2019) Chenglong Hou, Chensong Zhou, Kun Zhou, Jinan Sun, and Sisi Xuanyuan. 2019.  A survey of deep learning applied to story generation.  In _Smart Computing and Communication_ , pages 1–10. Springer International Publishing.
  * Howard and Ruder (2018) Jeremy Howard and Sebastian Ruder. 2018.  Universal language model fine-tuning for text classification.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 328–339, Melbourne, Australia. Association for Computational Linguistics.
  * Ippolito et al. (2019) Daphne Ippolito, David Grangier, Chris Callison-Burch, and Douglas Eck. 2019.  Unsupervised Hierarchical Story Infilling.  In _Proceedings of the First Workshop on Narrative Understanding_ , pages 37–43.
  * Keskar et al. (2019) Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019.  CTRL: A Conditional Transformer Language Model for Controllable Generation.  _arXiv preprint arXiv:1909.05858_.
  * Li et al. (2016) Jiwei Li, Will Monroe, Alan Ritter, and Dan Jurafsky. 2016.  Deep Reinforcement Learning for Dialogue Generation.  In _Conference on Empirical Methods in Natural Language Processing_ , pages 1192–1202.
  * Louis and Sutton (2018) Annie Louis and Charles Sutton. 2018.  Deep Dungeons and Dragons: Learning Character-Action Interactions from Role-Playing Game Transcripts.  In _Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)_ , pages 708–713.
  * Martin et al. (2018a) Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O. Riedl. 2018a.  Event Representations for Automated Story Generation with Deep Neural Nets.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , pages 868–875, New Orleans, Louisiana.
  * Martin et al. (2018b) Lara J. Martin, Srijan Sood, and Mark O. Riedl. 2018b.  Dungeons and DQNs: Toward Reinforcement Learning Agents that Play Tabletop Roleplaying Games.  In _Joint Workshop on Intelligent Narrative Technologies and Workshop on Intelligent Cinematography and Editing (INT-WICED)_ , Edmonton, AB, Canada.
  * Mathewson et al. (2019) Kory W. Mathewson, Pablo Samuel Castro, Colin Cherry, George Foster, and Marc G. Bellemare. 2019.  Shaping the Narrative Arc: An Information-Theoretic Approach to Collaborative Dialogue.  _arXiv preprint arXiv:1901.11528_ , page 20.
  * Mosig et al. (2020) Johannes E. M. Mosig, Vladimir Vlasov, and Alan Nichol. 2020.  Where is the context? - A critique of recent dialogue datasets.  _CoRR_ , abs/2004.10473.
  * Peng et al. (2021) Xiangyu Peng, Siyan Li, Sarah Wiegreffe, and Mark Riedl. 2021.  Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning.  In _Workshop on Narrative Understanding at NAACL-HLT 2021_.
  * Prabhumoye et al. (2019) Shrimai Prabhumoye, Khyathi Raghavi Chandu, Ruslan Salakhutdinov, and Alan W Black. 2019.  "My Way of Telling a Story": Persona based Grounded Story Generation.  In _Storytelling Workshop at ACL 2019_ , pages 11–21.
  * Rameshkumar and Bailey (2020) Revanth Rameshkumar and Peter Bailey. 2020.  Storytelling with Dialogue: A Critical Role Dungeons and Dragons Dataset.  In _Annual Meeting of the Association for Computational Linguistics (ACL)_ , pages 5121–5134.
  * Rashkin et al. (2020) Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, and Jianfeng Gao. 2020.  PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking.  In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4274–4295.
  * Roemmele (2018) Melissa Roemmele. 2018.  _Neural Networks for Narrative Continuation_.  Ph.D. thesis, University of Southern California.
  * Saleh et al. (2020) Abdelrhman Saleh, Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, and Rosalind Picard. 2020.  Hierarchical Reinforcement Learning for Open-Domain Dialog.  In _AAAI Conference on Artificial Intelligence (AAAI)_ , pages 8741–8748.
  * See et al. (2019) Abigail See, Aneesh Pappu, Rohun Saxena, Akhila Yerukola, and Christopher D. Manning. 2019.  Do Massively Pretrained Language Models Make Better Storytellers? In _Conference on Computational Natural Language Learning (CoNLL)_ , pages 843–861.
  * Serban et al. (2016) Iulian Vlad Serban, Ryan Lowe, Laurent Charlin, and Joelle Pineau. 2016.  Generative Deep Neural Networks for Dialogue: A Short Review.  In _Conference on Neural Information Processing Systems (NeurIPS), Workshop on Learning Methods for Dialogue_ , pages 1–6.
  * Smith et al. (2020) Eric Michael Smith, Diana Gonzalez-Rico, Emily Dinan, and Y-Lan Boureau. 2020.  Controlling Style in Generated Dialogue.  _arXiv preprint arXiv:2009.10855_.
  * Tambwekar et al. (2019) Pradyumna Tambwekar, Murtaza Dhuliawala, Lara J. Martin, Animesh Mehta, Brent Harrison, and Mark O. Riedl. 2019.  Controllable Neural Story Plot Generation via Reinforcement Learning.  In _International Joint Conference on Artificial Intelligence (IJCAI)_ , Macau, China.
  * Thoppilan et al. (2022) Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Kathleen S. Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed H. Chi, and Quoc Le. 2022.  LaMDA: Language Models for Dialog Applications.  _CoRR_ , abs/2201.08239.
  * Urbanek et al. (2019) Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Jason Weston. 2019.  Learning to Speak and Act in a Fantasy Text Adventure Game.  In _Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 673–683, Hong Kong, China. Association for Computational Linguistics.
  * Walton (2019) Nick Walton. 2019.  AI Dungeon.
  * Yao et al. (2019) Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019.  Plan-And-Write: Towards Better Automatic Storytelling.  In _AAAI Conference on Artificial Intelligence (AAAI)_ , volume 33(01), pages 7378–7385.
  * Yao et al. (2020) Shunyu Yao, Rohan Rao, Matthew Hausknecht, and Karthik Narasimhan. 2020.  Keep CALM and Explore: Language Models for Action Generation in Text-based Games.  In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 8736–8754.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018.  Personalizing Dialogue Agents: I have a dog, do you have pets too? In _Annual Meeting of the Association for Computational Linguistics (ACL)_ , pages 2204–2213, Melbourne, Australia.
  * Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019.  DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation.  _arXiv preprint arXiv:1911.00536_.

Player
(character) |  Game Dialogue |  D&D Game Description and
AI challenges
---|---|---
Griffin (DM) |  A dwarf named Gundren Rockseeker has hired you to transport a wagonload of provisions to the rough-and-tumble settlement of Phandalin, which is a couple days’ travel to the southeast. A day and a half after leaving, you turn off the high road that connects the major cities on the coast onto a smaller trail that will lead you to Phandalin. This trail is not as well maintained, and bandits and outlaws have been known to lurk along the trail. |  This game is based on the D&D starter adventure called “Lost Mine of Phadelver”. The adventure book is a mixture of rules and “boxed text” which is descriptive text for the DM to read aloud or paraphrase. See the appendix for the text that the DM is consulting. AI challenges: Generation of stories and descriptive text
Griffin (DM) |  Roll a perception check for me. Perception is a wisdom skill, so be sure to add your wisdom modifier. |  The previous text was descriptive text. Here the DM is asking the players to perform a game mechanic and referencing a game rule. This is called “out of character” dialogue.  AI challenges: Knowledge base population (extraction of rules from a rulebook)
Clint (out of character) |  I got an eight. |  Clint has rolled his dice. The number is low so his character fails the check. AI challenges: Multi-party dialogue
Justin (out of character) |  I got a six. |  Justin also fails. Neither character sees the thing that requires the perception check.
Travis (out of character) |  I rolled a natural twenty plus my wisdom modifier is 23. |  Travis rolls high number and succeeds on the check. AI challenges: Understanding rules, determining success or failure
Griffin (DM) |  With his eagle eyes, Magnus spots two dead horses lying in the middle of the road about 200 feet ahead of you. |  The DM describes what happens as a result of the success. AI challenges: Reasoning about consequences of success or failure, descriptive text generation
Travis
(in-character as Magnus) |  I stop the wagon and motion silently to get the attention of Merle and Taako, and kinda pull them up towards the front of the wagon. |  Travis is describing what he is doing using “in character” language. AI challenges: Persona-based chat
Griffin (DM) |  As you warn them that shit has gone south, you notice a few goblins crouching in a part of the shaded woods off to the side of the road. Two of the goblins begin charging your wagon. |  The DM describes the start of a battle with several monsters. AI challenges: State tracking (in combat v. out of combat).
Travis (out of character) |  How many goblins are there? |  AI challenges: Question answering, state tracking (how many monsters).
Griffin (DM) |  There are three goblins; two of them are rushing the group, one is pretty heavily obscured by the brush, probably about 40 feet out, sort of between you and the dead horses laying in the middle of the road. |  AI challenges: Question answering, Descriptive text generation from game state.
Clint (Merle) |  I will cast sacred flame at the nearest one. If it fails a dexterity saving throw, it takes 6 points of damage. |  Clint chooses an action based on what is allowed for his character class. He describes the rule that governs the spell in an out-of-character fashion. AI challenges: Intent detection (perform attack action against a particular goblin)
Griffin (DM) |  You attack. You launch some fire onto the goblin closest to the wagon. And with that, he looks like he is on death’s door. And the other goblin that you can see, the one that‘s not in the brush somewhere, just sort of stops in his tracks. What do you do next? |  The DM rolls for the monster, updates the state of its health meter, and describes the result of Merle’s attack. AI challenges: Reasoning about rules, state tracking (monster’s HIT points), descriptive text generation.
Table 8: Example dialogue from a game of D&D with explanations of what is
happening and comments on potential challenges for AI

##  Appendix A Example D&D Game Session

Instead of the game being a series of moves on a game board, RPGs D&D are
language-based. Players create characters that have a class (wizard, fighter,
thief, etc.) that denotes their abilities, and a fantasy race (elf, dwarf,
human, etc.). Players describe what they want their character to do and roll
dice to determine if they are successful. The dungeon master (DM) acts as the
narrator who shapes the overall story. The DM describes scenarios and
locations, and takes on the role of non-player characters (NPCs), and
monsters.

A common element to the game play is an encounter with monsters. Battles are
governed by rules, and unfold in a turn-based fashion where the DM controls
the monsters and each player controls their character. Each player and monster
has a health meter (called their HIT points), an armor class (which indicates
the threshold of the dice roll needed to damage them), and a set of possible
attack or move actions.

Table 8 provides example dialogue from a game of D&D being played between 3
players – Travis (playing a human fighter named Magnus Burnsides), Clint
(playing Merle Highchurch, a dwarf cleric), Justin (playing Taako an elf
wizard), and DM Griffin. We add comments about each dialogue turn to describe
what is happening in the game, and to highlight the challenges that would need
to be addressed if an AI system were to play the game either as a player or as
the DM.

The game session is taken from the podcast The Adventure Zone. In this
episode, the hosts are playing an adventure module called Lost Mine of
Phadelver, an expert of which is given in Appendix B. In the first episode of
the podcast666https://maximumfun.org/episodes/adventure-zone/ep-1-here-there-
be-gerblins-chapter-one/, the hosts explain the rules of D&D .

##  Appendix B Lost Mine of Phadelver Adventure

Here is an excerpt from the adventure book that the Dungeon Master was using
in our example game play. The adventure book provides boxed text, which is
descriptive text to be read aloud verbatim or to paraphrase. It also gives
details about the combat that is about to ensure, and links to relevant game
rules (like stealth checks, and statistics about the monsters that the
characters will be in combat with).

The adventure begins as the player characters are escorting a wagon full of
provisions and supplies from Neverwinter to Phandalin. The journey takes them
south along the High Road to the Triboar Trail, which heads east (as shown on
the overland map). When they’re a half-day’s march from Phandalin, they run
into trouble with goblin raiders from the Cragmaw tribe.

Read the boxed text when you’re ready to start. If you create a different
adventure hook, skip to the second paragraph and adjust the details as
necessary, ignoring the information about driving the wagon.

> In the city of Neverwinter, a dwarf named Gundren Rockseeker asked you to
> bring a wagonload of provisions to the rough-and-tumble settlement of
> Phandalin, a couple of days’ travel southeast of the city. Gundren was
> clearly excited and more than a little secretive about his reasons for the
> trip, saying only that he and his brothers had found “something big,” and
> that he’d pay you ten gold pieces each for escorting his supplies safely to
> Barthen’s Provisions, a trading post in Phandalin. He then set out ahead of
> you on horse, along with a warrior escort named Sildar Hallwinter, claiming
> he needed to arrive early to “take care of business.”
>
> You’ve spent the last few days following the High Road south from
> Neverwinter, and you’ve just recently veered east along the Triboar Trail.
> You’ve encountered no trouble so far, but this territory can be dangerous.
> Bandits and outlaws have been known to lurk along the trail.
>
> You’ve been on the Triboar Trail for about half a day. As you come around a
> bend, you spot two dead horses sprawled about fifty feet ahead of you,
> blocking the path. Each has several black-feathered arrows sticking out of
> it. The woods press close to the trail here, with a steep embankment and
> dense thickets on either side.

Four goblins are hiding in the woods, two on each side of the road. They wait
until someone approaches the bodies and then attack.

This will likely be the first of many combat encounters in the adventure. Here
are the steps you should follow to run it effectively:

  * •

Review the goblin stat block. Since the goblins are hiding, you’ll need their
Stealth skill modifier: +6.

  * •

Check to see who, if anyone, is surprised. The party can’t surprise the
goblins, but the goblins might surprise some or all of the characters. Make a
Dexterity (Stealth) check for the goblins: roll one d20 for all of them, add
their Stealth skill modifier (+6) to the roll, and compare the total to the
characters’ passive Wisdom (Perception) scores. A character whose score is
lower than the goblins’ check total is surprised and therefore can’t do
anything on his or her first turn in the combat (see “Surprise” in the Basic
Rules).

  * •

Use the initiative rules in the Basic Rules to determine who acts first,
second, third, and so on. Keep track of everyone’s initiative count on a piece
of paper.

  * •

When the time comes for the goblins to act, two of them rush forward and make
melee attacks while two goblins stand 30 feet away from the party and make
ranged attacks. The goblins’ stat block contains the information you need to
resolve these attacks. For more information on what the goblins can do on
their turn, see chapter 9, “Combat,” in the Basic Rules. When three goblins
are defeated, the last goblin attempts to flee, heading for the goblin trail

State variable | Model | Type | Multi-valued | Availability | Evaluation metric | Performance
---|---|---|---|---|---|---
Character | Span labeller | Text | No | 42% | - | -
Race | Classifier | Text | No |  $>$58% | Macro AUC | 0.45
Class | Classifier | Text | No |  $>$75% | Macro AUC | 0.71
Pronous | Classifier | Text | No | 42% | Macro AUC | 0.92
Inventory | Span labeller | Text | Yes | 11% | - | -
In combat? | Classifier | Score | No | 100% | Accuracy | 0.91
Action | Classifier | Text | Yes | 20% | Macro AUC | 0.92
Table 9: The estimated performance of our CNN classifier on predicting state
values for turns where our rule-based heuristics did not predict a value

##  Appendix C Estimated accuracy of predicted state variables

In addition to the heuristics that we used to recover state variables for each
turn in the game (described in Section 3.1), we used a CNN to fill in state
values when our heuristics did not fire. Table 9 estimates gives an estimate
of the CNN’s performance on filling in the state variables where the rule-
based heuristic did not extract a value. The CNN classifier only uses current
post text as input (no additional context).

##  Appendix D Annotation Guidelines and Annotation Interface

###  D.1 Annotation task

In this task, you will see part of a conversation between a few people playing
D&D . The players and their characters are listed at the beginning of the
conversation. The conversations that are shown as context are real
conversations from players. Your job is to read the context and then rate
different responses for a player/character given conversational context.
Please note that the context you are given represents only a part of the
players’ past conversations/interactions with one another during the game.

For each response, you would be asked the following questions.

  * •

Does the response make sense?

    * –

Use your common sense here. Is the response completely reasonable in terms of
the rules of D&D ?

    * –

The response “makes sense” if it is cohesive as a standalone statement,
consistent with the rules of the game, and the elements/entities mentioned are
plausible, given the prior context.

    * –

If anything seems off—not fluent, confusing, illogical, out of context, or
wrong according to the rules of D&D —then rate it as Does not make sense. If
in doubt, choose Does not make sense.

  * •

Is the response specific?

    * –

You may be asked to assess whether the response is specific to a given
context. In other words, do you think that the response represents a good
thing for the character to do now?

    * –

The response is "specific" if it flows logically from the narrative
established by the prior context.

      * *

Note: It is possible for a response to "make sense" (due to being cohesive,
consistent and plausible in and of itself), but be marked "not specific" when
it is not a logical next step in the overall game progression.

      * *

Note: "Specific" for the purposes of this task does not have to do with how
detailed the response is per se; a response can be fairly general in its
language, but still qualify as "specific" when it is a logical next step in
the overall game progression.

  * •

How interesting is the response?

    * –

You may be asked to score the response for its interestingness on a scale of
10.

Choose a high score for “Interesting” if the response would likely catch
someone’s attention or arouse curiosity in the game; or it is insightful,
creative, or witty with respect to the game. If the response is monotonous and
predictable, or if you’re unsure, then it is Less Interesting.

###  D.2 Annotation Interface

A mock up of the annotation user interface is given in Figure 3.

Figure 3: The user interface that our raters used to evaluate the quality of
our model’s next utterance prediction.

###  D.3 Survey of Raters

We recruited raters who had a background in role playing games and an
understanding of the fantasy genre. We surveyed our raters, asking them the
following questions:

  1. 1.

Have you ever played Dungeons and Dragons or another role playing game before?

  2. 2.

If so,

     * •

roughly how many times have you played

     * •

were you a player or a game master or both

  3. 3.

If not,

     * •

what kind of exposure do you have to Dungeons and Dragons? (For example, have
you seen it referred to in TV or movies)

  4. 4.

Are you a fan of the fantasy genre (like Lord of the Rings)?

Our 6 raters responded to the survey as follows: 5 out of the 6 have played
D&D or another role playing game before. All 5 of those who have played D&D
/other role playing games before have played more than 6 times. Of the 5 who
have played D&D /other role playing games before, 3 played as both Game Master
and Player. For the one who had not played D&D /other role playing games, they
indicated they had not had much exposure to D&D through TV or other channels.
All 6 answered that they were fans of the fantasy genre.