Analysis of maps usually involves comparing a baseline scenario, that is, the map as it is under the infuence of external drivers or just its own

**Fig. 6.1** Fuzzy Cognitive Map of deforestation in the Brazilian Amazon. (Source: Kok (2009))

**Fig. 6.2** Fuzzy Cognitive Map of the UK Humber region bio-based economy. (Source: Penn et al. (2013))

structure, with additional scenarios corresponding to changes, such as new external pressures or interventions. To do this, the map is transformed into a matrix corresponding to the links between factors and the edge weights (i.e. a table with the factors as both rows and columns, and the values on the edges that connect them in the cells where they 'cross'). This is the 'weight matrix'. The factors are all given a starting value, and by repeatedly multiplying and updating this list of factor values by the weight matrix, we get an output of the magnitude of the total infuences of all the factors feeding directly into any other factor. Don't worry if this sounds complicated: it is; the FCM software discussed below will do this for you. Importantly, in the causal approach, output factor values are often modifed with a thresholding or squashing function to keep values between 0 and 1 or −1 and 1. This is not required in the dynamical approach.

For the most part, rather than exploring how the actual values of factors or concepts change, FCM analysis explores how changes in a few factors propagate through the system. In the dynamical approach, a baseline scenario is often then produced by setting the initial values of any drivers in the map to 1 or less and the other values to zero. The model output is then calculated, propagating the change through the map using the values of edges and factors. This process of updating factors' values based on other factors' value changes and edge values is iterated until such point that a stable pattern can be seen in the values of factors. That is either values no longer change, or they change in a repeating cycle.

An example of the types of output this process produces can be seen in Fig. 6.3. The top left plot is the output from the map shown in Fig. 6.1; the other three plots are outputs from slightly modifed versions of this map structure intended to represent different policy scenarios. This output is dynamic in the sense that it is based on propagating changes to factor values; however, it is important to remember it does not represent changes through time. Rather, FCM practitioners often refer to changes through 'iterations' of the map. As this is a 'semi-quantitative' method, interpretation involves comparing only the relative ranking of factor values.

**Fig. 6.3** Example outputs from an FCM dynamic analysis. (Source Kok (2009))

There tend to be four types of outputs of the dynamic analysis: (i) the factor values all change and stabilise at new values; (ii) the factor values change and continue in one direction in a 'runaway change'-type scenario; (iii) values change but return to zero; or (iv) values change up and down in cycles. Looking for these types or calibrating the model to produce them can be a useful discussion tool in workshops. Calibration to achieve this involves tweaking edge weights, in particular, modifying the strength of short feedback loops which have a strong impact on the dynamics. Sensitivity testing may be done to determine which links cause the system to destabilise, but often an experienced modeller will have a sense of what sort of structure will stabilise or not and keep this in mind during map construction. This stage is the most technically demanding, and we recommend you practise and ensure familiarity with it before building maps you plan to use in your research.

In the causal approach, analysis tends to be focused on comparing the impacts of changes in the map on relative factor values. The factors are initialised, and the model run to equilibrium to produce a baseline. To examine different scenarios, one or more of the factors in the map has its initial value increased or decreased and 'clamped' at this new value (i.e. held artifcially at this point rather than changing dynamically). This represents an external change or an intervention. An output is produced of the relative change in fnal factor values under a given scenario, compared to the baseline, and all constrained by the squashing function. Again, the structure of the map is often modifed to add in an intervention or change and its connections to the extant system. This essentially shows how change propagates through the system as perceived by stakeholders but gives no idea of possible system dynamics.

Combined, the static map of factors, edges, and their values, and the dynamic or causal analysis based on these values is what FCM is. Terminology is fairly settled, there are no other names for the method which are regularly used, and though there can be differences in the names of concepts/factors/variables, and edges/connections, these are normally obvious and make no difference to the use of the method.

The actual practice of FCM does have some more important variety which can go unreported or underappreciated. The analysis can be done in different ways, as we have described above. Moving to building maps, this can be done in one of three ways. Researchers can build them themselves on whatever evidence or input they deem appropriate; multiple maps can be built in individual interviews with stakeholders, and then combined; or one map can be built in a workshop setting with multiple stakeholders. These modes can be combined in different ways, for example, with a researcher starting a map, then running a workshop, and then conducting some follow-up interviews.

During the construction of a map there are also subtle but important differences on how a map is started and constructed, which can have profound impacts on the nature of maps built. For example, we might have a list of twenty factors, and brainstorm in a workshop the most important connections ad hoc (i.e. asking stakeholders to start where they want), drawing and building out as we connect factors; or we may systematically go through each pair of factors in a table format and consider if there is a connection. Both approaches are valid, have pros and cons, but will result in different maps. More experienced FCM facilitators will also (sometimes subconsciously) guide the process in ways which avoid potential pitfalls later (more on this below).

#### How Do You Do Fuzzy Cognitive Mapping?

There are three basic stages to building FCM maps. First, we develop a list of the factors to go in the map; second, we construct the map and its connections; and third, we produce the analysis and interpret it with stakeholders or users. Let's consider each of these in turn:

1. **Develop list of factors to include**: This needs to be a well-thought through list of factors which are expressed as variables. Before the list can be made, a decision and agreement must be reached on what the map will be of, that is, what is the system. This system defnition can be very diffcult. Though factors can be anything, they normally need to have some level of comparability in their abstraction or simplifcation. In practice, it is useful to limit the number of factors at this stage to twenty, simply to reduce the time required to specify connections at the next stage. In multiple individual interviews, we might provide a list we create ourselves, or let interviewees build their own map with their own factors and then synthesise the maps and factors individuals created, decide when factors are the same and pick a name, remove duplicates, or work out what to do where there are differences. This process is like coding qualitative interviews where we look for themes; it requires a lot of researcher judgement. Guiding interviews based on what has happened in past interviews can reduce the potential for large differences and diffculty here. In workshops, this stage is normally the most time-consuming as participants must brainstorm, explain their thinking, then agree on choosing, merging, and removing duplicates from a fnal list. It is common to spend a large proportion of your workshop time on this stage.

causal thinking that the map contains. Thinking through these results considering how their own mental models of causal structure are producing them allows stakeholders to refect on, question, and confrm or change their beliefs about this structure.

To start doing FCM you need to make a choice about the materials you use to build the map, and then the software you use to visualise and analyse it. At the building stage, the main decision is whether to use pen and paper (and probably post-it notes), or to use software straight away. Using software can seem like an effciency saving for a map which you will need to digitise at some point; however, it comes at a big potential cost of engagement and inclusiveness when building maps in a group. Using software excludes people who are not confdent using computers or unfamiliar software, and if the facilitator operates the software alone, this makes them a bottleneck on the process. In our experience, participants will likely have lower levels of engagement and discussion building a map on a computer, but there will be exceptions to this.

Once you come to digitising the map there are a variety of software options including:

steep learning curve to using R, so we only recommend this to people who already use R, or who have the time to spend on learning it and are certain they want to use FCM. Given R's fexibility, power, and large user base, it is likely to offer the most functionality for building FCM maps, though we have not used it for this task ourselves.

#### Common Issues and 'Tricks of the Trade'

There are innumerable issues and questions you can come up against in an FCM project, too many to cover here in detail. Instead, we will consider some of the more salient and conceptual issues that apply to FCM.

With FCM it is often the dynamic analysis that users and stakeholders are most drawn to. Although all FCM practitioners refer to their method as 'semi-quantitative' and emphasise that it is not a simulation approach, in practice, there is always a temptation to over-interpret the model output. The analysis can seem like magic (especially when done live in a workshop) and offer a false sense of certainty, truth, and scientifc rigour. The outputs of any dynamic model are sensitive to the assumptions in that model, and the dynamic analysis of an FCM map built based on a participatory process is the same, but those assumptions have come from a place of (quick and dirty) consensus building and group deliberation, rather than cold methodical modelling. Additionally, the simple nature of the mathematics itself, whilst allowing rapid modelling, can hide unexpected pitfalls. In a dynamical FCM, edge weights and the presence and strength of feedback loops drive the output. The process of model calibration, altering these connections to obtain an equilibrium is intended to make the model output more interpretable by stakeholders, not to build a more accurate simulation.

If, on the other hand, a causal FCM approach is being used, with threshold functions on the factor values, then the model output is extremely sensitive to the mathematical form of these functions. In fact, when using this mathematical formulation, changing the form of the threshold function can have more of an impact on the results than the map structure, even to the degree of reversing model results. It is crucial therefore to do extensive sensitivity analysis when using a causal approach. We do not mean to suggest the model output has no value; it does. However, it is vital that it is used in the right way (i.e. as a discussion and thinking tool rather than as a forecast of what might happen), and those being shown it understand these caveats properly.

These, and other, common issues can normally be dealt with careful planning, an eye for detail, and a refexive approach. Iterating through map building, analysis, and interpretation will also be invaluable in ironing out issues. Iteration generally is a key theme to many of the approaches in this book—you will be sick of us talking about it soon!

There are also some tricks of the trade for FCM which can unlock potential issues. The frst, is to think about the analysis right from the start of map building. This is tricky at the beginning, but with practice you start to intuit whether a map structure will produce interesting outputs or not. As you build, look for feedbacks; are there any? Are there many short loops (i.e. with factor A affecting B and B affecting A)? Ideally you want to have a map with feedback loops, but these need to be questioned carefully so they refect reality and beliefs as best as possible, not just dropped into the map without much thought. Direct loops should always be questioned, are there intermediate factors which we should route a feedback via? A short, direct feedback will likely have a strong effect on the dynamic analysis, can we specify it so there are more steps?

The ability of FCM maps to include different types of factors is appealing. However, this can lead to issues if factors operate on different timescales. For example, if factor A infuences factor B over many years, and factor C also infuences B, but on a daily basis, the analysis of the map will treat them in the same way, which may produce misleading outputs. One solution may be to make the long-term infuence take a small value though this won't always make sense. A more reliable solution is to try to redefne the boundary of the map, and thus the system being considered, so that factors with very different timescales do not have to be artifcially included together.

Much of the value of FCM is in a process of capturing and then challenging stakeholders' mental models. In the dynamical approach, often altering the map live in a workshop to expose ways in which simple changes in system structure can alter outputs. Although this takes much experience to do live, a simple trick to try is to check some of the key outcome factors in the map and to ensure these are not represented as sinks (i.e. only with incoming arrows). Producing maps with this structure is common but leads to boring dynamics and is unlikely to refect reality.

#### What Is Fuzzy Cognitive Mapping Good and Bad At?

FCM's biggest strength (but also arguably the biggest risk in using it) is in its intuitive and quick descriptions of a system's structure and semiquantitative outputs. It does not require data or empirical evidence to inform or validate the map and its analysis. It is a form of quantitative storytelling, bringing together stakeholders and their narratives in a way which allows stakeholders to question and perhaps change their own assumptions about the system. The quick and dirty approach makes it ideal for a workshop context and use with stakeholders, and its ability to capture qualitative and quantitative aspects of a system mean that anything that matters on the ground can be included, making the method inclusive. The dynamical analysis, as well as being engaging and exciting to see, extrapolates and makes visible to stakeholders the logical implications of their beliefs about system structure, as it shows what factors are more highly driven, infuenced, or caused by external changes or interventions. Essentially allowing them to ask, 'does this factor or change actually cause what I think it does according to what I believe the system structure to be?'. This can lead to profound learning experiences.

The nature of the analysis required to work in this quick and dirty way, however, is the source of many problems. It creates the possibility of the analysis being easily misapplied or misunderstood. At worst, meaningless, but quantitative, analysis could be taken as truth by users and stakeholders. Although the dynamical analysis is often cited as a key hook, many practitioners say that the most important output is the map itself.

#### A Brief History of Fuzzy Cognitive Mapping

FCM frst appeared with Bart Kosko's (1986) paper outlining a modifed approach to cognitive maps (themselves frst outlined by Axelrod, 1976), applying fuzzy causal functions to connections (i.e. using −1 to 1 to represent the certainty of presence of a + or − causal link). Cognitive maps had been around for a little while and had been applied in a variety of domains; they had binary values for connections (i.e. either on or off, there or not) and were among the frst representations of causal connections between factors identifed by stakeholders, rather than researchers alone. Kosko's contribution in adding the 'fuzzy' was two-fold; to allow a more nuanced representation of causal reality, and to allow the maps to be 'computed' (i.e. with numerical values, to allow values of factors and edges to be combined to calculate the outcome of the values and the map structure).

Initially, most FCM research was reported in technical journals, with focus on the technical and mathematical details and methodological development. This meant FCM was slow to reach new audiences and potential application areas. While there is still much highly technical FCM work, since 2000, there has been both an increase in the number of published FCM works, and a spread into domain journals and a variety of felds, from social sciences through medicine and natural sciences. There does appear to be a clear divide between (i) work which uses FCM in a participatory mode, treating maps primarily as discussion and learning tools; and (ii) work which focuses on model discovery/inference/learning from data and treats maps as a form of neural network. Jetter and Kok (2014) provide a detailed history, with reference to many examples, for readers wanting to explore this history more fully.

Over time, however, as this method has been used more in participatory contexts, an ambiguity about whether certainty of cause and activation, or strength of causal link and magnitude of change, in factors is being investigated. Many authors use a causal mathematical modelling approach, but discuss strength of connections and size of effects, or switch between causal and dynamical terminology. This is potentially problematic as we are not actually calculating how values of factors change with this formulation and so a danger exists that a non-sensical model is produced and hence misinterpreted. A consistent causal approach could be used within stakeholder workshops by mapping certainty regarding cause rather than strength of effect on links. However, the ideas of factors being caused or not caused are by no means as intuitive as thinking about factors having an actual value or link weights representing a size of effect of one factor on another. This could cause issues within workshops when soliciting values for the map and indeed seems to cause issues in the literature. These issues are discussed in detail and clearly explained in the excellent working paper by Helfgott et al. (2015).

#### Getting Started with Fuzzy Cognitive Mapping Yourself

There is a noticeable dearth of resources and guides for getting started with FCM. However, we would recommend the following reading:

Beyond getting a frmer and more detailed grip on how to use the method, we would recommend two things. First, decide on the materials and software you are going to use to help you do FCM. This is an essential choice and making it will allow you to experiment with the method (either making 'test' maps in small groups with colleagues, or playing with different software) to get a real hands-on feel for it. Second, you need to fnd the windows of opportunity to apply FCM usefully. If you want to use it with stakeholders, speak to them and fnd out how it might be of value, what are the questions they have that might be amenable to FCM, what are the processes and workfows which an FCM project might feed into.

#### References

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

CHAPTER 7

# Bayesian Belief Networks

**Abstract** This chapter overviews Bayesian Belief Networks, an increasingly popular method for developing and analysing probabilistic causal models. We go into some detail to develop an accessible and clear explanation of what Bayesian Belief Networks are and how you can use them. We consider their strengths and weaknesses, outline a brief history of the method, and provide guidance on useful resources and getting started, including an overview of available software.

**Keywords** Bayesian Belief Networks • Bayesian networks

Bayesian Belief Networks (BBNs) were a real no-brainer to include in this book. Though they have been around for a little while now (30 years or so), we have seen a growing interest and excitement about them in a range of different felds, many concerned with social and policy questions. The roots of BBNs are in some of the more technical academic felds, such as computer science and statistics, but recognition of the value they can deliver has spread to many domains. There is a range of software options (more on these at the end of the chapter) to help you use BBNs, which mean that you don't need to have a deep understanding of the mathematics behind them. However, importantly, these don't give you 'too much power' without understanding, to risk doing genuinely inappropriate or misleading analysis. BBNs also fll an important niche in the landscape of different methods we explore in this book. They give us a method which has some of the best bits of quantifcation, allowing us to 'put numbers on things' while also incorporating uncertainty in a meaningful way.

The term 'Bayesian networks' was coined by Judea Pearl in the late 1980s. He is an interesting and vocal character (his Twitter account is well worth a follow for the illuminating debates he instigates), and his body of work on causal inference has had a growing infuence in recent years, most notably in economics, quantitative social science, and social data science. If you fnd this chapter interesting, and are looking for something more formal, we recommend also doing some reading around directed acyclic graphs (DAGs) and their use in causal inference.

Because of BBN's roots in computer science, there is a large literature on their use based around the idea that we can use learning algorithms to generate their networks (or maps) directly from data. In common with Fuzzy Cognitive Mapping, there is a parallel literature on developing them in more participatory modes, from stakeholder knowledge. We will focus more on the latter in this chapter, though we will touch on the concepts behind developing BBNs, and other system map types, from data in Chap. 9.

As in all the methods chapters in this book, we will now give as clear and accessible a description of what BBNs are and how to do them, as we can muster. We will avoid the use of mathematical notation to do this, there are other introductions which cover the maths well (e.g. Neapolitan & Jiang, 2016). We will then explore some common issues and tricks of the trade for using them, try to pin down what they are good and bad at, before outlining a brief history of the method and pointing to some key resources for getting started yourself.

#### What Are Bayesian Belief Networks?

BBNs, as with all networks, are made up of nodes (which here, represent variables, factors, or outcomes in a system) and edges (which represent the causal relations between these nodes). So far, so familiar, and like many of the methods in this book. What sets BBNs apart is that each node has some defned states (e.g. on or off, high or low, present or not present) and some associated likelihoods of being in each of those states. These likelihoods are based, in probabilistic fashion, on the states of the nodes that they are connected to, that is, the nodes from which they have arrows going into them. In the language of probability, nodes are 'conditionally dependent' on the states of the nodes that they have a causal relationship with. So, the BBN is the network and the collection of conditional probabilities (usually shown in simple tables or plots annotated onto a network diagram), denoting the likelihood of nodes taking different states. The last key point to mention is that BBNs are acyclic; that is, they do not have any cycles or feedbacks, and the arrows must fow all in one direction. This is an important distinction between other methods which do have cycles, such as Causal Loop Diagrams and Participatory System Maps.

It is useful to make this more tangible quickly, so let's look at an example. Figure 7.1 shows a simple BBN of the effects of 'rainfall' and 'forest cover' in a river catchment, and the links through to some different outcomes, such as 'angling potential' and 'farmer income'. This is a simple BBN, you can see the acyclic structure, and focus on outcomes we or others might care about.

The same BBN, but this time with the states of each node shown along with the probability distribution for them is shown in Fig. 7.2. This allows us to see, for example, if 'rainfall' and 'forest cover' are both high, the likelihood of all the other nodes taking specifc values.

We could explore different scenarios by setting the states of 'rainfall' and 'forest cover' differently and seeing how this affects the rest of the map. This is a common way of using BBNs, setting certain node states given our observations (or hypothetical scenarios we are interested in), and seeing what this implies about the probability of states in other nodes. With 'rainfall' and 'forest cover', we are setting values at the 'top' of the network (sometimes referred to as 'root nodes' or 'parent nodes', i.e. with

**Fig. 7.1** An example simple BBN. (Source: Bromley (2005))

**Fig. 7.2** An example BBN, now with nodes states and probability distributions. (Source: Bromley (2005))

**Table 7.1** An example conditional probability table based on the reservoir storage node in the BBN in Figs. 7.1 and 7.2

Source: Bromley (2005)

no nodes going into them) and looking causally 'down', but it can be done the other way round too; setting the states of outcomes (sometimes referred to as 'leaf nodes' or 'child nodes') and looking 'up' the network to see what might have contributed to that outcome. These are the two main types of insight the analysis of BBNs can provide: (i) assessing the probability of achieving outcomes, and (ii) quantifying the impacts on outcomes of changes elsewhere in the system. As with all the methods in the book, there is also huge potential value in the process of building a BBN, to generate discussion and learning about the topic.

These examples help us get a quick sense of what BBNs are about, but they are focused on the 'results' of the BBN; this is what is shown in the node tables. What is not present are the conditional probability tables that underpin these outputs. Table 7.1 shows what one of these tables might look like for one of the factors in this BBN, 'reservoir storage'. It shows different states of the parent node of 'reservoir storage', which is 'river fow', and the resulting probabilities of 'reservoir storage' taking each of its states. The numbers indicate the probability that 'reservoir storage' will take its values in the second column (i.e. if 'river fow' is good, then there is a 90% chance 'reservoir storage' is good, and 10% chance it is medium).

BBNs have some well-known and often-criticised constraints. First, they are acyclic; that is, there cannot be any feedback loops, of any length, in the network. This constraint is imposed for the calculations to work. In complex systems, it is rare for there to be no feedback loops; where there are feedback loops, these are often powerful drivers of dynamics in the system. It is possible to partially represent feedback loops by including multiple nodes for the same thing, but for different time points (we show an example of this below). BBNs that use this approach are often called 'dynamic' BBNs. A second constraint on BBNs developed with expert input is that most nodes cannot have more than two or three parent nodes (i.e. incoming connections) and nodes should not have more than a handful of states. This constraint is normally imposed so that the conditional probability tables, which are a key component of what is elicited from stakeholders to build the BBN, do not become unworkably large. By way of illustration, imagine a node with two states, and two parents each with two states themselves, this will require a 4 × 4 table. However, a node with three states, and three parents, each with three states themselves will need a 6 × 27 table. Imagine flling that in with stakeholders, cell by cell, with potentially important discussions at each step, and doing this for every node in the map. Combined, these two constraints mean that the underlying network in a BBN tends to end up being a relatively simplifed model of reality compared to some of the other systems mapping methods in this book. This is not necessarily a problem, but it is a constraint we should be aware of.

These constraints often invoke ire from researchers and practitioners who want to represent whole systems and take a complex systems worldview (including from us in the past!). However, one of the typically misunderstood, or simply missed, nuances with BBN is that the use of conditional probabilities means that we can still capture some elements of the wider system in the analysis and discussion around constructing maps, even if they are not in the network explicitly. To demonstrate, consider Table 7.2. Here, we can see the probability of an outcome occurring given the state of two interventions. Even when we have both interventions 'on'

**Table 7.2** Simple hypothetical conditional probability table for two interventions and an outcome

Source: Authors' creation

there is still a 0.1 probability the outcome does not happen, and conversely, when neither intervention is 'on' there is still a 0.2 probability that the outcome does happen. These non-zero probabilities represent 'everything else going on in the system'. They are often an important point of the elicitation process and allow us to capture infuence on the outcome, even if we do not formally put them in the network.

You may be wondering why BBNs are Bayesian. They are referred to as 'Bayesian' because of the use of the underlying logic of Bayesian statistics (which provides a way to update probabilities considering new data or evidence) rather than because they were developed by Thomas Bayes himself. Bayesian statistics, simply put, is a feld within statistics that revolves around the idea of probability expressing an expectation of likelihood based on prior knowledge or on a personal belief. This probability may be updated based on new information arriving about factors we believe to infuence that event. In a sense this operationalises how our belief about a particular probability should change rationally as we learn more. This is in opposition to the Frequentist view of probability which revolves around the idea that probability relates to the relative frequency of an event. We do not want to get into the large and ongoing debates within and between these two schools of thought. However, it is important to recognise that BBNs take that Bayesian idea of probability and implement it though the network structure and conditional probability tables; the parent nodes of any node hold the prior information we are using to update our beliefs about that node. We can also use new information about the states of child nodes to update our beliefs about parent nodes using Bayesian inference.

There is a lot of variety in how BBNs are built, either directly from data, or through participatory processes with experts and stakeholders. However, the object that is produced and the analysis that is done tend to be consistent. There are extensions, such as dynamic BBNs (as mentioned above), and hybrid BBNs (which allow us to include continuous variables as well as the categorical variables we have described above). Where there is more variety is in the terminology and jargon associated with BBN. Ironically, given the formalism of the method, this is one of the methods with the highest number of different names, but less surprisingly, the opaquest technical language.

You may see BBNs referred to as any of the following: 'Bayesian networks', 'probability networks', 'dependency models', 'infuence diagrams', 'directed graphical models', or 'causal probabilistic models'. We have also seen them referred to as 'Theory of Change maps' because of the similarities with these types of diagrams, that is, a focus on connections between inputs and outcomes, tendency to produce simple maps, and not include feedbacks of many causal infuences. This plethora of terms seems to refect the widespread use of BBNs in different domains rather than large differences in how they are used. Some of the key technical terms you may bump into might include 'prior probability distribution', or 'prior', which refers to the probability distribution that indicates our best guess about the probability that a state will take some value before new or additional evidence or data is taken into account; and 'posterior probability distribution', or 'posterior', which is the conditional probability we assign after taking into account new evidence or data. Note that the probability distributions assigned to states of root nodes are prior probabilities because they have no inputs on which their state is conditional.

#### How Do You Use Bayesian Belief Networks?

The main division in constructing BBNs is between approaches which generate the networks and conditional probabilities directly from data, using a range of different learning algorithms, and those which use stakeholder input. Here, we will focus on the latter, though we do consider the issue of developing system maps directly from data in Chap. 9. Even though we are focusing on the participatory mode of BBN, it is worth making clear that you will likely want to use purpose-built BBN software to implement your BBN. You can build the network and collect conditional probability information in standard ways (i.e. workshops, interviews) with general-purpose software or just pen and paper, but you will need the BBN software to run the analysis quickly and easily for you (more on software options at the end of the chapter).

Let's look at each of the main stages in developing and using BBNs:

**Fig. 7.3** An example dynamic BBN with a feedback between 'wood extraction' and 'wood stored'. (Source: Authors' creation based on an example in Landuyt et al. (2013))

• **Analysis**: now that you have your network and conditional probabilities, you can start doing some analysis. You will likely fnd value in iterating through a few rounds of analysis and discussion with stakeholders when you are working in a participatory mode. In practice, the main 'site' for your analysis will be the BBN software. It is normally easy to use the software to manipulate the BBN and address the questions you are interested in; most have simple point-and-click functionality to do this. To start with, your network will default to using the prior probabilities you have specifed for the root nodes (i.e. the nodes with no incoming connections) and use these to compute probability distributions for states for the rest of the nodes based on the conditional probabilities you have defned. You can then easily specify the states (or values if you have a hybrid BBN) of any nodes in the network, and the probability distributions of the states of all other nodes will update to refect this 'new' information. By setting states for different combinations of nodes you can create different scenarios and explore their implications in the system. The two most common types of insight we have seen this used for are: (i) to set intervention-type node(s) states and see what this implies for the chances of outcomes being achieved (i.e. what difference does turning an intervention 'on' make to the probabilities of an outcome happening?); or (ii) to set outcome-type node(s) states (perhaps based on what we have observed) and see what this implies about the states of other nodes in the network. Once you have explored the BBN and have some analysis, there are a range of ways you can export and present this. It is common to see full networks with states and probabilities defned, as in Fig. 7.2, but others often also choose to show the probabilities of different states for key outcome nodes, given different inputs, perhaps in a simple plot or table.

The stages we have described here are generic. You will need to design a bespoke process that fts your needs. This may emphasise more engagement and iteration with stakeholders or may be more streamlined with a focus on getting to analysis and producing outputs.

#### Common Issues and 'Tricks of the Trade'

As with all modelling that produces quantitative outputs, there is a common tendency to over-interpret the precision and reliability of the outputs of BBNs. It is easy for clients or other users of our BBN research to take away a few standalone numbers or plots and focus on these in isolation. Moreover, because BBN deals in probabilities, and is often touted as being useful in contexts of uncertainty (which it is), when results are taken out of context like this, they can still be interpreted as having taken into account uncertainty, creating an undue sense of confdence is using them glibly. We must encourage users to acknowledge that BBNs are always dependent on stakeholder opinion (unless developed based solely on data) and that removing outputs from that context, and not making clear either the process, or the network (i.e. the model), from which they are derived almost always dooms us to see them misinterpreted. Even in cases where outputs are not misused or misunderstood, the appeal of the diagram of a BBN with conditional probabilities annotated can also lead many to view BBN and its associated analysis as a product, rather than a process. Not recognising the value in the process of using this method is to ignore at best half its value, at worst, all its value.

The other common issue relating to the relatively formal nature of BBN is that it can make a participatory process demanding for stakeholders. The process of building a BBN can quickly become a long series of small steps, which individually are dull and/or conceptually dense. This is most common when eliciting conditional probabilities, which can create rather repetitive conversations around flling in conditional probability tables. Even during the building of the underlying network, which is typically one of the more creative and exciting stages in systems mapping, the constraints BBN imposes can become onerous and inhibit discussion, leading to frustration on the part of stakeholders.

To address these, and many other issues, we see four key 'tricks of the trade' being used regularly:

• **Emphasise iteration and learning from the process**: do not be afraid to emphasise the need for, and value in, iteration of BBN; build this into stakeholders', clients', and users' expectations. One of the reasons that iteration is so useful is that it allows us to 'fold-in' the learning we are generating as we go into the process and the products of BBN.

#### What Are Bayesian Belief Networks Good and Bad At?

Arguably, all the unique strengths of BBN relate to their ability to include probabilistic representations of the states of nodes in causal networks. The mere presence of numbers is not what is useful, rather it is the way the numbers are used. Despite our warnings about abuse of outputs, it is quite diffcult to produce meaningless analysis with BBN because it is based on extrapolating individual conditional probabilities (which are easy to defne in sensible ways) in a relatively low-risk way, rather than turning the network into a dynamic simulation, which can be fraught with danger. Essentially, unlike a simulation, a BBN retains its transparency even whilst generating numerical output. Everything that has gone into the model is clearly visible in the network structure and probabilities. There are no 'hidden' decisions taken by a facilitator or modeller on which the outputs might depend crucially. (For example, like parameter choices in Fuzzy Cognitive Mapping or System Dynamics.)

The use of probabilities also gives us the wiggle room to bring in wider contexts and causal infuences, which it may appear, from the network itself, have been simplifed away. Seen in this light, BBNs are one of the better methods for capturing the effect of 'everything else going on' in the system in a meaningful way. Because variables are connected by conditional probabilities, there is no need for any mechanistic model of how they change each other's values. This means that variables of many different types can be brought together in one model, allowing a BBN to cover multiple types of domains important in a problem. Taken together, these strengths hopefully make clear BBN is most useful when we want some quantifcation, but where it is useful to do this in a probabilistic form, and/or where we want it to be done in a relatively understandable and transparent way.

On the fip side, there are some key weaknesses of BBNs which may be problematic in some settings. They are not good at producing wholesystem views of a topic or system and will tend to lead to more simplifed models. This is not only in the sense of the breadth and size of map (though size is unconstrained, it is rare to see very large BBNs, i.e. 100 or more nodes) but also in individual connections because of the constraints on the number of parent nodes any one node can have. Where we want this fexibility, or the ability to capture a whole system, BBNs will be less useful. Though dynamic BBNs can capture feedbacks, this is an awkward and potentially incomplete solution (if we only capture one or two cycles of the feedback), so in contexts where there are multiple important feedbacks, BBNs may not be the best choice. Finally, it is worth noting that BBNs can also be time-consuming to update because of the additional conditional probability information that needs to be collected for new nodes, and anything they are connected to. Though they can be updated, in situations where we want a 'live-document' or an easily updateable system map, BBNs may be a poor choice.

#### A Brief History of Bayesian Belief Networks

The term 'Bayesian network' was frst coined by Judea Pearl in his 1988 book, *Probabilistic Reasoning in Intelligent Systems*. Together with Richard Neapolitan's 1989, and very similarly titled book, *Probabilistic Reasoning in Expert Systems*, the feld as it is recognised today was born. These two books refected much of the thinking that had been developing in the 1980s through a series of workshops (now a conference) on uncertainty in artifcial intelligence. Scholars from cognitive science, computer science, decision analysis, medicine, maths and statistics, and philosophy contributed to these developments. As the name of the workshop series suggests, the underlying aim was to incorporate uncertainty into the various types of computational analyses often referred to as 'knowledge-based systems' or 'expert systems'. Put simply, to include uncertainty in these techniques, which were being designed to solve complex problems, or make complex decisions, in human-like ways. These Bayesian networks built on the earlier work of Philip Dawd in the 1970s on the representations of and reasoning with probabilistic dependences. The word 'belief' was used in the name of the method right from the start (though not consistently) because this word is used to describe assertions about probabilities in Bayesian statistics. Note, 'belief' is not used because BBNs are used in subjective stakeholder participatory processes, though this is a sometimes useful coincidence.

From the start, there were two ways to build BBN, either from data or from stakeholder input. In the early days, data was only used to defne conditional probabilities (and then often checked by experts), but not the network structure. As the learning algorithms became more sophisticated, computing power more affordable, and data more plentiful, data could be used to develop the network structure too. More recently, as the value of BBN has been realised in a wider set of felds, and applied to practical policy problems, the use of BBN in a participatory mode has increased in popularity.

#### Getting Started with Bayesian Belief Networks

We hope you now have a clear picture of what BBN is and how you might use it. The big missing piece we have left undiscussed thus far is software. You will almost certainly want to use some BBN-specifc software and there are a few options you have. Let's now outline some of these software options, and their pros and cons, in Table 7.3. Note, there are more options available than those here; these are just the ones we have used and were recommended to us:

Beyond software, there are resources we would recommend to help you to get an even more detailed understanding of BBN and how to use them, including:

• **Bromley (**2005**) guidance**: this free and detailed guidance goes into a lot of details on the process of using BBN, including constructing

#### **Table 7.3** BBN software overview

Source: Authors' creation

them. The example used is water resource management, but it is useful for researchers working in any domain.

Hopefully, you now have everything you need to get started with BBN. You don't need to have a detailed grasp of Bayesian statistics, but we would recommend you try to get comfortable with the broad ideas. Beyond this, our other priorities before starting would be to play around with one of the software packages above, and to fnd a couple examples of BBN in similar domains, or used in similar ways, to what you are hoping to do. Good luck!

#### References

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# System Dynamics

**Abstract** This chapter introduces System Dynamics, a well-established method to developing full simulations of dynamic systems. We introduce the method and how it can be used to move from conceptual models to 'stocks and fows' models and simulations. We describe the steps involved in doing System Dynamics and consider common issues and tricks of the trade. We refect on what the method is good and bad at and present a brief history of use and debates in the feld. Finally, we give some advice on getting started yourself and point to some useful resources.

**Keywords** System Dynamics • Dynamic model • Stock and fow model

System Dynamics is probably one of the most widely known methods in this book, owing to its use in the Club of Rome commissioned report, Limits to Growth (1972). Though the report's modelling and conclusions were heavily debated and criticised, there is no doubt that it explored the dynamics of economic and population growth within the constraints of the natural world in a compelling and engaging way—it showcased the power of System Dynamics, and modelling more generally, both as analytic and thinking tools.

This high-profle example refects something more fundamental about the method; on a surface level, System Dynamics models are intuitive and often convincing. As a simulation method which produces results over time (results which can be interpreted as forecasts, or even predictions more on this later), it can create a black-box, magic-like quality; the lure of the simulated world convincing people of its truth. On a deeper level, however, discerning model observers and users can see the sensitivity of model results to an array of assumptions, and how the model can be used and abused by those wishing to use it to make their own arguments. Ultimately, though much energy was used on repetitive debates around Limits to Growth, and the model itself was criticised heavily, it likely did serve its purpose well—to raise awareness of an issue, and to facilitate and support more detailed thinking and strategic insight on it. Perhaps at a cost of System Dynamics itself, it also raised much discussion and thought on the different ways to model complex systems.

The rest of this chapter follows a now familiar structure. We frst describe as clearly as we can, what System Dynamics is, before outlining how it is done. In these sections, we try to capture the nuances in differences in practice and how these affect what is produced. We also cover tricks of the trade and common issues you might come up against. Next, we try to summarise what System Dynamics is good at, and what it is not so good at, and consider how people tend to react to it. Finally, having introduced the method, we outline a brief history, so readers can get a strong sense of the points of debate and the trajectory of the method's use.

#### What Is System Dynamics?

The short but jargon-flled defnition of System Dynamics (many variants of which you will fnd in the literature) is 'a computer simulation method for analysis of complex dynamic systems'. If we unpick and decode this a little, we can say it (i) is typically implemented on a computer using software to carry out the calculations we put into the model, (ii) allows us to prod and poke a model to run different scenarios and look at different outputs for comparison, and (iii) is normally used to look at systems which are dynamic or moreover have some interesting dynamic properties.

More concretely, a System Dynamics model is made up of stocks and fows, and the factors which affect fows. Stocks are any entity which accumulates or depletes over time. A fow is the rate of change in a stock (usually represented by a differential equation). Stocks take numerical values, and fows are defned by equations which can be affected by any number of other factors in the model. These equations may be relatively simple, using standard operations like addition, subtraction, multiplication, division, and sometimes exponents, or more complex involving functions and parameters which moderate how variables interact. Once a set of stocks, fows, and the factors which affect them are specifed in equations, the simulation can be run by choosing a starting point (i.e. a set of stock values) and computing how stock values change through repeated time steps (i.e. through being repeatedly changed by their associated fow). Figure 8.1 shows an example of a stock and fow System Dynamics model which represents tourism and pollution in the Maldives (it is taken from Kapmeier & Gonçalves, 2018). As with most of the methods in this book, the diagram is in essence made up of boxes and arrows. Common to many System Dynamics diagrams, in Fig. 8.1 we can see the stocks as rectangles, the fows as straight hollow arrows with hourglass-type symbols (these are meant to represent valves) on them, and the factors that affect fows represented by foating text and the solid (and mostly curved) arrows. The small circular feedback arrows with italic annotations are labels for the feedback loops represented by the arrows around them. B stands for a balancing feedback loop, meaning that it will tend to self-stabilise, R for a reinforcing feedback loop, which will tend to carry on increasing or decreasing. Not so common, but used here, are the dotted lines to

**Fig. 8.1** A System Dynamics stock and fow diagram of pollution and tourism in the Maldives. (Source: Kapmeier and Gonçalves (2018))

represent infuences which were not included in calibration processes due to missing data (more on calibration later).

To describe some of the equations in the model, let's take the example of the number of tourists. This is represented by the stock 'Tourists', which changes based on the fow 'Change in tourists', which is, in turn, determined by a constant historical tourist growth rate (we cannot see this in the diagram, but it is specifed in text which describes the model) modifed by the factor 'Attractiveness of the Maldives' (which we can see here). Essentially, the fow is a differential equation specifying the rate of change of tourists over time, as the historical rate of change is impacted by the changing attractiveness of the destination.

$$\frac{dT}{dt} = T \ast g\_h \ast \left(1 + A\right)^2$$

where T is number of Tourists, *gh* is Historical Tourist Growth Rate and *A* is Attractiveness of the Maldives.

'Attractiveness of the Maldives' is calculated based on fve inputs: tourist numbers, crowding, demand/supply balance of beds, prices, and tourist awareness of pollution. The ways these are specifed in an equation can vary from something simple to something more nuanced. In Kapmeier and Gonçalves (2018), they are calculated with the following equations:

$$\text{Attractiveness} = E\_{\text{R}} + E\_{\text{DS}} + E\_{\text{p}} + E\_{\text{p}}$$

*where ER = Effect of Resort Exclusivity on Resort Attractiveness; EDS = Effect of Demand/Supply Balance on Resort Attractiveness; EP = Effect of Pollution on Resort Attractiveness; Ep = Effect of Price on Resort Attractiveness*

> *E f Tourists Tourists reference value <sup>R</sup>* <sup>1</sup> / *E f Required BedCapacity Annual Bed Capacity DS* <sup>2</sup> / *E f Pollution Pollution reference value <sup>P</sup>* <sup>3</sup> / *E f Price Price reference value <sup>p</sup>* <sup>4</sup> /

where the *f* functions are decreasing s-shaped logistic functions, and reference values estimated from past data are used to normalise the input variables (i.e. dividing the current number of tourists by a reference value normalises it relative to that reference). The logistic functions bound the effects on attractiveness of any variable between specifc upper and lower limits, ensure that increasing values of numbers of tourists, demand/supply balance, pollution, and price decrease resort attractiveness and allow the system to be set up so that a high value of any one of these factors dominates over any positive effects from others. Similar sets of equations are described for each of the stocks and fows in the map and the infuences on them. These equations together allow the dynamical behaviour of the system to be simulated.

It is often assumed that System Dynamics models are deterministic using the same exact values each time and thus producing the same result every time; however, this is not true. Many models, and indeed very early examples, included stochastic elements. What types of things you can represent in stocks and fows is in theory broad. We do not have to have data on things in the model, and they don't even have to be quantifable. Indeed, the founding fgures of the method were clear that we did not need to revert to using highly quantitative factors with data to back them up. As long as it makes some sense for stocks to be talked about going 'up and down' and we can put some numbers to them and factors in equations without it being completely meaningless or inappropriate, we can make a model work.

However, in practice, people often favour more quantifable elements and avoid 'soft' (unquantifed) factors. This is perfectly understandable; the existing literature and data make it much easier to quickly implement a biophysical factor such as 'rainfall', for example, compared to a social factor such as 'Attractiveness of the Maldives' (as above). The natural and physical sciences tend to have more data and quantitative models of almost all the factors we might ever wish to use from these domains, whereas the social sciences do not. This data and models make it easy to implement these factors (i.e. we know their values, and we know what they are affected by). We must put in extra effort and take extra care to include important social and 'soft' factors in our System Dynamics models. Specifying mathematically how they interact may take a great deal of careful thought. All complex adaptive systems will have these components, and they will often play important roles in dynamics, so we cannot ignore them simply because they are more diffcult to specify.

Once a System Dynamics model is built, it is 'run', this produces outputs, such as those seen in Fig. 8.2. Here we see values for the stock 'Tourists', through time, under multiple different scenarios (labelled capacity, access, and price, there is also a baseline scenario and some real data plotted). A scenario represents a change in the model, either in the equations of fows or in the structure of the model (i.e. with a connection added or removed), both of which are intended to represent some intervention or difference in the system. Here the scenarios represent different policy options. Quite different sorts of system futures are visible under different scenarios. For example, increasing tourist access or capacity ultimately leads to a crash in tourist numbers as the resulting pollution from high tourist numbers destroys the destination's attractiveness. A policy of lowering prices does not have such large impact on tourist numbers and hence does not lead to a boom-and-bust dynamic.

There is ongoing debate in the System Dynamics community, and other simulation communities, as to whether we should ever (try to) interpret model outputs as predictions of future states. Many scholars contend that as the models are simplifcations of reality, and the reality we are modelling is a complex adaptive system with many components and interactions, models will never be able to provide reliable predictions of future states, so we should not attempt to use them in this way. Rather, they suggest, we should consider model outputs as hypothetical futures, or qualitative forecasts of the types of patterns and dynamics we might see. However, others take an opposing view, seeing prediction as one of the main purposes of models, and while acknowledging the diffculty of prediction in complex systems, still wish to attempt to predict using models like System Dynamics,

**Fig. 8.2** Number of tourists under different policy scenarios. (Source: Kapmeier and Gonçalves (2018))

and (partially) assess the quality of models against their ability to predict. Without some element of prediction, they see models as missing a key element of what defnes a 'good' model. We tend to sit on the fence on these debates, taking a pluralist view and seeing the merit in both approaches. As we have often said in this book, purpose drives the model. If our purpose is to predict, and we have designed with this in mind, it is not beyond the reach of methods like System Dynamics, but it is certainly very diffcult, and success may well be feeting.

As with many quantitative modelling methods, there are a range of rather technical activities that will be part of any System Dynamics process, which include parameterisation, calibration, validation, and sensitivity analysis. In practice, these elements often merge into one another, and the exact nature of how they are done, in what order and combination, and their importance in the modelling process, will depend on the purpose of each modelling project. We do not wish to go into the gory details of how they are done, which is beyond the scope of this chapter, but briefy, here are some simple (yet still debatable) defnitions of them: parameterisation is the process of picking values for parameters (e.g. stocks, factors, equations, modifying fows) in the model based on data or plausible values (i.e. 'plausible' is a loose idea, but here we mean values which an expert on the system would deem sensible); calibration is the process of picking values for parameters such that the model produces outputs that reproduce some target (e.g. changing values until the model produces a pattern seen in real data); validation is the process of assessing how well the model reproduces target outputs (and often involves elements of calibration); and sensitivity analysis is where different parameters, or structural components of the model (e.g. which connections and relationships are represented), are changed systematically to assess how sensitive model outputs are to them. Each of these activities can be done in informal qualitative ways, or in much more technically demanding quantitative ways.

There are some important variations in practice and terminology in System Dynamics. In the practice of System Dynamics modelling, these mostly revolve around how we arrive at a fully specifed and quantifed stock and fow diagram. Some system dynamicists dive straight into a stock and fow diagram, whereas others frst produce Causal Loop Diagrams (see Chap. 4) or other similar types of qualitative or semi-quantitative systems maps. These differences in where to start will likely have impacts on the models produced. Causal Loop Diagrams are a point of variation in terminology too. We have observed Causal Loop Diagrams referred to widely as 'system maps'. The way they are built appears to be consistent, but the different terminology can be confusing and unhelpful when the term 'system maps' also describes a variety of approaches, as we outline in this book. We have also seen stock and fow diagrams referred to as Causal Loop Diagrams and vice versa, though their intended use is normally clear.

Before we move on to how to do System Dynamics, we would like to emphasise one element in the method that is different to some of the others in this book but is not immediately obvious. Despite having 'system' in the name, System Dynamics models tend to be focused on specifc dynamical problems rather than whole systems. In practice, this means that models rarely aim to represent a whole system, but rather focus on carefully constrained sub-sets of a system with the relevant dynamics. This nuance in focus is often not understood, with many researchers and practitioners envisaging System Dynamics as taking a whole-systems view, where in fact it is the 'dynamics' that the method emphasises. Indeed, part of the 'art' of System Dynamics modelling is fnding and defning these knotty dynamical questions and problems which the method can contribute to signifcantly.

#### How Do You Do System Dynamics?

The process of conducting a System Dynamics modelling project can be broken up into six steps. These make sense conceptually, but are never actually done neatly in sequential order, rather much iteration and jumping backward and forward is the norm. They are:

and connections' model. Most commonly this is done with Causal Loop Diagrams (see Chap. 4 for an introduction and description of how to use this method, we don't reproduce it here in the interests of avoiding repetition) but may also be done with other qualitative systems mapping approaches akin to Participatory Systems Mapping (see Chap. 5) or Fuzzy Cognitive Mapping (Chap. 6). The point is to use something which is easy to build but which also provides the building blocks for the quantifed model of stocks and fows.

outputs and should look at similar examples you think work well, or follow more general guidance on data visualisation.

6. **Interpretation**: fnally, once we have results we can share (and importantly, our descriptions and explanations of them), it is time to interpret and use these results with stakeholders and end users. Again, this is a key and often underestimated stage. The most impactful and successful projects will give plenty of time for results to 'stew' and be interpreted and allow space for further iteration from there.

The exact detail of each of these steps and how you iterate within and between them will depend on many factors, most importantly your own preferences and the purpose of the project. We cannot emphasise this enough: do not be worried about jumping back and forward, the best modellers do, and are creative as they go.

As with most of the methods in this book, System Dynamics modelling can be done in a participatory setting, and this is common. Sometimes it is referred to as 'group model building', 'mediated modelling', or 'Participatory System Dynamics'. A key decision is what stages of the modelling to do with stakeholders; do we just do the problem defnition and model conceptualisation, then return with results, or do we also include them in the model formalisation stage? Both will be valid depending on your purpose. We would encourage you to attempt to include stakeholders at all stages, and not assume they do not want to get into technical details. The last thing we want in a participatory modelling project is for stakeholders to think there are 'highly technical and skilled' elements which they should not play a role in and thus passively wait for your return 'with the answers'. It may go against our instincts, but ideally, we want stakeholders to critique and attack our models, so we can uncover bad assumptions, misunderstandings, and the gaps that hamper consensus being reached. The trick is to have just enough legitimacy and respect with stakeholders so that they will take part and engage, but not so much that they won't be sceptical and criticise you.

There are many software options available for a budding system dynamicist. Point-and-click options are common, and you will not need to write the computer code for a model if you do not want to. Some of the most popular and powerful include (in alphabetical order):

Though it is no longer in use, the DYNAMO simulation language is worth a special mention. This was the language used for many early System Dynamics models, including the Limits to Growth model, and is still spoken about with affection. Some efforts exist to bring it back to (some form of) life, for more, see https://github.com/bfx/dynamo.

#### Common Issues and 'Tricks of the Trade'

We touched on one of the more fundamental but common issues above struggling to fnd a dynamical problem or question which is well-suited to a System Dynamics model. We often begin a project with a broad topic or interest, and perhaps a method in mind, and need to fnd a way in which the latter can be of use in the former. This is a diffcult art. Some rules of thumb for defning the 'target' of the model (i.e. the thing we want to model) include:

It is relatively easy to start building System Dynamics models, but much harder to produce 'good' models and develop genuine insights. This can create a common sense of frustration for intelligent but inexperienced modellers. While the technical craft can be picked up quickly, you will likely bump into many conceptual issues and questions which are more diffcult to 'solve', and you will also start to see a range of issues and weaknesses in your model and the purpose you are putting it to. While as modellers we are often told not to 'fall in love' with our model, it can be just as common and problematic for new modellers to become disillusioned at the diffcultly of good modelling and lose interest. This is normal and part of the learning process. This can happen with any of the methods in this book, but it is particularly relevant here because System Dynamics is a simulation method with many components to be specifed, calibrated, a model to be validated and scenarios to be designed and run—in short, there are simply more points at which to run aground.

One of the keys to avoiding this issue, and many others, is to iterate quickly and aggressively. You will see calls for iteration everywhere in research and modelling, and especially for work on complex adaptive systems, where iteration is a key heuristic in researching and engaging with systems. But how do you actually do it? What does it look like? Most simply, it means jumping backwards and forward between the six steps we describe above. There is a diffcult balance to doing this though, it can be done too slow and too fast. The trick is in having the confdence to move between steps openly and with purpose, but to do so in a systematic way rather than in a panic.

#### What Is System Dynamics Good and Bad At?

System Dynamics' real strength is in helping us develop high-level strategic insights on dynamical questions. It can help us fnd counter-intuitive dynamics, to rigorously play out the implications of an intervention or theory, or to play a facilitating role in doing this with stakeholders. It also helps us think about dynamics rather than individual events or static models. As well as producing outputs showing how variables change dynamically, many systems dynamics software packages allow you to see how dynamics emerge from interacting feedback loops as they change in strength and importance as a model unfolds. This breaking down of dynamics into component parts can allow more intuitive understanding of how dynamics arise for stakeholders and modellers alike. Thinking in a dynamical way is a very different way of thinking for many people, which can be hard to develop; System Dynamics will help you do it. As a simulation method, it is also well-suited to helping us test out assumptions and the potential impacts of interventions; to exploring those 'what if?' questions.

Despite being a quantitative method, it is not useful for operational or more local, micro, questions, or more generally in detailed analysis of systems. It normally represents systems at an aggregate level and focuses on dynamical subsystems. It requires a relatively large amount of time to conduct compared to other methods in this book. This can be a serious barrier to its use in applied and participatory contexts. It can also be diffcult to include social and 'soft' factors in a model, where we don't often have the help of previous modelling and quantitation of these ideas to inform our implementation of them.

#### A Brief History of System Dynamics

The method appeared in the 1950s, developed by Jay Forrester, who used it to explore the success and failure of businesses and industrial dynamics. Legend has it, Forrester used 'hand simulations' (we can only assume these were calculations done on paper, rather than physical representations of the stocks and fows!) to explore employment dynamics with managers at General Electric. Forrester and his team quickly moved on to modelling on computers, and his approach was applied to multiple business and management questions. By the late 1960s and into the 1970s, the method started to spread beyond these roots and was famously used in the Limits to Growth work described above. Since then, it has been applied in many domains and enjoys a large and active community of practitioners and researchers.

Because of the method's name, and partly because of its approach, people often assume there are strong connections with systems science and thinking; however, the method did not emerge out of these felds, so the links can sometimes be weaker than we might expect or hope. Nonetheless, the underpinning interest in complex adaptive systems is present in both. Prominent fgures in the feld have suggested progress has been held back by the lack of connection to other modelling and research domains, and this has led to a lack of development in the method itself (e.g. Sterman, 2018).

There have been, and continue to be, many debates in the community. They have revolved around questions such as: whether we should aim to make point predictions; whether we have to actually build a simulation, rather than just a static model, to be 'doing' System Dynamics; and what types of validation are appropriate and how quantitative they should be. There are also different schools of thought on how the method should be used, either as a participatory tool, or as a purely analytical tool. All of these sorts of debates are common in simulation methods, though they do appear to be had rather more forcefully in this community as opposed to others. We believe there should be room for all the different ways of doing System Dynamics to exist under the one umbrella. Modelling should be driven by purpose, and as long as there are different purposes behind System Dynamics projects, there will be justifcations for doing it in different ways; with or without a simulation, with stakeholders or as a highly academic endeavour; with modern quantitative validation methods, or with more qualitative and heuristic validation.

#### Getting Started with System Dynamics

Because System Dynamics is somewhat more formal than some of the other methods in this book, we are little more cautious than our usual 'dive in and see what happens' approach to getting started. The benefts of doing some more reading, and maybe even attending a course, are likely to be high for this method.

Perhaps, the most obvious place to start is the System Dynamics Society. The society was setup in 1983 with Forrester as president and has continues to be the main voice and organising force in the feld. Its journal— System Dynamics Review—is one of the main places to fnd current work using the method, and its annual conference is worth considering too. Its website is easy to get lost in, has a plethora of resources, and can point you to up-to-date information on events and training, including an online course catalogue, readings, and videos.

We would recommend looking at some of the following further reading and resources:

start on more academic and detailed pieces. John Sterman's introduction is an excellent, if very strong, description of some of the history of the feld and a call to arms to improve practice and link with other domains.

• For those who like to return to the classics of a feld, Industrial Dynamics by Jay Forrester (1968), and Business Dynamics by John Sterman (Sterman, 2000) are the obvious choices, and both still considered relevant to budding and experienced system dynamicists.

Despite the need to take a little more care with System Dynamics than some of the other methods in this book, we hope you are not discouraged. You can still easily have a play with some of the software above or online models such as isee's COVID-19 simulator (https://exchange.iseesystems.com/public/isee/covid-19-simulator). Dip your toe in and see how it takes you. The joy of producing some dynamic output can really capture your imagination. It is also worth emphasising, it is fne to be self-taught in more formal methods like System Dynamics. Most researchers and practitioners are. Don't be put off by the fact you are not going to enrol on a masters or PhD programme in System Dynamics. If nothing else, giving System Dynamics a go will help you to understand what it means to 'think dynamically'. In our experience, this is a rare, diffcult, and valuable skill.

#### References

Forrester, J. W. (1968). *Industrial dynamics*. Productivity Press.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# What Data and Evidence Can You Build System Maps From?

**Abstract** This chapter takes a step back from individual systems mapping methods and considers what evidence and data we might use to underpin the design of system maps. It presents four broad types—data from participatory processes, qualitative data, existing evidence, and quantitative data—and outlines the pros and cons of each, considers how you can use them, and makes a call, ideally, for using them in combination and in creative ways.

**Keywords** Systems mapping • Information • Evidence • Data

In each of the methods chapters in this book we have considered how exactly you can use a method; what things you need to do, what you need to collect, how you create a map or model, and do analysis. We have tried to be agnostic throughout about whether the use of methods should be based on quantitative data, formal evidence, qualitative data, or participatory workshops and processes (though some bias towards participatory processes may have slipped in since this is what we are most experienced in). In practice, any of these approaches can be used, and often are combined and overlap with one another.

The fact that these types of information for building maps overlap means it is worth trying to be clear about what they are. Figure 9.1 attempts to do this in a simple way. It roughly defnes the four types. We can see some of the overlaps; participatory processes can also collect

**Fig. 9.1** Types of information for building system maps, and their overlaps. Source: authors' creation

qualitative data, and that evidence can be based on quantitative and/or qualitative data. Arguably, evidence can also be based on participatory processes, but it is rarely framed in this way.

This chapter takes a step back and seeks to refect on the question of what types of data and evidence we can use to build maps, to identify to some of the key pros and cons of each, and to consider how we might go about using them. Chapter 10 goes into detail on how to run systems mapping workshops, so we won't cover that in this chapter. Rather, we frst put forward a defence of the, sometimes critiqued, use of stakeholder opinion to build maps in a participatory mode. We follow this with a more practical consideration of how we can use more traditional data (qualitative and quantitative) and existing evidence to develop maps, including issues we need to take care around. We conclude with a few remarks on combining different types of data and evidence, and on the question of how to choose methods within different data availability contexts.

#### Defending the Use of a Participatory Process to Build and Use Your Map

One of the most common questions we get when running workshops or presenting on systems mapping is 'what evidence do you use to back up the model?' or, the closely related, 'how do you validate the model?' These are important questions, but questions that people from diverse backgrounds, and who have used different modelling types, approach in a variety of ways. Even the technical term 'validation' can have different meanings, nuances, and interpretations in different modelling domains. It is not always clear whether people think qualitative and workshopgenerated data can be used to build a system map, but when asked, we often assume these types of question are based on a belief that they cannot, or that quantitative data would be preferable. The assumption appears to be that we need quantitative data or scientifc evidence to trust or draw value from our maps, from our models. This refects a strong belief held by some researchers that only quantitatively validated models, of any type, are valid or useful.

We must be upfront about a fundamental belief that many systems mapping practitioners have. That is, when working in a genuine complex adaptive system, it is highly unlikely we will have access to the breadth or depth of quantitative data or evidence we need to formally validate every node and edge in a system map, or to validate simulation outputs from something like System Dynamics. Even where we do have some data or evidence, it is likely to be patchy, with systematic reasons determining the areas we have data and evidence for, and those that we do not. The reasons for absent or patchy evidence are multiple. Many of the important components in a system are social or behavioural; they are actual practice on the ground, peoples' perceptions, tacit, and local knowledge determine what they do. Data is not generally collected on many of these things; indeed it is diffcult and expensive to do so. Moreover, the amount of time needed to collect data to validate on the ground knowledge is prohibitive in contexts in which decisions need to be made with reasonable haste, that is, in policy-making contexts. Genuine complex adaptive systems are also open, causal connections may come from many different domains, and we cannot always determine in advance what is relevant. In these systems, things are also always changing; focusing on onerous data collection may make us reticent to update maps with new knowledge. Therefore, the task of validating a model based on quantitative data or evidence will always be an uphill battle, if possible at all.

This should not mean we abandon any hope of using systems mapping, or that we build narrow models that only include things we have quantitative data or evidence on. Indeed, many of the methods in this book work best, relative to other methods, in data-poor contexts. Vitally, we believe there is still huge potential value in maps that are not underpinned directly by quantitative evidence. There are many situations in which we need to make decisions quickly, on the ground, in data-poor situations, but in which we think it is likely that system interconnections are present and likely to be important. Thus, we still want to think through system interconnections more thoroughly, even without data. The pragmatic response here is to use participatory approaches to illustrate that system effects might indeed be important and to help us both raise awareness of this and start to think things through. We are likely to make better decisions with a map like this than without, so it is still worth doing. We should also note, these methods have an important function in generating new questions, for example, what sort of systems effects might be present, and might we need to take account of? How do we adjust our strategies to take account of these possibilities? What extra data do we need to gather to see what is actually happening? These are often hugely valuable and may directly inform new data collection processes.

So, rather than unfairly critique or abandon systems mapping efforts in data-poor contexts, we should proceed with the right amount of caution, and when working in a participatory mode, with stakeholders in the system, emphasise the value of their beliefs in constructing useful models. More than simply emphasising the value of participatory models, in contexts where there is long-standing acceptance and deference to the idealised standards of what science is (i.e. falsifability, empiricism, etc), we may need to be robust and ambitious in our advocacy of these forms of information.

The main sources of value in using maps and models built in participatory ways are as discussion and thinking tools. In reality, all models ultimately have this purpose—they are there to improve the quality of our thinking and discussion—even if we become overly focused on forecasting or related pursuits (note, we do not intend to dismiss forecasting based on validated models, as an activity, but rather to make clear, it is not the only thing we can do with models).

Maps built in a participatory fashion help us to surface and explore peoples' mental models of a system or issue. By building them together, we surface assumptions and beliefs as a group that otherwise might be hidden or left undiscussed. The maps become 'boundary objects' (see Star & Griesemer, 1989) around which stakeholders and researchers can learn. They can help build consensus and capacity to make decisions around an issue, but also to fnd the places where disagreements are, and help us work through these and preserve ideas and represent both sides, if they are not to be resolved. Maps, and the researchers working on them, become 'interested amateurs' (Dennett, 2014; Johnson, 2015) in the system at hand; actors or objects that can be critiqued and improved by participants without the need to offend other stakeholders and their opinions. These types of use and value of participatory maps can also be developed from their analysis and outputs, not just their construction; analysis should always be seen as a participatory and iterative activity too.

In any applied situation, we need to fully understand the context and needs of stakeholders, users, and clients of our systems mapping research. These, along with our purpose and aims, will shape much of our work. A deep understanding of these contexts and needs is only likely to come from serious and iterative interaction with these people. This means, even where we are building maps from data and evidence, and perhaps are enjoying the opportunity to 'geek-out' on more formal methods, we still need to place this in a human context of use and value in which there will be some element of participation in a map's use, even if we don't really acknowledge it.

This is as far as we will go into the debate about the validity and value of participatory models. Others have covered this ground before (though these debates are often found in discipline-specifc spaces, so more general lessons can be missed), for example, Voinov and Bousquet (2010), in an infuential paper, outline and defne core concepts for modelling with stakeholders, provide a detailed defence of drawing value from the process of modelling, rather than the results, and identify principles for modelling with stakeholders; Voinov et al. (2016) update and build on this, with a review on the topics covered in 2010; Hurlbert and Gupta (2015) consider when participation can be useful in research more generally, and what it can achieve in different contexts, many of these arguments apply to participatory modelling too; Prell et al. (2007) provide an in-depth consideration of why we might use participatory modelling, and how we can do it; fnally, Voinov et al. (2018) provide a detailed assessment of the process of choosing different participatory modelling approaches. If you are going to be using systems mapping, or any modelling, in a participatory mode, we suggest you become familiar with some of this literature and the arguments in it.

#### Using Qualitative Data to Build Your Map

It is not uncommon to see system maps that have been built based on qualitative data. That is, data which may have been collected from interviews or focus groups, but where the map was not built during that interaction, but rather based on the recording, transcript, or analysis of it. In this approach, the textual data is converted into the boxes and connections of a systems map by the researcher. We are looking for assertions of what factors exist and how they are connected in the data. These assertions may be put quite simply (i.e. an interview says X is caused by Y) or may need to be extracted from participants' descriptions and narratives of processes and actors within a system.

One of the key advantages of using qualitative data is that it is typically both rich and broad. It can go into a high level of detail and description about a system but can also easily cover the full spectrum of relevant issues and domains in a system. This means that in theory its coverage of a system can be both broad and detailed, though probably only where we have a lot of data. A second key advantage, similar to participatory processes, is that if we are collecting the data ourselves in an interview or focus group, we can adapt our questions and prompts in ways we see ft, guiding the collection process so that it meets our mapping needs.

There are disadvantages to using qualitative data to build system maps. Foremost, it puts a lot of power and responsibility in the judgement of the researcher or modeller to translate data into a map. This is rarely a straightforward process, especially when we are using data that was already collected or was collected with additional purposes in mind. There will be many dozens of decisions about how to create factors and connections that meet the 'rules' of the method while also refecting what was said. There is also often disagreement or contradictions in what different stakeholders have said in interviews. These will need to be resolved or preserved by the researcher. Finally, it often happens that one interviewee describes something that can give rise to one map, another interviewee gives us something else, and when we combine them we have a new map, which no individual actually described. The map we end up with is a composite that no stakeholder has described or had the chance to react to and comment on. This is not necessarily a problem, but is an important point to refect on; is this map valid, does it refect the mental models of our participants?

There are several software options that can help us develop system maps from qualitative data. Almost all the software we mention in this book can be used to build a map, but those which allow us to connect to or tag qualitative data for a map in useful ways are what are of value here. Below, we describe three well-used qualitative data analysis software packages, and one purpose-built application, which allow us to do this. However, there are important constraints on their functionality, particularly around exporting maps in usable data format, and none are free to use.

qualitative data. It is still in development, but a beta version is available. We have not used it ourselves, but it appears to have all the functionality we could want for developing, analysing, and exporting maps. It is not free, there is a free trial, but prices start at £490 a year; there is an R package forthcoming.

#### Using Existing Evidence to Build Your Map

Before we consider quantitative data, it is worth refecting on how we might use existing evidence to inform a map. By 'existing evidence' we mean any study or analysis which has already been done based on any type of data. This might include peer-reviewed academic studies, or grey literature from reputable sources. We would need to defne some inclusion criteria, but we would expect that it would include similar sources to those which make it into systematic reviews, meta-analyses, or rapid evidence reviews. In this mode, we are in effect doing a systematic review or evidence review focused on building a causal description of a system.

One of the main advantages of doing this will be that we have strong evidential support for the map we create. Depending on how strict the inclusion criteria we set are, we will be able to make pretty strong assertions about the validity of our map if it is backed up by peer-reviewed studies. Given the formalism of the studies and analysis we will likely use, and the pervasive use of concepts such as variables and causal relationships, there will also be less of a need for a modeller or researcher to make lots of decisions in translating individual pieces of evidence into nodes and edges in a map; this conversion should be more straightforward than the equivalent process for qualitative data.

However, there will still be some researcher judgement involved, and again the issue of combining pieces of evidence which perhaps contradict each other. More important though, will be the issue of coverage of a system. In many systems, there simply will not be much relevant evidence we can use. Where there is evidence there will likely be systematic reasons for what is covered and what is not, with corresponding risks of systematic bias in the resulting map. We will need to be careful about areas of a system that are not covered by existing studies, and how we can account for them. This means this approach is only really applicable in domains where there is a strong tradition of evidence gathering in formal studies of all aspects of the system, or where we are only hoping to use evidence to inform part of a map, not all of it.

The process of using evidence in this way will start with doing the early stages of a more traditional evidence review or systematic review. We recommend you look at some of the guidance for using these approaches in the domains in which you are working and think about how you could use them to gather and classify evidence in your system. For systematic reviews, Petticrew and Roberts (2005) is accessible and thorough; for rapid evidence reviews, you may fnd Crawford et al. (2015) useful, though it is applied to health care, so may need translating for other contexts; fnally, for evidence mapping, try O'Leary et al. (2017) for inspiration. You will likely not need to complete the full process for these approaches, but just do suffcient to collect and classify the evidence relevant to you. From here, the process will involve translating each piece of evidence into the nodes and edges it can underpin and adding these to your map. As you progress it will likely be useful to have a version of your map which is annotated in some way to record and visualise what evidence is underpinning what bits. You might also want to use this in the analysis of the map, for example, to choose your focus (perhaps on bits with less evidence).

#### Using Quantitative Data to Build Your Map

There are two different ways to use quantitative data directly in building your map. Both rely on the idea that the variables we have in a dataset are appropriate to use directly as the boxes in a system map. This is not a silly assumption but should be checked and thought through—you may want to consider whether the variables are at similar scales and whether there are likely to be lots of important factors not present in your data.

In the frst mode of using quantitative data, you may have data of any quantitative form (e.g. time series with data points for multiple variables through time, or cross-sectional with only data at only one time point) that allows you to do analysis on the statistical and/or causal relationship(s) between variables. This will likely use traditional statistical approaches (e.g. regression) or causal inference methods (e.g. difference-indifferences—see Cunningham, 2021, for an introduction) to focus on which variables we might fnd an association or causal relationship between. When you fnd a relationship between variables, you can add a connection between them. Depending on the method you use, you may be able to use the analysis to tell you the direction of the arrow, but in many cases, you will have to use other knowledge or theory, or leave the connection undirected. You will need to decide and use a threshold at which point a relationship is 'signifcant' enough to draw a connection between two variables.

In a second mode, if you have time series data across a wide range of variables you think may be relevant in your system, you can analyse these as a whole to extract causal networks directly using one or several of a variety of methods for 'estimating networks'. These methods rely on a range of measures, such as 'conditional independence'. This is an important but tricky concept; take, for example, two variables, a child's height and the number of words they know. These may appear to be related, but it is actually the child's age which has a strong infuence on them both; they are not causally related. So, a simple two-variable analysis may make them appear related, but if we include age, we will conclude they are 'conditionally independent'. This process of looking at two variables, considering others, is useful for constructing networks from data. This mode of using quantitative data is less widespread, and still in its infancy in the social sciences, but has the potential to be powerful and quick to use.

The main advantage of using quantitative data in either of these ways is that we have a direct and fully transparent connection between the map and the information we have based it on. At the 'information-to-map' stage of the process, there is the least amount of researcher judgement required compared to other modes. We have conducted the analysis, perhaps collected the data, and do not have to make much, if any, interpretation to convert quantitative data to nodes and edges. Nodes will be the variables we have in our data, and connections will be drawn where our analysis meets some threshold (likely defned using existing norms and standards) in suggesting some association or causal relationship. However, we must not forget that a lot of researcher judgement is involved earlier in the process when we curate datasets and choose methods. Quantitative datasets are often treated as objective truth, but we must always keep in mind they are constructed and collected based on judgements, can contain errors, and offer only a snapshot of some part of a system.

The most salient disadvantages, or risks, to using quantitative data to directly inform maps are similar to those for existing evidence. It restricts our map to only those things which are both quantifable and which we have data on. As we have repeatedly stated, this is often a serious restriction; we do not typically have a wealth of (good or even usable) data on a system, and where we do have it, there will be biases in what it includes and excludes. When using quantitative data, we must have ways to account for the aspects of a system for which data is diffcult or impossible to collect and use. We should also be aware of the choices we make in what system to look at, when these are driven, at least in part, by what data we think is available. We must also be careful with our language and describe the arrows in our map using the language that the statistical or causal inference method uses. So, for example, if we use traditional statistical method, which does not allow us to talk about cause, we should only describe connections as showing relationships or associations, rather than causes.

Health warnings noted, let's think about how you might actually use quantitative data. To underpin the inclusion of individual nodes and connections, you will frst need to get hold of the data, and then use one, or several, of the many statistical and causal inference methods that exist. There are dozens, if not hundreds, of these methods (too many to mention here). They can be grouped in different ways, from those that only assert association, such as standard linear regression, through to causal inference methods such as difference-in-differences and instrumental variables (again, see Cunningham, 2021, for an introduction). Methods can also be grouped by the types of data they can be used on. If you have cross-sectional data, then 'structural equation models' are one of the most obvious approaches to use; there is a range of methods within this broad umbrella, for an introduction we recommend the introductory materials on the UK National Centre for Research Methods website—https:// www.ncrm.ac.uk/resources/online/all/?id=10416. If you have time series data, then 'Granger causality' approaches are one of the most popular (Granger, 1980). Finally, if you have panel data (i.e. longitudinal data where the observations are from the same subjects in each time period), you can use causal inference methods such as difference-in-differences.

It is not within the remit of this book, nor do we have space, to introduce these methods. If you are not familiar with them already, we strongly recommend you develop a basic understanding of the range of methods, their pros and cons, and then dive deeper into the methods you think you might want to use. To help you start on this learning journey, we recommend the following: Cunningham (2021) for an accessible introduction, Pearl et al. (2016) for a more technical but short introduction, and Peters et al. (2017) for a more traditional textbook. Once you have the data and the method(s), it is a simple task to create your map (i.e. draw edges between nodes) when you fnd casual relationships. You can do this stepby-step on each two-variable set you have, slowly building the map up.

If you are lucky enough to have lots of time series data relevant to your system, there are some emerging methods which make the process more streamlined. These methods are referred to with different terms, but most common are 'network estimation' or 'causal discovery'. The frst step, again, will be getting hold of data. You will need to collect as much time series data as possible for your system. You should aim to collect data on as many domains as possible and look to maximise the number of time periods covered by the data. The methods themselves include basic correlation thresholding approaches, Granger causality approaches, statistical structure learning approaches including causal graphical models (i.e. Bayesian networks) and structural causal models, inner composition alignment, and cross-convergence mapping, to name a few. They all have different pros and cons revolving around what data they can use, what they can assert about causality, and what types of maps they give you (i.e. directed or not, weighted or not, cyclic or acyclic). Ospina-Forero et al. (2020) provide an excellent overview of these and other methods in the context of the sustainable development goals (SDGs). SDGs are an interesting way to frame systems because the scale of measurement (typically national) and good data collection around them mean they are often amenable to this type of analysis. Systems which are appropriate to defne at a national level, or other scales at which there is lots of data collected and available, may be one of the most appropriate to develop quantitative data-driven system maps because of this data availability. Most of these methods will produce maps directly for you. The 'fnal' task thus becomes combining these and/or converting them into a form compliant with whichever systems mapping method you are hoping to use.

#### Using Different Types of Data and Evidence in Practice

Though we have outlined the use of different types of data and evidence individually in turn, we will often want to combine them. Indeed, given enough time and resources, it is hard to think of a reason why we would not want to combine them wherever possible. In a similar fashion to the combination of methods, as described in Chap. 11, combining data sources and evidence will help us approach a topic more holistically and give us helpful points around which to cross-compare, triangulate, and iterate. Different skills are needed to use different types of information, and this should not be underestimated as a barrier. Using quantitative data is technically demanding, whereas running participatory processes requires strong facilitation and communication skills.

Perhaps the most obvious combination of types of information is to develop a map in participatory mode, and then refne it with further information from different data sources and existing evidence (as is sometimes done with Bayesian Belief Networks). These could be used to annotate a map, validate it, or to inform quantifcation decisions (i.e. what the conditional probabilities are in a Bayesian Belief Network, or what the equations are in a System Dynamics model). The second use we see as holding most value is to use qualitative data and participatory processes to address the potential gaps in quantitative data and existing evidence. This replicates the logic of much mixed methods research in using the strengths of approaches to covers the weaknesses of others. Combining sources in any way will involve additional work in map visualisation and may create parallel streams in your analysis and use of maps.

As we often do, we want to fnish with a plea for creativity. It is tempting to think of different types of data working better with certain methods. For example, you might think quantitative data will work best with Bayesian Belief Networks or System Dynamics because of their quantitative and more formal approach. However, we believe it is important to keep an open mind here; any of the methods can, and have, been used with different data sources, evidence, and processes. Moreover, some of the room for innovation, and potential new insights, may be in more unexpected and creative combinations. It may be interesting to ask, what might a Rich Picture of quantitative data look like? Or, how might a Theory of Change based on evidence look different to a Theory of Change from the mental models of a policy team? Any combination of method and information source is valid, as long as we are aware of the potential gaps and omissions in the information we use, and either address them directly, or adjust our aims and claims accordingly.

#### References

Crawford, C., Boyd, C., Jain, S., Khorsan, R., & Jonas, W. (2015). Rapid Evidence Assessment of the Literature (REAL©): Streamlining the systematic review process and creating utility for evidence-based health care. *BMC Research Notes, 8*(1), 631–640.

Cunningham, S. (2021). *Causal inference: The mixtape*. Yale University Press.

Dennett, D. (2014). *Intuition pumps and other tools for thinking*. Penguin.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

CHAPTER 10

# Running Systems Mapping Workshops

**Abstract** This chapter explores the practicalities of organising and facilitating systems mapping workshops. It considers how you should plan them; how to choose venues, materials, and technology; how to record them; and how to improve your facilitation skills. It provides readers with guidance for dealing with some common issues, such as power dynamics, disagreement, confusion, disengagement, and facilitator burnout, and points to useful resources for specifc methods.

**Keywords** Systems mapping • Workshops • Stakeholders • Facilitation

Mapping workshops are key milestones in any systems mapping project. In participatory projects they are vital spaces for engagement, discussion, and the creation of maps. Workshops will inform and energise the entire project. They are moments around which participatory projects can live or die. Everyone involved, including participants, will want them to be a success. This means they feel like high-pressure moments for researchers. Organising and facilitating workshops is also a skill in its own right; it is diffcult to do. It takes thought and practice before you can expect to run workshops well, and to not feel exhausted by the process.

In this chapter, we will try to shed some light on how to plan and run successful workshops. Much of the craft of running workshops relies on tacit knowledge and subtle inter-personal and communication skills, so we can't cover everything. But we will be doggedly practical and refect on our own experiences. The style of this chapter is a little different to others; there are so many things to think about that we have used a more bulletpoint-heavy structure to help you use this chapter like a checklist in your planning.

It is worth reminding ourselves that individual workshops are always part of a wider project process. You can, and should, take care in designing this process. Your purpose, and the needs of users and stakeholders, will drive the initial process design, and the process may need to change as the project develops. Considerations around process design for each method are discussed in individual method chapters, and we have developed our own practical guide for Participatory Systems Mapping processes in Penn and Barbrook-Johnson (2022).

#### Planning Workshops

It is possible to over-plan and over-think workshops and to stress yourself and others needlessly. A lot of thinking and adjusting can be done during the workshop (especially as you become more experienced), so try to avoid the urge to plan everything in microscopic detail. That said, it is vital to have a plan for workshops and the wider process they ft into, and to develop this with users and stakeholders. Key things to plan are:

will be busy. If there are other relevant processes, events, or publications being launched, think about how you would like to relate to them; should you go before them and try to inform them, or do you want to go after and use them as inputs?

• **Have a time-plan document**: the key document you will create for planning will be a time-plan of the workshop. We recommend having a detailed but fexible time-plan. If you have a three-hour session, you probably want to know what you are doing in each ffteenminute section (sections may last longer than ffteen minutes but use it as your 'unit of time-planning'), if you have a full day, then planning in thirty- or sixty-minute chunks is suffcient. The point is not to have a rigid plan you must stick to but rather to have thought through all the things you want to do and to be realistic about what can ft in your time. A written plan is an invaluable resource during a session, to check progress, adjust thinking, and ensure you get done what you need to. We normally combine this time-plan with all the prompts and questions you plan on using.

#### Venue, Materials, and Technology for Workshops

Having the right venue, materials, and technology can unlock all sorts of potential problems and transform a low-energy workshop into a major success. Things to consider include:

right size, and chairs which can be moved aside easily. Beware many modern offce tables have holes for cables, or power sockets built it, these can easily get in the way, when all you want is a good-sized fat surface. Discuss these needs with your venue ahead of time and make sure you have time during the day to set up the room if needed.

to digitise maps (i.e. enter them into our software of choice for that project) and then bring large printouts of these (A1 or A0 paper size). You can then overlay a large piece of transparent whiteboard paper and continue map building with pens and post-its. Having the actual original paper and post-its to hand can be useful too, to refer to, but we don't tend to use this to continue map development directly as paper gets tatty and things inevitably move and fall off.

#### Facilitating Discussion

The actual act of facilitating discussion between people is based on a range of tacit and subtle inter-personal skills. You will have many of these skills already, but they will take time and practice to further develop and refne for the context of mapping workshops. On the upside, these skills are easily transferable; if you have facilitated other types of workshops, or conducted interviews, then these skills will stand you in good stead for systems mapping workshops. Once you have run a few workshops, you will feel more comfortable with other types of facilitation and public speaking. You can improve your facilitation by:

these clear in your mind beforehand so you can answer any questions about the method, but more importantly so you can focus on facilitation rather than technicalities.

adjust if needed. Participants will need breaks too. Have a rough idea of when breaks will be, but allow a little bit of fex so that you don't stop at very productive moments, or feel the need to go on when energy is low or you have reached a natural breakpoint.

• **Working in pairs**: working in pairs can be a useful way to make facilitation easier. For any given section of the workshop, you can assign roles, one person can be in active facilitation mode, and the other can be in a participant observer role, watching and refecting, but also contributing when needed. You can then switch roles during different sections. Refecting on how things are going, and managing energy levels, become much easier with someone else too. However, beware of working with too many people. We have found facilitation can get a bit muddled if there are maybe four or fve people who participants perceive as facilitators. Side conversations can pop up, which we normally hope to avoid, or people can go 'offmessage' if the purpose is not simple and clear.

#### Capturing and Recording Workshops

It is vital, but diffcult, to capture the discussion during a workshop. The map you create itself is not the only output. The discussions are equally valuable, and it can be useful to refer to these later. You have two basic options for capturing a session: you can either take an audio (and maybe even video) recording, and then transcribe or make notes re-watching it; or you can take written notes. We normally opt for the latter and have a dedicated note-taker present for the workshop. We ask them to take as detailed notes as possible, and then we look at these soon after the session and write them up into something more formal, which we can refer to in the future. If you wait more than 24 hours to write up notes, we fnd it is increasingly diffcult to remember the nuances of the discussions. We also take photos of, and keep, all the fipchart notes made, and post-its which don't make it into the fnal map.

Using an audio recording removes the needs for a note-taker and gives you a more accurate record of the session, but creates two potential issues. First, you may inhibit conversation if people feel they are 'on the record'. Second, you will have a signifcant task in converting the recording into a transcript or notes. Transcription can easily take fve or six times the length of the recording when there are multiple people involved.

Lastly, if you are physically drawing your map, we highly recommend you take photos of the map throughout the process. Photos of the map at the end will be vital to help you digitise (i.e. input into whatever software you want to use), but photos during will also help you refect on the process, how the map developed, and maybe even jog your memory of the discussion.

#### Post-workshop

After a workshop you may just want to collapse in a heap and have a sleep! You should take some time to rest if the process is tiring, which it often is. However, it is also important to take advantage of the energy and raised interest a workshop will often generate. As well as processing notes and photos, and starting the process of digitising maps, you should contact participants to thank them for their time, explain what will happen next, and invite them for bilateral conversations should they want them. We have found that participants are often interested in following up one-toone, and these can be some of the most useful spaces to develop ideas for analysis and use of maps. A de-brief with your team will also likely be useful; make some dedicated time to refect on what worked and what didn't, how you might change future workshops, or the project design.

#### Common Issues

Workshops are not always all sweetness and light, there can be thorny issues you will need to resolve. The most important we have come across are the following:

• **Diffcult power dynamics**: there are all sorts of group dynamics that can emerge during a workshop, or that can exist from previous interactions. You may not be aware of many of these, and they rarely cause serious issues. However, when there are uneven power dynamics in a group, it can undermine a workshop. You may fnd that the most senior or powerful person in a group dominates, and the mapping just refects their views; or you may fnd that even when a powerful person does not want to dominate, others are still hesitant to say much in front of them. In either situation, you need to fnd ways to ensure the people who are less powerful, or more hesitant, can contribute. You may have to irritate the more powerful people to do this, but it is likely worth the risk. It can be helpful to position yourself as an outsider, or to justify your request using the method, rather than directly saying someone is dominating too much. If there are foreseeable issues here, with regards to say employees not likely to talk freely in front of a boss, or groups who have conficted relationships then it can be useful to organise separate workshops for different people or groups.

will think they can bend the 'rules' of the method if they want to. They may view the purpose of the session simply to have a discussion, and not see as much value in the actual map you are producing. Misunderstandings can go unnoticed until quite far into a workshop. You need to be patient with people, avoid positioning yourself as the expert and them as the lay people. It can be useful to use prompts and summaries as people are adding things to the map to ensure understanding is correct, rather than giving long presentations at the start, or giving handouts with the 'rules'. For example, if someone explains, 'I think X should be connected to Y with a negative arrow', you can prompt and summarise as it is drawn down by saying, 'OK, so that is because as X increases Y tends to go down?'. If that person actually meant 'X is bad for Y', this will quickly be realised, and you can explain again without the sense of you being patronising or needlessly concerned with the details.

fdence in the purpose or quality of the work. You might fnd you start to develop nerves before and during sessions, when previously you did not, or you may begin to feel unreasonably anxious about workshops. This is totally normal. Sometimes it is a good sign you are being refexive and thoughtful about your work. The simplest way to deal with this is, of course, to give yourself some time not working on, or thinking about, the project. A little time and refection can work wonders. You may also want to talk with team members about how you are feeling; they may be feeling the same way, and/or can offer reassurance.

#### Running Online Workshops

As we write this, in late 2021 in the UK, we have spent the last year and a half largely unable to run workshops in person due to the pandemic. No one knows with any certainty what will happen in the future, if and when meeting in groups in person for workshops will become the norm again. Though in 'normal' circumstances we would almost never want to run a workshop virtually, given the changing norms brought on by the pandemic, it is worth considering how we they can be facilitated online. We have spent the last eighteen months doing this with mixed success, here are some of our refections and tips:

parallel discussions. It is useful to increase the facilitator to participant ratio here, so that, for example, someone is monitoring the chat, another person leading the discussion, and another keeping an overview of the interface you are using. Group size should also be reduced to help reduce these problems. You may fnd it useful to make use of breakout rooms for certain discussion-heavy parts of a workshop whilst keeping other parts in the whole group. All of this creates more work, for example, in creating multiple versions of a map which must be discussed and merged.

you should make every effort to accommodate them, rather than excluding them. Ideally, build in a 'tech-ice-breaker' session at the start of a frst workshop in which people get to use the software in an introductory activity.

As the world has adapted to working online, there have been several useful discussions and guides developed for running workshops online, you may fnd the following helpful: Fowler (2020) gives some excellent broad-level suggestions; Khuri and Reed (2020) go into more detail on how to generate more interaction, and consider some of the generalpurpose software tools available, some of which include functionalities like qualitative systems mapping; fnally, Dialogue Matters (2020) provide more formal and detailed guidance, though technically it is targeted at online meetings rather than workshops, it provides another thorough list of considerations and tips.

#### Getting Started Yourself

Embarking on a workshop or series of workshops can be intimidating the frst time you are tasked with it. There are many resources available for general guidance on facilitation and running workshops; numerous books on the subject, and many blogs and web-guides. But to be honest, we have used few of these, and don't have any we would particularly recommend. Have a search yourself and see if there are any produced by organisations or authors working in domains relevant for you. We have learnt the most, and developed the skills to run workshops, from attending workshops ourselves, from discussing with colleagues with more experience, and through practice, both in 'test-workshops' with colleagues, friends, or family, and in the real thing.

We recommend organising a practice workshop(s) with some willing and friendly volunteers. This allows you to build skills and confdence and iron out problems in your approach. Give them a topic which makes sense for them, not necessarily the one you will use in the real thing. Ask them to treat you frmly and offer critique and questions, you want to be pushed a bit, not just have a fun few hours with friends.

There are guides for workshops for individual methods, which tend to focus less on skills of facilitation and communication, and more on the technical details of a project or workshop or refect on applying a method in a participatory mode. These are outlined in the individual methods chapters.

 We have found it a rewarding and interesting thing to do, you get to meet all sorts of different people, and you get a real feel for the value systems mapping can generate for people. You may be nervous, but you will enjoy it, promise!

#### References

www.fasttrackimpact.com/post/tips-and-tools-for-making-your-onlinemeetings-and-workshops-more-interactive

Penn, A. S., & Barbrook-Johnson, P. (2022). How to design a Participatory Systems Mapping process. CECAN toolkit. https://www.cecan.ac.uk/ resources/toolkits/

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/ by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# Comparing, Choosing, and Combining Systems Mapping Methods

**Abstract** This chapter explores (i) a detailed but usable comparison of the system mapping methods in this book; (ii) how we might choose which ones are appropriate given our and our project's needs, and the nature of the system we are working in; and (iii) how we might combine different sets of methods, both sequentially within a project, and in hybrid forms, to approach problems more holistically, and innovate methodologically.

**Keywords** Systems mapping • Method choice • Appropriateness

Making good comparisons and choosing which systems mapping methods to use is one of the most important things you will do in using systems mapping. This plays a big part in determining whether a participatory process is useful to stakeholders or not and whether a map can provide genuinely useful insights on the system of interest. Frequently, the choice also determines whether a good quality mapping process is even possible. Paradoxically, it is often done without much thought, if it is done at all. Often, we chose methods because they are what we know, have used before, or are asked for, based on rather random historical reasons. However, different methods have different strengths, weaknesses, and requirements, answer different sorts of questions, and work well in different situations. This makes it worthwhile broadening the range of methods in your toolkit and selecting them more thoughtfully.

Helping researchers and practitioners to make better comparisons and choices is one of our main motivations for this book. Comparisons need to be made on solid grounds, with a good understanding of the variety of methods, of how maps and models are built but also in how they are analysed and used. We also need to make decisions with an appreciation of the fexibility of many methods, which means they can sometimes accommodate requirements that on face value may seem diffcult to meet. This potentially infnite fexibility needs to be tempered by an understanding of where and when methods work best, and what their real strengths are. We want to be able to fnd the most appropriate method for our, and our stakeholders', needs, not just those that are adequate given several tweaks and adaptations.