Simpl framework, big impact!
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Lee, Joseph
Eisenstein, Jane
Drawing upon decades of simulation and higher education experience, the Alfred West Jr. Learning Lab at the Wharton School set out to create it’s own in-house developed simulation platform to accelerate our simulation delivery capabilities. However, what began as a white boarding session two years ago, morphed into the first of its kind open source simulation platform. Simpl will not only fuel the next generation of simulations written at the Wharton School, but it is our hope it will serve the foundation for countless other simulations at other schools around the world. Our open source platform puts tools in the hands of any developer who has the curiosity to create a simulation, tools that were previously unobtainable due to cost. The Learning Lab partnered with RevSys, and relying upon Python/Django and a slew of other open source tools, we were able to deliver our first simulation written in Simpl in November of 2017. Simpl‘s server side is built on Python/Django and open source WAMP technologies. On the front end, Simpl uses JavaScript frameworks including React, Redux, and Autobahn. Combined, these constitute a framework for developing single and multiplayer simulations. In this talk you will learn about simulations in the higher education space, how we developed and implemented Simpl, a demo of our first Simpl simulation, and why we think this Python/Django fueled simulation platform will have a seismic impact on the burgeoning edtech field. Tentative outline of talk: About the Learning Lab and simulations The barrier to develop simulations Why we chose to develop Simpl and make it open source Simpl design process Experience writing first Simpl simulation Technical deep dive Demo of first Simpl sim Where we are, next steps
 Great, thank you. So yeah, my name is Joe Lee. I'm the director of the Alfred West Junior Learning Lab at the Wharton School of the University of Pennsylvania. This is my colleague Jane Isastine, who's the lead developer of Simple, and we're really excited to share Simple with you today, which we recently open sourced, so we're really excited about that. I do want to preface this talk a little bit beforehand. You don't need to be a higher ed developer. You don't need to be a simulation developer or game developer to get something out of this talk. I want to give a big shout out to Kendall and Henry's talk. I don't know if they're here. There they are. From yesterday about Django Channels. If that was interesting to you, I think a lot of that content will resonate with you as well during this presentation. We're doing a lot of similar things, but we're building games with it, so I think it's really interesting and I hope you find it interesting as well. So before we get into the actual platform about the Learning Lab, you don't work at a higher ed university unless you have an overwrought mission statement. So this is ours. The Learning Lab's been around since 2001. We have 33 active simulations that we write in support, about two-thirds of which are ones that we have written. They really span tons of different languages. We have simulations written in Perl. We have simulations written in Python, Java, ColdFusion, and simulations written off the shelf like commercially available simulation platforms as well. And these simulations really run the gamut in terms of subject matter. We have simulations in the accounting space, the finance space, marketing, entrepreneurship. We try to touch at least every department at the school with the simulation. And the simulations themselves can range from 10-minute simulations that you use your mobile device for to simulations that are an entire class. We have a simulation that we run in late August for all incoming MBA students, which is 864 people and we run them through a simulation in the course of four days, which is really an intense experience, it's really awesome. And it really speaks to kind of the reach of the Learning Lab at the school. So we have over 20, our simulations are played over 20,000 times a year. And they're also played over 10,000 times a year around the country and the world. We have three simulations that are for sale on Harvard's, Harvard Business Publishing, which does a simulation platform. So simulations are available to any university that wants to use it. And it really speaks to kind of the reach of the lab and the impact of the lab has with a fairly small team. So Jane's really the only full-time developer that I have. We work with vendors, which I'll talk about later. But we don't have a lot of development resources. So simple, really one of the reasons we wanted to develop it was that it would help us build these games faster. We work with a lot of enthusiastic faculty that come to us and are like, oh, can you build this game? I want to use it in a month. And no, simple doesn't help you do it quite that fast, but it does help you do it faster. So that's one of the things that we're really excited about sharing. So what do we mean by a simulation? Well, for us, a simulation is a web-based multi-user application based on a mathematical model. So you could think of it like this. We collaborate with faculty, and we come up with, they either have a piece of research or a piece of pedagogy in their class, and they'll build a mathematical model around it. So that model can take the form of a 60-page PDF with all these different formula that, you know, we translate into Python, or they could be an Excel spreadsheet that they've been using for years that they want to build a game out of. So Jane will take that information, build a mathematical model, and then we'll build a game in Python, and we'll build a game around that. So that's what we talk about there when it comes to simulations. And business education is really a great fit for simulations. When you think about it, we have students that will be managing millions of dollars worth of assets in the real world, and it's really much better for them to fail in a simulation than it is to fail in the real world. And then kind of bringing that pedagogy home is one of the things that simulations are really good at, and that's why one of the reasons why we use it a lot at the school, and I think it could be used in a lot of other places that don't previously use simulations. So what are the benefits of a framework? Well, kind of the basic stuff. We don't need to start from scratch every time we build a game. When we build, a lot of our games have been built from scratch before, before we had a platform to build on. And those games could take anywhere from six months to a year to make because games are still really complicated things to make, and just keeping data in sync and all the plumbing required to make an interactive experience is not easy. And even with technology having, you know, grown dramatically in the last few years, that's still not an easy thing to do. Just kind of the talk yesterday and channels, but it still takes a lot of plumbing to make all that happen seamlessly, and our platform gives you that plumbing for you so you can concentrate on making the game. So making, you know, keeping data in sync between users and things like that are already done for you. So you just concentrate on building the game. So there are some existing platforms in the space that you can build games upon. There's a few open source ones that aren't really super robust, but they're out there. And then there's also commercial offerings, which a lot of our simulations are built. Again, there isn't a lot of players in this space, but there definitely are platforms out there that help you build games. But we're talking about licensing fees, northwards of, you know, five figures per year. They require, you know, a specific set of skills that just really aren't easily transferable. You need to be, if you're good at programming, it'll help you learn these platforms, but those aren't skills that you're going to be able to take to your next job. And it makes it really challenging for us to recruit and retain developers when they have to work on these platforms where they're really uncertain about what they can do next. And then it creates a lot of stratification in higher ed. You know, you have universities like Penn that can afford the resources to build these simulations, but a lot of other schools don't have those resources available. And we've done this, a version of this talk about four or five times now. And we've done it at other universities in the northeast. And you know, people come up to us afterwards and they're like, oh, wow, I've been using this pen and paper exercise in my class for two decades. It would be really cool if we could build like an interactive game around this. And there just isn't a lot of resources out there for them to do that. You know, I tell them it's not, it's not, simple doesn't make it easy to do that, you know, pun not intended. But it definitely helps get you much further down the road than you could previously. And Python is really prevalent in higher ed from the research angle and reachers perspective. So those Python, people with Python skill are out there and there's no reason why they can't take our platform, clone our repo, do a couple of tutorials and build a game in a fairly short period of time. So that's really exciting. So why do we want to build it? Well, kind of building on the points I just made, we want to control over the platform. We've run into a lot of issues just working within the confines of our existing tools before we built Simple. There are things we just couldn't do. And as our needs evolved, the platform, we actually, we ran out of functionality in those platforms to make them viable for us to build games in the future. So we wanted to build Simple, draw upon all the experience that we had and build a platform that was flexible enough to accommodate not only games that we built today, but also hopefully games that we build in the future as well. And we wanted to build a community around this. Like I said, there isn't a lot of players in this space, but I think there's a lot of untapped demand. There's a lot of awesome opportunities out there for people to work on simulations, build simulations, and they just don't have the tools for it. And I hope Simple gives them those tools and will help build that community around the tool, which will be really awesome. And we didn't want to be locked into the vendor. We have a lot of technical debt in the learning lab of those 33 simulations, maintaining them, and they're all different programming languages, and professors use them forever, like decades and decades, and they expect it to work forever. So building on this really, I think, really rock solid platform will help us in the future as well. So what did we start? So we started in the spring of 2016. Shout out to the RevSys folks. They were our vendor partner building Simple, Flavio, Frank, Steven, Jeff, they were all integral in us building this platform from scratch. Shout out to Sarah Tombs, who was my predecessor at the learning lab. It was her idea to build the platform to begin with, and her idea to open source it, so I won't be standing here giving this talk without her. So thanks, Sarah. So how did we start? Well, we did a lot of whiteboarding. So we took our games, and we, which have a lot of different use cases, and we kind of just mapped out what a model would look like that could accommodate all these games, and still be flexible enough for us to build upon, but not so opinionated that it would, you know, harm our flexibility going forward. So it was a lot of work on a lot of whiteboarding. The RevSys folks really brought in, you know, their perspective, and they really helped us build this model up from scratch, so their diverse viewpoints really helped us as well. So we launched our first simulation, which is the one that Jane will be sharing with you today, Rules of Engagement, which is a simulation that we do in a marketing class for about 300 people. We run it in the fall. We ran it in fall 2017. It may or may not have exploded during the due date for said simulation. I always, I've worked in higher ed for over 10 years now, and I'm still astonished by just how close the deadline students will wait to do an assignment. And we underestimated just the amount of load, and it just kind of blew up on our face. We were able to, you know, limp over the finish line and delivered for the professor. And despite all that, I still thought it was a success because, you know, we did it. We got there. We generated tons of very interesting data about the platform itself and how we wanted to use it and how we wanted it to evolve. So that was really cool. And then we basically had to sit down and look at the areas where it fell flat, and we had to rip out a lot of the plumbing and do a lot of refactoring. So it was really our goal to open source it in the first half of this year or late last year, and we just open sourced it now because of all the changes that we made, which really really makes it now a very robust platform. You know, one of the things that we had to do was move the entire platform, you know, to use asynchronous requests. We also built a profiling framework that actually helped us simulate, you know, the WebSocket 300 folks going through the, you know, the WebSockets at a, you know, at simulating a deadline scenario. So we were actually able to profile against our production server that had the old code, and then the development server that had the new code, actually see that the gains we were getting and we were on the right path in terms of refactoring all of that code. So we went open source, which is awesome. So we opened up the repos a few weeks ago. Our mayor may not be checking the GitHub traffic stats on the Games API to see who's cloned it. It's been at least two people, so it's really exciting. And maybe there'll be more than two people after this talk. We opened it up on GitHub under GPLv2, and really I couldn't be prouder of the team. RevSys, you know, Jane, and the entire learning lab team that really helped make this possible. It's over two years worth of work, and it was really awesome to see it now live and out in the wild. One of the great milestones we had last week is we had a developer who had a React background, which we'll get into, it was super valuable when looking at Simple. But they were able to clone all the repos, go through the tutorials, and in the span of about a week, we were able to make a rudimentary version of a simulation that took us over a year to make from scratch. So that was really a great milestone for us. Like, hey, this thing actually works, which you're in your little bubble of development, and you're like, oh, yeah, I think this thing works, but actually someone else says it works, too. It's like, oh, cool, it does work. That's really awesome. So we were really happy about that. So now I'm going to pass it over to Jane, who's going to go kind of more into the weeds of Simple and going to show you rules of engagement. Jane. Okay. All right. Thanks, Joe. Yeah, I'm Jane Eisenstein. I'm a senior application developer at the Wharton Learning Lab. I have been working for the Learning Lab for four and a half years, and I joined the team because it sounded like fun to create games that help people learn. And I also am really proud that we're able to take something that we developed for our internal use and offer it to the wider community. So I'm going to spend a couple of slides just kind of going over the concepts of what we mean by a game, because we've got some cool architecture, but Simple is really focused on helping us create these games. So we built it to be flexible and support the needs of these games. So as Joe had already covered, games are built around a mathematical model. User decisions go into the model. It cranks and is stepped through, and then it produces results. And this can happen iteratively. You know, new decisions, results over several periods. These periods with their associated input decisions and output results are grouped together into scenarios. Users, and by that I mean Django users, participate in a game by being assigned to a run of the game as either a leader or a player. So a run is an instance of a game for a particular group of users, and the way we use it is if the professor has a class or it has a section of a class and they want them to all be playing the game at the same time, like two o'clock on Friday the 13th, then they're going to put them in the same run, and so that they will have that same timing that the run gives them. Players in a run may be grouped into worlds they don't need to be, but often if they are grouped into worlds, that's just a team of players who typically either are collaborating together or competing against each other, but the decisions they put in of all the players goes into the model and the results come out and they're shared by everyone in the world. So depending on your UI, you could see every other player in the world. You could see all the decisions they had made, all the players had made. You could see all the results that came out, and it really is a UI decision about how you want to design your game. The thing about both runs and worlds is they're independent of each other. One run of a game is totally independent of another run of a game. It might have a different set of parameters. It will definitely have a different set of users, at least players, and worlds also are independent of each other. What goes on within a world is hidden from what goes on in other worlds within a run. All right. Players are often assigned roles when they come into a game and that defines what they're able to see and it also can be used to, may define the decisions they may make. And we'll see that with rules of engagement when we do the demo. Run interactions occur in phases, what is a typical pattern for us is it'll be a set up phase. Oh, wait a second. I need to. I didn't talk about leaders and players. I talked about leaders briefly, like teachers are primary leaders, so are admin people. A leader can log into a game and they can see multiple runs and they can see all the data about a run. When a player logs into a game, they just see information about their private scenarios or the one world that they belong to. And that is quite a focus. So again, going back to phases, set up phase, leaders would log in. There would be entering parameters. They might be adding players to the game. And a leader moves the run to play phase, that's when players can get in and they can start entering those decisions and seeing the results. And then when the play phase is over, a leader again will move the game to a debrief phase. And that's what I think really to me the secret sauce of a lot of what we do with the simulations at Wharton. Players run these models, they get experience using them, but then the professor has a debrief session and he looks at results from all the different players and all the different worlds within a run and he really can use that as a pedagogical tool. I got the word pedagogical in there. So that leads us to the simple model in the Django sense. And I believe all the concepts that I had mentioned in the previous slides appear as a model class except the run user one down over sort of midway down on the right. The run user is really a model that represents a many to many relationship between the Django users and runs within the simple database. So a user may be assigned to multiple runs either within a game or cross games and a run may have multiple users. But for any particular run, one user is associated with one run in a one to one relationship and they either are a leader or a player. And the other thing here is that I don't know if you can see it, but there's a lot of one to in relationships going down. So when the model is actually instantiated at runtime, you have a tree structure. Even though it looks from here like, oh gee, a scenario could be associated with the world or a scenario could be associated with the run user, that's not really what the platform supports. If a user, either a leader or a player wants to run the model in kind of a private environment, then they have their own private scenarios. But if the scenarios associated with the world are shared across all the members of the world. So this guarantees that every model instance has a single parent which allows you to kind of traverse down from any place up to the top of the game. And it allows you to pull out subtrees out of the model instances, which is really an advantage. And I'll just go to the next slide that talks about this. When we are running games, we are pulling all the data, the game data that's relevant to a particular user into the browser and storing it in a Redux state. And we really want to hone in and bring in only information that's relevant to the user and not have them getting updates from things that have nothing to do with them. So it's really cool that we can do this, but how does this work? And since this is a Django conference, when I'm going to focus a little bit more on the front-end service than we might do otherwise, a simple game is composed of a front-end service and a model service that are unique to that game. And it uses the database service that is shared across all the games. And in the front-end service, there is actually a very small Django application. And its responsibilities are authenticating users. And it is, once the user is authenticated, then the view for that Django app, well, it's not shown here, but the front-end will actually query the simple database to find out what runs the user is associated with and whether they're associated as a leader or a player. And based on that, it figures out what subtree of all the data instances for that game it should pull back and put into the React store. So once it's done that query, it hands it off to a single-page application and the rest of the UI logic is handled in React in Redux. And from then on, any time when the React wants to know something about the database or make changes in the database, it asks that it communicates with the model service over WAMP, which is Web Application Messaging Protocol, using WebSockets. And so all that communication goes through the model service to the database service. Any updates that happen in the database will trigger webhooks, which fire and go back to the model service, and which then may broadcast back to the subscribers for the particular portions of the data model. Let me see. What else I want to cover? Yeah, so the model service is, again, written by the developer. It's specific to the game, it implements the simulation model in Python, and it also creates custom classes with methods that can be accessed via WAMP, and it does that by subclassing off these scope objects that are defined in the simple model, the model service package. So when the model service starts up, it queries the database about all the active runs, it pulls in that information, it creates these scope objects that basically it creates a copy of the active runs of the game in memory. And that means when the browser just wants to refresh a page, that nobody has to hit the database. It just says, hey, tell me about that, you know, tell me about this portion of the data objects again, and then it can display it, which keeps the load off the shared database service. And so I think I've covered, to me, the primary things that you, as a developer, you need to know about the simple model service is it provides these scope classes that can be subclassed to create the business features that you need for your particular game, things like to submit a decision. You do that by creating a subclass of a scope object and say, when I submit a decision, you need to, you know, create a decision in the database. And also the web callbacks. And now I will go on and take a quick look at the technology stack. We went out the door with the 0.7 release using Python 3.6, Django 111. I'm not sure of the Django REST framework version. The web hooks are implemented using Thorn. The WAMP is implementing using cross bar IO on the Java side, Java side, on the Python side and Audubon on the JavaScript side. And again, we're using React and Redux to implement our single page application. It is now time for the live demo. Please send good thoughts. So, all right. This first part of the demo is running on my local machine. It is just to give you an idea of what goes on when you've logged in and how, what information is needed to pull the information into the Redux state that's specific to this user. So this stuff they got, so, but, okay, slow down. Rules of engagement is a tipbook. It's a multiplayer game. It has worlds. Each world comprises three players with three different roles. And then it has a leader. And the leader, as you can see here, can be looking at multiple runs. It's called classes, but there's a default run and a Django run. And they're in different phases. The default run is still in setup. And here is where someone can go in and change the start date and open date. There's a whole bunch of parameters that I don't think we're ever going to change, but it's nice to know that they could be changed. And that allows me to mention an important point. We have this flexible kind of generic model, but each game is going to have data that's specific to that game. And the way it's stored on the model is that the model classes all have a data property which is stored as JSON. So when I create a decision, the role of the decision, the name of the decision, that's all part of it. Those are all model properties. But when I want to say like, what the heck, the decision, you know, the values of the decision, that's all stored as JSON. So just when you, we were looking at those game settings, that's all JSON data that gets stored on the run. So when you log in, some of the queries are that for a leader, it looks for what runs the leader's associated with, and it creates topics that are at the run level. So that's model.run.run.1, model.colon.run.2. But for a player, again, because the player's associated with the world, we want to know stuff about that particular run user, such as what are their private scenarios. And we also want to know about what worlds they belong to, which is one. So the topics for a player are going to be, if I can get that queued up. Their topics are going to be, okay, I want to know anything that happens about the run user that represents this particular logged in user. And I want to know anything that happens about what goes on with the world that this run user's been assigned to. Now the thing that got me so excited about Simple early on, and I really didn't understand it for ages, is the AutoMagic property. My first blog about Simple was talking about its AutoMagic properties. So it's now time to open play. In fact, it is a bit late to open play. It was supposed to open at 11.30, but we had this silly talk we had to get through first. So now I'm going to move this run to play. And boom, suddenly this player gets to play. There's no polling involved. There's no pushing involved. It's fantastic. I get to create a new test. If I was imaginative, I'd call it something besides tests, but my adrenaline's going, so I'm not going to. I'll just put 2008 to show that I can do something that isn't already in my presets. So to back off a little bit and say something about this game, when a player comes in here, they're really simulating the Canadian Children's Serial Company case study. Each player, I'll go back to the dashboard. I got too excited about this. So my role is I'm playing the product manager for Kellogg, and there are two other product managers. The competition is post and Kellogg's. So my goal is to create a budget that's going to knock the socks off the competition, and their goal is to create budgets that are going to knock the socks off the competition. And they all have a different case history. They all have basically more or less resources. And the reason this game is called Rules of Engagement rather than Canadian Children's Serial Company is because you define the budget for the current period, but you also create rules that are going to set future budget. So this game is a little weird for us because it runs over many periods, but there's only one set of decisions, and those decisions affect the budget that is for the future periods, but you're not creating new decisions with each period. So I'm not going to mess with anything, but there's all sorts of options. You can keep it fixed, which is the default. You can react to the highest, the lowest by percentages above or below whatever they did the last period. And I'm going to just save that and run, but it doesn't want to save because I don't have a name. So I'm going to save this and run it. And I'm not going to show you how cool it is. And on the model service it's saying, oh, I ran this. But anyway, I can step it through multiple periods. And because everything was fixed, it's just everything's flatlined. But I can also say, OK, so I can create as a rules of engagement player. I can create multiple test scenarios. I can do what ifs. This is something the professors have been asking about for years, and we weren't able to give it to them with the previous simulation platform that we had. And then when I'm done, and I've figured out what my best deal is given my role, I can then say, OK, I'm going to submit this as my final decision. And basically I've done everything I have to do in the game as a player. So at some point, the time will be up, and it's going to be time for the professor to debrief the students on the results of their decisions. So we just moved it to debrief. If you see now, the user can no longer make corrections on their final decisions because the play period is over. And so if we go in here now, then we see the, it's only two worlds in this little test run. But we can say I want to increment it like five periods and step it forward just like we did with the test scenario. And there's a lot of other detail which I'm not going to dig into. But what I'm going to do now is switch over and trusting that the talk gods are with us. I just want to do a quick refresh. Well, no, it's good. This actually is our production server. And what we're seeing here on the top four rows are the runs from last year. And what we're seeing on the bottom four rows are the empty runs that have been set up for the coming for this semester, which we'll start running in the end of next week. So never mind. Never mind. So I'm going to go in here and say that when we first started telling the professor that the old version, because this is a re-implementation and it's an improvement on the previous version of the simulation, when we told him that basically the sands of time are running out on the old version and we were going to give him a new version, he said, well, I got to know that I can have 300 students log on at the same time and the system isn't going to choke. And I got to know that if I take one of these runs, which has 78 students, which means 26 worlds in it, and step at five periods, it'll finish in less than 10 seconds. So we were near there a year ago. And I don't know if we're going to be here this minute, but we are here more often than not this year. So here we go. This is like 26 worlds, all being step five periods. And I'm going to show the JavaScript because that is, let me see how long it took. Okay, we ran it in 6.76 seconds and it took a couple of seconds to refresh. And one of the things you got to know is that we do a lot of stuff automagically, but in order to get this response time, not necessarily on the server for running the model, but to get the refresh, we're not waiting for the cycle to go to the database and the web hook fires and then it comes back and filters back to the Redux store. And then the page would, that was like, that's why there are all these thrashing notices because there was a lot of thrashing and the refreshes would fail. So what we do is we run it, we scoop up a JSON, we serialize the data and we send it back to the front end and that's how we're doing the refresh. So now I'm going to switch back to Joe who's going to take us home. Here you are. There you are. That moment when you hit refresh on a live demo and to pray that it works that time, so that was awesome. Awesome, Jane. Thank you. So where are we now? So we're beginning work on our next simple simulation. It's already in development. We've written a model for it. It's another marketing simulation that we're going to be running with the idea of delivering it into the classroom for over 200 executive MBA students next fall. Really excited about that. The repos are available. They're open source, github.com, simple world. We have documentation. We have tutorials. We have a beautiful website, HTTP right there, simple.world. Shout out to Chad Whitman. He's not here today. He works at work computing. He did an amazing job on the website. It looks really like totally legit. And so it's really, really cool. And we have t-shirts, so that's really awesome too. Next up for us, we want to do this. This is the largest talk that we've done so far. We want to take this thing on the road, go to other places, maybe to PyCon. We'll see. But we also want to do workshops locally in the Philadelphia area at Penn. For those folks in the university at other schools, I might want to use this. So we're going to have some workshops there, just free workshops in the region. And maybe we'll have SimpleCon someday. We'll see. But we're really excited about the prospects. If you want to learn more about this, I think we're going to have a really few minutes for questions, but you can reach out to us, learninglab.worton.upn.edu. Is the email address for our team? The homepage is there. The github link is there. If you go on the webpage at the very bottom, there's a place to sign up for our mailing list. So we're going to be, as we build content out, as we make changes, as we have workshops, we'll be broadcasting that information via the mailing list. So I hope you got something out of this. It's a really exciting platform for us and a really big milestone for the learning lab. We really think this could be a pretty impactful project at higher ed, but also we're really excited to see folks, all the diverse folks here at JangoCon and beyond who can take it and maybe think about using it in ways that we haven't even thought of yet. So that's a really exciting thing about having an open source project. So thank you for your time. I think we have a few questions. We have time for a few questions, but yeah, thank you. In regards to your live demo today, so I noticed you played for one player and the other players had results as well. So how did that work and how do you deal with students who might not do their homework? Is that part of that? That's a great question. So when the students submit decisions, when they do some test scenarios, they're entering decisions for all three roles and they're kind of simulating what their opponents or what their other competitors might do. So when we provision the game and load it up, we assign folks to every role in the world. We create the worlds. And when the professor moves the game to debrief and is cranking forward those worlds, all the decisions for that world are being run at that time. If someone doesn't do it, the assignment, there is just default behavior that's in there to prevent the model service from blowing up. I was curious where you guys plan to take this next if you're going to keep it in the finance world or if there's other areas that you're hoping to break into. Yeah, I think, you know, so right now, we've given this talk at a few places at Penn. We've given this talk at a couple universities around the Philadelphia area. Business is a natural fit because of what I mentioned about simulations. But I think that there could be simulations built around a lot of other disciplines, non-business related, non-finance related. I could see this being really effective tool in the engineering world where simulations could be effective. I could see this in even liberal arts. I could see this at a lot of other places. I could also see it as well. We've talked to a few folks after our talks in the research world. So maybe using this in a research capacity in some way, I could see that happening as well. So really, we'll see what kind of traction it gets outside of business. But I'm pretty optimistic that we'll have some folks that are using this in other disciplines that we haven't seen before. All right, so thank you. We'll be around the rest of today. And again, if you have any other questions, feel free to email us. We'll share the slides out. Well, this talk will be online. So hopefully you enjoyed it. Thank you. doomed.
 Great, thank you.  So yeah, my name is Joe Lee.  I'm the director of the Alfred West Junior Learning Lab at the Wharton School of the  University of Pennsylvania.  This is my colleague Jane Isastine, who's the lead developer of Simple, and we're really  excited to share Simple with you today, which we recently open sourced, so we're really  excited about that.  I do want to preface this talk a little bit beforehand.  You don't need to be a higher ed developer.  You don't need to be a simulation developer or game developer to get something out of  this talk.  I want to give a big shout out to Kendall and Henry's talk.  I don't know if they're here.  There they are.  From yesterday about Django Channels.  If that was interesting to you, I think a lot of that content will resonate with you  as well during this presentation.  We're doing a lot of similar things, but we're building games with it, so I think it's really  interesting and I hope you find it interesting as well.  So before we get into the actual platform about the Learning Lab, you don't work at  a higher ed university unless you have an overwrought mission statement.  So this is ours.  The Learning Lab's been around since 2001.  We have 33 active simulations that we write in support, about two-thirds of which are  ones that we have written.  They really span tons of different languages.  We have simulations written in Perl.  We have simulations written in Python, Java, ColdFusion, and simulations written off the  shelf like commercially available simulation platforms as well.  And these simulations really run the gamut in terms of subject matter.  We have simulations in the accounting space, the finance space, marketing, entrepreneurship.  We try to touch at least every department at the school with the simulation.  And the simulations themselves can range from 10-minute simulations that you use your mobile  device for to simulations that are an entire class.  We have a simulation that we run in late August for all incoming MBA students, which is 864  people and we run them through a simulation in the course of four days, which is really  an intense experience, it's really awesome.  And it really speaks to kind of the reach of the Learning Lab at the school.  So we have over 20, our simulations are played over 20,000 times a year.  And they're also played over 10,000 times a year around the country and the world.  We have three simulations that are for sale on Harvard's, Harvard Business Publishing,  which does a simulation platform.  So simulations are available to any university that wants to use it.  And it really speaks to kind of the reach of the lab and the impact of the lab has with  a fairly small team.  So Jane's really the only full-time developer that I have.  We work with vendors, which I'll talk about later.  But we don't have a lot of development resources.  So simple, really one of the reasons we wanted to develop it was that it would help us build  these games faster.  We work with a lot of enthusiastic faculty that come to us and are like, oh, can you  build this game?  I want to use it in a month.  And no, simple doesn't help you do it quite that fast, but it does help you do it faster.  So that's one of the things that we're really excited about sharing.  So what do we mean by a simulation?  Well, for us, a simulation is a web-based multi-user application based on a mathematical  model.  So you could think of it like this.  We collaborate with faculty, and we come up with, they either have a piece of research  or a piece of pedagogy in their class, and they'll build a mathematical model around  it.  So that model can take the form of a 60-page PDF with all these different formula that,  you know, we translate into Python, or they could be an Excel spreadsheet that they've  been using for years that they want to build a game out of.  So Jane will take that information, build a mathematical model, and then we'll build  a game in Python, and we'll build a game around that.  So that's what we talk about there when it comes to simulations.  And business education is really a great fit for simulations.  When you think about it, we have students that will be managing millions of dollars worth  of assets in the real world, and it's really much better for them to fail in a simulation  than it is to fail in the real world.  And then kind of bringing that pedagogy home is one of the things that simulations are  really good at, and that's why one of the reasons why we use it a lot at the school,  and I think it could be used in a lot of other places that don't previously use simulations.  So what are the benefits of a framework?  Well, kind of the basic stuff.  We don't need to start from scratch every time we build a game.  When we build, a lot of our games have been built from scratch before, before we had a  platform to build on.  And those games could take anywhere from six months to a year to make because games are  still really complicated things to make, and just keeping data in sync and all the plumbing  required to make an interactive experience is not easy.  And even with technology having, you know, grown dramatically in the last few years,  that's still not an easy thing to do.  Just kind of the talk yesterday and channels, but it still takes a lot of plumbing to make  all that happen seamlessly, and our platform gives you that plumbing for you so you can  concentrate on making the game.  So making, you know, keeping data in sync between users and things like that are already  done for you.  So you just concentrate on building the game.  So there are some existing platforms in the space that you can build games upon.  There's a few open source ones that aren't really super robust, but they're out there.  And then there's also commercial offerings, which a lot of our simulations are built.  Again, there isn't a lot of players in this space, but there definitely are platforms  out there that help you build games.  But we're talking about licensing fees, northwards of, you know, five figures per year.  They require, you know, a specific set of skills that just really aren't easily transferable.  You need to be, if you're good at programming, it'll help you learn these platforms, but  those aren't skills that you're going to be able to take to your next job.  And it makes it really challenging for us to recruit and retain developers when they  have to work on these platforms where they're really uncertain about what they can do next.  And then it creates a lot of stratification in higher ed.  You know, you have universities like Penn that can afford the resources to build these  simulations, but a lot of other schools don't have those resources available.  And we've done this, a version of this talk about four or five times now.  And we've done it at other universities in the northeast.  And you know, people come up to us afterwards and they're like, oh, wow, I've been using  this pen and paper exercise in my class for two decades.  It would be really cool if we could build like an interactive game around this.  And there just isn't a lot of resources out there for them to do that.  You know, I tell them it's not, it's not, simple doesn't make it easy to do that, you  know, pun not intended.  But it definitely helps get you much further down the road than you could previously.  And Python is really prevalent in higher ed from the research angle and reachers perspective.  So those Python, people with Python skill are out there and there's no reason why they  can't take our platform, clone our repo, do a couple of tutorials and build a game  in a fairly short period of time.  So that's really exciting.  So why do we want to build it?  Well, kind of building on the points I just made, we want to control over the platform.  We've run into a lot of issues just working within the confines of our existing tools  before we built Simple.  There are things we just couldn't do.  And as our needs evolved, the platform, we actually, we ran out of functionality in those  platforms to make them viable for us to build games in the future.  So we wanted to build Simple, draw upon all the experience that we had and build a platform  that was flexible enough to accommodate not only games that we built today, but also hopefully  games that we build in the future as well.  And we wanted to build a community around this.  Like I said, there isn't a lot of players in this space, but I think there's a lot of  untapped demand.  There's a lot of awesome opportunities out there for people to work on simulations, build  simulations, and they just don't have the tools for it.  And I hope Simple gives them those tools and will help build that community around the  tool, which will be really awesome.  And we didn't want to be locked into the vendor.  We have a lot of technical debt in the learning lab of those 33 simulations, maintaining them,  and they're all different programming languages, and professors use them forever, like decades  and decades, and they expect it to work forever.  So building on this really, I think, really rock solid platform will help us in the future  as well.  So what did we start?  So we started in the spring of 2016.  Shout out to the RevSys folks.  They were our vendor partner building Simple, Flavio, Frank, Steven, Jeff, they were all  integral in us building this platform from scratch.  Shout out to Sarah Tombs, who was my predecessor at the learning lab.  It was her idea to build the platform to begin with, and her idea to open source it, so I  won't be standing here giving this talk without her.  So thanks, Sarah.  So how did we start?  Well, we did a lot of whiteboarding.  So we took our games, and we, which have a lot of different use cases, and we kind of  just mapped out what a model would look like that could accommodate all these games, and  still be flexible enough for us to build upon, but not so opinionated that it would, you  know, harm our flexibility going forward.  So it was a lot of work on a lot of whiteboarding.  The RevSys folks really brought in, you know, their perspective, and they really helped us  build this model up from scratch, so their diverse viewpoints really helped us as well.  So we launched our first simulation, which is the one that Jane will be sharing with  you today, Rules of Engagement, which is a simulation that we do in a marketing class  for about 300 people.  We run it in the fall.  We ran it in fall 2017.  It may or may not have exploded during the due date for said simulation.  I always, I've worked in higher ed for over 10 years now, and I'm still astonished by  just how close the deadline students will wait to do an assignment.  And we underestimated just the amount of load, and it just kind of blew up on our face.  We were able to, you know, limp over the finish line and delivered for the professor.  And despite all that, I still thought it was a success because, you know, we did it.  We got there.  We generated tons of very interesting data about the platform itself and how we wanted  to use it and how we wanted it to evolve.  So that was really cool.  And then we basically had to sit down and look at the areas where it fell flat, and we had  to rip out a lot of the plumbing and do a lot of refactoring.  So it was really our goal to open source it in the first half of this year or late last  year, and we just open sourced it now because of all the changes that we made, which really  really makes it now a very robust platform.  You know, one of the things that we had to do was move the entire platform, you know,  to use asynchronous requests.  We also built a profiling framework that actually helped us simulate, you know, the WebSocket  300 folks going through the, you know, the WebSockets at a, you know, at simulating a  deadline scenario.  So we were actually able to profile against our production server that had the old code,  and then the development server that had the new code, actually see that the gains we were  getting and we were on the right path in terms of refactoring all of that code.  So we went open source, which is awesome.  So we opened up the repos a few weeks ago.  Our mayor may not be checking the GitHub traffic stats on the Games API to see who's  cloned it.  It's been at least two people, so it's really exciting.  And maybe there'll be more than two people after this talk.  We opened it up on GitHub under GPLv2, and really I couldn't be prouder of the team.  RevSys, you know, Jane, and the entire learning lab team that really helped make this possible.  It's over two years worth of work, and it was really awesome to see it now live and  out in the wild.  One of the great milestones we had last week is we had a developer who had a React background,  which we'll get into, it was super valuable when looking at Simple.  But they were able to clone all the repos, go through the tutorials, and in the span  of about a week, we were able to make a rudimentary version of a simulation that took us over  a year to make from scratch.  So that was really a great milestone for us.  Like, hey, this thing actually works, which you're in your little bubble of development,  and you're like, oh, yeah, I think this thing works, but actually someone else says it works,  too.  It's like, oh, cool, it does work.  That's really awesome.  So we were really happy about that.  So now I'm going to pass it over to Jane, who's going to go kind of more into the weeds  of Simple and going to show you rules of engagement.  Jane.  Okay.  All right.  Thanks, Joe.  Yeah, I'm Jane Eisenstein.  I'm a senior application developer at the Wharton Learning Lab.  I have been working for the Learning Lab for four and a half years, and I joined the team  because it sounded like fun to create games that help people learn.  And I also am really proud that we're able to take something that we developed for our  internal use and offer it to the wider community.  So I'm going to spend a couple of slides just kind of going over the concepts of what we  mean by a game, because we've got some cool architecture, but Simple is really focused  on helping us create these games.  So we built it to be flexible and support the needs of these games.  So as Joe had already covered, games are built around a mathematical model.  User decisions go into the model.  It cranks and is stepped through, and then it produces results.  And this can happen iteratively.  You know, new decisions, results over several periods.  These periods with their associated input decisions and output results are grouped together  into scenarios.  Users, and by that I mean Django users, participate in a game by being assigned to a run of the  game as either a leader or a player.  So a run is an instance of a game for a particular group of users, and the way we use it is if  the professor has a class or it has a section of a class and they want them to all be playing  the game at the same time, like two o'clock on Friday the 13th, then they're going to  put them in the same run, and so that they will have that same timing that the run gives  them.  Players in a run may be grouped into worlds they don't need to be, but often if they are  grouped into worlds, that's just a team of players who typically either are collaborating  together or competing against each other, but the decisions they put in of all the players  goes into the model and the results come out and they're shared by everyone in the world.  So depending on your UI, you could see every other player in the world.  You could see all the decisions they had made, all the players had made.  You could see all the results that came out, and it really is a UI decision about how you  want to design your game.  The thing about both runs and worlds is they're independent of each other.  One run of a game is totally independent of another run of a game.  It might have a different set of parameters.  It will definitely have a different set of users, at least players, and worlds also are  independent of each other.  What goes on within a world is hidden from what goes on in other worlds within a run.  All right.  Players are often assigned roles when they come into a game and that defines what they're  able to see and it also can be used to, may define the decisions they may make.  And we'll see that with rules of engagement when we do the demo.  Run interactions occur in phases, what is a typical pattern for us is it'll be a set  up phase.  Oh, wait a second.  I need to.  I didn't talk about leaders and players.  I talked about leaders briefly, like teachers are primary leaders, so are admin people.  A leader can log into a game and they can see multiple runs and they can see all the data  about a run.  When a player logs into a game, they just see information about their private scenarios  or the one world that they belong to.  And that is quite a focus.  So again, going back to phases, set up phase, leaders would log in.  There would be entering parameters.  They might be adding players to the game.  And a leader moves the run to play phase, that's when players can get in and they can  start entering those decisions and seeing the results.  And then when the play phase is over, a leader again will move the game to a debrief phase.  And that's what I think really to me the secret sauce of a lot of what we do with the simulations  at Wharton.  Players run these models, they get experience using them, but then the professor has a debrief  session and he looks at results from all the different players and all the different worlds  within a run and he really can use that as a pedagogical tool.  I got the word pedagogical in there.  So that leads us to the simple model in the Django sense.  And I believe all the concepts that I had mentioned in the previous slides appear as  a model class except the run user one down over sort of midway down on the right.  The run user is really a model that represents a many to many relationship between the Django  users and runs within the simple database.  So a user may be assigned to multiple runs either within a game or cross games and a  run may have multiple users.  But for any particular run, one user is associated with one run in a one to one relationship  and they either are a leader or a player.  And the other thing here is that I don't know if you can see it, but there's a lot of one  to in relationships going down.  So when the model is actually instantiated at runtime, you have a tree structure.  Even though it looks from here like, oh gee, a scenario could be associated with the world  or a scenario could be associated with the run user, that's not really what the platform  supports.  If a user, either a leader or a player wants to run the model in kind of a private environment,  then they have their own private scenarios.  But if the scenarios associated with the world are shared across all the members of the world.  So this guarantees that every model instance has a single parent which allows you to kind  of traverse down from any place up to the top of the game.  And it allows you to pull out subtrees out of the model instances, which is really an  advantage.  And I'll just go to the next slide that talks about this.  When we are running games, we are pulling all the data, the game data that's relevant  to a particular user into the browser and storing it in a Redux state.  And we really want to hone in and bring in only information that's relevant to the user  and not have them getting updates from things that have nothing to do with them.  So it's really cool that we can do this, but how does this work?  And since this is a Django conference, when I'm going to focus a little bit more on the  front-end service than we might do otherwise, a simple game is composed of a front-end service  and a model service that are unique to that game.  And it uses the database service that is shared across all the games.  And in the front-end service, there is actually a very small Django application.  And its responsibilities are authenticating users.  And it is, once the user is authenticated, then the view for that Django app, well, it's  not shown here, but the front-end will actually query the simple database to find out what  runs the user is associated with and whether they're associated as a leader or a player.  And based on that, it figures out what subtree of all the data instances for that game it  should pull back and put into the React store.  So once it's done that query, it hands it off to a single-page application and the rest  of the UI logic is handled in React in Redux.  And from then on, any time when the React wants to know something about the database or make  changes in the database, it asks that it communicates with the model service over WAMP, which is  Web Application Messaging Protocol, using WebSockets.  And so all that communication goes through the model service to the database service.  Any updates that happen in the database will trigger webhooks, which fire and go back to  the model service, and which then may broadcast back to the subscribers for the particular  portions of the data model.  Let me see.  What else I want to cover?  Yeah, so the model service is, again, written by the developer.  It's specific to the game, it implements the simulation model in Python, and it also creates  custom classes with methods that can be accessed via WAMP, and it does that by subclassing off  these scope objects that are defined in the simple model, the model service package.  So when the model service starts up, it queries the database about all the active runs, it  pulls in that information, it creates these scope objects that basically it creates a  copy of the active runs of the game in memory.  And that means when the browser just wants to refresh a page, that nobody has to hit the  database.  It just says, hey, tell me about that, you know, tell me about this portion of the data  objects again, and then it can display it, which keeps the load off the shared database  service.  And so I think I've covered, to me, the primary things that you, as a developer, you need  to know about the simple model service is it provides these scope classes that can be  subclassed to create the business features that you need for your particular game, things  like to submit a decision.  You do that by creating a subclass of a scope object and say, when I submit a decision,  you need to, you know, create a decision in the database.  And also the web callbacks.  And now I will go on and take a quick look at the technology stack.  We went out the door with the 0.7 release using Python 3.6, Django 111.  I'm not sure of the Django REST framework version.  The web hooks are implemented using Thorn.  The WAMP is implementing using cross bar IO on the Java side, Java side, on the Python  side and Audubon on the JavaScript side.  And again, we're using React and Redux to implement our single page application.  It is now time for the live demo.  Please send good thoughts.  So, all right.  This first part of the demo is running on my local machine.  It is just to give you an idea of what goes on when you've logged in and how, what information  is needed to pull the information into the Redux state that's specific to this user.  So this stuff they got, so, but, okay, slow down.  Rules of engagement is a tipbook.  It's a multiplayer game.  It has worlds.  Each world comprises three players with three different roles.  And then it has a leader.  And the leader, as you can see here, can be looking at multiple runs.  It's called classes, but there's a default run and a Django run.  And they're in different phases.  The default run is still in setup.  And here is where someone can go in and change the start date and open date.  There's a whole bunch of parameters that I don't think we're ever going to change, but  it's nice to know that they could be changed.  And that allows me to mention an important point.  We have this flexible kind of generic model, but each game is going to have data that's  specific to that game.  And the way it's stored on the model is that the model classes all have a data property  which is stored as JSON.  So when I create a decision, the role of the decision, the name of the decision, that's  all part of it.  Those are all model properties.  But when I want to say like, what the heck, the decision, you know, the values of the  decision, that's all stored as JSON.  So just when you, we were looking at those game settings, that's all JSON data that  gets stored on the run.  So when you log in, some of the queries are that for a leader, it looks for what runs  the leader's associated with, and it creates topics that are at the run level.  So that's model.run.run.1, model.colon.run.2.  But for a player, again, because the player's associated with the world, we want to know  stuff about that particular run user, such as what are their private scenarios.  And we also want to know about what worlds they belong to, which is one.  So the topics for a player are going to be, if I can get that queued up.  Their topics are going to be, okay, I want to know anything that happens about the run  user that represents this particular logged in user.  And I want to know anything that happens about what goes on with the world that this run  user's been assigned to.  Now the thing that got me so excited about Simple early on, and I really didn't understand  it for ages, is the AutoMagic property.  My first blog about Simple was talking about its AutoMagic properties.  So it's now time to open play.  In fact, it is a bit late to open play.  It was supposed to open at 11.30, but we had this silly talk we had to get through first.  So now I'm going to move this run to play.  And boom, suddenly this player gets to play.  There's no polling involved.  There's no pushing involved.  It's fantastic.  I get to create a new test.  If I was imaginative, I'd call it something besides tests, but my adrenaline's going,  so I'm not going to.  I'll just put 2008 to show that I can do something that isn't already in my presets.  So to back off a little bit and say something about this game, when a player comes in here,  they're really simulating the Canadian Children's Serial Company case study.  Each player, I'll go back to the dashboard.  I got too excited about this.  So my role is I'm playing the product manager for Kellogg, and there are two other product  managers.  The competition is post and Kellogg's.  So my goal is to create a budget that's going to knock the socks off the competition, and  their goal is to create budgets that are going to knock the socks off the competition.  And they all have a different case history.  They all have basically more or less resources.  And the reason this game is called Rules of Engagement rather than Canadian Children's  Serial Company is because you define the budget for the current period, but you also create  rules that are going to set future budget.  So this game is a little weird for us because it runs over many periods, but there's only  one set of decisions, and those decisions affect the budget that is for the future periods,  but you're not creating new decisions with each period.  So I'm not going to mess with anything, but there's all sorts of options.  You can keep it fixed, which is the default.  You can react to the highest, the lowest by percentages above or below whatever they did  the last period.  And I'm going to just save that and run, but it doesn't want to save because I don't have  a name.  So I'm going to save this and run it.  And I'm not going to show you how cool it is.  And on the model service it's saying, oh, I ran this.  But anyway, I can step it through multiple periods.  And because everything was fixed, it's just everything's flatlined.  But I can also say, OK, so I can create as a rules of engagement player.  I can create multiple test scenarios.  I can do what ifs.  This is something the professors have been asking about for years, and we weren't able  to give it to them with the previous simulation platform that we had.  And then when I'm done, and I've figured out what my best deal is given my role, I can  then say, OK, I'm going to submit this as my final decision.  And basically I've done everything I have to do in the game as a player.  So at some point, the time will be up, and it's going to be time for the professor to  debrief the students on the results of their decisions.  So we just moved it to debrief.  If you see now, the user can no longer make corrections on their final decisions because  the play period is over.  And so if we go in here now, then we see the, it's only two worlds in this little test run.  But we can say I want to increment it like five periods and step it forward just like  we did with the test scenario.  And there's a lot of other detail which I'm not going to dig into.  But what I'm going to do now is switch over and trusting that the talk gods are with us.  I just want to do a quick refresh.  Well, no, it's good.  This actually is our production server.  And what we're seeing here on the top four rows are the runs from last year.  And what we're seeing on the bottom four rows are the empty runs that have been set up for  the coming for this semester, which we'll start running in the end of next week.  So never mind.  Never mind.  So I'm going to go in here and say that when we first started telling the professor that  the old version, because this is a re-implementation and it's an improvement on the previous version  of the simulation, when we told him that basically the sands of time are running out on the old  version and we were going to give him a new version, he said, well, I got to know that  I can have 300 students log on at the same time and the system isn't going to choke.  And I got to know that if I take one of these runs, which has 78 students, which means 26  worlds in it, and step at five periods, it'll finish in less than 10 seconds.  So we were near there a year ago.  And I don't know if we're going to be here this minute, but we are here more often than  not this year.  So here we go.  This is like 26 worlds, all being step five periods.  And I'm going to show the JavaScript because that is, let me see how long it took.  Okay, we ran it in 6.76 seconds and it took a couple of seconds to refresh.  And one of the things you got to know is that we do a lot of stuff automagically, but in  order to get this response time, not necessarily on the server for running the model, but to  get the refresh, we're not waiting for the cycle to go to the database and the web hook  fires and then it comes back and filters back to the Redux store.  And then the page would, that was like, that's why there are all these thrashing notices  because there was a lot of thrashing and the refreshes would fail.  So what we do is we run it, we scoop up a JSON, we serialize the data and we send it  back to the front end and that's how we're doing the refresh.  So now I'm going to switch back to Joe who's going to take us home.  Here you are.  There you are.  That moment when you hit refresh on a live demo and to pray that it works that time,  so that was awesome.  Awesome, Jane.  Thank you.  So where are we now?  So we're beginning work on our next simple simulation.  It's already in development.  We've written a model for it.  It's another marketing simulation that we're going to be running with the idea of delivering  it into the classroom for over 200 executive MBA students next fall.  Really excited about that.  The repos are available.  They're open source, github.com, simple world.  We have documentation.  We have tutorials.  We have a beautiful website, HTTP right there, simple.world.  Shout out to Chad Whitman.  He's not here today.  He works at work computing.  He did an amazing job on the website.  It looks really like totally legit.  And so it's really, really cool.  And we have t-shirts, so that's really awesome too.  Next up for us, we want to do this.  This is the largest talk that we've done so far.  We want to take this thing on the road, go to other places, maybe to PyCon.  We'll see.  But we also want to do workshops locally in the Philadelphia area at Penn.  For those folks in the university at other schools, I might want to use this.  So we're going to have some workshops there, just free workshops in the region.  And maybe we'll have SimpleCon someday.  We'll see.  But we're really excited about the prospects.  If you want to learn more about this, I think we're going to have a really few minutes  for questions, but you can reach out to us, learninglab.worton.upn.edu.  Is the email address for our team?  The homepage is there.  The github link is there.  If you go on the webpage at the very bottom, there's a place to sign up for our mailing  list.  So we're going to be, as we build content out, as we make changes, as we have workshops,  we'll be broadcasting that information via the mailing list.  So I hope you got something out of this.  It's a really exciting platform for us and a really big milestone for the learning lab.  We really think this could be a pretty impactful project at higher ed, but also we're really  excited to see folks, all the diverse folks here at JangoCon and beyond who can take it  and maybe think about using it in ways that we haven't even thought of yet.  So that's a really exciting thing about having an open source project.  So thank you for your time.  I think we have a few questions.  We have time for a few questions, but yeah, thank you.  In regards to your live demo today, so I noticed you played for one player and the other players  had results as well.  So how did that work and how do you deal with students who might not do their homework?  Is that part of that?  That's a great question.  So when the students submit decisions, when they do some test scenarios, they're entering  decisions for all three roles and they're kind of simulating what their opponents or  what their other competitors might do.  So when we provision the game and load it up, we assign folks to every role in the world.  We create the worlds.  And when the professor moves the game to debrief and is cranking forward those worlds, all the  decisions for that world are being run at that time.  If someone doesn't do it, the assignment, there is just default behavior that's in there  to prevent the model service from blowing up.  I was curious where you guys plan to take this next if you're going to keep it in the  finance world or if there's other areas that you're hoping to break into.  Yeah, I think, you know, so right now, we've given this talk at a few places at Penn.  We've given this talk at a couple universities around the Philadelphia area.  Business is a natural fit because of what I mentioned about simulations.  But I think that there could be simulations built around a lot of other disciplines, non-business  related, non-finance related.  I could see this being really effective tool in the engineering world where simulations  could be effective.  I could see this in even liberal arts.  I could see this at a lot of other places.  I could also see it as well.  We've talked to a few folks after our talks in the research world.  So maybe using this in a research capacity in some way, I could see that happening as  well.  So really, we'll see what kind of traction it gets outside of business.  But I'm pretty optimistic that we'll have some folks that are using this in other disciplines  that we haven't seen before.  All right, so thank you.  We'll be around the rest of today.  And again, if you have any other questions, feel free to email us.  We'll share the slides out.  Well, this talk will be online.  So hopefully you enjoyed it.  Thank you.  doomed.
DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_0_119.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_120_251.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_252_335.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_336_1703.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_1704_5159.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_5160_11303.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_11304_13391.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_13392_30299.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_30300_37079.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_37080_38087.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_38088_38423.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_38424_38519.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_38520_41771.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_41772_42791.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_42792_43523.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_43524_43715.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_43716_44111.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_44112_44555.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_44556_45467.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_45468_45779.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_45780_46991.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_46992_47207.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_47208_48119.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_48120_48227.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_48228_48419.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_48420_48791.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_48792_48971.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_48972_49019.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_49020_49343.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_49344_50471.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_50472_50519.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_50520_51887.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_51888_52031.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_52032_52079.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_52080_53699.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_53700_53819.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_53820_53879.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_53880_54131.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_54132_56051.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_56052_57491.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_57492_60995.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_60996_61055.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_61056_61199.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_61200_61247.png DjangoCon_US_2018_-_Simpl_framework__big_impact__by_Joseph_Lee___Jane_Eisenstein_0GhCvBGfYfA_61248_61315.png