Collaborative Mapping using GeoGit
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Clark, Scott
The The Rapid Open Geospatial User-Driven Enterprise (ROGUE) Joint Capability Technology Demonstration (JCTD) is focused on supporting humanitarian assistance and disaster response efforts in the SOUTHCOM area of responsibility. ROGUE is addressing some of the core challenges in the geospatial community right now - distributed collaboration, disconnected editing workflows, and provenance of data. All of this is being delivered as open source software, and based on open standards to encourage adoption by partners. This talk will explore how the ROGUE team is using GeoNode, GeoGit, and the OpenGeo Suite to provide a collaborative editing environment that maintains provenance of the data. In addition to developing GeoGit, the ROGUE technical team has demonstrated practical application of the technology through mobile and web applications (Arbiter & MapLoom). Both of these projects are available as open source as well. The discussion will include an overview of how the technology is being used operationally in Honduras and for risk assessment and response. A short demo will wrap up the talk.
 We've been working on for the last couple of years called Rogue. And the team has been in Element Solutions and Boundless as well. Rogue is actually a backer name. We came up with the name and then we decided to, what that means is Rapid Open G, a spatial user driven enterprise, just because if you're an OSD program, you have to have an acronym, right? So Rogue is a joint capability technology demonstration. And what that means is it's an office of secretary of defense mechanism for bringing somewhat mature technologies to operational problems. So basically you identify an operational problem, you say, hey, I have a technical solution for that. And then you basically have two years to basically proof it out in the field. So the problem was really collaboration with geospatial information. And when you put in the context of humanitarian assistance and disaster response, then you make the problem a little bit more interesting or complex at the same time. The key with disaster responses, you don't really know who's going to show up to the table, right? If something happens, then different countries come to the table, different NGOs, depending on what part of the world is in, different agencies, and of course whichever military or military's respond. Common situation awareness is obviously needed. When Haiti came, went down, when Yolanda happened, everybody was trying to grasp and get it. Like what's the actual situation on the ground, and then everybody needs to share that information. So it's where you're going to radical sharing almost in some cases. And they each have a piece of the puzzle, right? So the folks that are, for example USAID, know where all the different projects are and investments on the ground. The local response agency actually has an idea of where the most vulnerable areas are, and they can probably tell you where the most likely areas are, where the worst impact has been. And that all needs to be combined somehow. They're all distributed just by the nature of it, and sometimes disconnected, right? So the communications have this kind of bad habit of not being available when you need them the most. And the last one was actually driven by Southern Command themselves as the operational manager, is all the partners need to have direct access to the technology themselves. So one of the unique things about the Southern Command AOR is that they want all the countries to be technologically enabled. It's not a help for them if a country doesn't have technology and they can't really share information. So really that's our focus. Our focus is to share geospatial information and to do that in a very distributed fashion and handle disconnected cases. And I always have this slide up here to make sure I give credit where credit is due. The U.S. Army Corps of Engineers is the technical manager for this, and they're the ones that have been really driving us forward. I mentioned Southern Command as well, but Department of State, HIU, is also involved, and we're transitioning to Pacific Disaster Center. And I'll talk a little bit more about Honduras actually throughout this. So we're using quite a few components. We're kind of using the OpenGew suite plus this thing called GeoNote. It's a portal for discovering geospatial information. It's all open source. This cool thing called GeoGit that you just heard about, so thanks to Juan for setting that up really well. And then we have a couple of custom components. We actually modified GeoNote a little bit because we've been adding it so that GeoGetAware, and then we have a mobile app for data collection that's basically specialized to be able to go disconnected, load the data, go disconnected, collect the data, come back, reconnect, push your changes, and then just a thing called MapLume for weaving all that data together in a web map. So there's a cool thing called GeoGit you already heard of. So one of the things that we do in Rogue is we store all of our data in GeoGit databases for the most part. Some of the stuff is still in PostGIS. G-server lets you get away with using a lot of different data sources, and we take advantage of that. But the key for us was the multi-user, kind of multi-organizational aspect, right? And then the other key was being able to actually sync repositories, along with the conflict resolution. So that was really key. So if you've got different groups of people that are actually working on data together and they go disconnected, they're going to diverge very quickly, right? Think about word docs before track changes. I send you the word doc, and now you have to painstakingly go through and manually figure out what I changed. Or I have to tell you what I changed. If I did 50 things, then we're all in trouble, right? So that's kind of like the state where you are now if you're shipping files around. So I'm a junior nerd, maybe even a new geographer. I'm not sure if I qualify or not. But from my point of view, OpenStreetMap is kind of like epiphany of this great kind of concept of sharing data and making it available. I'm also an analyst, or was an analyst a long time ago. An analyst will worry about their data. So this is kind of what they worry about, right? You can laugh. This is meant to be lighthearted. So this is actually what happens when you let three developers take the UI and play with the data and start making commits. But that's actually pretty cool because it's 2000 commits. Everything's versioned. When this data was actually still live, you could actually go through all the different versions of the shark of snooze because there was a little edit wherever that one. So really what it gets to is the prominence is key. So prominence is key to actually being able to have trust. So if I want to give you my data, what are you going to do with it? Especially if I'm going to let you edit the same data that I'm editing, what's going to happen, right? Well, the established trust one is I got to know who did it. And that's what you get, let's us do. And understand the lineage, actually what specifically changed and when it changed. And it's also key to being able to work distributed because when you're distributed, when you come back online, you start pushing this information with each other, the data, the software itself has to know it changed. And we used a mobile app we built called Arbiter. It's open source as well. If you want to use it, you can go and grab it. Not quite as slick as the one we saw earlier, the map tool. But it's meant for distributed data or disconnected data collection. So you cache the data, you can go edit it, you come back, you push your changes, you can make photos and associate them to the features. And you can see those in the map as well. And we don't care about the comms as long as it's pretty decent, right? It's got to be 3G or Wi-Fi. So I promised you get the Honduras and how we actually put this in use. So each one of those nodes, you see Pacific Disaster Center on the left. Up on the top was Joint Task Force Bravo, so Connoir Base. And at the bottom was Capeco. Capeco is the FEMA equivalent in Honduras. So they're the ones that are responsible for resilience as well as response. And so what we did is an exercise based around their Independence Day support. And we had data syncing using GeoGit between all three locations. And the back end or the front end, I'm sorry, was GeoNode with our map inset on the JTF Bravo, Capeco. But over on the PDC side, it was actually disaster-aware. Because that's already a situation awareness tool for event management and response. So basically we could let users at Southcom and other agencies be able to see the data and take advantage of this user-generated content without having to change the tool they're using. And we tried to make the other tools as simple as possible. And the other beautiful thing about GeoGit is because of the fact that it keeps track of all the commits you're on, you can sync basically all three directions at once. And all three of these nodes can keep track with each other. If you drop connection, for example, between PDC and Capeco, all the changes will go through JTF Bravo. And it's okay that you're actually syncing with the two at both the same time. So the operational demonstration that we're going to have next month is actually based on Hurricane Mitch. The one we did previous last year was actually for the Independence Day parades that they had. And going up the main boulevard there into the stadium, they had all their teams situated along the parade route and it was very much kind of an event management kind of thing. But the key was it was very much corollary to what they have to do manually with their logs. So if somebody opens an event, a child is lost. Boom, this time stamp, child was put lost, this location is exactly what they'd write. At this time stamp, parents were found, child was reunited, okay, keep track of that. So basically we just did the same thing using GeoGit. They would just go and they would actually just use the tablets and just change the reports. And it's only pushing the WFST, so it could be really any tool that's doing it. And on the back end, you're getting all the version that goes along with it. So we actually did have people reunited. We actually had a traffic accident reported. Heat exhaustion was actually more common as well. But it was actually quite successful in that case. Kind of a little bit different concept than our mapping and data management concepts that we talked about. So assessment and response is kind of one of the big focuses that they have down there. Building damage assessments, they have reports that started out as Excel spreadsheets that we turned into layers. There were 50 some odd fields with certain conditionals on them and you'd have to go down and do that report. And it becomes an official government report. That's now a layer. We can also leverage OSM. The picture on the right is actually the hot layer, the humanitarian open street map layer for Yolanda. So that was one of the first experiments with pulling OSM using DuGet. And just being able to grab that and pull that in is one of our main use cases. So this ties in with the Department of State because they have this great program called Images of the Crowd, where they make data available to the volunteers for Ulpa Street Map. And people can digitize, but the key is you can also grab that now and you can pull that down. And so you can go disconnected with that as well. And also down in Honduras, they're going to start using it for their social development layers. There's a lot of poverty in Honduras, right? So going to a house, what do you need? What is your house need? Do you need water? Do you need utilities? Do you need a roof? Do you need all three, etc.? And just keeping the history of all that. So there's a website that's got all of our slides and links available to it. So you can go and get that. And there's actually even a demo server that's live you can play with. So hopefully the resolution is pretty good here. So this is a GeoNode with MapLume inset in it. And we've got some of the test data that we've pushed up on here. Getting to the history is actually pretty straightforward. We've got it by the layer and by the feature. So if I pull this up, so this is a GeoGit layer, I've actually got a history button. And this gets me to a list of now I can see all the changes that have been made, who made them and when. All the way back to the beginning of the feature. And if I want to see a specific change, I can go in here and it'll actually tell me, I can dig down through here. And this one I just changed to one. So we kind of follow a norm of green means the new, something that was new. They're added, yellow means it's changed, and are modified, and then red means a delete. So this one I came and I fixed the name because it was actually typed in incorrectly earlier by me. And one of the cool things that GeoGit lets you do is it's actually called the blame command. We decided to call it show author so that people wouldn't be freaked out by it. But I can also get everything about each individual attribute within a feature because you think about the lifetime of a feature, it's going to have a lot of different authors that are contributing to those different pieces of data. And so you can actually get that information as well. And then if I want to actually have the big red button for undoing it. We also have a conflict resolution in here. So the conflict resolution is actually very similar. You have three panels. You have the original, then the chain, or the, basically yours and theirs. And then in the middle is the one that's going to be the final. We have the ability to set up syncs. So we could run a sync through here. And then if I go in, we wanted people to be able to add features pretty straightforward fashion as well. So now it just gets added to the database. For the map, we're just making every edit a commit so it's going in almost immediately. And then for the mobile app, it will actually push however many commits you've, or how many changes you made during the time you've been working. And then notifications were key because it's always great to know that you're getting the latest and greatest changes from people. But what if you don't know what actually changed? That was kind of one of the problems. So we made it easy to get to the notifications and figure out when something was added. And it basically has all the same abilities as what I showed earlier. And then like I said, there's actually a full layer history as well. And this lets you get through all the history of the layer itself. So that's just kind of a real quick tour. There is actually a website online. And if you want, you could actually go up and play with the server yourself. There's a gentleman from the Netherlands who's playing with it. If you want to get to the hub, you can just get to it on GitHub as well. So we've been fulfilling our mandate of making sure everything was open source. Okay. Thank you.
 We've been working on for the last couple of years called Rogue.  And the team has been in Element Solutions and Boundless as well.  Rogue is actually a backer name.  We came up with the name and then we decided to, what that means is Rapid Open G,  a spatial user driven enterprise, just because if you're an OSD program,  you have to have an acronym, right?  So Rogue is a joint capability technology demonstration.  And what that means is it's an office of secretary of defense mechanism  for bringing somewhat mature technologies to operational problems.  So basically you identify an operational problem, you say, hey, I have a technical solution for that.  And then you basically have two years to basically proof it out in the field.  So the problem was really collaboration with geospatial information.  And when you put in the context of humanitarian assistance and disaster response,  then you make the problem a little bit more interesting or complex at the same time.  The key with disaster responses, you don't really know who's going to show up to the table, right?  If something happens, then different countries come to the table, different NGOs,  depending on what part of the world is in, different agencies,  and of course whichever military or military's respond.  Common situation awareness is obviously needed.  When Haiti came, went down, when Yolanda happened, everybody was trying to grasp and get it.  Like what's the actual situation on the ground, and then everybody needs to share that information.  So it's where you're going to radical sharing almost in some cases.  And they each have a piece of the puzzle, right?  So the folks that are, for example USAID, know where all the different projects are  and investments on the ground.  The local response agency actually has an idea of where the most vulnerable areas are,  and they can probably tell you where the most likely areas are, where the worst impact has been.  And that all needs to be combined somehow.  They're all distributed just by the nature of it, and sometimes disconnected, right?  So the communications have this kind of bad habit of not being available when you need them the most.  And the last one was actually driven by Southern Command themselves as the operational manager,  is all the partners need to have direct access to the technology themselves.  So one of the unique things about the Southern Command AOR is that they want all the countries to be technologically enabled.  It's not a help for them if a country doesn't have technology and they can't really share information.  So really that's our focus.  Our focus is to share geospatial information and to do that in a very distributed fashion  and handle disconnected cases.  And I always have this slide up here to make sure I give credit where credit is due.  The U.S. Army Corps of Engineers is the technical manager for this,  and they're the ones that have been really driving us forward.  I mentioned Southern Command as well, but Department of State, HIU, is also involved,  and we're transitioning to Pacific Disaster Center.  And I'll talk a little bit more about Honduras actually throughout this.  So we're using quite a few components.  We're kind of using the OpenGew suite plus this thing called GeoNote.  It's a portal for discovering geospatial information.  It's all open source.  This cool thing called GeoGit that you just heard about,  so thanks to Juan for setting that up really well.  And then we have a couple of custom components.  We actually modified GeoNote a little bit because we've been adding it so that GeoGetAware,  and then we have a mobile app for data collection that's basically specialized to be able to go disconnected,  load the data, go disconnected, collect the data, come back, reconnect,  push your changes, and then just a thing called MapLume for weaving all that data together in a web map.  So there's a cool thing called GeoGit you already heard of.  So one of the things that we do in Rogue is we store all of our data in GeoGit databases for the most part.  Some of the stuff is still in PostGIS.  G-server lets you get away with using a lot of different data sources, and we take advantage of that.  But the key for us was the multi-user, kind of multi-organizational aspect, right?  And then the other key was being able to actually sync repositories, along with the conflict resolution.  So that was really key.  So if you've got different groups of people that are actually working on data together and they go disconnected,  they're going to diverge very quickly, right?  Think about word docs before track changes.  I send you the word doc, and now you have to painstakingly go through and manually figure out what I changed.  Or I have to tell you what I changed.  If I did 50 things, then we're all in trouble, right?  So that's kind of like the state where you are now if you're shipping files around.  So I'm a junior nerd, maybe even a new geographer.  I'm not sure if I qualify or not.  But from my point of view, OpenStreetMap is kind of like epiphany of this great kind of concept of sharing data and making it available.  I'm also an analyst, or was an analyst a long time ago.  An analyst will worry about their data.  So this is kind of what they worry about, right?  You can laugh.  This is meant to be lighthearted.  So this is actually what happens when you let three developers take the UI and play with the data and start making commits.  But that's actually pretty cool because it's 2000 commits.  Everything's versioned.  When this data was actually still live, you could actually go through all the different versions of the shark of snooze because there was a little edit wherever that one.  So really what it gets to is the prominence is key.  So prominence is key to actually being able to have trust.  So if I want to give you my data, what are you going to do with it?  Especially if I'm going to let you edit the same data that I'm editing, what's going to happen, right?  Well, the established trust one is I got to know who did it.  And that's what you get, let's us do.  And understand the lineage, actually what specifically changed and when it changed.  And it's also key to being able to work distributed because when you're distributed, when you come back online, you start pushing this information with each other, the data, the software itself has to know it changed.  And we used a mobile app we built called Arbiter.  It's open source as well.  If you want to use it, you can go and grab it.  Not quite as slick as the one we saw earlier, the map tool.  But it's meant for distributed data or disconnected data collection.  So you cache the data, you can go edit it, you come back, you push your changes, you can make photos and associate them to the features.  And you can see those in the map as well.  And we don't care about the comms as long as it's pretty decent, right?  It's got to be 3G or Wi-Fi.  So I promised you get the Honduras and how we actually put this in use.  So each one of those nodes, you see Pacific Disaster Center on the left.  Up on the top was Joint Task Force Bravo, so Connoir Base.  And at the bottom was Capeco.  Capeco is the FEMA equivalent in Honduras.  So they're the ones that are responsible for resilience as well as response.  And so what we did is an exercise based around their Independence Day support.  And we had data syncing using GeoGit between all three locations.  And the back end or the front end, I'm sorry, was GeoNode with our map inset on the JTF Bravo, Capeco.  But over on the PDC side, it was actually disaster-aware.  Because that's already a situation awareness tool for event management and response.  So basically we could let users at Southcom and other agencies be able to see the data  and take advantage of this user-generated content without having to change the tool they're using.  And we tried to make the other tools as simple as possible.  And the other beautiful thing about GeoGit is because of the fact that it keeps track of all the commits  you're on, you can sync basically all three directions at once.  And all three of these nodes can keep track with each other.  If you drop connection, for example, between PDC and Capeco, all the changes will go through JTF Bravo.  And it's okay that you're actually syncing with the two at both the same time.  So the operational demonstration that we're going to have next month is actually based on Hurricane Mitch.  The one we did previous last year was actually for the Independence Day parades that they had.  And going up the main boulevard there into the stadium, they had all their teams situated along the parade route  and it was very much kind of an event management kind of thing.  But the key was it was very much corollary to what they have to do manually with their logs.  So if somebody opens an event, a child is lost.  Boom, this time stamp, child was put lost, this location is exactly what they'd write.  At this time stamp, parents were found, child was reunited, okay, keep track of that.  So basically we just did the same thing using GeoGit.  They would just go and they would actually just use the tablets and just change the reports.  And it's only pushing the WFST, so it could be really any tool that's doing it.  And on the back end, you're getting all the version that goes along with it.  So we actually did have people reunited.  We actually had a traffic accident reported.  Heat exhaustion was actually more common as well.  But it was actually quite successful in that case.  Kind of a little bit different concept than our mapping and data management concepts that we talked about.  So assessment and response is kind of one of the big focuses that they have down there.  Building damage assessments, they have reports that started out as Excel spreadsheets that we turned into layers.  There were 50 some odd fields with certain conditionals on them and you'd have to go down and do that report.  And it becomes an official government report.  That's now a layer.  We can also leverage OSM.  The picture on the right is actually the hot layer, the humanitarian open street map layer for Yolanda.  So that was one of the first experiments with pulling OSM using DuGet.  And just being able to grab that and pull that in is one of our main use cases.  So this ties in with the Department of State because they have this great program called Images of the Crowd,  where they make data available to the volunteers for Ulpa Street Map.  And people can digitize, but the key is you can also grab that now and you can pull that down.  And so you can go disconnected with that as well.  And also down in Honduras, they're going to start using it for their social development layers.  There's a lot of poverty in Honduras, right?  So going to a house, what do you need?  What is your house need? Do you need water? Do you need utilities? Do you need a roof? Do you need all three, etc.?  And just keeping the history of all that.  So there's a website that's got all of our slides and links available to it.  So you can go and get that. And there's actually even a demo server that's live you can play with.  So hopefully the resolution is pretty good here.  So this is a GeoNode with MapLume inset in it.  And we've got some of the test data that we've pushed up on here.  Getting to the history is actually pretty straightforward. We've got it by the layer and by the feature.  So if I pull this up, so this is a GeoGit layer, I've actually got a history button.  And this gets me to a list of now I can see all the changes that have been made, who made them and when.  All the way back to the beginning of the feature.  And if I want to see a specific change, I can go in here and it'll actually tell me, I can dig down through here.  And this one I just changed to one. So we kind of follow a norm of green means the new, something that was new.  They're added, yellow means it's changed, and are modified, and then red means a delete.  So this one I came and I fixed the name because it was actually typed in incorrectly earlier by me.  And one of the cool things that GeoGit lets you do is it's actually called the blame command.  We decided to call it show author so that people wouldn't be freaked out by it.  But I can also get everything about each individual attribute within a feature because you think about the lifetime of a feature,  it's going to have a lot of different authors that are contributing to those different pieces of data.  And so you can actually get that information as well.  And then if I want to actually have the big red button for undoing it.  We also have a conflict resolution in here.  So the conflict resolution is actually very similar.  You have three panels. You have the original, then the chain, or the, basically yours and theirs.  And then in the middle is the one that's going to be the final.  We have the ability to set up syncs. So we could run a sync through here.  And then if I go in, we wanted people to be able to add features pretty straightforward fashion as well.  So now it just gets added to the database.  For the map, we're just making every edit a commit so it's going in almost immediately.  And then for the mobile app, it will actually push however many commits you've,  or how many changes you made during the time you've been working.  And then notifications were key because it's always great to know that you're getting the latest and greatest changes from people.  But what if you don't know what actually changed? That was kind of one of the problems.  So we made it easy to get to the notifications and figure out when something was added.  And it basically has all the same abilities as what I showed earlier.  And then like I said, there's actually a full layer history as well.  And this lets you get through all the history of the layer itself.  So that's just kind of a real quick tour.  There is actually a website online.  And if you want, you could actually go up and play with the server yourself.  There's a gentleman from the Netherlands who's playing with it.  If you want to get to the hub, you can just get to it on GitHub as well.  So we've been fulfilling our mandate of making sure everything was open source.  Okay.  Thank you.
Collaborative_Mapping_using_GeoGit__720p__0_167.png Collaborative_Mapping_using_GeoGit__720p__168_3995.png Collaborative_Mapping_using_GeoGit__720p__3996_5543.png Collaborative_Mapping_using_GeoGit__720p__5544_5639.png Collaborative_Mapping_using_GeoGit__720p__5640_6971.png Collaborative_Mapping_using_GeoGit__720p__6972_8315.png Collaborative_Mapping_using_GeoGit__720p__8316_9179.png Collaborative_Mapping_using_GeoGit__720p__9180_10055.png Collaborative_Mapping_using_GeoGit__720p__10056_12467.png Collaborative_Mapping_using_GeoGit__720p__12468_14507.png Collaborative_Mapping_using_GeoGit__720p__14508_16439.png Collaborative_Mapping_using_GeoGit__720p__16440_17699.png Collaborative_Mapping_using_GeoGit__720p__17700_17771.png Collaborative_Mapping_using_GeoGit__720p__17772_17951.png Collaborative_Mapping_using_GeoGit__720p__17952_18779.png Collaborative_Mapping_using_GeoGit__720p__18780_19559.png Collaborative_Mapping_using_GeoGit__720p__19560_20243.png Collaborative_Mapping_using_GeoGit__720p__20244_20291.png Collaborative_Mapping_using_GeoGit__720p__20292_20999.png Collaborative_Mapping_using_GeoGit__720p__21000_22535.png