Postgis in the Open Cloud
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Jarvinen, Ryan
Instant Mapping Applications with PostGIS and Nodejs Learn how to write PostgreSQL-backed applications that can be automatically cloned, built, monitored, and scaled in a matter of minutes. We'll walk you through all of the steps involved in developing and scaling your own Node.js-powered PostGIS mapping application on the OpenShift cloud hosting platform. By the end of this talk, you should be able to instantly deploy your own PostgreSQL-backed network applications to your own public or private cloud in a single command. This talk includes a brief architectural overview of OpenShift, a freely-available Open Source hosting platform. Folks who want to follow along should bring their own laptop with git, rubygems, and nodejs installed & available.
 The principal's been so far. Pretty good. Yeah? Cool. I've been working on these slides, so I've been to, did the last minute, like, get pushed this stuff live to the web, like, about 20 minutes ago. So it's all live, but I missed the other talks, unfortunately. I know, it's great. It's really nice, yeah. All right, I get started here. So today I'm doing a talk on post-GIS and the open cloud, and I'll define kind of what I mean by open cloud in a minute. And we're specifically going to be writing highly portable open source mapping applications using leaflet and a little bit of OpenShift. There's a bit.ly URL and a full URL if you guys are interested in following along. You can literally open up these slides, and there is no excuses. All of you guys could have a live mapping app by the end of this talk. We could have, however many people here, all fully deployed and sharing URLs. So really cool stuff. I don't know if you've ever set up a full post-GRIS database in under an hour or deployed a whole app on top of that. So this is a lot of cool content, and I'll try to cram it all into the 45 minutes here. Oh, yeah. You bet. Or is it somewhere else like a complete cramming approach? You know what? I could try to write it up on this whiteboard here if there was some chalk. Here is the bit.ly URL. Bit.ly 1TP BQGA. And it's case sensitive. So watch out for that. There's the slides. And let's go. So I'm Ryan. You can also find me as Ryan J on IRC, Twitter, GitHub, or RyanJ at redhat.com. I work at Red Hat on the web. I work at Red Hat on the web. I work at Red Hat on their OpenShift team doing their cloud stuff. And we've got a lot of really exciting stuff. I'm going to show you a little bit of our stuff and a lot of the application technology that I built on top of it using a lot of post-GRIS, post-GIS stuff. So I'll start off with that open cloud overview. I'll give you guys a little bit of background for people who are new to open source cloud technologies. If you guys want to follow along and try to build an app during this workshop, you could select from many different languages. I'm mostly, since we have a limited amount of time, I'm just going to focus on JavaScript and Node.js. So hopefully that's interesting for you guys. And if not, we have the same solution written in Python and the same solution written in Java. And so you could refer to a couple different example codes and take a look and hopefully get something useful out of it. We'll add leaflet for our client-side map interactions. We'll build a simple API, install and configure post-GRIS and post-GIS extensions. Use environment variables, bootstrap our database, launch an instant mapping solution, and learn about tuning your database. I really hope I could fit all this stuff in. There's a lot of content. So I'll flip through these slides quick. If you have any questions, feel free to shout them out, raise your hand, whatever. And I'll do my best to answer it. So the cloud, what is it? What is the cloud? There's a lot of different explanations, mostly cat photos maybe. But when you're talking about building the cloud with open source tools, there's a couple big components you want to use. So infrastructure as a service is a term hopefully you guys have heard of before. There's companies like Amazon Web Services at EC2 basically offers this. There's a lot of other companies that offer a whole box on demand or a virtual machine on demand. Usually these are virtualized environments. It's composed of a bunch of hardware, of course. And then the software you're going to use is going to be something like OpenStack. Eucalyptus is another open source. Infrastructure as a service. Cloud Stack, Salt Stack, these are all good options. My favorite is OpenStack. And the end result of infrastructure as a service is you have easy on demand Linux environments or whole boxes. But my criticism of infrastructure as a service is it's kind of like a conversation between one ops guy and another ops guy, where one's a vendor and one's a consumer, and they've really kind of left developers completely out of the conversation. Here's a whole box that's not configured at all. You go ahead and configure it and then do the work to maintain it. And that's the real, like, when you run into a dead end on these things is you can come out with, you could quickly customize a box, but what happens when you need to maintain that and keep it up to date becomes a lot more work. So Platforms is a service. That really helps open source the workflows between your operations team and your development team. Usually there's a lot of contention between these two groups because they have kind of competing goals. The operations team wants to optimize around uptime, not using too many resources, not spinning up too much hardware. They want to keep the costs low. And developers want to just do everything as quickly as possible and not have to wait for the ops team to respond. So we're going to basically show you workflows for building, hosting, and scaling. Those are the main three things that Platform as a Service is going to try to solve for you. And there's a lot of technical underpinnings of how that works. I could give you more information if you want. I have some slightly, a little bit more details in these slides. So in this case, Darth Vader is the operations guy and Luke Skywalker. I don't know if that's how you perceive things, but Luke Skywalker is running around trying to mix it up. And these guys can finally work together, each really working towards their own goals without hurting each other. And the great thing about this is the discussion ends up shifting towards good policies for how to do auto scaling in response to demand. Instead of saying, hey, are you done with that VM? I need the memory back or the CPUs back. You don't have this fighting back and forth because your hardware scales automatically in response to the amount of network load or CPU load. So that's kind of what we're trying to do. Here's some terminology warning. I put this on a red slide because this is red hat specific. So we have a couple pieces to how we do Platform as a Service. We have two different machine types. We've got a broker that does orchestration. And then we have another machine type called a node. And what we do with those nodes, it's basically a whole box. And we carve out SE Linux containers. And then we use C groups to make a user permission or user resource scope. And each application that's running on top of this Platform is kind of in its own, we call that collection of resources built with SE Linux and C groups. We call that a gear, for lack of a better word. And so your application is going to run inside this gear environment. And then we could clone those and load balance across multiple gears. So that's the idea. Cartridges are kind of our metaphor for extending what you have on a gear. And sometimes, for example, if I was running Postgres, I would probably want to put my application in a scalable configuration. What that'll do is it'll put the DB on a separate gear. So it has the full amount of resource allocation just for the DB. And it's not trying to share it with the web server. If you do non-scale, they'll both be bundled together on the same gear. So you have low latency. But it's not quite as scalable. Also, with that scalable configuration, like I said, we'll clone the front end. We don't clone the back end yet, but I'll have some slides on that later. So we do have an open cartridge format. If anyone's interested in developing add-ons for Open Shift, click on this link. I made this graphic myself. That's not the Nintendo for Nintendo Nerds. It's not the Nintendo Seal of Approval. I edited it. It says Open Source Seal. Anyone recognize this meme? Normally, it's a trap. That's one thing you want to worry about. There's a common meme on the internet, Admiral Ackbar saying, it's a trap. Do you want to invest all your time of reformatting your code to work on a platform as a service? Maybe, maybe not. Maybe whole boxes are good enough for you. The reason why he's saying seems legit to me is we have no lock-in, and you could use it. It's open source. We have three different releases of Open Shift. There's Origin. That's all of our open source code. We have a hosted version called Open Shift Online, and then there's an enterprise version that you could run in your own data center. And I think that is all I have for, well, maybe a little bit more. But I'll try to keep the Open Shift content to a minimum and keep it on the post-GIS stuff. So frameworks. These are my starting points for you guys to get up and going with a new application. So for Node.js, I recommend using, these are all micro frameworks. It's a term that gets thrown around. But I like using Restify for Node.js. If you wanted to get started, this has kind of like a really simple base application in here. I also have a really simple Flask base application that basically just has a web server, a dependencies file, and it already has a basic API that just responds on index.html to serve up your main web page. And it also has static assets, support for serving static assets. So those are a couple great starting points that you're going to want to use. You could also, Flask is available. For PHP, I like using Sylex. It's really simple. For Ruby, there's Sinatra. So those are all great starting points. And we'll get into specifically looking at, where did that go? Open, and I think my full screen is blocking me. So here is the base. I have some config stuff. Ooh, my screen's getting chopped. Some config stuff over here. Static assets, I'll go in this folder. And there's an index.html file. And then the two main really critical things for Node.js is you want to have a package.json file. That's going to indicate your list of dependencies. This has dependencies on the config module and on restify. So really simple. It also has something in here that tells your server how to start itself. It's going to run whenever you run npm space start, that'll automatically run Node server.js. OpenShift actually reads from this one here, main. So you want to have whatever your main script is annotated there in your package.json. And by default, the default is to load server.js. So you could leave those lines out where you're saying specifically server.js. As long as you name your server, server.js. If you want to name anything else, you want to go back in package.json and edit it. Here's a basic, you could see I'm using require up at the top. That's a global keyword in Node.js to load npm packages. So I'm requiring the config module, restify, and then FS is a built-in module for reading and writing to the file system. This is one of the things that is unique to Node.js, that you don't get in standard JavaScript, both the require as well as standard JavaScript you can't interact with file system at all. So that's some of the nice stuff that server-side JavaScript will give you access to. So you could see here we have a couple routes defined. There's a status page that just sends an OK response or a status API endpoint. And then on our main endpoint, slash, we just return index.html and do some little switcheroo with the host import setting. I think this is for local dev, though. You also have down here, this is loading up, serving all your static. Anything that's in static gets served up if it's CSS, JavaScript, or image files. And here's how we start the server. The only thing really unique to OpenShift in here is that we're passing in the IP address in addition to the port. That's not always defined with Node.js, but OpenShift runs your each app on a different IP, virtual IP. So you're going to want to make sure to include the IP address. And that gets, we'll see the config file a little bit later. But that's a brief intro to what you can expect from some of these micro frameworks. So you could do getClone on this and have that be your starting point if you wanted to try to hack along with the slides and copy and paste content in. So language specific dependencies. So Ruby has RubyGems, Python has Python eggs, Node.js has these MPM modules. OpenShift will automatically, if you have them detailed, will automatically pull down these requirements and make them available to your app. You could also, I think, check these dependencies into your source code. And there's a debate on, do you want to check in your dependencies or not? For Node.js, it's actually recommended to check in your module dependencies into your source code. But if they're not checked in, we'll populate those as part of the build process. So you want to also add some language specific database bindings. So for Node.js, this is how you would install something from the command line inside your project. NPM install, pgquery-save. If you run that, it'll add it to your package.json file and list it as a dependency. So that helps keep, whether you check in your Node modules, that's where the things get loaded to, whether you check all those in or not, use the dash-save and always keep your package.json up to date. Same thing for Python. There's either settings.py or requirements.text. You want to have your dependencies listed in there in order to maximize portability. So for Node.js, I'm going to be using, there's a guy on GitHub, Brian C. We'll be using Brian C's pgquery module. And that really makes this stuff. It's just standard Postgres calls and then a function handler. And it's really simple. So for local development, if you wanted to spin up a local server with a base project or with a completed project, you'd run npm install to install dependencies locally if you don't have them yet. And then npm start. That spins up whatever is dictated in your package.json file. All right, leaflet. So how many people have used leaflet before? Anyone heard of leaflet? Only, wow, OK. All right, leaflet is one of my favorite mapping solutions. It's fully open source. It uses open street maps for the tiles. But you can also modify it to use other map tiles. It has a very simple interface and excellent documentation. I would highly recommend that you guys take a look at their docs, because they've got a lot of great stuff. And we'll take a look at some of their code next. If you wanted to use leaflet, you could just slap this into your index.html, put this in your header, and it basically has a style sheet and some JavaScript that leaflet's going to use. And I'm loading this right from leaflet cdn here. So to initialize your map, this you want to put in a script tag. I put this near the bottom of my index file. But there's basically a map variable that you're going to initialize. You'll set the view to some coordinates. Here I just have it hard coded to my neighborhood in the San Francisco Bay Area. And then there's the final value here is the amount of zoom you want. And we'll take a look at what this looks like in a minute. You also have, it looks like this is adding a tile layer. So here I'm using Stamen tiles instead of OpenStreetMap tiles. All I did was change this URL. So if you want to use OpenStreetMaps, you could put their URL in. There's a variety of different tile solutions you could load in here. And then I have some attribution here. You want to make sure you give people credit, especially OpenSource projects. Give them credit for all the work they're doing. So I give credit here to Stamen. And the data that's encoded into the map, I think the data that they're talking about is the city names are scaled relative to population size. And so that's the data, quote unquote. It's just a city name. But there's some data processing that went into scaling the name. And that was basically data from OpenStreetMap. So next up, we actually want to redraw the map whenever someone does a zoom, drag, or when the map loads. Here's our three up top. We've got these three actions that we want to respond to. And I have them down here. Here's the actual functions. I'm setting up some event triggers. And I'm saying whenever leaflet detects a zoom event or a drag event or a new map comes on, whenever any of those three events fire off, I am going to run this getPins function. And it's passing in e is the event that fired off in the browser. I'm going to ask the map to get its bounds and say, hey, what's the top right and the bottom left coordinate? And it'll get those two mapping points. So I'll pull that from the screen. And then it'll construct a URL here. And then we'll just run getURL. I think this is jQuery or something. Yeah, jQuery. And then we'll use this pin the map function to process the result of whatever comes back from our server. So we haven't defined this endpoint yet. We'll be looking for parks slash within is the endpoint we're going to write on the server side. And so for now, this is going to fail. But yeah, eventually when these map pins come back, we'll clear the current pins and then add the new ones to the map. Pretty simple, right? Any questions about this? Not much to ask about. It's real clean. So let's build an API for our map to talk to. Here, it looks similar to the base application that I started with. We have a config. We have RESTify. We have FS. The only thing that's new here is I have a module called db.js that's going to handle all of our db interaction. I didn't want to muck up my main server file with all this db-specific code. So separated that into a different module. And then here, I'm doing create server. Cores is to allow cross-site requests. I have a couple endpoints here. Here, let's get all the parks within. And we'll just run db select box whenever the parks within endpoint is hit. So this is just mapping a URL to a function call. And we'll take a look at that function in a minute. But this is how easy it is to define API endpoints with a micro framework. And this is true whether you're using Flask, whether you're using Sylex for PHP, whether you're using Jax RS for Java, or Sinatra for Ruby. It's going to look similar to this. I already talked about the other stuff in this file. So I'll skip ahead. We'll take a look at setting up Postgres. So if you already have an existing OpenShift application environment, you could run a command like this, cartridge add. RHC is our command line tool. You could install that with RubyGems. But you run gem install RHC. You sign up for an OpenShift account or host your own OpenShift cluster somewhere. And then you could say add Postgres. 8.4 was the version we started with. And we're currently have 9.2 available. I don't know the specific sub release on that if it's 9.2.12 or something. But we could look it up if you guys need specifics on that. Yeah, there's more information in that Postgres 9.2 blog post. But that's how easy it is to, if someone from the, basically these cartridges allow developers to say, I need this dependency. And it becomes available as part of their application. Here is the, so let's just assume that just worked. Postgres is available, right? It seems like I skipped over something there, right? But the database is available. Let's look at that dv file that I defined and see how we're going to run our queries. So here we have that select box function that I mentioned whenever we run slash parks slash within. It's going to hit this function here. So here this request, response, and next is kind of a standard function parameters on any API response from either express or restify those micro frameworks. The rec variable is going to be the request. Rest is your response. You could name those whatever you want. And yeah, so, and I think next is if you have something like a web socket, you could kind of keep a continued socket open on here. So I am going to basically look up the query string from the request. We'll parse the URL. I'll try to pull out the lat1 and long1, lat2 and long2. And I also check if someone passed in a limit parameter. That way the clients can do throttling. If you're on a mobile device, they could say, I only want 20 map points at a time. Don't give me 15,000 map points. You're going to overwhelm my browser. So I put that limit parameter in for client side throttling. And I think I see is it in the code here? I have in the, oh yeah, yeah, here. Query limit 40. So I put a default of 40. Someone could override the default query limit, but I thought it was a sensible default. It at least gives you 40 map pins is something to look at. And then people can manually bump it up if they want more. So here is the response I'm going to send back in case they missed any of these parameters. If they don't have lat long, that stuff's missing. I send them an error response. And if it's all available, I use that pg module to make my query. So here pg is the module that I required earlier. And then I'm just going to run a select. Here I have a transform. It's trying, yeah, it does. Sorry, I'm really tired. I was up super late on this stuff. It basically does the geospatial query. And the 4326 is the format of the data, I think, right? Yeah, yeah, OK. All right, good. Check with him if you have questions. Any other questions about this slide? Then it's just going to send back the rows that we got. Pretty simple. Map the query, get the rows, and then return them back to the map front end. So how did all this get wired together? How am I talking to Postgres with just this pg variable? I didn't specify the IP address. I didn't specify the admin password or the username, any of that stuff. That hasn't been in any of our code, right? We want to have clean and portable code. And there's nothing hard-coded so far, which is good, but some major gaps, right? So let's take a look at where did that stuff, what happened to that stuff? So here, I'll do a RHC app create, and I'll say MyApp and Node.js. Get rid of that. Too many argue. Spelled create wrong. Thanks. OK, so here it's going to create my application. I just did Node.js here. But that's already going to spin up a new app for me, and it's going to have some environment variables pre-defined within this application. You saw how I was connecting to the IP address of the server. There was also a port number in the server.listen call when we set up our server earlier. So this will come back with some environment variables that we'll be able to use when that command completes. You could also look up any of the environments later by running RHC app show. In fact, I might be able to do that already. OK, here app is ready. Here it's making the DNS available. Here is MyApp2 was the name of the app. I picked this namespace earlier for all of my applications. You could use your GitHub username, your IRC handle, maybe your email address, whatever you want for your namespace there. And then if you wanted to do team collaboration, you could assign a team member write privileges or update privileges SSH access on your subdomain there. So here you could see we have a URL already available. There's a SSH address. We could SSH into our app. We have a git remote and a local copy of our source here in the MyApp2 folder. Here's our local source. I could do RHC SSH to connect to this thing over SSH. And I could also run RHC app show to see the details. Here's the same details from earlier. Echoed back. If I had Postgres installed, it would also have, I'll just do that right now, RHC cartridge add Postgres 9.2. And we'll add that to our Node.js app. Postgresql. That's not a full list there. That's just the list of guesses. It's chopping off the side of my, let me know if I lose part of the text there. But that's a partial list of cartridges based on what I typed. OK, here we go. We got our root user, root password, database name, connection string. That came up pretty quick, right? And now if I go back to this SSH environment, I'm going to need to log in again because the environment variables for connecting to Postgres got added after this shell session got started. So here I could SSH back in. I could use just standard SSH as well, but RHC SSH is kind of a shortcut that runs SSH and then the long connection string that you guys saw earlier. So here I could do ENV and then grep for something like DB. So here's all my DB connection strings that my application's going to use. So you write your applications to refer to environment variables in order to keep your code clean. And so let's take a look at the config file I have. That's RESTifyBase. Let's find RESTifyPostGIS. Actually, I have this in the slides as well. Here is the config file. I basically created as a module. This is what the config module for Node.js expects. And I have a similar file for Flask, my Flask application, and my Ruby application as well. And so this has a lot of conditional assignments. First, it checks for process.envport. For Node.js, process is your main thread that's running. Env is your environment relative to that process. And OpenShift doesn't define.port. We don't define port. Heroku does. So theoretically, this might be able to run on Heroku, but maybe not. I haven't actually tried it, so probably not. But I would love to have these be super portable applications to any open cloud platform, not just a Red Hat one. But this is how you could shim in as many different environment variables as you want in order to populate the config.port variable. We're also going to populate an IP address. And we're going to have a connection string for Postgres. All of these have a fallback at the end. Here's our sensible fallback for local development. So if I run this same code in local dev, it's going to start up on 127.0.0.1. Port 3000. If I run it on OpenShift and the environment variables are available, it'll take advantage of the environment variables. If I run it somewhere else and they have different environment variables, I include it in this file. And my code instantly can run on that other platform. So this is one of the keys to writing your ultra portable code that'll run just about anywhere. If you could also set custom environment variables, so one thing that a lot of applications have a Google Analytics tracker key. You don't necessarily want to hard code that into your source, especially if you're going to deploy to several different URLs. So you can specify this as here I have a secret token with a bunch of junk in here. I could run ENV set secret token. It sets my environment variable. I could do ENV list. I could see it's populated there. And if I go back to SSH and connect, I have to reload the session. But I could check in there and that secret key would also be populated in here. ENV grep secret. There's the same secret token. So that's another way to keep your code super nice and clean. If you want any help with that, RHC ENV help has a lot of details. Yeah? I have a question really, a lot of questions. Are you doing all this work locally, any local development before you get to the application or server? I'm doing this with OpenShift Online. So this is live right now. Anyone can go to this URL up top right now with your cell phone, anything. Anyone can go. When you're running the commands, you're running the commands on your computer. Oh, yeah. The laptop must be getting replicated into the remote. Yeah, this here is running on my laptop here, setting the environment. And I also had a shell open as well. So here's. Yeah, I didn't find the local way to get that. I mean, you don't have to know. Yep, yeah, yeah, yeah. And the way that works is it uses this command line tool I'm using is talking to OpenShift has a REST API. So there's a lot more that you could do with the REST API if you really wanted to do heavy automation. That REST API has the most number of features. The command line has a little bit less. And then we have a web interface as well. The web interface, you can't set these environment variables yet. So that's one thing I'd really love to see added into the web interface. Yeah, good question, though. OK, so let's get back to slides. All right, environment variables. OK, here you could even supply additional environment variables during your app creation process. So unfortunately, this is a demo I was working on for Code for America. They have an app called adoptahydrant. And it basically maps all the fire hydrants in town. And their problem was when it snowed, these fire hydrants would get covered up in a bunch of snow. And someone would need to dig them out. And so they had neighborhood representatives that would volunteer, I'm going to adopt this hydrant. And then they had kind of a social app where you could see, hey, so and so is maintaining that they've all these neighborhood volunteers are filling the gap for a lack of government services. And you really have citizen-driven applications that are really powerful. And this is exactly the type of thing you would want to be portable. They also spun this up in Hawaii. And instead of, there's no snow there. But they have these sirens that alert you if there's a tsunami warning. And they had all these birds that were nesting in these sirens. And they would need to go keep these tsunami alert sirens clean of all these nesting birds. And so that was another similar source code, slightly adapted. But hopefully, they could have a common base and then just a different style sheet on top. Or just minor differences. So this one actually, you could see how you could specify an environment variable here. But yeah, this one, it takes too long to build and it fails on OpenShift. We need to clean up our build process to run a little bit quicker or allow for longer build times. At least when you're doing your app create statement. If I do this in two steps, where I create a Node.js or a Ruby app, add Postgres, and then add the code, it works just fine. But this one liner is where it gets really powerful. So if you're doing really advanced DB services, let's say you have a local development machine and you want to talk to your remote Postgres. Here, I'm back in my local development. I could run RHC port forward. And that will make some local connections to my remote DB available. Also my remote server. So I could go here, there's my mouse. I could go here and load up this URL. And it will proxy through to my remote environment. Same thing with the Postgres. So if you have PG admin or something locally, boom, there's your URL. You can also, I have a blog post on how to use local development and develop against your remote OpenShift server. So if you're on a locked down laptop where you're not allowed to install Postgres, spin up an OpenShift environment, set up the port forward, boom, you're done. So yeah, good stuff there as well. We also have a post on how to connect to existing Postgres services from an OpenShift front end. So if you had a big inner, your own PG database that's large, you could talk to your own PG database instead of these small OpenShift ones. We're kind of spinning up a separate single serve PG for each development environment. And if you wanted to talk to a central Postgres master or one of your slaves, you hit that second link there and we'll have more information on how to get that done. So automating your DB setup. These are still some remaining work here. We've got just about everything done, but we haven't enabled Postgres. We haven't created our table schema. We haven't added our geospatial index. We haven't injected our data into our application. Let's take a look at how that stuff might work. So here I have, there's a dot OpenShift folder where we'll allow you to define some action hooks. Action hooks are basically just scripts. This one is a bash script. If it's a Ruby script, that's fine. We just try to run the script if it's defined. And we have a couple different scripts. This one is called deploy. So this gets run on every deploy. There's also a one called build and prebuild and postbuild. Depending on what you want to automate, you could hook into several different parts of the process and run an arbitrary script to help set up your database. I actually relocated all of the code into something that's actually run via my package.json. I didn't want my code to be really specific to OpenShift. I wanted anyone to be able to initialize their DB, even if they weren't an OpenShift user. So let's jump back to the package.json and see where that init thing is defined. Wrong file. That's all of our data here, package.json. So here you could see in scripts, instead of just the start script, we have a couple other ones. There's build. There's flush DB, init DB. So there's a couple different things. And this just runs npm run init DB, which runs node bin bootstrap. So let's go take a look at bin bootstrap. I have some of that code here in the slides. So here, similar to what we saw before, we're relying on that config to not refer to anything about how to log in to our database. We're using PGQuery for super clean, easy queries. Here I'm finding the remote connection string from config and a table name. I'm going to make my DB connection here. I'm going to grab all of our points that's in a JSON file. And I'm going to start loading these into the database. So the first function we call is init DB. init DB basically just adds the post GIS extension. And then runs this callback afterwards. Create DB schema. So these basically chain together. Node.js does everything asynchronously. So if you want to run things in order, you use callbacks like this. So here, we're creating the DB schema next. And here, it checks for errors. And then goes and runs this create table. It creates all the table schema here. And finally, we end up adding the spatial index here. And then importing our map points. Let's see. I think I do a small amount of translation here. Yeah, I did some kind of translation. There was a quote marks in the file that was breaking the query. So I did a little search and replace to put double quotes around the single quote. I don't know. That's the only kind of tweak I needed to make with our input data. So that's it. We've got results for Node.js. Here is the resulting application. And if you guys want to give that a try for Python Flask, you can go to flask-post-gis. These are all the fully completed apps with everything that's in there. We also have a Java and hibernate solution. I don't know if that one works as an instant app. And I'll show you real quick. And we're pretty much out of time. But here is what it would look like. Here's any kind of app creation for OpenShift. You type in RHC app create, the name of the app, however many cartridge dependencies you need. You could also specify your source code. And then so specifically here, we could run something like this. In order to spin up a new, I'll call this Parks4 maybe. I don't know how many of these Parks apps I have. I've got a lot of them. Here's a live demo of the completed result. So here are map points set on Boston. When I drag out, we got some more points added. As I pan over, any time I pan or zoom, it's redrawing all the points. With a limit of 40. Right? Pretty fast. And the Parks data is just national parks and historic sites. I could click on any of these pins and see what any of these points are. I also added for how many of you guys have seen the clone me on GitHub banners? I'm a fan of the clone me on GitHub kind of ribbon that people have, but I modified it. This says run me on OpenShift. So with this click, this goes right into our web-based workflow for creating apps. So here I could create Parks5. Here's my Git source. I could say, hey, I need extra memory. Give me a larger size environment. It already has Python and Postgres listed as my dependencies. All this stuff was just encoded into the link. Here in the URL. So yeah, create application, turn scaling on and off. That's one, two clicks. Click on the ribbon. Click on create if they already have an OpenShift account. And you can host your own OpenShift. So really, really cool stuff there with that web-based workflow for creating apps. I have a post on how to customize that workflow and construct those URL strings. So whether you like the ribbon or not, if you want to give someone a link that says launch my app, look up that post and we've got info on how you can really compose these apps based on source code and a list of cartridge dependencies. Here's a post on how to make those ribbons and add the ribbons to your app. Tuning Postgres. I'm out of time, but I'm going to run like two minutes over if you guys are OK with that. All right. This is some of the good stuff here. Where the heck did my pghba.conf go? And where's postgresql.conf? As a developer, I don't usually, like our theory is developers shouldn't have to mess with this stuff. This should be stuff that the operations team and the developers are collaborating on, a conversation about how we scale. And do we scale horizontally in small chunks at a time, or do we scale in larger chunks at a time? But either way, we should have a focus on horizontal scaling and redundancy. So let's get the developers out of these files, give them easy on-demand environments, and the ops team can tune the Postgres cartridge. Developers can also tune it a bit. We have some environment variables. This open shift Postgresql shared buffers allows you to set the number of shared buffers. You can also set the number of max connections. These are all setable just with that RHCE and Vset for setting environment variables. And then that'll go into effect. You may need to reload your database to pick those up, but pretty easy. And a lot of the stuff you see in this wiki page on tuning your Postgres server is pretty easily doable on open shift. Other advanced configurations. So here's one issue. I was talking with the guys from Crunchy Data. Have you guys heard of anyone familiar with Crunchy Data? We'll check this out. These guys gave me a warning that they had some issues with the Postgres 9.2 cartridge with the statistics collector. And I don't know if this was after a million records or after a certain amount. It's a scale that I haven't hit yet with my tiny little demo apps. But they say after a certain number of iterations, the statistics collector was a little bit buggy in our Postgres 9.2 cartridge. So what did they do? They created their own Postgres 9.3 cartridge for open shift. This is something that you can run right away. You know how I did that RHC cartridge ad? You could specify the URL of this and add Postgres 9.3, even though we don't support it. We won't answer the phone calls. And when you say, hey, my Postgres 9.3 is busted, we're going to be like, hey, it's not in our list of supported stuff. But you can opt in to running arbitrary cartridges from GitHub. So that's definitely something good to look at. This Postgres 9.3 cart from Crunchy Data. Also, they have a 9.4 Devel with the RLS patch available. And if you wanted to do highly available Postgres on open shift in your enterprise environment, they've got solutions for that as well. So they just had a big launch announcement. And they'll do load balancing, geographic failover, master slave replication, and other things. But yeah, there's a link to this post if you want information on bringing all this technology in-house into your own data center. I think that's about it. We accept pull requests. We want help with everything. We use PEPs for our major, that's what Python uses. It's a project enhancement proposal. We use that for our major feature enhancements. You could track everything we do is public, everything we do is open source. Just check us out online and you could see what we're up to. We also have this third party index for, and here's where you could find, you go to originley.com, you could add your own cartridge solutions or find other third party cartridges that people like Crunchy Data are building. So there's a link to those cartridges. And let's see if I could wrap this up. Oh, we've got some really cool stuff for Docker coming up in the future. Keep an eye on OpenShift and our Docker support. OpenShift Origin is our upstream, like I said, online three apps. Everyone can run that. And that's about it. If anyone clicked on that ribbon in the mapping app, they ought to be live with their own mapping solution. Click there and you could be live with your own app. All right? I think that's it for me. Thanks for sticking around.
 The principal's been so far.  Pretty good.  Yeah?  Cool.  I've been working on these slides, so I've been to,  did the last minute, like, get pushed this stuff live  to the web, like, about 20 minutes ago.  So it's all live, but I missed the other talks,  unfortunately.  I know, it's great.  It's really nice, yeah.  All right, I get started here.  So today I'm doing a talk on post-GIS and the open cloud,  and I'll define kind of what I mean by open cloud  in a minute.  And we're specifically going to be writing  highly portable open source mapping applications using  leaflet and a little bit of OpenShift.  There's a bit.ly URL and a full URL if you guys are  interested in following along.  You can literally open up these slides, and there is no  excuses.  All of you guys could have a live mapping app by the end of  this talk.  We could have, however many people here, all fully  deployed and sharing URLs.  So really cool stuff.  I don't know if you've ever set up a full post-GRIS database  in under an hour or deployed a whole app on top of that.  So this is a lot of cool content, and I'll try to  cram it all into the 45 minutes here.  Oh, yeah.  You bet.  Or is it somewhere else like a complete cramming  approach?  You know what?  I could try to write it up on this whiteboard here if there  was some chalk.  Here is the bit.ly URL.  Bit.ly 1TP BQGA.  And it's case sensitive.  So watch out for that.  There's the slides.  And let's go.  So I'm Ryan.  You can also find me as Ryan J on IRC, Twitter, GitHub, or  RyanJ at redhat.com.  I work at Red Hat on the web.  I work at Red Hat on the web.  I work at Red Hat on their OpenShift team doing their  cloud stuff.  And we've got a lot of really exciting stuff.  I'm going to show you a little bit of our stuff and a lot  of the application technology that I built on top of it  using a lot of post-GRIS, post-GIS stuff.  So I'll start off with that open cloud overview.  I'll give you guys a little bit of background for people who  are new to open source cloud technologies.  If you guys want to follow along and try to build an app  during this workshop, you could select from many different  languages.  I'm mostly, since we have a limited amount of time, I'm  just going to focus on JavaScript and Node.js.  So hopefully that's interesting for you guys.  And if not, we have the same solution written in Python and  the same solution written in Java.  And so you could refer to a couple different example codes  and take a look and hopefully get something useful out of it.  We'll add leaflet for our client-side map interactions.  We'll build a simple API, install and configure post-GRIS  and post-GIS extensions.  Use environment variables, bootstrap our database, launch  an instant mapping solution, and learn about  tuning your database.  I really hope I could fit all this stuff in.  There's a lot of content.  So I'll flip through these slides quick.  If you have any questions, feel free to shout them out,  raise your hand, whatever.  And I'll do my best to answer it.  So the cloud, what is it?  What is the cloud?  There's a lot of different explanations, mostly cat  photos maybe.  But when you're talking about building the cloud with  open source tools, there's a couple big components you  want to use.  So infrastructure as a service is a term hopefully you guys  have heard of before.  There's companies like Amazon Web Services at EC2 basically  offers this.  There's a lot of other companies that offer a whole box  on demand or a virtual machine on demand.  Usually these are virtualized environments.  It's composed of a bunch of hardware, of course.  And then the software you're going to use is going to be  something like OpenStack.  Eucalyptus is another open source.  Infrastructure as a service.  Cloud Stack, Salt Stack, these are all good options.  My favorite is OpenStack.  And the end result of infrastructure as a service  is you have easy on demand Linux environments or  whole boxes.  But my criticism of infrastructure as a service is  it's kind of like a conversation between one ops guy  and another ops guy, where one's a vendor and one's a  consumer, and they've really kind of left developers  completely out of the conversation.  Here's a whole box that's not configured at all.  You go ahead and configure it and then do the work to  maintain it.  And that's the real, like, when you run into a dead end on  these things is you can come out with, you could quickly  customize a box, but what happens when you need to  maintain that and keep it up to date becomes a lot more work.  So Platforms is a service.  That really helps open source the workflows between your  operations team and your development team.  Usually there's a lot of contention between these two  groups because they have kind of competing goals.  The operations team wants to optimize around uptime, not  using too many resources, not spinning up too much  hardware.  They want to keep the costs low.  And developers want to just do everything as quickly as  possible and not have to wait for the ops team to respond.  So we're going to basically show you workflows for  building, hosting, and scaling.  Those are the main three things that Platform as a  Service is going to try to solve for you.  And there's a lot of technical underpinnings of how  that works.  I could give you more information if you want.  I have some slightly, a little bit more details in these  slides.  So in this case, Darth Vader is the operations guy and Luke  Skywalker.  I don't know if that's how you perceive things, but Luke  Skywalker is running around trying to mix it up.  And these guys can finally work together, each really  working towards their own goals without  hurting each other.  And the great thing about this is the discussion ends up  shifting towards good policies for how to do auto  scaling in response to demand.  Instead of saying, hey, are you done with that VM?  I need the memory back or the CPUs back.  You don't have this fighting back and forth because your  hardware scales automatically in response to the amount of  network load or CPU load.  So that's kind of what we're trying to do.  Here's some terminology warning.  I put this on a red slide because this is red hat  specific.  So we have a couple pieces to how we do Platform as a  Service.  We have two different machine types.  We've got a broker that does orchestration.  And then we have another machine type called a node.  And what we do with those nodes, it's basically a whole box.  And we carve out SE Linux containers.  And then we use C groups to make a user permission or user  resource scope.  And each application that's running on top of this  Platform is kind of in its own, we call that collection of  resources built with SE Linux and C groups.  We call that a gear, for lack of a better word.  And so your application is going to run inside this gear  environment.  And then we could clone those and load balance across  multiple gears.  So that's the idea.  Cartridges are kind of our metaphor for extending what  you have on a gear.  And sometimes, for example, if I was running Postgres, I  would probably want to put my application in a scalable  configuration.  What that'll do is it'll put the DB on a separate gear.  So it has the full amount of resource allocation just for  the DB.  And it's not trying to share it with the web server.  If you do non-scale, they'll both be bundled together on  the same gear.  So you have low latency.  But it's not quite as scalable.  Also, with that scalable configuration, like I said, we'll  clone the front end.  We don't clone the back end yet, but I'll have some  slides on that later.  So we do have an open cartridge format.  If anyone's interested in developing add-ons for Open  Shift, click on this link.  I made this graphic myself.  That's not the Nintendo for Nintendo Nerds.  It's not the Nintendo Seal of Approval.  I edited it.  It says Open Source Seal.  Anyone recognize this meme?  Normally, it's a trap.  That's one thing you want to worry about.  There's a common meme on the internet, Admiral Ackbar  saying, it's a trap.  Do you want to invest all your time of reformatting your  code to work on a platform as a service?  Maybe, maybe not.  Maybe whole boxes are good enough for you.  The reason why he's saying seems legit to me is we have  no lock-in, and you could use it.  It's open source.  We have three different releases of Open Shift.  There's Origin.  That's all of our open source code.  We have a hosted version called Open Shift Online, and  then there's an enterprise version that you could run in  your own data center.  And I think that is all I have for, well, maybe a little  bit more.  But I'll try to keep the Open Shift content to a minimum  and keep it on the post-GIS stuff.  So frameworks.  These are my starting points for you guys to get up and  going with a new application.  So for Node.js, I recommend using, these are all  micro frameworks.  It's a term that gets thrown around.  But I like using Restify for Node.js.  If you wanted to get started, this has kind of like a really  simple base application in here.  I also have a really simple Flask base application that  basically just has a web server, a dependencies file, and  it already has a basic API that just responds on  index.html to serve up your main web page.  And it also has static assets, support for  serving static assets.  So those are a couple great starting points that you're  going to want to use.  You could also, Flask is available.  For PHP, I like using Sylex.  It's really simple.  For Ruby, there's Sinatra.  So those are all great starting points.  And we'll get into specifically looking at, where did  that go?  Open, and I think my full screen is blocking me.  So here is the base.  I have some config stuff.  Ooh, my screen's getting chopped.  Some config stuff over here.  Static assets, I'll go in this folder.  And there's an index.html file.  And then the two main really critical things for Node.js  is you want to have a package.json file.  That's going to indicate your list of dependencies.  This has dependencies on the config module and on restify.  So really simple.  It also has something in here that tells your server how to  start itself.  It's going to run whenever you run npm space start, that'll  automatically run Node server.js.  OpenShift actually reads from this one here, main.  So you want to have whatever your main script is  annotated there in your package.json.  And by default, the default is to load server.js.  So you could leave those lines out where you're saying  specifically server.js.  As long as you name your server, server.js.  If you want to name anything else, you want to go back in  package.json and edit it.  Here's a basic, you could see I'm using require up at the  top.  That's a global keyword in Node.js to load npm packages.  So I'm requiring the config module, restify, and then FS is  a built-in module for reading and writing to the file system.  This is one of the things that is unique to Node.js, that  you don't get in standard JavaScript, both the require  as well as standard JavaScript you can't interact with file  system at all.  So that's some of the nice stuff that server-side  JavaScript will give you access to.  So you could see here we have a couple routes defined.  There's a status page that just sends an OK response or a  status API endpoint.  And then on our main endpoint, slash, we just return  index.html and do some little switcheroo with the host  import setting.  I think this is for local dev, though.  You also have down here, this is loading up, serving all  your static.  Anything that's in static gets served up if it's CSS,  JavaScript, or image files.  And here's how we start the server.  The only thing really unique to OpenShift in here is that  we're passing in the IP address in addition to the port.  That's not always defined with Node.js, but OpenShift runs  your each app on a different IP, virtual IP.  So you're going to want to make sure to include the IP  address.  And that gets, we'll see the config file a little bit  later.  But that's a brief intro to what you can expect from some  of these micro frameworks.  So you could do getClone on this and have that be your  starting point if you wanted to try to hack along with the  slides and copy and paste content in.  So language specific dependencies.  So Ruby has RubyGems, Python has Python eggs, Node.js has  these MPM modules.  OpenShift will automatically, if you have them detailed,  will automatically pull down these requirements and make  them available to your app.  You could also, I think, check these dependencies into  your source code.  And there's a debate on, do you want to check in your  dependencies or not?  For Node.js, it's actually recommended to check in your  module dependencies into your source code.  But if they're not checked in, we'll populate those as part  of the build process.  So you want to also add some language specific  database bindings.  So for Node.js, this is how you would install something  from the command line inside your project.  NPM install, pgquery-save.  If you run that, it'll add it to your package.json file and  list it as a dependency.  So that helps keep, whether you check in your Node modules,  that's where the things get loaded to, whether you check  all those in or not, use the dash-save and always keep  your package.json up to date.  Same thing for Python.  There's either settings.py or requirements.text.  You want to have your dependencies listed in there  in order to maximize portability.  So for Node.js, I'm going to be using, there's a guy on  GitHub, Brian C. We'll be using Brian C's pgquery module.  And that really makes this stuff.  It's just standard Postgres calls and then a function  handler.  And it's really simple.  So for local development, if you wanted to spin up a local  server with a base project or with a completed project, you'd  run npm install to install dependencies locally if you  don't have them yet.  And then npm start.  That spins up whatever is dictated in your package.json  file.  All right, leaflet.  So how many people have used leaflet before?  Anyone heard of leaflet?  Only, wow, OK.  All right, leaflet is one of my favorite mapping solutions.  It's fully open source.  It uses open street maps for the tiles.  But you can also modify it to use other map tiles.  It has a very simple interface and excellent documentation.  I would highly recommend that you guys take a look at their  docs, because they've got a lot of great stuff.  And we'll take a look at some of their code next.  If you wanted to use leaflet, you could just slap this  into your index.html, put this in your header, and it  basically has a style sheet and some JavaScript that  leaflet's going to use.  And I'm loading this right from leaflet cdn here.  So to initialize your map, this you want to put in a  script tag.  I put this near the bottom of my index file.  But there's basically a map variable that you're going to  initialize.  You'll set the view to some coordinates.  Here I just have it hard coded to my neighborhood in the  San Francisco Bay Area.  And then there's the final value here is the amount of  zoom you want.  And we'll take a look at what this looks like in a minute.  You also have, it looks like this is adding a tile layer.  So here I'm using Stamen tiles instead of OpenStreetMap  tiles.  All I did was change this URL.  So if you want to use OpenStreetMaps, you could  put their URL in.  There's a variety of different tile solutions you  could load in here.  And then I have some attribution here.  You want to make sure you give people credit, especially  OpenSource projects.  Give them credit for all the work they're doing.  So I give credit here to Stamen.  And the data that's encoded into the map, I think the  data that they're talking about is the city names are  scaled relative to population size.  And so that's the data, quote unquote.  It's just a city name.  But there's some data processing that went into  scaling the name.  And that was basically data from OpenStreetMap.  So next up, we actually want to redraw the map whenever  someone does a zoom, drag, or when the map loads.  Here's our three up top.  We've got these three actions that we want to respond to.  And I have them down here.  Here's the actual functions.  I'm setting up some event triggers.  And I'm saying whenever leaflet detects a zoom event or a  drag event or a new map comes on, whenever any of those  three events fire off, I am going to run this  getPins function.  And it's passing in e is the event that fired off in the  browser.  I'm going to ask the map to get its bounds and say, hey,  what's the top right and the bottom left coordinate?  And it'll get those two mapping points.  So I'll pull that from the screen.  And then it'll construct a URL here.  And then we'll just run getURL.  I think this is jQuery or something.  Yeah, jQuery.  And then we'll use this pin the map function to process the  result of whatever comes back from our server.  So we haven't defined this endpoint yet.  We'll be looking for parks slash within is the endpoint we're  going to write on the server side.  And so for now, this is going to fail.  But yeah, eventually when these map pins come back, we'll  clear the current pins and then add the new ones to the map.  Pretty simple, right?  Any questions about this?  Not much to ask about.  It's real clean.  So let's build an API for our map to talk to.  Here, it looks similar to the base application that I  started with.  We have a config.  We have RESTify.  We have FS.  The only thing that's new here is I have a module called  db.js that's going to handle all of our db interaction.  I didn't want to muck up my main server file with all this  db-specific code.  So separated that into a different module.  And then here, I'm doing create server.  Cores is to allow cross-site requests.  I have a couple endpoints here.  Here, let's get all the parks within.  And we'll just run db select box whenever the  parks within endpoint is hit.  So this is just mapping a URL to a function call.  And we'll take a look at that function in a minute.  But this is how easy it is to define API endpoints with a  micro framework.  And this is true whether you're using Flask, whether  you're using Sylex for PHP, whether you're using Jax RS  for Java, or Sinatra for Ruby.  It's going to look similar to this.  I already talked about the other stuff in this file.  So I'll skip ahead.  We'll take a look at setting up Postgres.  So if you already have an existing OpenShift  application environment, you could run a command like  this, cartridge add.  RHC is our command line tool.  You could install that with RubyGems.  But you run gem install RHC.  You sign up for an OpenShift account or host your own  OpenShift cluster somewhere.  And then you could say add Postgres.  8.4 was the version we started with.  And we're currently have 9.2 available.  I don't know the specific sub release on that if it's 9.2.12  or something.  But we could look it up if you guys need specifics on that.  Yeah, there's more information in that Postgres 9.2 blog post.  But that's how easy it is to, if someone from the, basically  these cartridges allow developers to say, I need  this dependency.  And it becomes available as part of their application.  Here is the, so let's just assume that just worked.  Postgres is available, right?  It seems like I skipped over something there, right?  But the database is available.  Let's look at that dv file that I defined and see how we're  going to run our queries.  So here we have that select box function that I mentioned  whenever we run slash parks slash within.  It's going to hit this function here.  So here this request, response, and next is kind of a  standard function parameters on any API response from either  express or restify those micro frameworks.  The rec variable is going to be the request.  Rest is your response. You could name those whatever you want.  And yeah, so, and I think next is if you have something like  a web socket, you could kind of keep a continued socket  open on here.  So I am going to basically look up the query string from the  request.  We'll parse the URL.  I'll try to pull out the lat1 and long1, lat2 and  long2.  And I also check if someone passed in a limit parameter.  That way the clients can do throttling.  If you're on a mobile device, they could say, I only want  20 map points at a time.  Don't give me 15,000 map points.  You're going to overwhelm my browser.  So I put that limit parameter in for client side  throttling.  And I think I see is it in the code here?  I have in the, oh yeah, yeah, here.  Query limit 40.  So I put a default of 40.  Someone could override the default query limit, but I  thought it was a sensible default.  It at least gives you 40 map pins is something to look at.  And then people can manually bump it up if they want more.  So here is the response I'm going to send back in case  they missed any of these parameters.  If they don't have lat long, that stuff's missing.  I send them an error response.  And if it's all available, I use that pg module to make my  query.  So here pg is the module that I required earlier.  And then I'm just going to run a select.  Here I have a transform.  It's trying, yeah, it does.  Sorry, I'm really tired.  I was up super late on this stuff.  It basically does the geospatial query.  And the 4326 is the format of the data, I think, right?  Yeah, yeah, OK.  All right, good.  Check with him if you have questions.  Any other questions about this slide?  Then it's just going to send back the rows that we got.  Pretty simple.  Map the query, get the rows, and then return them back to  the map front end.  So how did all this get wired together?  How am I talking to Postgres with just this pg variable?  I didn't specify the IP address.  I didn't specify the admin password or the username, any  of that stuff.  That hasn't been in any of our code, right?  We want to have clean and portable code.  And there's nothing hard-coded so far, which is good, but  some major gaps, right?  So let's take a look at where did that stuff, what happened  to that stuff?  So here, I'll do a RHC app create, and I'll say MyApp  and Node.js.  Get rid of that.  Too many argue.  Spelled create wrong.  Thanks.  OK, so here it's going to create my application.  I just did Node.js here.  But that's already going to spin up a new app for me, and  it's going to have some environment variables pre-defined  within this application.  You saw how I was connecting to the IP address of the server.  There was also a port number in the server.listen call when  we set up our server earlier.  So this will come back with some environment variables that  we'll be able to use when that command completes.  You could also look up any of the environments later by  running RHC app show.  In fact, I might be able to do that already.  OK, here app is ready.  Here it's making the DNS available.  Here is MyApp2 was the name of the app.  I picked this namespace earlier for all of my applications.  You could use your GitHub username, your IRC handle,  maybe your email address, whatever you want for your  namespace there.  And then if you wanted to do team collaboration, you could  assign a team member write privileges or update privileges  SSH access on your subdomain there.  So here you could see we have a URL already available.  There's a SSH address.  We could SSH into our app.  We have a git remote and a local copy of our source here  in the MyApp2 folder.  Here's our local source.  I could do RHC SSH to connect to this thing over SSH.  And I could also run RHC app show to see the details.  Here's the same details from earlier.  Echoed back.  If I had Postgres installed, it would also have, I'll just do  that right now, RHC cartridge add  Postgres 9.2.  And we'll add that to our Node.js app.  Postgresql.  That's not a full list there.  That's just the list of guesses.  It's chopping off the side of my, let me know if I lose part  of the text there.  But that's a partial list of cartridges based on  what I typed.  OK, here we go.  We got our root user, root password, database name,  connection string.  That came up pretty quick, right?  And now if I go back to this SSH environment, I'm going to  need to log in again because the environment variables for  connecting to Postgres got added after this shell  session got started.  So here I could SSH back in.  I could use just standard SSH as well, but RHC SSH is kind  of a shortcut that runs SSH and then the long connection  string that you guys saw earlier.  So here I could do ENV and then grep for something like  DB.  So here's all my DB connection strings that my  application's going to use.  So you write your applications to refer to  environment variables in order to keep your code clean.  And so let's take a look at the config file I have.  That's RESTifyBase.  Let's find RESTifyPostGIS.  Actually, I have this in the slides as well.  Here is the config file.  I basically created as a module.  This is what the config module for Node.js expects.  And I have a similar file for Flask, my Flask application,  and my Ruby application as well.  And so this has a lot of conditional assignments.  First, it checks for process.envport.  For Node.js, process is your main thread that's running.  Env is your environment relative to that process.  And OpenShift doesn't define.port.  We don't define port.  Heroku does.  So theoretically, this might be able to run on Heroku, but  maybe not.  I haven't actually tried it, so probably not.  But I would love to have these be super portable applications  to any open cloud platform, not just a Red Hat one.  But this is how you could shim in as many different  environment variables as you want in order to populate the  config.port variable.  We're also going to populate an IP address.  And we're going to have a connection string for  Postgres.  All of these have a fallback at the end.  Here's our sensible fallback for local development.  So if I run this same code in local dev, it's going to  start up on 127.0.0.1.  Port 3000.  If I run it on OpenShift and the environment variables are  available, it'll take advantage of the environment  variables.  If I run it somewhere else and they have different  environment variables, I include it in this file.  And my code instantly can run on that other platform.  So this is one of the keys to writing your ultra portable  code that'll run just about anywhere.  If you could also set custom environment variables, so one  thing that a lot of applications have a Google  Analytics tracker key.  You don't necessarily want to hard code that into your  source, especially if you're going to deploy to several  different URLs.  So you can specify this as here I have a secret token with  a bunch of junk in here.  I could run ENV set secret token.  It sets my environment variable.  I could do ENV list.  I could see it's populated there.  And if I go back to SSH and connect, I have to reload the  session.  But I could check in there and that secret key would also be  populated in here.  ENV grep secret.  There's the same secret token.  So that's another way to keep your code super nice and clean.  If you want any help with that, RHC ENV help has a lot of  details.  Yeah?  I have a question really, a lot of questions.  Are you doing all this work locally, any local development  before you get to the application or server?  I'm doing this with OpenShift Online.  So this is live right now.  Anyone can go to this URL up top right now with your  cell phone, anything.  Anyone can go.  When you're running the commands, you're running the  commands on your computer.  Oh, yeah.  The laptop must be getting replicated into the remote.  Yeah, this here is running on my laptop here, setting the  environment.  And I also had a shell open as well.  So here's.  Yeah, I didn't find the local way to get that.  I mean, you don't have to know.  Yep, yeah, yeah, yeah.  And the way that works is it uses this command line tool I'm  using is talking to OpenShift has a REST API.  So there's a lot more that you could do with the REST API if  you really wanted to do heavy automation.  That REST API has the most number of features.  The command line has a little bit less.  And then we have a web interface as well.  The web interface, you can't set these environment  variables yet.  So that's one thing I'd really love to see added into the  web interface.  Yeah, good question, though.  OK, so let's get back to slides.  All right, environment variables.  OK, here you could even supply additional environment  variables during your app creation process.  So unfortunately, this is a demo I was working on for  Code for America.  They have an app called adoptahydrant.  And it basically maps all the fire hydrants in town.  And their problem was when it snowed, these fire hydrants  would get covered up in a bunch of snow.  And someone would need to dig them out.  And so they had neighborhood representatives that would  volunteer, I'm going to adopt this hydrant.  And then they had kind of a social app where you could  see, hey, so and so is maintaining that they've all  these neighborhood volunteers are filling the gap for a  lack of government services.  And you really have citizen-driven applications  that are really powerful.  And this is exactly the type of thing you would want to be  portable.  They also spun this up in Hawaii.  And instead of, there's no snow there.  But they have these sirens that alert you if there's a  tsunami warning.  And they had all these birds that were nesting in these  sirens.  And they would need to go keep these tsunami alert sirens  clean of all these nesting birds.  And so that was another similar source code, slightly  adapted.  But hopefully, they could have a common base and then just a  different style sheet on top.  Or just minor differences.  So this one actually, you could see how you could specify an  environment variable here.  But yeah, this one, it takes too long to build and it fails  on OpenShift.  We need to clean up our build process to run a little bit  quicker or allow for longer build times.  At least when you're doing your app create statement.  If I do this in two steps, where I create a Node.js or a  Ruby app, add Postgres, and then add the code, it works  just fine.  But this one liner is where it gets really powerful.  So if you're doing really advanced DB services, let's  say you have a local development machine and you want to  talk to your remote Postgres.  Here, I'm back in my local development.  I could run RHC port forward.  And that will make some local connections to my remote DB  available.  Also my remote server.  So I could go here, there's my mouse.  I could go here and load up this URL.  And it will proxy through to my remote environment.  Same thing with the Postgres.  So if you have PG admin or something locally, boom,  there's your URL.  You can also, I have a blog post on how to use local  development and develop against your remote  OpenShift server.  So if you're on a locked down laptop where you're not  allowed to install Postgres, spin up an OpenShift  environment, set up the port forward, boom, you're done.  So yeah, good stuff there as well.  We also have a post on how to connect to existing  Postgres services from an OpenShift front end.  So if you had a big inner, your own PG database that's  large, you could talk to your own PG database instead of  these small OpenShift ones.  We're kind of spinning up a separate single serve PG for  each development environment.  And if you wanted to talk to a central Postgres master or  one of your slaves, you hit that second link there and  we'll have more information on how to get that done.  So automating your DB setup.  These are still some remaining work here.  We've got just about everything done, but we  haven't enabled Postgres.  We haven't created our table schema.  We haven't added our geospatial index.  We haven't injected our data into our application.  Let's take a look at how that stuff might work.  So here I have, there's a dot OpenShift folder where we'll  allow you to define some action hooks.  Action hooks are basically just scripts.  This one is a bash script.  If it's a Ruby script, that's fine.  We just try to run the script if it's defined.  And we have a couple different scripts.  This one is called deploy.  So this gets run on every deploy.  There's also a one called build and prebuild and  postbuild.  Depending on what you want to automate, you could hook into  several different parts of the process and run an  arbitrary script to help set up your database.  I actually relocated all of the code into something that's  actually run via my package.json.  I didn't want my code to be really specific to OpenShift.  I wanted anyone to be able to initialize their DB, even if  they weren't an OpenShift user.  So let's jump back to the package.json and see where  that init thing is defined.  Wrong file.  That's all of our data here, package.json.  So here you could see in scripts, instead of just the  start script, we have a couple other ones.  There's build.  There's flush DB, init DB.  So there's a couple different things.  And this just runs npm run init DB, which runs node  bin bootstrap.  So let's go take a look at bin bootstrap.  I have some of that code here in the slides.  So here, similar to what we saw before, we're relying on  that config to not refer to anything about how to log in  to our database.  We're using PGQuery for super clean, easy queries.  Here I'm finding the remote connection string from config  and a table name.  I'm going to make my DB connection here.  I'm going to grab all of our points that's in a JSON file.  And I'm going to start loading these into the database.  So the first function we call is init DB.  init DB basically just adds the post GIS extension.  And then runs this callback afterwards.  Create DB schema.  So these basically chain together.  Node.js does everything asynchronously.  So if you want to run things in order, you use callbacks  like this.  So here, we're creating the DB schema next.  And here, it checks for errors.  And then goes and runs this create table.  It creates all the table schema here.  And finally, we end up adding the spatial index here.  And then importing our map points.  Let's see.  I think I do a small amount of translation here.  Yeah, I did some kind of translation.  There was a quote marks in the file that was breaking the  query.  So I did a little search and replace to put double quotes  around the single quote.  I don't know.  That's the only kind of tweak I needed to make with our  input data.  So that's it.  We've got results for Node.js.  Here is the resulting application.  And if you guys want to give that a try for Python Flask,  you can go to flask-post-gis.  These are all the fully completed apps with everything  that's in there.  We also have a Java and hibernate solution.  I don't know if that one works as an instant app.  And I'll show you real quick.  And we're pretty much out of time.  But here is what it would look like.  Here's any kind of app creation for OpenShift.  You type in RHC app create, the name of the app, however  many cartridge dependencies you need.  You could also specify your source code.  And then so specifically here, we could run something like  this.  In order to spin up a new, I'll call this Parks4 maybe.  I don't know how many of these Parks apps I have.  I've got a lot of them.  Here's a live demo of the completed result.  So here are map points set on Boston.  When I drag out, we got some more points added.  As I pan over, any time I pan or zoom, it's redrawing all  the points.  With a limit of 40.  Right?  Pretty fast.  And the Parks data is just national parks and historic  sites.  I could click on any of these pins and see what any of these  points are.  I also added for how many of you guys have seen the clone me on  GitHub banners?  I'm a fan of the clone me on GitHub kind of ribbon that people  have, but I modified it.  This says run me on OpenShift.  So with this click, this goes right into our web-based  workflow for creating apps.  So here I could create Parks5.  Here's my Git source.  I could say, hey, I need extra memory.  Give me a larger size environment.  It already has Python and Postgres listed as my  dependencies.  All this stuff was just encoded into the link.  Here in the URL.  So yeah, create application, turn scaling on and off.  That's one, two clicks.  Click on the ribbon.  Click on create if they already have an OpenShift account.  And you can host your own OpenShift.  So really, really cool stuff there with that web-based  workflow for creating apps.  I have a post on how to customize that workflow and  construct those URL strings.  So whether you like the ribbon or not, if you want to give  someone a link that says launch my app, look up that post  and we've got info on how you can really compose these  apps based on source code and a list of cartridge  dependencies.  Here's a post on how to make those ribbons and add the  ribbons to your app.  Tuning Postgres.  I'm out of time, but I'm going to run like two minutes  over if you guys are OK with that.  All right.  This is some of the good stuff here.  Where the heck did my pghba.conf go?  And where's postgresql.conf?  As a developer, I don't usually, like our theory is  developers shouldn't have to mess with this stuff.  This should be stuff that the operations team and the  developers are collaborating on, a conversation about how  we scale.  And do we scale horizontally in small chunks at a time, or  do we scale in larger chunks at a time?  But either way, we should have a focus on horizontal  scaling and redundancy.  So let's get the developers out of these files, give them  easy on-demand environments, and the ops team can tune  the Postgres cartridge.  Developers can also tune it a bit.  We have some environment variables.  This open shift Postgresql shared buffers allows you to  set the number of shared buffers.  You can also set the number of max connections.  These are all setable just with that RHCE and Vset for  setting environment variables.  And then that'll go into effect.  You may need to reload your database to pick those up,  but pretty easy.  And a lot of the stuff you see in this wiki page on tuning  your Postgres server is pretty easily doable on open shift.  Other advanced configurations.  So here's one issue.  I was talking with the guys from Crunchy Data.  Have you guys heard of anyone familiar with Crunchy Data?  We'll check this out.  These guys gave me a warning that they had some issues with  the Postgres 9.2 cartridge with the statistics collector.  And I don't know if this was after a million records or  after a certain amount.  It's a scale that I haven't hit yet with my tiny little demo  apps.  But they say after a certain number of iterations,  the statistics collector was a little bit buggy in our  Postgres 9.2 cartridge.  So what did they do?  They created their own Postgres 9.3 cartridge for open shift.  This is something that you can run right away.  You know how I did that RHC cartridge ad?  You could specify the URL of this and add Postgres 9.3,  even though we don't support it.  We won't answer the phone calls.  And when you say, hey, my Postgres 9.3 is busted,  we're going to be like, hey, it's not in our list of  supported stuff.  But you can opt in to running arbitrary cartridges from  GitHub.  So that's definitely something good to look at.  This Postgres 9.3 cart from Crunchy Data.  Also, they have a 9.4 Devel with the RLS patch available.  And if you wanted to do highly available Postgres on open  shift in your enterprise environment,  they've got solutions for that as well.  So they just had a big launch announcement.  And they'll do load balancing, geographic failover,  master slave replication, and other things.  But yeah, there's a link to this post if you want  information on bringing all this technology in-house into  your own data center.  I think that's about it.  We accept pull requests.  We want help with everything.  We use PEPs for our major, that's what Python uses.  It's a project enhancement proposal.  We use that for our major feature enhancements.  You could track everything we do is public,  everything we do is open source.  Just check us out online and you could see what we're up to.  We also have this third party index for,  and here's where you could find,  you go to originley.com,  you could add your own cartridge solutions or find other  third party cartridges that people like Crunchy Data are  building.  So there's a link to those cartridges.  And let's see if I could wrap this up.  Oh, we've got some really cool stuff for Docker coming up  in the future.  Keep an eye on OpenShift and our Docker support.  OpenShift Origin is our upstream, like I said,  online three apps.  Everyone can run that.  And that's about it.  If anyone clicked on that ribbon in the mapping app,  they ought to be live with their own mapping solution.  Click there and you could be live with your own app.  All right?  I think that's it for me.  Thanks for sticking around.
Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_0_10259.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_10260_11951.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_11952_14603.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_14604_15839.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_15840_16343.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_16344_18179.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_18180_18491.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_18492_18551.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_18552_19151.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_19152_23087.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_23088_30587.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_30588_33227.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_33228_45587.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_45588_46139.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_46140_47363.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_47364_47591.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_47592_48119.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_48120_48947.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_48948_48995.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_48996_52763.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_52764_52811.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_52812_52943.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_52944_53255.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_53256_53795.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_53796_54131.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_54132_54479.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_54480_55571.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_55572_59111.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_59112_59627.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_59628_59687.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_59688_60083.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_60084_60587.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_60588_61463.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_61464_61955.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_61956_62015.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_62016_62483.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_62484_63119.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_63120_63443.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_63444_63731.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_63732_63899.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_63900_63995.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_63996_67067.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_67068_70199.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_70200_70343.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_70344_70775.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_70776_73643.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_73644_74771.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_74772_76583.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_76584_77195.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_77196_78347.png Writing_portable_postgreSQL-backed_applications_for_the_Open_Cloud_2_78348_78616.png