{"action":"created","comment":{"url":"https://api.github.com/repos/linode/docs/pulls/comments/83505804","pull_request_review_id":4348765,"id":83505804,"diff_hunk":"@@ -0,0 +1,1095 @@\n+---\n+author:\n+    name: Karthik Shiraly\n+    email: docs@linode.com\n+description: 'Deploy Storm cluster on Linode cloud for real-time analytics on streaming datasets.'\n+keywords: 'storm,analytics,big data,zookeeper'\n+license: '[CC BY-ND 4.0](http://creativecommons.org/licenses/by-nd/4.0/)'\n+published: 'Thursday, March 24th, 2016'\n+modified: Thursday, March 24th, 2016\n+modified_by:\n+    name: Linode\n+title: 'Big Data in the Linode Cloud: Streaming Data Processing with Apache Storm'\n+contributor:\n+    name: Karthik Shiraly\n+    link: https://github.com/pathbreak\n+external_resources:\n+- '[Apache Storm project website](http://storm.apache.org/)'\n+- '[Apache Storm documentation](http://storm.apache.org/documentation.html)'\n+- '[Storm - Distributed and Fault-Tolerant Real-time Computation](http://www.infoq.com/presentations/Storm-Introduction)'\n+---\n+\n+[Apache Storm](http://storm.apache.org/) is a big data technology that enables software, data, and infrastructure engineers to process high velocity, high volume data in real time and extract useful information. Any project that involves processing high velocity data streams in real time can benefit from it.\n+\n+[Zookeeper](https://zookeeper.apache.org/) is a critical distributed systems technology that Storm depends on to function correctly. \n+\n+Some use cases where Storm is a good solution:\n+\n+-  Twitter data analytics (for example, trend prediction or sentiment analysis)\n+-  Stock market analysis\n+-  Analysis of server logs\n+-  Internet of Things (IoT) sensor data processing\n+\n+This guide explains how to create Storm clusters on the Linode cloud using a set of shell scripts that use Linode's Application Programming Interfaces (APIs) to programmatically create and configure large clusters. The deployed architecture will look like this:\n+\n+![Architecture](/docs/assets/storm-architecture-650px.png)\n+\n+![Storm Topology and Deployment](/docs/assets/storm-topology.png)\n+\n+The application flow begins, from the client side, with the Storm client, which provides a user interface. This contacts a *Nimbus* node, which is central to the operation of the Storm cluster. The Nimbus node gets the current state of the cluster, including a list of the supervisor nodes and *topologies* from the Zookeeper cluster. The Storm cluster's supervisor nodes constantly update their states to the Zookeeper nodes, which ensure that the system remains synced.\n+\n+The method by which Storm handles and processes data is called a *topology*. A topology is a network of components that perform individual operations, and is made up of *spouts*, which are sources of data, and *bolts*, which accept the incoming data and perform operations such as running functions or transformations. The data itself, called a *stream* in Storm terminology, comes in the form of unbounded sequences of tuples.\n+\n+This guide will explain how to configure a working Storm cluster and its Zookeeper nodes, but it will not provide information on how to develop custom topologies for data processing. For more information on creating and deploying Storm topologies, see the [Apache Storm tutorial](https://storm.apache.org/releases/current/Tutorial.html).\n+\n+## Before You Begin\n+\n+### OS Requirements\n+\n+-  This article assumes that the workstation used for the initial setup of the Cluster Manager Linode is running Ubuntu 14.04 LTS or Debian 8. Other distributions and operating systems have not been tested. \n+-  After the initial setup, any SSH capable workstation can be used to log in to the Cluster Manager Linode or cluster nodes.\n+-  The Cluster Manager Linode can have either Ubuntu 14.04 LTS or Debian 8 installed.\n+-  A Zookeeper or Storm cluster can have either Ubuntu 14.04 LTS or Debian 8 installed on its nodes. Its distribution does not need to be the same one as the one installed on the Cluster Manager Linode.\n+\n+{: .note}\n+>The steps in this guide and in the baSH scripts referenced require root privileges. Be sure to run the steps below as `root`. For more information on privileges, see our [Users and Groups](/docs/tools-reference/linux-users-and-groups) guide.\n+\n+### Naming Conventions\n+\n+Throughout this guide, we will use the following names as examples that refer to the images and clusters we will be creating:\n+\n+-  `zk-image1` - Zookeeper image\n+-  `zk-cluster1` - Zookeeper cluster\n+-  `storm-image1` - Storm image\n+-  `storm-cluster1` - Storm cluster\n+\n+These are the names we'll use, but you are welcome to choose your own when creating your own images and clusters. This guide will use these names in all example commands, so be sure to substitute your own names where applicable.\n+\n+### Get a Linode API Key\n+\n+Follow the steps in [Generating an API Key](/docs/platform/api/api-key) and save your key securely. It will be entered into configuration files in upcoming steps.\n+\n+If the key expires or is removed, remember to create a new one and update the `api_env_linode.conf` API environment configuration file on Cluster Manager Linode. This will be explained further in the next section.\n+\n+## Set Up the Cluster Manager\n+\n+The first step is setting up a central *Cluster Manager* to store details of all Storm clusters, and enable authorized users to create, manage or access those clusters. This can be a local workstation or a Linode.\n+\n+You can set up multiple Cluster Manager nodes if necessary to satisfy organizational or security needs. For example, two departments can each have their own Cluster Manager nodes to manage their respective clusters and keep data access restricted to their own members.\n+\n+1.  The scripts communicate with Linode's API using Python. Install Git, Python 2.7 and curl on your Ubuntu 14.04 LTS or Debian 8 workstation or Linode:\n+\n+        sudo apt-get install python2.7 curl git\n+\n+2.  Download the project git repository:\n+\n+        git clone \"https://github.com/pathbreak/storm-linode\"\n+        cd storm-linode\n+        git checkout $(git describe $(git rev-list --tags='release*' --max-count=1))\n+\n+3.  Make the shell and Python scripts executable:\n+\n+        chmod +x *.sh *.py\n+\n+4.  Make a working copy of the API environment configuration file:\n+\n+        cp api_env_example.conf api_env_linode.conf\n+\n+5.  Open `api_env_linode.conf` in a text editor, and set `LINODE_KEY` to the API key previously created (see [Get a Linode API key](#get-a-linode-api-key)).\n+\n+    {: .file-excerpt}\n+    ~/storm-linode/api_env_linode.conf\n+    :   ~~~\n+        export LINODE_KEY=Enter your API key here\n+        ~~~\n+\n+    Save and close the editor.\n+\n+6.  Open `~/storm-linode/cluster_manager.sh` in a text editor and change the following configuration settings to customize where and how the Cluster Manager Linode is created:\n+\n+    -  `ROOT_PASSWORD`:\n+        This is the root user's password for the Cluster Manager Linode and is **required** to create the node. Set this to a secure password of your choice. Linode requires the root password to contain at least 2 of these 4 character types – lower case characters, upper case characters, numeric characters and symbolic characters.\n+\n+        If you have spaces in your password, make sure the entire password is enclosed in double quotes. If you have double quotes, dollar characters or backslashes in your password, escape each of them with a backslash (`\\`).\n+\n+    -  `PLAN_ID`:\n+        The default value of `1` creates the Cluster Manager Linode as a 2GB node, the smallest plan. This is usually sufficient. However, if you want a more powerful Linode, use the following commands to see a list of all available plans and their IDs:\n+\n+            source ~/storm-linode/api_env_linode.conf\n+            ~/storm-linode/linode_api.py plans\n+\n+    -  `DATACENTER`:\n+        This specifies the Linode datacenter where the Cluster Manager Linode is created. Set it to the ID of the datacenter that is nearest to your location, to reduce network latency. It's also recommended to create the cluster manager node in the same datacenter where the images and cluster nodes will be created, so that it can communicate with them using low latency private IP addresses and reduce data transfer usage.\n+\n+        To view the list of datacenters and their IDs:\n+\n+            source ~/storm-linode/api_env_linode.conf\n+            ~/storm-linode/linode_api.py datacenters table\n+\n+    -  `DISTRIBUTION`:\n+        This is the ID of the distribution to install on the Cluster Manager Linode. This guide has been tested only on Ubuntu 14.04 or Debian 8; other distributions are not supported.\n+\n+        The default value of `124` selects Ubuntu 14.04 LTS 64-bit. If you'd like to use Debian 8 instead, change this value to `140`.\n+\n+    -  `KERNEL`:\n+        This is the ID of the Linux kernel to install on the Cluster Manager Linode. The default value of `138` selects the latest 64-bit Linux kernel available from Linode. It is recommended not to change this setting.   \n+\n+    -  `DISABLE_SSH_PASSWORD_AUTHENTICATION`:\n+        This disables SSH password authentication and allows only key-based SSH authentication for the Cluster Manager Linode. Password authentication is considered less secure, and is hence disabled by default. To enable password authentication, you can change this value to `no`.\n+\n+    {: .note}\n+    > The options shown in this section are generated by the `linode_api.py` script, and differ slightly from the options shown using the Linode CLI tool. Do not use the Linode CLI tool to configure your Manager Node.\n+\n+    When you've finished making changes, save and close the editor.\n+\n+7.  Now, create and set up the Cluster Manager Linode:\n+\n+        ./cluster_manager.sh create-linode api_env_linode.conf\n+\n+    When the node is created, you should see output like this:\n+\n+    ![Cluster Manager creation](/docs/assets/storm-clustermgr-creation-2.png)\n+\n+    Note the public IP address of the Cluster Manager Linode. You will need this when you log into the cluster manager to create or manage clusters.\n+\n+8.  The `cluster_manager.sh` script that was run in the previous step creates three users on the Cluster Manager Linode, and generates authentication keypairs for all of them on your workstation, as shown in this illustration:\n+    \n+    [![Security Overview](/docs/assets/storm-clustermgrkeys-650px.png)](/docs/assets/storm-clustermgrkeys.png)\n+\n+    -  `~/.ssh/clustermgrroot` is the private key for Cluster Manager Linode's *root* user. Access to this user's credentials should be as restricted as possible.\n+\n+    -  `~/.ssh/clustermgr` is the private key for the Cluster Manager Linode's *clustermgr* user. This is a privileged administrative user who can create and manage Storm or Zookeeper clusters. Access to this user's credentials should be as restricted as possible.\n+\n+    -  `~/.ssh/clustermgrguest` is the private key for Cluster Manager Linode's *clustermgrguest* user. This is an unprivileged user for use by anybody who need information about Storm clusters, but not the ability to manage them. These are typically developers, who need to know a cluster's client node IP address to submit topologies to it.\n+\n+    SSH password authentication to the cluster manager is disabled by default. It is recommended to leave the default setting. However, if you want to enable password authentication for just *clustermgrguest* users for convenience, log in as root and append the following line to the *end* of `/etc/ssh/sshd_config`:\n+\n+    {: .file-excerpt}\n+    /etc/ssh/sshd_config\n+    :   ~~~ conf\n+        Match User clustermgrguest\n+          PasswordAuthentication yes\n+        ~~~\n+\n+    Restart the SSH service to enable this change:\n+\n+        service ssh restart\n+\n+    {: .caution}\n+    >\n+    > Since access to the cluster manager provides access to all Storm and Zookeeper clusters and any sensitive data they are processing, its security configuration should be considered critical, and access should be as restrictive as possible.\n+\n+9.  Log in to the Cluster Manager Node as the `root` user, using the public IP address that was shown when you created the node:\n+\n+        ssh -i ~/.ssh/clustermgrroot root@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+\n+10.  Change the hostname to something more descriptive. Here, we are changing it to *clustermgr*, but you may substitute a different name if you like:\n+\n+         sed -i -r \"s/127.0.1.1.*$/127.0.1.1\\tclustermgr/\" /etc/hosts\n+         echo clustermgr > /etc/hostname\n+         hostname clustermgr\n+\n+11.  Set passwords for the *clustermgr* and *clustermgrguest* users:\n+\n+         passwd clustermgr\n+         passwd clustermgrguest\n+\n+        Any administrator logging in as the *clustermgr* user should know this password because they will be asked to enter the password when attempting a privileged command.\n+\n+12.  Delete `cluster_manager.sh` from root user's directory and close the SSH session:\n+\n+         rm cluster_manager.sh\n+         exit\n+\n+13.  Log back in to the Cluster Manager Linode - this time as *clustermgr* user - using its public IP address and the private key for *clustermgr* user:\n+\n+         ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+\n+14.  Navigate to your `storm-linode` directory and make a working copy of `api_env_example.conf`. In this example, we'll call it `api_env_linode.conf`:\n+\n+         cd storm-linode\n+         cp api_env_example.conf api_env_linode.conf\n+\n+15.  Open the newly created `api_env_linode.conf` in a text editor and set `LINODE_KEY` to your API key. Set `CLUSTER_MANAGER_NODE_PASSWORD` to the password you set for the *clustermgr* user in Step 11.\n+\n+        {: .file-excerpt}\n+        ~/storm-linode/api_env_linode.conf\n+        :   ~~~ conf\n+            export LINODE_KEY=Enter your API key here\n+            ...\n+            export CLUSTER_MANAGER_NODE_PASSWORD=Enter a password for the clustermgr user\n+            ~~~\n+\n+            Save your changes and close the editor.\n+\n+16.  The Cluster Manager Linode is now ready to create Apache Storm clusters. Anyone who will manage the clusters should have their public keys added to `/home/clustermgr/.ssh/authorized_keys` so that they can connect via SSH to the Cluster Manager Linode as user *clustermgr*.\n+\n+## Create a Storm Cluster\n+\n+Creating a new Storm cluster involves four main steps, some of which are necessary only the very first time and can be skipped when creating subsequent clusters.\n+\n+### Create a Zookeeper Image\n+\n+A *Zookeeper image* is a master disk image with all necessary Zookeeper softwares and libraries installed. The benefits of using a Zookeeper image include:\n+\n+-  Quick creation of a Zookeeper cluster by simply cloning it to create as many nodes as required, each a perfect copy of the image\n+-  Distribution packages and third party software packages are identical on all nodes, preventing version mismatch errors\n+-  Reduced network usage, because downloads and updates are executed only once when preparing the image instead of repeating them on each node\n+\n+{: .note}\n+>If a Zookeeper image already exists, this step is not mandatory. Multiple Zookeeper clusters can share the same Zookeeper image. In fact, it's a good idea to keep the number of images low because image storage is limited to 10GB.\n+>\n+>When creating an image, you should have `clustermgr` authorization to the Cluster Manager Linode.\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory:\n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Choose a unique name for your image and create a configuration directory for the new image using `new-image-conf` command. In this example, we'll call our new image `zk-image1`:\n+\n+        ./zookeeper-cluster-linode.sh new-image-conf zk-image1\n+\n+    This creates a directory named `zk-image1` containing the files that make up the image configuration:\n+\n+    -  **zk-image1.conf** - This is the main image configuration file, and the one you'll be modifying the most. Its properties are described in the next step. This file is named `zk-image1.conf` in our example, but if you chose a different image name, yours may vary.\n+\n+    -  **zoo.cfg** - This is the primary Zookeeper configuration file. See the official [Zookeeper Configuration Parameters documentation](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_configuration) for details on what parameters can be customized. It's not necessary to enter the cluster's node list in this file. That's done automatically by the script during cluster creation.\n+\n+    -  **log4j.properties** - This file sets the default logging levels for Zookeeper components. You can also customize these at a node level when a cluster is created.\n+\n+    -  **zk-supervisord.conf** - The Zookeeper daemon is run under supervision so that if it shuts down unexpectedly, it's automatically restarted by Supervisord. There is nothing much to customize here, but you can refer to the [Supervisord Configuration documentation](http://supervisord.org/configuration.html) if you want to learn more about the options.\n+\n+3.  Open the image configuration file (in this example, `./zk-image1/zk-image1.conf`) in an editor. Enter or edit values of configuration properties as required. Properties that *must* be entered or changed from their default values are marked as **REQUIRED**:\n+\n+    -  `DISTRIBUTION_FOR_IMAGE`\n+\n+        Specify either Ubuntu 14.04 or Debian 8 to use for this image. This guide has *not* been tested on any other versions or distributions.\n+\n+        All nodes of all clusters created from this image will have this distribution. The default value is `124` corresponding to Ubuntu 14.04 LTS 64-bit. For Debian 8 64-bit, change this value to `140`.\n+\n+    -  `LABEL_FOR_IMAGE`\n+\n+        A label to help you differentiate this image from others. This name will be shown if you edit or view your images in the Linode Manager.\n+\n+    -  `KERNEL_FOR_IMAGE`\n+\n+        The kernel version provided by Linode to use in this image. The default value is `138`, corresponding to the latest 64-bit kernel provided by Linode. It is recommended that you leave this as the default setting.\n+\n+    -  `DATACENTER_FOR_IMAGE`\n+\n+        The Linode datacenter where this image will be created. This can be any Linode datacenter, but cluster creation is faster if the image is created in the same datacenter where the cluster will be created. It's also recommended to create the image in the same datacenter as the Cluster Manager Linode. Select a datacenter that is geographically close to your premises, to reduce network latency.\n+\n+        This value can either be the datacenter's ID or location or abbreviation. To see a list of all datacenters:\n+\n+            ./zookeeper-cluster-linode.sh datacenters api_env_linode.conf\n+\n+    -  `IMAGE_ROOT_PASSWORD` - **REQUIRED**\n+\n+        The default root user password for the image. All nodes of any clusters created from this image will have this as the root password, unless it's overridden in a cluster's configuration file.\n+\n+    -  `IMAGE_ROOT_SSH_PUBLIC_KEY` and `IMAGE_ROOT_SSH_PRIVATE_KEY`\n+\n+        The keypair files for SSH public key authentication as root user. Any user who logs in with this private key will be authenticated as root.\n+\n+        By default, the `cluster_manager.sh` setup has already created a keypair named `clusterroot` and `clusterroot.pub` under `~/.ssh/`. If you wish to replace these with your own keypair, you may create your own keys and set their full paths here.\n+\n+    -  `IMAGE_DISABLE_SSH_PASSWORD_AUTHENTICATION`\n+\n+        This disables SSH password authentication and allows only key based SSH authentication for the cluster nodes. Password authentication is considered less secure, and is hence disabled by default. To enable password authentication, you can change this value to `no`.\n+\n+    -  `IMAGE_ADMIN_USER`\n+\n+        Administrators or developers may have to log in to the cluster nodes for maintenance. Instead of logging in as root users, it's better to log in as a privileged non-root user. The script creates a privileged user with this name in the image (and in all cluster nodes based on this image).\n+\n+    -  `IMAGE_ADMIN_PASSWORD` - **REQUIRED**\n+\n+        Sets the password for the `IMAGE_ADMIN_USER`.\n+\n+    -  `IMAGE_ADMIN_SSH_AUTHORIZED_KEYS`\n+\n+        A file that contains public keys of all personnel authorized to log in to cluster nodes as `IMAGE_ADMIN_USER`. This file should be in the same format as the standard SSH *authorized_keys* file. All the entries in this file are appended to the image's `authorized_keys` file, and get inherited into all nodes based on this image.\n+\n+        By default, the `cluster_manager.sh` setup creates a new `clusteradmin` keypair, and this variable is set to the path of the public key. You can either retain this generated keypair and distribute the generated private key file `~/.ssh/clusteradmin` to authorized personnel. Alternatively, you can collect public keys of authorized personnel and append them to `~/.ssh/clusteradmin.pub`.\n+\n+    -  `IMAGE_DISK_SIZE`\n+\n+        The size of the image disk in MB. The default value of 5000MB is generally sufficient, since the installation only consists of the OS with Java and Zookeeper software installed.\n+\n+    -  `UPGRADE_OS`\n+\n+        If `yes`, the distribution's packages are updated and upgraded before installing any software. It is recommended to leave the default setting to avoid any installation or dependency issues.\n+\n+    -  `INSTALL_ZOOKEEPER_DISTRIBUTION`\n+\n+        The Zookeeper version to install. By default, `cluster_manager.sh` has already downloaded version 3.4.6. If you wish to install a different version, download it manually and change this variable. However, it is recommended to leave the default value as this guide has not been tested against other versions.\n+\n+    -  `ZOOKEEPER_INSTALL_DIRECTORY`\n+\n+        The directory where Zookeeper will be installed on the image (and on all cluster nodes created from this image). \n+\n+    -  `ZOOKEEPER_USER`\n+\n+        The username under which the Zookeeper daemon runs. This is a security feature to avoid privilege escalation by exploiting some vulnerability in the Zookeeper daemon.\n+        \n+    -  `ZOOKEEPER_MAX_HEAP_SIZE`\n+\n+        The maximum Java heap size for the JVM hosting the Zookeeper daemon. This value can be either a percentage, or a fixed value. If the fixed value is not suffixed with any character, it is interpreted as bytes. If it is suffixed with `K`, `M`, or `G`, it is interpreted as kilobytes, megabytes or gigabytes, respectively.\n+\n+        If this is too low, it may result in out of memory errors, and cause data losses or delays in the Storm cluster. If it is set too high, the memory for the OS and its processes will be limited, resulting in disk thrashing, which will have a significant negative impact on Zookeeper's performance. \n+\n+        The default value is 75%, which means at most 75% of the Linode's RAM can be reserved for the JVM, and remaining 25% for the rest of the OS and other processes. It is *strongly* recommended not to change this default setting.\n+\n+    -  `ZOOKEEPER_MIN_HEAP_SIZE`\n+    \n+        The minimum Java heap size to commit for the JVM hosting the Zookeeper daemon. This value can be either a percentage, or a fixed value. If the fixed value is not suffixed with any character, it is interpreted as bytes. If it is suffixed with `K`, `M`, or `G`, it is interpreted as kilobytes, megabytes, or gigabytes, respectively.\n+        \n+        If this value is lower than `ZOOKEEPER_MAX_HEAP_SIZE`, this amount of memory is *committed*, and additional memory up to `ZOOKEEPER_MAX_HEAP_SIZE` is allocated only when the JVM requests it from OS. This can lead to memory allocation delays during operation. So do not set it too low.\n+\n+        This value should never be more than `ZOOKEEPER_MAX_HEAP_SIZE`. If it is, the Zookeeper daemon will not start.\n+\n+        The default value is 75%, which means 75% of the Linode's RAM is committed – not just reserved - to the JVM and unavailable to any other process. It is *strongly* recommended not to change this default setting.\n+\n+    When you've finished making changes, save and close the editor.\n+\n+4.  Create the image using `create-image` command, specifying the name of the newly created image and the API environment file:\n+\n+        ./zookeeper-cluster-linode.sh create-image zk-image1 api_env_linode.conf\n+\n+    If the image is created successfully, the output will look something like this at the end:\n+\n+        Deleting the temporary linode xxxxxx\n+\n+        Finished creating Zookeeper template image yyyyyy\n+\n+    If the process fails, ensure that you do not already have an existing Zookeeper with the same name in the Linode Manager. If you do, delete it and run the command again, or recreate this image with a different name.\n+\n+    {: .note}\n+    >\n+    > During this process, a temporary, short-lived 2GB Linode is created and deleted. This will entail a small cost in your monthly invoice and trigger an event notification email to be sent to the address you have registered with Linode. This is expected behavior.\n+\n+### Create a Zookeeper Cluster\n+\n+In this section, you will learn how to create a new Zookeeper cluster in which every node is a replica of an existing Zookeeper image. If you have not already created a Zookeeper image, do so first by following [Create a Zookeeper image](#create-a-zookeeper-image).\n+\n+{: .note}\n+>\n+>If a Zookeeper cluster already exists, this step is not mandatory. Multiple Storm clusters can share the same Zookeeper cluster. \n+>\n+>When creating a cluster, you should have `clustermgr` authorization to the Cluster Manager Linode.\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory:\n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Choose a unique name for your image and create a configuration directory for the new image using the `new-cluster-conf` command. In this example, we'll call our new image `zk-cluster1`:\n+\n+        ./zookeeper-cluster-linode.sh new-cluster-conf zk-cluster1\n+\n+    This creates a directory named `zk-cluster1` that contains the main configuration file, `zk-cluster1.conf`, which will be described in the next step. If you chose a different name when you ran the previous command, your directory and configuration file will be named accordingly. \n+\n+3.  Open the newly created `zk-cluster1.conf` file and make changes as described below. Properties that must be entered or changed from their default values are marked as **REQUIRED**:\n+\n+    -  `DATACENTER_FOR_CLUSTER`\n+\n+        The Linode datacenter where the nodes of this cluster will be created. All nodes of a cluster have to be in the same datacenter; they cannot span multiple datacenters since they will use private network traffic to communicate.\n+\n+        This can be any Linode datacenter, but cluster creation may be faster if it is created in the same datacenter where the image and Cluster Manager Linode are created. It is recommended to select a datacenter that is geographically close to your premises to reduce network latency.\n+\n+        This value can either be the datacenter's ID or location or abbreviation. To see a list of all datacenters:\n+\n+            ./zookeeper-cluster-linode.sh datacenters api_env_linode.conf\n+\n+    -  `CLUSTER_SIZE`\n+\n+        The types and number of nodes that constitute this cluster. The syntax is: \n+\n+        `plan:count plan:count ... `\n+\n+        A `plan` is one of `2GB | 4GB | ... | 120GB` (see [Linode plans](/pricing) for all plans) and `count` is the number of nodes with that plan.\n+\n+        Examples:\n+\n+        -  For a cluster with three 4GB nodes:\n+\n+                CLUSTER_SIZE=\"4GB:3\"\n+\n+        -  For a cluster with three nodes of different plans:\n+\n+                CLUSTER_SIZE=\"2GB:1 4GB:1 8GB:1\"\n+\n+        The total number of nodes in a Zookeeper cluster *must* be an odd number. Although cluster can have nodes of different plans, it's recommended to use the same plan for all nodes. It is recommended to avoid very large clusters. A cluster with 3-9 nodes is sufficient for most use cases. 11-19 nodes would be considered \"large\". Anything more than 19 nodes would be counterproductive, because at that point, Zookeeper would slow down all the Storm clusters that depend on it.\n+\n+        Size the cluster carefully, because as of version 3.4.6, Zookeeper does not support dynamic expansion. The only way to resize would be to take it down and create a new cluster, creating downtime for any Storm clusters that depend on it.\n+\n+    -  `ZK_IMAGE_CONF` - **REQUIRED**\n+\n+        Path of the Zookeeper image directory or configuration file to use as a template for creating nodes of this cluster. Every node's disk will be a replica of this image.\n+\n+        The path can either be an absolute path, or a path that is relative to the cluster configuration directory. Using our example, the absolute path would be `/home/clustermgr/storm-linode/zk-image1` and the relative path would be `../zk-image1`.\n+\n+    -  `NODE_DISK_SIZE`\n+\n+        Size of each node's disk in MB. This must be at least as large as the selected image's disk, otherwise the image will not copy properly.\n+\n+    -  `NODE_ROOT_PASSWORD`\n+\n+        Optionally, you can specify a root password for the nodes. If this is empty, the root password will be the `IMAGE_ROOT_PASSWORD` in the image configuration file.\n+\n+    -  `NODE_ROOT_SSH_PUBLIC_KEY` and `NODE_ROOT_SSH_PRIVATE_KEY`\n+\n+        Optionally, you can specify a custom SSH public key file and private key file for root user authentication. If this is empty, the keys will be the keys specified in image configuration file.\n+\n+        If you wish to specify your own keypair, select a descriptive filename for this new keypair (example: *zkcluster1root*), generate them using `ssh-keygen`, and set their full paths here.\n+\n+    -  `PUBLIC_HOST_NAME_PREFIX`\n+    \n+        Every Linode in the cluster has a *public IP address*, which can be reached from anywhere on the Internet, and a *private IP address*, which can be reached only from other nodes of the same user inside the same datacenter.\n+\n+        Accordingly, every node is given a *public hostname* that resolves to its public IP address. Each node's public hostname will use this value followed by a number (for example, public-host1, public-host2, etc.) If the cluster manager node is in a different Linode datacenter from the cluster nodes, it uses the public hostnames and public IP addresses to communicate with cluster nodes.\n+\n+    -  `PRIVATE_HOST_NAME_PREFIX`\n+\n+        Every Linode in the cluster is given a *private hostname* that resolves to its private IP address. Each node's private hostname will use this value followed by a number (for example, private-host1, private-host2, etc.). All the nodes of a cluster communicate with one another through their private hostnames. This is also the actual hostname set for the node using the host's `hostname` command and saved in `/etc/hostname`. \n+\n+    -  `CLUSTER_MANAGER_USES_PUBLIC_IP`\n+\n+        Set this value to `false` if the cluster manager node is located in the *same* Linode datacenter as the cluster nodes. This is the recommended value. Change to `true` **only** if the cluster manager node is located in a *different* Linode datacenter from the cluster nodes.\n+\n+        {: .caution}\n+        > It's important to set this correctly to avoid critical cluster creation failures.\n+\n+    -  `ZOOKEEPER_LEADER_CONNECTION_PORT`\n+    \n+        The port used by a Zookeeper node to connect its followers to the leader. When a new leader is elected, each follower opens a TCP connection to the leader at this port. There's no need to change this unless you plan to customize the firewall. \n+\n+    -  `ZOOKEEPER_LEADER_ELECTION_PORT`\n+\n+        The port used for Zookeeper leader election during quorum. There's no need to change this, unless you plan to customize the firewall.\n+\n+    -  `IPTABLES_V4_RULES_TEMPLATE`\n+\n+        Absolute or relative path of the IPv4 iptables firewall rules file. Modify this if you plan to customize the firewall configuration.\n+\n+    -  `IPTABLES_V6_RULES_TEMPLATE`\n+\n+        Absolute or relative path of the IPv6 iptables firewall rules file. IPv6 is completely disabled on all nodes, and no services listen on IPv6 addresses. Modify this if you plan to customize the firewall configuration.\n+\n+    When you've finished making changes, save and close the editor.\n+\n+4.  Create the cluster using the `create` command:\n+\n+        ./zookeeper-cluster-linode.sh create zk-cluster1 api_env_linode.conf\n+\n+    If the cluster is created successfully, a success message is printed:\n+\n+        Zookeeper cluster successfully created\n+\n+    Details of the created cluster can be viewed using the `describe` command:\n+\n+        ./zookeeper-cluster-linode.sh describe zk-cluster1\n+\n+    Cluster nodes are shut down soon after creation. They are started only when any of the Storm clusters starts.\n+\n+### Create a Storm Image\n+\n+A *Storm image* is a master disk with all necessary Storm software and libraries downloaded and installed. The benefits of creating a Storm image include:\n+\n+-  Quick creation of a Storm cluster by simply cloning it to create as many nodes as required, each a perfect copy of the image \n+-  Distribution packages and third party software packages are identical on all nodes, and prevent version mismatch errors\n+-  Reduced network usage, because downloads and updates are executed only once when preparing the image, instead of repeating them on each node\n+\n+{: .note}\n+>If a Storm image already exists, this step is not mandatory. Multiple Storm clusters can share the same Zookeeper image. In fact, it's a good idea to keep the number of images low because image storage is limited to 10GB.\n+>\n+>When creating an image, you should have `clustermgr` authorization to the Cluster Manager Linode.\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory:\n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Choose a unique name for your image and create a configuration directory for the new image using `new-image-conf` command. In this example, we'll call our new image `storm-image1`:\n+\n+        ./storm-cluster-linode.sh new-image-conf storm-image1\n+\n+    This creates a directory named `storm-image1` containing the files that make up the image configuration:\n+\n+    -  **storm-image1.conf** - This is the main image configuration file, and the one you'll be modifying the most. Its properties are described in later steps.  \n+\n+    The other files are secondary configuration files. They contain reasonable default values, but you can always open them in an editor and modify them to suit your needs:  \n+\n+    -  **template-storm.yaml** - The Storm configuration file. See the official [Storm Configuration](http://storm.apache.org/documentation/Configuration.html) documentation for details on what parameters can be customized.\n+\n+    -  **template-storm-supervisord.conf** - The Storm daemon is run under supervision so that if it shuts down unexpectedly, it's automatically restarted by Supervisord. There is nothing much to customize here, but review the [Supervisord Configuration documentation](http://supervisord.org/configuration.html) if you do want to customize it.\n+\n+3.  Open the image configuration file (in this example, `~/storm-linode/storm-image1/storm-image1.conf`) in an editor. Enter or edit values of configuration properties as required. Properties that must be entered or changed from their default values are marked as **REQUIRED**:\n+\n+    -  `DISTRIBUTION_FOR_IMAGE`\n+\n+        Specify either Ubuntu 14.04 or Debian 8 to use for this image. This guide has *not* been tested on any other versions or distributions.\n+\n+        All nodes of all clusters created from this image will have this distribution. The default value is `124` corresponding to Ubuntu 14.04 LTS 64-bit. For Debian 8 64-bit, change this value to `140`.\n+\n+    -  `LABEL_FOR_IMAGE`\n+\n+        A label to help you differentiate this image from others. This name will be shown if you edit or view your images in the Linode Manager.\n+\n+    -  `KERNEL_FOR_IMAGE`\n+\n+        The kernel version provided by Linode to use in this image. The default value is `138` corresponding to the latest 64-bit kernel provided by Linode. It is recommended that you leave this as the default setting.\n+\n+    -  `DATACENTER_FOR_IMAGE`\n+\n+        The Linode datacenter where this image will be created. This can be any Linode datacenter, but cluster creation is faster if the image is created in the same datacenter where the cluster will be created. It's also recommended to create the image in the same datacenter as the Cluster Manager Linode. Select a datacenter that is geographically close to you to reduce network latency.\n+\n+        This value can either be the datacenter's ID or location or abbreviation. To see a list of all datacenters:\n+\n+            ./zookeeper-cluster-linode.sh datacenters api_env_linode.conf \n+\n+    -  `IMAGE_ROOT_PASSWORD` - **REQUIRED**\n+\n+        The default root user password for the image. All nodes of any clusters created from this image will have this as the root password, unless it's overridden in a cluster's configuration file.\n+\n+    -  `IMAGE_ROOT_SSH_PUBLIC_KEY` and `IMAGE_ROOT_SSH_PRIVATE_KEY`\n+\n+        The keypair files for SSH public key authentication as root user. Any user who logs in with this private key will be authenticated as root.\n+\n+        By default, the `cluster_manager.sh` setup has already created a keypair named `clusterroot` and `clusterroot.pub` under `~/.ssh/`. If you wish to replace them with your own keypair, you may create your own keys and set their full paths here.\n+\n+    -  `IMAGE_DISABLE_SSH_PASSWORD_AUTHENTICATION`\n+\n+        This disables SSH password authentication and allows only key based SSH authentication for the cluster nodes. Password authentication is considered less secure, and is hence disabled by default. To enable password authentication, you can change this value to `no`.\n+\n+    -  `IMAGE_ADMIN_USER`\n+\n+        Administrators or developers may have to log in to the cluster nodes for maintenance. Instead of logging in as root users, it's better to log in as a privileged non-root user. The script creates a privileged user with this name in the image (and in all cluster nodes based on this image).\n+\n+    -   `IMAGE_ADMIN_PASSWORD` - **REQUIRED**\n+\n+        Sets the password for the `IMAGE_ADMIN_USER`.\n+\n+    -  `IMAGE_ADMIN_SSH_AUTHORIZED_KEYS`\n+\n+        A file that contains public keys of all personnel authorized to log in to cluster nodes as `IMAGE_ADMIN_USER`. This file should be in the same format as the standard SSH *authorized_keys* file. All the entries in this file are appended to the image's `authorized_keys` file, and get inherited into all nodes based on this image.\n+\n+        By default, the `cluster_manager.sh` setup creates a new `clusteradmin` keypair, and this variable is set to the path of the public key. You can either retain this generated keypair and distribute the generated private key file `~/.ssh/clusteradmin` to authorized personnel. Alternatively, you can collect public keys of authorized personnel and append them to `~/.ssh/clusteradmin.pub`.\n+\n+    -  `IMAGE_DISK_SIZE`\n+    \n+        The size of the image disk in MB. The default value of 5000MB is generally sufficient, since the installation only consists of the OS with Java and Storm software installed.\n+        \n+    -  `UPGRADE_OS`\n+\n+        If `yes`, the distribution's packages are updated and upgraded before installing any software. It is recommended to leave the default setting to avoid any installation or dependency issues.\n+\n+    -  `INSTALL_STORM_DISTRIBUTION`\n+\n+        The Storm version to install. By default, the `cluster_manager.sh` setup has already downloaded version 0.9.5. If you wish to install a different version, download it manually and change this variable. However, it is recommended to leave the default value as this guide has not been tested against other versions.\n+\n+    -  `STORM_INSTALL_DIRECTORY`\n+\n+        The directory where Storm will be installed on the image (and on all cluster nodes created from this image).\n+\n+    -  `STORM_YAML_TEMPLATE`\n+    \n+        The path of the template `storm.yaml` configuration file to install in the image. By default, it points to the `template-storm.yaml` file under the image directory. Administrators can either customize this YAML file before creating the image, or set this variable to point to another `storm.yaml` of their choice.\n+        \n+    -  `STORM_USER`\n+\n+        The username under which the Storm daemon runs. This is a security feature to avoid privilege escalation by exploiting some vulnerability in the Storm daemon.\n+\n+    -  `SUPERVISORD_TEMPLATE_CONF`\n+\n+        The path of the template supervisor configuration file to install in the image. By default, it points to the `template-storm-supervisord.conf` file in the Storm image directory. Administrators can modify this file before creating the image, or set this variable to point to any other `storm-supervisord.conf` file of their choice.\n+\n+    Once you've made changes, save and close the editor.\n+\n+4.  Create the image using the `create-image` command, specifying the name of the newly created image and the API environment file:\n+\n+        ./storm-cluster-linode.sh create-image storm-image1 api_env_linode.conf\n+\n+    If the image is created successfully, the output will look something like this towards the end:\n+\n+        ....\n+        Deleting the temporary linode xxxxxx\n+\n+        Finished creating Storm template image yyyyyy\n+\n+    If the process fails, ensure that you do not already have an existing Storm image with the same name in the Linode Manager. If you do, delete it and run the command again, or recreate this image with a different name.\n+\n+    {: .note}\n+    >\n+    > During this process, a short-lived 2GB linode is created and deleted. This will entail a small cost in the monthly invoice and trigger an event notification email to be sent to the address you have registered with Linode. This is expected behavior.\n+\n+### Create a Storm Cluster\n+\n+In this section, you will learn how to create a new Storm cluster in which every node is a replica of an existing Storm image. If you have not created any Storm images, do so first by following [Create a Storm image](#create-a-storm-image).\n+\n+{: .note}\n+>\n+>When creating a cluster, you should have `clustermgr` authorization to the Cluster Manager Linode.\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory:\n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Choose a unique name for your image and create a configuration directory for the new image using the `new-cluster-conf` command. In this example, we'll call our new image `storm-cluster1`:\n+\n+        ./storm-cluster-linode.sh new-cluster-conf storm-cluster1\n+\n+    This creates a directory named `storm-cluster1` that contains the main configuration file, `storm-cluster1.conf`, which will be described in the next step. If you chose a different name when you ran the previous command, your directory and configuration file will be named accordingly.\n+\n+3.  Open the newly created `storm-cluster1.conf` file and make changes as described below. Properties that must be entered or changed from their default values are marked as **REQUIRED**:\n+\n+    -  `DATACENTER_FOR_CLUSTER`\n+\n+        The Linode datacenter where the nodes of this cluster will be created. All nodes of a cluster have to be in the same datacenter; they cannot span multiple datacenters since they will use private network traffic to communicate.\n+\n+        This can be any Linode datacenter, but cluster creation may be faster if it is created in the same datacenter where the image and Cluster Manager Linode are created. It is recommended to select a datacenter that is geographically close to your premises to reduce network latency.\n+\n+        This value can either be the datacenter's ID or location or abbreviation. To see a list of all datacenters:\n+\n+            ./zookeeper-cluster-linode.sh datacenters api_env_linode.conf\n+\n+    -  `NIMBUS_NODE`\n+\n+        This specifies the Linode plan to use for the Nimbus node, which is responsible for distributing and coordinating a Storm topology to supervisor nodes.\n+\n+        It should be one of `2GB | 4GB | ... | 120GB` (see [Linode plans](/pricing) for all plans). The default size is 2GB, but a larger plan is strongly recommended for the Nimbus node.\n+\n+    -  `SUPERVISOR_NODES`\n+\n+        Supervisor nodes are the workhorses that execute the spouts and bolts that make up a Storm topology. \n+\n+        The size and number of supervisor nodes should be decided based on how many topologies the cluster should run concurrently, and the computational complexities of their spouts and bolts. The syntax is:\n+\n+        `plan:count plan:count ... `\n+\n+        A `plan` is one of `2GB | 4GB| ....| 120GB` (see [Linode plans](/pricing) for all plans) and `count` is the number of supervisor nodes with that plan. Although a cluster can have supervisor nodes of different sizes, it's recommended to use the same plan for all nodes.\n+\n+        The number of supervisor nodes can be increased later using the `add-nodes` command (see [Expand Cluster](#expand-a-storm-cluster)).\n+\n+        Examples: \n+\n+        -  Create three 4GB nodes:\n+\n+                SUPERVISOR_NODES=\"4GB:3\"\n+\n+        -  Create six nodes with three different plans:\n+\n+                SUPERVISOR_NODES=\"2GB:2 4GB:2 8GB:2\"\n+\n+    -  `CLIENT_NODE`\n+\n+        The client node of a cluster is used to submit topologies to it and monitor it. This should be one of `2GB | 4GB | ... | 120GB` (see [Linode plans](/pricing) for all plans). The default value of 2GB is sufficient for most use cases.\n+\n+    -  `STORM_IMAGE_CONF` - **REQUIRED**\n+\n+        Path of the Storm image directory or configuration file to use as a template for creating nodes of this cluster. Every node's disk will be a replica of this image.\n+\n+        The path can either be an absolute path, or a path that is relative to this cluster configuration directory. Using our example, the absolute path would be `/home/clustermgr/storm-linode/storm-image1` and the relative path would be `../storm-image1`.\n+\n+    -  `NODE_DISK_SIZE`\n+\n+        Size of each node's disk in MB. This must be at least as large as the selected image's disk, otherwise the image will not copy properly.\n+\n+    -  `NODE_ROOT_PASSWORD`\n+\n+        Optionally, you can specify a root password for the nodes. If this is empty, the root password will be the `IMAGE_ROOT_PASSWORD` in the image configuration file.\n+\n+    -  `NODE_ROOT_SSH_PUBLIC_KEY` and `NODE_ROOT_SSH_PRIVATE_KEY`\n+\n+        Optionally, you can specify a custom SSH public key file and private key file for root user authentication. If this is empty, the keys will be the keys specified in image configuration file.\n+\n+        If you wish to specify your own keypair, select a descriptive filename for this new keypair (example: *zkcluster1root*), generate them using `ssh-keygen`, and set their full paths here.\n+\n+    -  `NIMBUS_NODE_PUBLIC_HOSTNAME`, `SUPERVISOR_NODES_PUBLIC_HOSTNAME_PREFIX` and  `CLIENT_NODES_PUBLIC_HOSTNAME_PREFIX`\n+\n+        Every Linode in the cluster has a *public IP address*, which can be reached from anywhere on the Internet, and a *private IP address*, which can be reached only from other nodes of the same user inside the same datacenter.\n+\n+        Accordingly, every node is given a *public hostname* that resolves to its public IP address. Each node's public hostname will use this value followed by a number (for example, public-host1, public-host2, etc.) If the cluster manager node is in a different Linode datacenter from the cluster nodes, it uses the public hostnames and public IP addresses to communicate with cluster nodes.\n+\n+    -  `NIMBUS_NODE_PRIVATE_HOSTNAME`, `SUPERVISOR_NODES_PRIVATE_HOSTNAME_PREFIX` and  `CLIENT_NODES_PRIVATE_HOSTNAME_PREFIX`\n+\n+        Every Linode in the cluster is given a *private hostname* that resolves to its private IP address. Each node's private hostname will use this value followed by a number (for example, private-host1, private-host2, etc.). All the nodes of a cluster communicate with one another through their private hostnames. This is also the actual hostname set for the node using the host's `hostname` command and saved in `/etc/hostname`. \n+\n+    -  `CLUSTER_MANAGER_USES_PUBLIC_IP`\n+\n+        Set this value to `false` if the cluster manager node is located in the *same* Linode datacenter as the cluster nodes. This is the recommended value and is also the default. Change to `true` **only** if the cluster manager node is located in a *different* Linode datacenter from the cluster nodes.\n+\n+        {: .caution}\n+        >\n+        > It's important to set this correctly to avoid critical cluster creation failures.\n+\n+    -  `ZOOKEEPER_CLUSTER` - **REQUIRED**\n+\n+        Path of the Zookeeper cluster directory to be used by this Storm cluster.\n+\n+        This can be either an absolute path or a relative path that is relative to this Storm cluster configuration directory. Using our example, the absolute path would be `/home/clustermgr/storm-linode/zk-cluster1`, and the relative path would be `../zk-cluster1`.\n+\n+    -  `IPTABLES_V4_RULES_TEMPLATE`\n+\n+        Absolute or relative path of the IPv4 iptables firewall rules file applied to Nimbus and Supervisor nodes. Modify this if you plan to customize their firewall configuration.\n+\n+    -  `IPTABLES_CLIENT_V4_RULES_TEMPLATE`\n+\n+        Absolute or relative path of the IPv4 iptables firewall rules file applied to Client node. Since the client node hosts a cluster monitoring web server and should be accessible to administrators and developers, its rules are different from those of other nodes. Modify this if you plan to customize its firewall configuration.\n+\n+        Default: `../template-storm-client-iptables-rules.v4`\n+\n+    -  `IPTABLES_V6_RULES_TEMPLATE`\n+\n+        Absolute or relative path of the IPv6 iptables firewall rules file followed for all nodes, including client node. IPv6 is completely disabled on all nodes, and no services listen on IPv6 addresses. Modify this if you plan to customize the firewall configuration.\n+\n+    When you've finished making changes, save and close the editor.\n+\n+9.  Create the cluster using the `create` command:\n+\n+        ./storm-cluster-linode.sh create storm-cluster1 api_env_linode.conf\n+\n+    If the cluster is created successfully, a success message is printed:\n+\n+        Storm cluster successfully created\n+\n+    Details of the created cluster can be viewed using the `describe` command:\n+\n+        ./storm-cluster-linode.sh describe storm-cluster1\n+\n+    Cluster nodes are shut down soon after creation. \n+\n+## Start a Storm Cluster\n+\n+This section will explain how to start a Storm cluster. Doing so will also start any Zookeeper clusters on which it depends, so they do not need to be started separately.\n+\n+{: .note}\n+>\n+>When starting a cluster, you should have `clustermgr` authorization to the Cluster Manager Linode.\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory:\n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Start the Storm cluster using the `start` command. This example uses the `storm-cluster1` naming convention from above, but if you chose a different name you should replace it in the command:\n+\n+        ./storm-cluster-linode.sh start storm-cluster1 api_env_linode.conf\n+\n+3.  If cluster is being started for the very first time, see the next section for how to [authorize users to monitor a Storm cluster](#monitor-a-storm-cluster).\n+\n+## Monitor a Storm Cluster\n+\n+Every Storm cluster's client node runs a Storm UI web application for monitoring that cluster, but it can be accessed only from whitelisted workstations.\n+\n+The next two sections explain how to whitelist workstations and monitor a cluster from the web interface.\n+\n+### Whitelist Workstations to Monitor a Storm Cluster\n+\n+When performing the steps in this section, you should have `clustermgr` authorization to the Cluster Manager Linode.\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory:\n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Open the `your-cluster/your-cluster-client-user-whitelist.ipsets` file (using our example from above, `storm-cluster1/storm-cluster1-client-user-whitelist.ipsets`) file in a text editor. \n+\n+    This file is an [ipsets](http://ipset.netfilter.org/ipset.man.html) list of whitelisted IP addresses. It consists of one master ipset and multiple child ipsets that list whitelisted machines by IP addresses or other attributes such as MAC IDs.\n+\n+    The master ipset is named *your-cluster-uwls*. By default, it's completely empty, which means nobody is authorized.\n+\n+    [![Master ipset](/docs/assets/storm-user-whitelist-1-650px.png)](/docs/assets/storm-user-whitelist-1.png)\n+\n+3.  To whitelist an IP address:\n+\n+    -  Uncomment the line that creates the *your-cluster-ipwl* ipset\n+    -  Add the IP address under it\n+    -  Add *your-cluster-ipwl* to the master ipset *your-cluster-uwls*\n+\n+    These additions are highlighted below:\n+\n+    [![Whitelist entries](/docs/assets/storm-user-whitelist-2-650px.png)](/docs/assets/storm-user-whitelist-2.png)\n+\n+    {: .note}\n+    >\n+    > Any IP address that is being included in the file should be a *public facing IP* address of the network.\n+    > For example, company networks often assign local addresses like 10.x.x.x or 192.x.x.x addresses to employee workstations, which are then NATted to a public IP address while sending requests outside the company network.\n+    > Since the cluster client node is in the Linode cloud outside your company network, it will see monitoring requests as arriving from this public IP address. So it's the public IP address that should be whitelisted.\n+\n+4.  Any number or type of additional ipsets can be created, as long as they are added to the master ipset.\n+\n+    See the **Set Types** section in the [ipset manual](http://ipset.netfilter.org/ipset.man.html#lbAS) for available types of ipsets. Note that some of the types listed in the manual may not be available on the client node because the ipset version installed on it using Ubuntu or Debian package manager is likely to be an older version.\n+\n+5.  Enter all required ipsets, save the file, and close the editor.\n+\n+6.  Activate the new ipsets with the `update-user-whitelist` command:\n+\n+        ./storm-cluster-linode.sh update-user-whitelist storm-cluster1\n+\n+7.  Log in to the client node from the Cluster Manager Linode:\n+\n+        ssh -i ~/.ssh/clusterroot root@storm-cluster1-private-client1\n+\n+    Verify that the new ipsets have been configured correctly:\n+\n+        ipset list\n+\n+    You should see output similar to the following (in addition to custom ipsets if you added them, and the ipsets for the Storm and Zookeeper cluster nodes):\n+\n+    [![ipset output](/docs/assets/storm-user-whitelist-3.png)](/docs/assets/storm-user-whitelist-3.png)\n+\n+    Disconnect from the client node and navigate back to the `storm-linode` directory on the cluster manager node:\n+\n+        exit\n+\n+8.  From the cluster manager node, get the public IP address of the client node. This IP address should be provided to users authorized to access the Storm UI monitoring web application. To show the IP addresses, use the `describe` command:\n+\n+        ./storm-cluster-linode.sh describe storm-cluster1\n+\n+9.  Finally, verify that the Storm UI web application is accessible by opening `http://public-IP-of-client-node` in a web browser on each whitelisted workstation. You should see the Storm UI web application, which looks like this:\n+\n+    [![Storm UI](/docs/assets/storm-ui-650px.png)](/docs/assets/storm-ui-large.png)\n+\n+    The Storm UI displays the list of topologies and the list of supervisors executing them:\n+\n+    [![Storm UI monitoring](/docs/assets/storm-ui-monitor-650px.png)](/docs/assets/storm-ui-monitor.png)\n+\n+    If the cluster is executing any topologies, they are listed under the **Topology summary** section. Click on a topology to access its statistics, supervisor node logs, or actions such as killing that topology.\n+\n+## Test a New Storm Cluster\n+\n+1.  Log in to the Cluster Manager Linode as `clustermgr` and navigate to the `storm-linode` directory: \n+\n+        ssh -i ~/.ssh/clustermgr clustermgr@PUBLIC-IP-OF-CLUSTER-MANAGER-LINODE\n+        cd storm-linode\n+\n+2.  Get the private IP address (this is preferred for security and to minimize impact on the data transfer quota, but the public IP address works as well) of the client node of the target cluster.        \n+\n+        ./storm-cluster-linode.sh describe storm-cluster1\n+\n+3.  Log in to the client node as its `IMAGE_ADMIN_USER` user (the default is `clusteradmin`, configured in the Storm image configuration file) via SSH using an authorized private key: \n+\n+        ssh -i ~/.ssh/clusteradmin clusteradmin@PUBLIC-IP-OF-CLIENT-NODE\n+\n+4.  Run the following commands to start the preinstalled word count example topology:\n+\n+        cd /opt/apache-storm-0.9.5/bin\n+        ./storm jar ../examples/storm-starter/storm-starter-topologies-0.9.5.jar storm.starter.WordCountTopology \"wordcount\"\n+\n+5.  A successful submission should produce output similar to this:\n+\n+        Running: java -client -Dstorm.options= -Dstorm.home=/opt/apache-storm-0.9.5 -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /opt/apache-storm-0.9.5/lib/disruptor-2.10.1.jar:/opt/apache-storm-0.9.5/lib/minlog-1.2.jar:/opt/apache-storm-0.9.5/lib/commons-io-2.4.jar:/opt/apache-storm-0.9.5/lib/clj-time-0.4.1.jar:/opt/apache-storm-0.9.5/lib/clout-1.0.1.jar:/opt/apache-storm-0.9.5/lib/ring-devel-0.3.11.jar:/opt/apache-storm-0.9.5/lib/tools.macro-0.1.0.jar:/opt/apache-storm-0.9.5/lib/ring-jetty-adapter-0.3.11.jar:/opt/apache-storm-0.9.5/lib/jetty-util-6.1.26.jar:/opt/apache-storm-0.9.5/lib/commons-exec-1.1.jar:/opt/apache-storm-0.9.5/lib/tools.cli-0.2.4.jar:/opt/apache-storm-0.9.5/lib/objenesis-1.2.jar:/opt/apache-storm-0.9.5/lib/jetty-6.1.26.jar:/opt/apache-storm-0.9.5/lib/ring-servlet-0.3.11.jar:/opt/apache-storm-0.9.5/lib/storm-core-0.9.5.jar:/opt/apache-storm-0.9.5/lib/hiccup-0.3.6.jar:/opt/apache-storm-0.9.5/lib/clojure-1.5.1.jar:/opt/apache-storm-0.9.5/lib/commons-codec-1.6.jar:/opt/apache-storm-0.9.5/lib/servlet-api-2.5.jar:/opt/apache-storm-0.9.5/lib/compojure-1.1.3.jar:/opt/apache-storm-0.9.5/lib/json-simple-1.1.jar:/opt/apache-storm-0.9.5/lib/commons-logging-1.1.3.jar:/opt/apache-storm-0.9.5/lib/math.numeric-tower-0.0.1.jar:/opt/apache-storm-0.9.5/lib/asm-4.0.jar:/opt/apache-storm-0.9.5/lib/commons-lang-2.5.jar:/opt/apache-storm-0.9.5/lib/clj-stacktrace-0.2.2.jar:/opt/apache-storm-0.9.5/lib/kryo-2.21.jar:/opt/apache-storm-0.9.5/lib/logback-classic-1.0.13.jar:/opt/apache-storm-0.9.5/lib/slf4j-api-1.7.5.jar:/opt/apache-storm-0.9.5/lib/reflectasm-1.07-shaded.jar:/opt/apache-storm-0.9.5/lib/ring-core-1.1.5.jar:/opt/apache-storm-0.9.5/lib/joda-time-2.0.jar:/opt/apache-storm-0.9.5/lib/logback-core-1.0.13.jar:/opt/apache-storm-0.9.5/lib/snakeyaml-1.11.jar:/opt/apache-storm-0.9.5/lib/carbonite-1.4.0.jar:/opt/apache-storm-0.9.5/lib/tools.logging-0.2.3.jar:/opt/apache-storm-0.9.5/lib/core.incubator-0.1.0.jar:/opt/apache-storm-0.9.5/lib/chill-java-0.3.5.jar:/opt/apache-storm-0.9.5/lib/jgrapht-core-0.9.0.jar:/opt/apache-storm-0.9.5/lib/jline-2.11.jar:/opt/apache-storm-0.9.5/lib/commons-fileupload-1.2.1.jar:/opt/apache-storm-0.9.5/lib/log4j-over-slf4j-1.6.6.jar:../examples/storm-starter/storm-starter-topologies-0.9.5.jar:/opt/apache-storm-0.9.5/conf:/opt/apache-storm-0.9.5/bin -Dstorm.jar=../examples/storm-starter/storm-starter-topologies-0.9.5.jar storm.starter.WordCountTopology wordcount\n+        \n+        1038 [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...\n+        1061 [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar ../examples/storm-starter/storm-starter-topologies-0.9.5.jar to assigned location: /var/lib/storm/nimbus/inbox/stormjar-3a9e3c47-88c3-44c2-9084-046f31e57668.jar\n+        Start uploading file '../examples/storm-starter/storm-starter-topologies-0.9.5.jar' to '/var/lib/storm/nimbus/inbox/stormjar-3a9e3c47-88c3-44c2-9084-046f31e57668.jar' (3248678 bytes)\n+        [==================================================] 3248678 / 3248678\n+        File '../examples/storm-starter/storm-starter-topologies-0.9.5.jar' uploaded to '/var/lib/storm/nimbus/inbox/stormjar-3a9e3c47-88c3-44c2-9084-046f31e57668.jar' (3248678 bytes)\n+        1260 [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /var/lib/storm/nimbus/inbox/stormjar-3a9e3c47-88c3-44c2-9084-046f31e57668.jar\n+        1261 [main] INFO  backtype.storm.StormSubmitter - Submitting topology wordcount in distributed mode with conf {\"topology.workers\":3,\"topology.debug\":true}\n+        2076 [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: wordcount\n+\n+5.  Verify that the topology is running correctly by opening the Storm UI in a web browser. The \"wordcount\" topology should be visible in the **Topology Summary** section.\n+\n+The above instructions will use the sample \"wordcount\" topology, which doesn't provide a visible output to show the results of the operations it is running. However, this topology simply counts words in generated sentences, so the number under \"Emitted\" is the actual word count.\n+\n+For a more practical test, feel free to download another topology, such as the [Reddit Comment Sentiment Analysis Topology](https://github.com/pathbreak/reddit-sentiment-storm), which outputs a basic list of threads within given subreddits, based upon which have the most positive and negative comments over time. If you do choose to download a third party topology, be sure it is from a trustworthy source and that you download it to the correct directory.\n+\n+## Start a New Topology\n+\n+If you or a developer have created a topology, perform these steps to start a new topology on one of your Linode Storm clusters:\n+\n+{: .note}\n+> The developer should have `clusteradmin` (or `clusterroot`) authorization to log in to the client node of the target Storm cluster.\n+>\n+> Optionally, to get the IP address of client node, the developer should have `clustermgrguest` (or `clustermgrroot`) authorization to log in to the Cluster Manager Linode. If the IP address is known by other methods, this authorization is not required.\n+\n+1.  Package your topology along with all the third party classes on which they depend into a single JAR (Java Archive) file. \n+\n+2.  If multiple clusters are deployed, select the target Storm cluster to run the topology on. Get the public IP address of the client node of the target cluster. See [cluster description](#describe-a-storm-cluster) for details on how to do this.\n+\n+3.  Transfer the topology JAR from your local workstation to client node:\n+\n+        scp -i ~/.ssh/private-key local-topology-path clusteradmin@public-ip-of-client-node:topology-jar\n+\n+    Substitute `private-key` for the private key of the Storm client, `local-topology-path` for the local filepath of the JAR file, `PUBLIC-IP-OF-CLIENT-NODE` for the IP address of the Storm client, and `topology-jar` for the filepath you'd like to use to store the topology on the client node.\n+\n+4.  Log in to the client node as `clusteradmin`, substituting the appropriate values:\n+\n+        ssh -i ~/.ssh/private-key clusteradmin@PUBLIC-IP-OF-CLIENT-NODE\n+\n+5.  Submit the topology to the cluster:\n+\n+        cd /opt/apache-storm-0.9.5/bin\n+        ./storm jar topology-jar.jar main-class arguments-for-topology\n+\n+    Substitute `topology-jar.jar` for the path of the JAR file you wish to submit, `main-class` with the main class of the topology, and `arguments-for-topology` for the arguments accepted by the topology's main class.\n+\n+6.  [Monitor the execution of the new topology.](#monitor-a-storm-cluster-using-storm-ui)\n+\n+{: .note}\n+> The Storm UI will show only information on the topology's execution, not the actual data it is processing. The data, including its output destination, is handled in the topology's JAR files. \n+\n+## Other Storm Cluster Operations\n+\n+In this section, we'll cover additional operations to manage your Storm cluster once it's up and running.\n+\n+All commands in this section should be performed from the `storm-linode` directory on the cluster manager Linode. You will need `clustermgr` privileges unless otherwise specified.\n+\n+### Expand a Storm Cluster\n+\n+If the supervisor nodes of a St","path":"docs/applications/big-data/big-data-in-the-linode-cloud-streaming-data-processing-with-apache-storm.md","position":null,"original_position":1087,"commit_id":"e11e8e7b4a4c86828d4fb4a8506ba28ed9eaf6fa","original_commit_id":"e11e8e7b4a4c86828d4fb4a8506ba28ed9eaf6fa","user":{"login":"alexfornuto","id":2349184,"avatar_url":"https://avatars.githubusercontent.com/u/2349184?v=3","gravatar_id":"","url":"https://api.github.com/users/alexfornuto","html_url":"https://github.com/alexfornuto","followers_url":"https://api.github.com/users/alexfornuto/followers","following_url":"https://api.github.com/users/alexfornuto/following{/other_user}","gists_url":"https://api.github.com/users/alexfornuto/gists{/gist_id}","starred_url":"https://api.github.com/users/alexfornuto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexfornuto/subscriptions","organizations_url":"https://api.github.com/users/alexfornuto/orgs","repos_url":"https://api.github.com/users/alexfornuto/repos","events_url":"https://api.github.com/users/alexfornuto/events{/privacy}","received_events_url":"https://api.github.com/users/alexfornuto/received_events","type":"User","site_admin":false},"body":"I think we should leave the example in, but explain as you just did that source and target are reverse from normal `cp`, and maybe include the syntax example you noted. Since I've already give you a PR, I'll leave that to you to add after you merge it. I *hate* dealing with merge conflicts.","created_at":"2016-10-14T21:34:46Z","updated_at":"2016-10-14T21:34:46Z","html_url":"https://github.com/linode/docs/pull/419#discussion_r83505804","pull_request_url":"https://api.github.com/repos/linode/docs/pulls/419","_links":{"self":{"href":"https://api.github.com/repos/linode/docs/pulls/comments/83505804"},"html":{"href":"https://github.com/linode/docs/pull/419#discussion_r83505804"},"pull_request":{"href":"https://api.github.com/repos/linode/docs/pulls/419"}}},"pull_request":{"url":"https://api.github.com/repos/linode/docs/pulls/419","id":63965696,"html_url":"https://github.com/linode/docs/pull/419","diff_url":"https://github.com/linode/docs/pull/419.diff","patch_url":"https://github.com/linode/docs/pull/419.patch","issue_url":"https://api.github.com/repos/linode/docs/issues/419","number":419,"state":"open","locked":false,"title":"Big Data in the Linode cloud - Part 1: Streaming Data Processing using Apache Storm","user":{"login":"pathbreak","id":5163416,"avatar_url":"https://avatars.githubusercontent.com/u/5163416?v=3","gravatar_id":"","url":"https://api.github.com/users/pathbreak","html_url":"https://github.com/pathbreak","followers_url":"https://api.github.com/users/pathbreak/followers","following_url":"https://api.github.com/users/pathbreak/following{/other_user}","gists_url":"https://api.github.com/users/pathbreak/gists{/gist_id}","starred_url":"https://api.github.com/users/pathbreak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pathbreak/subscriptions","organizations_url":"https://api.github.com/users/pathbreak/orgs","repos_url":"https://api.github.com/users/pathbreak/repos","events_url":"https://api.github.com/users/pathbreak/events{/privacy}","received_events_url":"https://api.github.com/users/pathbreak/received_events","type":"User","site_admin":false},"body":"I'd like to contribute this article I wrote that describes how to create Storm clusters on the Linode cloud using a set of shell scripts that use Linode's powerful Application Programming Interfaces (APIs) to programmatically create and configure large clusters. ","created_at":"2016-03-23T22:51:30Z","updated_at":"2016-10-14T21:34:46Z","closed_at":null,"merged_at":null,"merge_commit_sha":"fd36e8297513921304b8c9320951c1cb92c0cf69","assignee":{"login":"pbzona","id":16768769,"avatar_url":"https://avatars.githubusercontent.com/u/16768769?v=3","gravatar_id":"","url":"https://api.github.com/users/pbzona","html_url":"https://github.com/pbzona","followers_url":"https://api.github.com/users/pbzona/followers","following_url":"https://api.github.com/users/pbzona/following{/other_user}","gists_url":"https://api.github.com/users/pbzona/gists{/gist_id}","starred_url":"https://api.github.com/users/pbzona/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pbzona/subscriptions","organizations_url":"https://api.github.com/users/pbzona/orgs","repos_url":"https://api.github.com/users/pbzona/repos","events_url":"https://api.github.com/users/pbzona/events{/privacy}","received_events_url":"https://api.github.com/users/pbzona/received_events","type":"User","site_admin":false},"assignees":[{"login":"pbzona","id":16768769,"avatar_url":"https://avatars.githubusercontent.com/u/16768769?v=3","gravatar_id":"","url":"https://api.github.com/users/pbzona","html_url":"https://github.com/pbzona","followers_url":"https://api.github.com/users/pbzona/followers","following_url":"https://api.github.com/users/pbzona/following{/other_user}","gists_url":"https://api.github.com/users/pbzona/gists{/gist_id}","starred_url":"https://api.github.com/users/pbzona/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pbzona/subscriptions","organizations_url":"https://api.github.com/users/pbzona/orgs","repos_url":"https://api.github.com/users/pbzona/repos","events_url":"https://api.github.com/users/pbzona/events{/privacy}","received_events_url":"https://api.github.com/users/pbzona/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/linode/docs/pulls/419/commits","review_comments_url":"https://api.github.com/repos/linode/docs/pulls/419/comments","review_comment_url":"https://api.github.com/repos/linode/docs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/linode/docs/issues/419/comments","statuses_url":"https://api.github.com/repos/linode/docs/statuses/e11e8e7b4a4c86828d4fb4a8506ba28ed9eaf6fa","head":{"label":"pathbreak:bigdata-storm","ref":"bigdata-storm","sha":"e11e8e7b4a4c86828d4fb4a8506ba28ed9eaf6fa","user":{"login":"pathbreak","id":5163416,"avatar_url":"https://avatars.githubusercontent.com/u/5163416?v=3","gravatar_id":"","url":"https://api.github.com/users/pathbreak","html_url":"https://github.com/pathbreak","followers_url":"https://api.github.com/users/pathbreak/followers","following_url":"https://api.github.com/users/pathbreak/following{/other_user}","gists_url":"https://api.github.com/users/pathbreak/gists{/gist_id}","starred_url":"https://api.github.com/users/pathbreak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pathbreak/subscriptions","organizations_url":"https://api.github.com/users/pathbreak/orgs","repos_url":"https://api.github.com/users/pathbreak/repos","events_url":"https://api.github.com/users/pathbreak/events{/privacy}","received_events_url":"https://api.github.com/users/pathbreak/received_events","type":"User","site_admin":false},"repo":{"id":54596763,"name":"docs","full_name":"pathbreak/docs","owner":{"login":"pathbreak","id":5163416,"avatar_url":"https://avatars.githubusercontent.com/u/5163416?v=3","gravatar_id":"","url":"https://api.github.com/users/pathbreak","html_url":"https://github.com/pathbreak","followers_url":"https://api.github.com/users/pathbreak/followers","following_url":"https://api.github.com/users/pathbreak/following{/other_user}","gists_url":"https://api.github.com/users/pathbreak/gists{/gist_id}","starred_url":"https://api.github.com/users/pathbreak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pathbreak/subscriptions","organizations_url":"https://api.github.com/users/pathbreak/orgs","repos_url":"https://api.github.com/users/pathbreak/repos","events_url":"https://api.github.com/users/pathbreak/events{/privacy}","received_events_url":"https://api.github.com/users/pathbreak/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/pathbreak/docs","description":"Linode guides and tutorials.","fork":true,"url":"https://api.github.com/repos/pathbreak/docs","forks_url":"https://api.github.com/repos/pathbreak/docs/forks","keys_url":"https://api.github.com/repos/pathbreak/docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pathbreak/docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pathbreak/docs/teams","hooks_url":"https://api.github.com/repos/pathbreak/docs/hooks","issue_events_url":"https://api.github.com/repos/pathbreak/docs/issues/events{/number}","events_url":"https://api.github.com/repos/pathbreak/docs/events","assignees_url":"https://api.github.com/repos/pathbreak/docs/assignees{/user}","branches_url":"https://api.github.com/repos/pathbreak/docs/branches{/branch}","tags_url":"https://api.github.com/repos/pathbreak/docs/tags","blobs_url":"https://api.github.com/repos/pathbreak/docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pathbreak/docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pathbreak/docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/pathbreak/docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pathbreak/docs/statuses/{sha}","languages_url":"https://api.github.com/repos/pathbreak/docs/languages","stargazers_url":"https://api.github.com/repos/pathbreak/docs/stargazers","contributors_url":"https://api.github.com/repos/pathbreak/docs/contributors","subscribers_url":"https://api.github.com/repos/pathbreak/docs/subscribers","subscription_url":"https://api.github.com/repos/pathbreak/docs/subscription","commits_url":"https://api.github.com/repos/pathbreak/docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/pathbreak/docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/pathbreak/docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/pathbreak/docs/issues/comments{/number}","contents_url":"https://api.github.com/repos/pathbreak/docs/contents/{+path}","compare_url":"https://api.github.com/repos/pathbreak/docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pathbreak/docs/merges","archive_url":"https://api.github.com/repos/pathbreak/docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pathbreak/docs/downloads","issues_url":"https://api.github.com/repos/pathbreak/docs/issues{/number}","pulls_url":"https://api.github.com/repos/pathbreak/docs/pulls{/number}","milestones_url":"https://api.github.com/repos/pathbreak/docs/milestones{/number}","notifications_url":"https://api.github.com/repos/pathbreak/docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pathbreak/docs/labels{/name}","releases_url":"https://api.github.com/repos/pathbreak/docs/releases{/id}","deployments_url":"https://api.github.com/repos/pathbreak/docs/deployments","created_at":"2016-03-23T22:17:30Z","updated_at":"2016-03-19T06:57:50Z","pushed_at":"2016-10-14T17:23:40Z","git_url":"git://github.com/pathbreak/docs.git","ssh_url":"git@github.com:pathbreak/docs.git","clone_url":"https://github.com/pathbreak/docs.git","svn_url":"https://github.com/pathbreak/docs","homepage":"https://www.linode.com/docs/","size":122392,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":1,"forks":0,"open_issues":1,"watchers":0,"default_branch":"master"}},"base":{"label":"linode:master","ref":"master","sha":"0690a81aa2d06ddaaf17c67f13a4775c6d6bfcc9","user":{"login":"linode","id":5184305,"avatar_url":"https://avatars.githubusercontent.com/u/5184305?v=3","gravatar_id":"","url":"https://api.github.com/users/linode","html_url":"https://github.com/linode","followers_url":"https://api.github.com/users/linode/followers","following_url":"https://api.github.com/users/linode/following{/other_user}","gists_url":"https://api.github.com/users/linode/gists{/gist_id}","starred_url":"https://api.github.com/users/linode/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/linode/subscriptions","organizations_url":"https://api.github.com/users/linode/orgs","repos_url":"https://api.github.com/users/linode/repos","events_url":"https://api.github.com/users/linode/events{/privacy}","received_events_url":"https://api.github.com/users/linode/received_events","type":"Organization","site_admin":false},"repo":{"id":22214885,"name":"docs","full_name":"linode/docs","owner":{"login":"linode","id":5184305,"avatar_url":"https://avatars.githubusercontent.com/u/5184305?v=3","gravatar_id":"","url":"https://api.github.com/users/linode","html_url":"https://github.com/linode","followers_url":"https://api.github.com/users/linode/followers","following_url":"https://api.github.com/users/linode/following{/other_user}","gists_url":"https://api.github.com/users/linode/gists{/gist_id}","starred_url":"https://api.github.com/users/linode/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/linode/subscriptions","organizations_url":"https://api.github.com/users/linode/orgs","repos_url":"https://api.github.com/users/linode/repos","events_url":"https://api.github.com/users/linode/events{/privacy}","received_events_url":"https://api.github.com/users/linode/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/linode/docs","description":"Linode guides and tutorials.","fork":false,"url":"https://api.github.com/repos/linode/docs","forks_url":"https://api.github.com/repos/linode/docs/forks","keys_url":"https://api.github.com/repos/linode/docs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/linode/docs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/linode/docs/teams","hooks_url":"https://api.github.com/repos/linode/docs/hooks","issue_events_url":"https://api.github.com/repos/linode/docs/issues/events{/number}","events_url":"https://api.github.com/repos/linode/docs/events","assignees_url":"https://api.github.com/repos/linode/docs/assignees{/user}","branches_url":"https://api.github.com/repos/linode/docs/branches{/branch}","tags_url":"https://api.github.com/repos/linode/docs/tags","blobs_url":"https://api.github.com/repos/linode/docs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/linode/docs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/linode/docs/git/refs{/sha}","trees_url":"https://api.github.com/repos/linode/docs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/linode/docs/statuses/{sha}","languages_url":"https://api.github.com/repos/linode/docs/languages","stargazers_url":"https://api.github.com/repos/linode/docs/stargazers","contributors_url":"https://api.github.com/repos/linode/docs/contributors","subscribers_url":"https://api.github.com/repos/linode/docs/subscribers","subscription_url":"https://api.github.com/repos/linode/docs/subscription","commits_url":"https://api.github.com/repos/linode/docs/commits{/sha}","git_commits_url":"https://api.github.com/repos/linode/docs/git/commits{/sha}","comments_url":"https://api.github.com/repos/linode/docs/comments{/number}","issue_comment_url":"https://api.github.com/repos/linode/docs/issues/comments{/number}","contents_url":"https://api.github.com/repos/linode/docs/contents/{+path}","compare_url":"https://api.github.com/repos/linode/docs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/linode/docs/merges","archive_url":"https://api.github.com/repos/linode/docs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/linode/docs/downloads","issues_url":"https://api.github.com/repos/linode/docs/issues{/number}","pulls_url":"https://api.github.com/repos/linode/docs/pulls{/number}","milestones_url":"https://api.github.com/repos/linode/docs/milestones{/number}","notifications_url":"https://api.github.com/repos/linode/docs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/linode/docs/labels{/name}","releases_url":"https://api.github.com/repos/linode/docs/releases{/id}","deployments_url":"https://api.github.com/repos/linode/docs/deployments","created_at":"2014-07-24T13:57:47Z","updated_at":"2016-10-14T05:04:43Z","pushed_at":"2016-10-11T12:17:12Z","git_url":"git://github.com/linode/docs.git","ssh_url":"git@github.com:linode/docs.git","clone_url":"https://github.com/linode/docs.git","svn_url":"https://github.com/linode/docs","homepage":"https://www.linode.com/docs/","size":149723,"stargazers_count":467,"watchers_count":467,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":471,"mirror_url":null,"open_issues_count":40,"forks":471,"open_issues":40,"watchers":467,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/linode/docs/pulls/419"},"html":{"href":"https://github.com/linode/docs/pull/419"},"issue":{"href":"https://api.github.com/repos/linode/docs/issues/419"},"comments":{"href":"https://api.github.com/repos/linode/docs/issues/419/comments"},"review_comments":{"href":"https://api.github.com/repos/linode/docs/pulls/419/comments"},"review_comment":{"href":"https://api.github.com/repos/linode/docs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/linode/docs/pulls/419/commits"},"statuses":{"href":"https://api.github.com/repos/linode/docs/statuses/e11e8e7b4a4c86828d4fb4a8506ba28ed9eaf6fa"}}}}
{"actor":{"display_login":"alexfornuto"}}