title: Deploy Storm topology remotely using "storm jar" command on Windows
body: <p>I am trying to deploy a storm topology to a remote machine using the storm jar command on Windows</p>
  
  <pre><code>storm jar jarName.jar mainClass
  </code></pre>
  
  <p>Here is what I have in my storm.yaml file:</p>
  
  <pre><code>storm.zookeeper.servers:
       - "HostName"
  
  nimbus.host: "HostName"
  nimbus.thrift.port: 6627
  </code></pre>
  
  <p>But whenever I try to deploy, it always tries to deploy to my local machine.</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>I tried putting the following configurations but it does not work:</p>
  
  <pre><code>conf.put(Config.NIMBUS_HOST, "HostName");
  conf.put(Config.NIMBUS_THRIFT_PORT,6627);
  conf.put(Config.STORM_ZOOKEEPER_SERVERS,Arrays.asList(new String[]{"HostName"}));
  conf.put(Config.STORM_ZOOKEEPER_PORT,2181);
  </code></pre>
  
  <p>I also tried the -c option on the command line as:</p>
  
  <pre><code>storm -c nimbus.host=HostName jar jarName.jar mainClass 
  </code></pre>
  
  <p>But it does not work; I get:</p>
  
  <pre><code>Error: Could not create the Java Virtual Machine.
  Error: A fatal exception has occurred. Program will exit.
  </code></pre>
  
  <p>I tried placing the -c part at different places but nothing seems to work.</p>
  
  <p>Also, I get </p>
  
  <pre><code>The field Config.NIMBUS_HOST is deprecated
  </code></pre>
  
  <p>when I set:</p>
  
  <pre><code>conf.put(Config.NIMBUS_HOST, "HostName");
  </code></pre>
  
  <p>I am using Storm version <strong>0.10.0-beta</strong></p>
  
  <p><strong>EDIT2</strong></p>
  
  <p>Submitting works perfectly fine from IDE after I use</p>
  
  <pre><code>System.setProperty("storm.jar", "path to jar"); 
  </code></pre>
  
  <p>however, it does not work from command line.</p>
  
  <p><strong>Edit3</strong></p>
  
  <p>Thanks Matt for all your help. I was able to deploy the topology using the java command that eclipse executes. I am still unable to deploy using the storm jar command however. It complains that the topology already exists even though i dont see anything on the Storm UI. Also, when i deploy using Eclipse or the Java command from command line, the topology deploys successfully. How should i compare the difference between executing the java command and what the storm jar command is doing? i tried to echo the variables in this section of the cmd file </p>
  
  <pre><code> if %storm-command% == jar (
      set STORM_OPTS=%STORM_CLIENT_OPTS% %STORM_OPTS% -Dstorm.jar=%2
      echo %STORM_OPTS%
      set CLASSPATH=%CLASSPATH%;%2
  
      echo %CLASSPATH%
  
      set CLASS=%3
      set args=%4
      goto start
      :start
      shift
      if [%4] == [] goto done
      set args=%args% %4
  
      echo args: %args%
  
      goto start
  
      :done
      echo %args%
      set storm-command-arguments=%args%
  
    )
  </code></pre>
  
  <p>but i dont see anything unusual. Also, i see the following exception in the stacktrace</p>
  
  <pre><code>    Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
  </code></pre>
  
  <p>notice </p>
  
  <pre><code> topologies:[]
  </code></pre>
  
  <p>and then it complains that the topology  name exists</p>
  
  <pre><code>Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
          at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:342)
          at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:226)
          at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
  
          at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
  
          at mainClass.submitTopology(mainClass.java:127)
          at mainClass.main(mainClass.java:57)
  Caused by: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
          at backtype.storm.generated.ClusterSummary.validate(ClusterSummary.java:515)
          at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
  read(ClusterSummary.java:613)
          at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
  read(ClusterSummary.java:549)
          at backtype.storm.generated.ClusterSummary.read(ClusterSummary.java:473)
  
          at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
  resultStandardScheme.read(Nimbus.java:16546)
          at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
  resultStandardScheme.read(Nimbus.java:16531)
          at backtype.storm.generated.Nimbus$getClusterInfo_result.read(Nimbus.java:16470)
          at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)
  
          at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:569)
          at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:557)
          at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:333)
          ... 5 more
  </code></pre>
  
  <p>finally, here is the entire stacktrace.</p>
  
  <pre><code> X:\abc\xyz\target&gt;ERROR StatusLogger Invalid URL C:/asd/ewt/apache-storm 0.10.0-beta/log4j2/cluster.xml java.net.MalformedURLException: unknown protocol: c
              at java.net.URL.&lt;init&gt;(URL.java:593)
              at java.net.URL.&lt;init&gt;(URL.java:483)
              at java.net.URL.&lt;init&gt;(URL.java:432)
              at java.net.URI.toURL(URI.java:1089)
              at org.apache.logging.log4j.core.config.ConfigurationFactory.getInputFromUri(ConfigurationFactory.java:296)
              at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:395)
              at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:254)
              at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:419)
              at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:138)
              at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:147)
              at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
              at org.apache.logging.log4j.LogManager.getContext(LogManager.java:175)
              at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:102)
              at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
              at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
              at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
              at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:277)
              at org.apache.log4j.Category.&lt;init&gt;(Category.java:56)
              at org.apache.log4j.Logger.&lt;init&gt;(Logger.java:35)
              at org.apache.log4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:59)
              at org.apache.log4j.Logger.getLogger(Logger.java:39)
              at mainClass.&lt;clinit&gt;(mainClass.java:32)
  Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protocol.TProtocolException: Required field nimbus_uptime_secs' is unset!Struct:ClusterSummary(supervisors:SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
                  at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:342)
                  at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:226)
                  at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
  
                  at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
  
                  at mainClass.submitTopology(mainClass.java:127)
                  at mainClass.main(mainClass.java:57)
          Caused by: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:[SupervisorSummary(host:HostName, uptime_secs:76688, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
                  at backtype.storm.generated.ClusterSummary.validate(ClusterSummary.java:515)
                  at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
          read(ClusterSummary.java:613)
                  at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.
          read(ClusterSummary.java:549)
                  at backtype.storm.generated.ClusterSummary.read(ClusterSummary.java:473)
  
                  at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
          resultStandardScheme.read(Nimbus.java:16546)
                  at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_
          resultStandardScheme.read(Nimbus.java:16531)
                  at backtype.storm.generated.Nimbus$getClusterInfo_result.read(Nimbus.java:16470)
                  at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)
  
                  at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:569)
                  at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:557)
                  at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:333)
                  ... 5 more
  </code></pre>
  
  <p>and by the way what is that weird logging exception/error at the top of the stacktrace? could that have something to do with the weird behavior of storm jar command? </p>
  
  <p>if i do </p>
  
  <pre><code>java -cp jarName.jar;C:\abc\asdasd\apache-storm-0.10.0-beta\lib\* mainClass
  </code></pre>
  
  <p>i get </p>
  
  <pre><code>513  [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
  582  [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
  590  [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for MD
  5-digest: -6100635753051151148:-8346710213172614614
  591  [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
  599  [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeM
  s [2000] the maxSleepTimeMs [60000] the maxRetries [5]
  626  [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeM
  s [2000] the maxSleepTimeMs [60000] the maxRetries [5]
  Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.protoc
  ol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:SupervisorSummary(host:HostName, uptime_secs:83775, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
          at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:342)
          at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:226)
          at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
  
          at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
  
          at com.rbccm.th2015.storm.SampleTopologyX.submitTopology(SampleTopologyX.java:127)
          at com.rbccm.th2015.storm.SampleTopologyX.main(SampleTopologyX.java:57)
  Caused by: org.apache.thrift7.protocol.TProtocolException: Required field 'nimbus_uptime_secs' is unset! Struct:ClusterSummary(supervisors:SupervisorSummary(host:HostName,uptime_secs:83775, num_workers:2, num_used_workers:0, supervisor_id:7e4b13c5-05d5-4944-9566-d71f2c2de504, version:0.10.0.2.3.0.0-2557)], nimbus_uptime_secs:0, topologies:[])
          at backtype.storm.generated.ClusterSummary.validate(ClusterSummary.java:515)
          at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.read(ClusterSummary.java:613)
          at backtype.storm.generated.ClusterSummary$ClusterSummaryStandardScheme.read(ClusterSummary.java:549)
          at backtype.storm.generated.ClusterSummary.read(ClusterSummary.java:473)
  
          at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_resultStandardScheme.read(Nimbus.java:16546)
          at backtype.storm.generated.Nimbus$getClusterInfo_result$getClusterInfo_resultStandardScheme.read(Nimbus.java:16531)
          at backtype.storm.generated.Nimbus$getClusterInfo_result.read(Nimbus.java:16470)
          at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)
  
          at backtype.storm.generated.Nimbus$Client.recv_getClusterInfo(Nimbus.java:569)
          at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:557)
          at backtype.storm.StormSubmitter.topologyNameExists(StormSubmitter.java:333)
          ... 5 more
  </code></pre>
  
  <p>so its complaining about the same thing but via a different route. could this have something to do with my YAML? and what is this business about nimbus_uptime_secs?</p>
  
  <p><strong>Final Edit</strong></p>
  
  <p>ok, time to put this to rest. For now at least, it does not look like the sotrm jar command is suitable for windows. I can clearly see a difference in the command that storm.py tries to execute in Windows and the one that it tries to execute in Linux. Basically, the paths get screwed up in windows and that must be what's causing all of these weird exceptions.</p>
  
  <p>Thanks Matt for all of your help and support. I really learned a lot.</p>