title: How to access a web URL using a spark context
body: <p>I am trying to use a WEB URL from spark-shell using textFile method, but getting error. Probably this is not the right way. So can someone please tell me how to access a web URL from spark context.</p>
  
  <p>I am using spark version 1.3.0 ; Scala version 2.10.4 and Java 1.7.0_21</p>
  
  <p><div class="snippet" data-lang="js" data-hide="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>hduser@ubuntu:~$ spark-shell
  Spark assembly has been built with Hive, including Datanucleus jars on classpath
  Welcome to
        __              
       / /   / /
      \ \/  \/  `/ _/  '_/
     /___/ .__/\_,_/_/ /_/\_\   version 1.3.0
        /_/
  
  Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_21)
  Type in expressions to have them evaluated.
  Type :help for more information.
  Spark context available as sc.
  SQL context available as sqlContext.
  
  scala&gt; val pagecount = sc.textFile( "https://www.google.co.in/?gws_rd=ssl" )
  pagecount: org.apache.spark.rdd.RDD[String] = https://www.google.co.in/?gws_rd=ssl MapPartitionsRDD[1] at textFile at &lt;console&gt;:21
  
  scala&gt; pagecount.count()
  java.io.IOException: No FileSystem for scheme: https
   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1383)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
   at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
   at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:176)
   at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
   at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:203)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
   at scala.Option.getOrElse(Option.scala:120)
   at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
   at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:32)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:219)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:217)
   at scala.Option.getOrElse(Option.scala:120)
   at org.apache.spark.rdd.RDD.partitions(RDD.scala:217)
   at org.apache.spark.SparkContext.runJob(SparkContext.scala:1511)
   at org.apache.spark.rdd.RDD.count(RDD.scala:1006)
   at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.&lt;init&gt;(&lt;console&gt;:24)
   at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.&lt;init&gt;(&lt;console&gt;:29)
   at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.&lt;init&gt;(&lt;console&gt;:31)
   at $iwC$$iwC$$iwC$$iwC$$iwC.&lt;init&gt;(&lt;console&gt;:33)
   at $iwC$$iwC$$iwC$$iwC.&lt;init&gt;(&lt;console&gt;:35)
   at $iwC$$iwC$$iwC.&lt;init&gt;(&lt;console&gt;:37)
   at $iwC$$iwC.&lt;init&gt;(&lt;console&gt;:39)
   at $iwC.&lt;init&gt;(&lt;console&gt;:41)
   at &lt;init&gt;(&lt;console&gt;:43)
   at .&lt;init&gt;(&lt;console&gt;:47)
   at .&lt;clinit&gt;(&lt;console&gt;)
   at .&lt;init&gt;(&lt;console&gt;:7)
   at .&lt;clinit&gt;(&lt;console&gt;)
   at $print(&lt;console&gt;)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)
   at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338)
   at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)
   at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
   at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
   at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:856)
   at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:901)
   at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:813)
   at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:656)
   at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:664)
   at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:669)
   at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:996)
   at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:944)
   at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:944)
   at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
   at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:944)
   at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1058)
   at org.apache.spark.repl.Main$.main(Main.scala:31)
   at org.apache.spark.repl.Main.main(Main.scala)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)
   at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)</code></pre>
  </div>
  </div>
  </p>