title: Using Spark to extract part of the string in CSV format
body: <p>Spark newbie here and hopefully you guys can give me some help. Thanks!</p>
  
  <p>I am trying to extract a URL from a CSV file and the URL is located at the 16th column. The problem is that the URLs were written in a strange format as you can see from the print out from the code below. What is the best approach to get a the URL in correct format? </p>
  
  <pre><code>case class log(time_stamp: String, url: String )
  
  val logText = sc.textFile("hdfs://...").map(s =&gt; s.split(",")).map( s =&gt; log(s(0).replaceAll("\"", ""),s(15).replaceAll("\"", ""))).toDF()
  
  
  logText.registerTempTable("log")
  
  val results = sqlContext.sql("SELECT * FROM log")
  results.map(s =&gt; "URL: " + s(1)).collect().foreach(println)
  
  URL: /XXX/YYY/ZZZ/http/www.domain.com/xyz/xyz
  URL: /XX/YYY/ZZZ/https/sub.domain.com/xyz/xyz/xyz/xyz
  URL: /XXX/YYY/ZZZ/http/www.domain.com/
  URL: /VV/XXXX/YYY/ZZZ/https/sub.domain.com/xyz/xyz/xyz
  </code></pre>