title: using Jsoup to sign in and crawl data
body: <p>I want to use <code>Jsoup</code> to crawl a page that is only available when I signed in. I guess it means I need to sign in on one page and send cookies to another page.<br>
  I read some earlier post <a href="https://stackoverflow.com/questions/6432970/jsoup-posting-and-cookie">here</a> and write the following code:</p>
  
  <pre><code>public static void main(String[] args) throws IOException {
      Connection.Response res = Jsoup.connect("login.yahoo.com")
          .data("login", "myusername", "passwd", "mypassword")
          .method(Method.POST)
          .execute();
  
  Document doc=res.parse();
  String sessionId = res.cookie("SESSIONID");
  
  Document doc2 = Jsoup.connect("http://health.groups.yahoo.com/group/asthma/messages")
          .cookie("SESSIONID", sessionId)
          .get();
  
  Elements Eles=doc2.getElementsByClass("message");
  
  String content=Eles.first().text();
  
  System.out.println(content);
  </code></pre>
  
  <p>My question is how I can know my cookie name (i.e. "SESSIONID") here for sending my login info? I used the <code>.cookies()</code> method to get all the cookies from the login page:</p>
  
  <blockquote>
    <p>B<br>
    DK<br>
    YM<br>
    T<br>
    PH<br>
    Y<br>
    F</p>
  </blockquote>
  
  <p>I tried them one by one but none worked. I could get sessionId from some of them, but I could not successfully get nodes from the second page, which means I didn't successfully sign in. Could anyone give me some suggestions? Many thanks!</p>