title: Browser doesnt support frames
body: <p>I am trying to create a java program that performs a login against an <a href="http://www.achievo.org/" rel="nofollow">achievo</a> instance. I am trying to use <a href="http://en.wikipedia.org/wiki/Screen_scraping#Screen_scraping" rel="nofollow">Screen Scraping</a>. </p>
  
  <p>I manage to login using the following code:</p>
  
  <pre><code>@Test
  public void testLogin() throws Exception {
      HashMap&lt;String, String&gt; data = new HashMap&lt;String, String&gt;();
      data.put("auth_user", "user");
      data.put("auth_pw", "password");
      doSubmit("https://someurl.com/achievo/index.php", data);
  }
  
  private void doSubmit(String url, HashMap&lt;String, String&gt; data) throws Exception {
      URL siteUrl = new URL(url);
      HttpsURLConnection conn = (HttpsURLConnection) siteUrl.openConnection();
      conn.setRequestMethod("POST");
      conn.setDoOutput(true);
      conn.setDoInput(true);
      //conn.setRequestProperty( "User-agent", "spider" );
      //conn.setRequestProperty("User-agent", "Opera/9.80 (X11; Linux i686; U; en) Presto/2.7.62 Version/11.01");
  
      conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 1.2.30703)");
  
      DataOutputStream out = new DataOutputStream(conn.getOutputStream());
  
      Set&lt;String&gt; keys = data.keySet();
      Iterator&lt;String&gt; keyIter = keys.iterator();
      StringBuilder content = new StringBuilder("");
      for(int i=0; keyIter.hasNext(); i++) {
          Object key = keyIter.next();
          if(i!=0) {
              content.append("&amp;");
          }
          content.append(key + "=" + URLEncoder.encode(data.get(key), "UTF-8"));
      }
      System.out.println(content.toString());
  
      out.writeBytes(content.toString());
      out.flush();
      out.close();
      BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
      String line = "";
      while((line=in.readLine())!=null) {
          System.out.println(line);
      }
      in.close();
  }
  </code></pre>
  
  <p>However, when achievo successfully logs-in, I get redirected to the main page where it says:</p>
  
  <pre><code>&lt;head&gt;
      &lt;META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"&gt;
      &lt;title&gt;Achievo&lt;/title&gt;
    &lt;/head&gt;
      &lt;frameset rows="113,*" frameborder="0" border="0"&gt;
      &lt;frame name="top" scrolling="no" noresize src="top.php?atklevel=-1&amp;atkprevlevel=0&amp;achievo=37b552462afdfd248a21fedbf0eebe43" marginwidth="0" marginheight="0"&gt;
      &lt;frameset cols="210,*" frameborder="0" border="0"&gt;
        &lt;frame name="menu" scrolling="no" noresize src="menu.php?atklevel=-1&amp;atkprevlevel=0&amp;achievo=37b552462afdfd248a21fedbf0eebe43" marginwidth="0" marginheight="0"&gt;
        &lt;frame name="main" scrolling="auto" noresize src="dispatch.php?atknodetype=pim.pim&amp;atkaction=pim&amp;atklevel=-1&amp;atkprevlevel=0&amp;achievo=37b552462afdfd248a21fedbf0eebe43" marginwidth="0" marginheight="0"&gt;
      &lt;/frameset&gt;
      &lt;noframes&gt;
        &lt;body bgcolor="#CCCCCC" text="#000000"&gt;
          &lt;p&gt;Your browser doesnt support frames, but this is required to run Achievo&lt;/p&gt;
        &lt;/body&gt;
      &lt;/noframes&gt;
    &lt;/frameset&gt;
  </code></pre>
  
  <p>Obviously I get the <strong>Your browser doesnt support frames, but this is required to run Achievo</strong>.</p>
  
  <p>I have tried to directly access the dispatch.php frame, as this is what I probably want, however, it reports that my session has expired, and that I need to re-login.</p>
  
  <p>Is there someway to fake a frame? Or somehow keep the connection, change the url, and try to get the dispatch.php frame?</p>
  
  <hr>
  
  <p>Using HtmlUnit, I have done the following:</p>
  
  <pre><code>WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3);
  HtmlPage page = webClient.getPage("https://someurl.com/index.php");
  System.out.println(page.asXml());
  
  List&lt;HtmlForm&gt; forms = page.getForms();
  assertTrue(forms != null &amp;&amp; !forms.isEmpty());
  
  HtmlForm form = forms.get(0);
  HtmlSubmitInput submit = form.getInputByName("login");
  HtmlInput inputUsername = form.getInputByName("auth_user");
  HtmlInput inputPw = form.getInputByName("auth_pw");
  
  inputUsername.setValueAttribute("foo");
  inputPw.setValueAttribute("bar");
  
  HtmlPage page2 = submit.click();
  
  CookieManager cookieManager = webClient.getCookieManager();
  Set&lt;Cookie&gt; cookies = cookieManager.getCookies();
  System.out.println("Is cookie " + cookieManager.isCookiesEnabled());
  
  for(Cookie cookie : cookies) {
      System.out.println(cookie.toString());
  }
  
  System.out.println(page2.asXml());
  webClient.closeAllWindows();
  </code></pre>
  
  <p>Here I get the form, I submit it, and I retrieve the same message. When I also print out, I can see that I have a cookie. Now the question is, how do I proceed to get the dispatch.php frame using the logged in cookie?</p>