title: HTTP Post to API return 403 FORBIDDEN
body: <p>I have to upload an XML file to an API. This is API is secured by a signed certificate, which I got from the issuer of the API.</p>
  
  <p>Now, I got two use cases. First, I have to download some files from this API. This is working perfectly with the following code:</p>
  
  <pre><code>final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ip, port));
  
  final URL url = new URL(linkToFile);
  final HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(proxy);
  conn.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
  try (
              InputStream inputStream = zipUrlConn.getInputStream();
              ZipInputStream stream = new ZipInputStream(inputStream);) {
  
      // Do stuff with ZipInputStream here
  }
  </code></pre>
  
  <p>The <code>createSSLContext()</code> method looks as follows:</p>
  
  <pre><code>public SSLContext createSSLContext() throws NoSuchAlgorithmException, CertificateException, IOException, KeyStoreException,
          UnrecoverableKeyException, KeyManagementException {
  
      final KeyStore clientStore = KeyStore.getInstance("PKCS12");
      clientStore.load(new FileInputStream(this.certificateResource.getFile()), this.p12PW.toCharArray());
  
      final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
      keyManagerFactory.init(clientStore, this.p12PW.toCharArray());
      final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();
  
      final KeyStore trustStore = KeyStore.getInstance("JKS");
      trustStore.load(new FileInputStream(this.trustStoreResource.getFile()), this.trustStorePW.toCharArray());
  
      final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
      trustManagerFactory.init(trustStore);
      final TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
  
      final SSLContext sslContext = SSLContext.getInstance("TLS");
      sslContext.init(keyManagers, trustManagers, new SecureRandom());
  
      return sslContext;
  }
  </code></pre>
  
  <p>I was following a guideline, which I got from the issuer, which showed how to do this with a cUrl command:</p>
  
  <pre><code>curl --cert Certificate-&lt;id&gt;.pem[:pem_password] https://api.url.com/
  </code></pre>
  
  <p>So I was basically trying to rebuild this command in java, which is working.</p>
  
  <p>Now for the part that's not working, which is the file upload. Again, I was given a cUrl command which I have to rebuild:</p>
  
  <pre><code>curl --cert Certificate-&lt;id&gt;.pem[:pem_password] -F upload=@&lt;Path_to_file&gt;\DS&lt;PartnerId&gt;_&lt;Timestamp&gt;.XML https://api.url.com/in/upload.php
  </code></pre>
  
  <p>I tried several things to achieve that:</p>
  
  <ol>
  <li><strong>"Normal" Java</strong></li>
  </ol>
  
  <p>Firstly, I tried it with the standard <code>HttpsURLConnection</code> as follows:</p>
  
  <pre><code>final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.1.25", 3128));
  final HttpsURLConnection connection = (HttpsURLConnection) new URL(ARGE_UPLOAD_URL).openConnection(proxy);
  
  connection.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
  connection.setDoOutput(true);
  connection.setRequestProperty("Accept-Charset", "UTF-8");
  connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
  
  try (OutputStream outputStream = connection.getOutputStream()) {
      outputStream.write(Files.readAllBytes(new File("src/main/resources/XML/example.xml").toPath()));
  }
  
  final InputStream result = connection.getInputStream();
  </code></pre>
  
  <p>But this always results in <code>java.io.IOException: Server returned HTTP response code: 403 for URL: https://api.url.com/in/upload.php</code>, even though I'm using the same configuration, with which I am able to download from the API.</p>
  
  <ol start="2">
  <li><strong>Apache HttpClient</strong></li>
  </ol>
  
  <p>I found some resources claiming that the <code>HttpClient</code> is a lot easier to configure and use, so I gave it a try:</p>
  
  <pre><code>final CloseableHttpClient httpClient = HttpClientBuilder
              .create()
              .setSSLContext(this.http.createSSLContext())
              .setProxy(new HttpHost(InetAddress.getByName(ip), port))
              .build();
  final HttpEntity requestEntity = MultipartEntityBuilder
              .create()
              .addBinaryBody("upload=@example.xml", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
              .build();
  
  final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
  post.setEntity(requestEntity);
  
  try (CloseableHttpResponse response = httpClient.execute(post)) {
      this.logger.info(response.getStatusLine().toString());
      EntityUtils.consume(response.getEntity());
  }
  </code></pre>
  
  <p>Resulting in <code>HTTP/1.1 403 FORBIDDEN</code></p>
  
  <ol start="3">
  <li><strong>HttpClient (FileEntity instead of MultipartEntity)</strong></li>
  </ol>
  
  <p>As a last thing I tried to a <code>FileEntity</code>:</p>
  
  <pre><code>final HttpPost httpPost = new HttpPost(https://api.url.com/in/upload.php);
  httpPost.addHeader("content-type", "application/x-www-form-urlencoded;charset=utf-8");
  
  final FileEntity fileEntity = new FileEntity(new File("src/main/resources/XML/example.xml"));
  httpPost.setEntity(fileEntity);
  
  System.out.println("executing request " + httpPost.getRequestLine() + httpPost.getConfig());
  try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
      final HttpEntity responseEntity = response.getEntity();
  
      System.out.println("Status: " + response.getStatusLine());
      if (responseEntity != null) {
          System.out.println("Entity: " + EntityUtils.toString(responseEntity));
      }
  }
  </code></pre>
  
  <p>Resulting in <code>Status: HTTP/1.1 403 FORBIDDEN</code></p>
  
  <p>I just don't understand, how I can be able to download from the API, but not upload to it, despite using exactly the same configuration.</p>
  
  <p>If you need any more information, I'll be happy to provide them.</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>As suggested by <a href="https://stackoverflow.com/users/4861882/oli">oli</a>, I used Fiddler to capture the HTTPS request. This is the result for method 1 (Normal Java):</p>
  
  <pre><code>POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
  Accept-Charset: utf-8
  Accept: */*
  Content-Type: application/x-www-form-urlencoded;charset=utf-8
  User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
  Host: hrbaxml.arbeitsagentur.de
  Connection: keep-alive
  Content-Length: 6738
  </code></pre>
  
  <p>And this is the result from the manual upload through Google Chrome:</p>
  
  <pre><code>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
  Accept-Encoding: gzip, deflate, br
  Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
  Cache-Control: max-age=0
  Connection: keep-alive
  Content-Length: 6948
  Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryxZxIJ5h19MEFbZQs
  Cookie: cookie_consent=accepted
  Host: hrbaxml.arbeitsagentur.de
  Origin: https://hrbaxml.arbeitsagentur.de
  Referer: https://hrbaxml.arbeitsagentur.de/in/
  Upgrade-Insecure-Requests: 1
  User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
  </code></pre>
  
  <p><strong>EDIT 2</strong></p>
  
  <p>Just to add, this is the result using method 2 (HttpClient with MultipartEntity):</p>
  
  <pre><code>POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
  Content-Length: 7025
  Content-Type: multipart/form-data; boundary=1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
  Host: hrbaxml.arbeitsagentur.de
  Connection: Keep-Alive
  User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_161)
  Accept-Encoding: gzip,deflate
  
  --1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
  Content-Disposition: form-data; name="DSV000306700_2018-08-23_09-00-00.xml";    filename="DSV000306700_2018-08-23_09-00-00.xml"
  Content-Type: application/octet-stream
  Content-Transfer-Encoding: binary
  </code></pre>
  
  <p><strong>EDIT 3</strong></p>
  
  <p>I tried copying all the HTTP Headers from the Chrome request, so that my request from Java looks like this:</p>
  
  <pre><code>POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
  Accept: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
  Accept-Encoding: gzip, deflate, br
  Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
  Cache-Control: max-age=0
  Content-Type: multipart/form-data; boundary=1535095530678
  Cookie: cookie_consent=accepted
  Referer: https://hrbaxml.arbeitsagentur.de/in/
  Upgrade-Insecure-Requests: 1
  User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
  Host: hrbaxml.arbeitsagentur.de
  Connection: keep-alive
  Content-Length: 6944
  
  --1535095530678
  Content-Disposition: form-data; name="uploadFile"; filename="DSV000306700_2018-08-23_09-00-00.xml"
  Content-Type: application/xml
  Content-Transfer-Encoding: binary
  
  .. xml data ..
  
  --1535095530678--
  </code></pre>
  
  <p>But still, without success. Any other possible solutions? Maybe it isn't a problem with the upload but something else?</p>