title: StreamSets HTTP Client
body: <p>I'm working with StreamSets on a Cloudera Distribution, trying to ingest some data from this website <a href="http://files.data.gouv.fr/sirene/" rel="nofollow noreferrer">http://files.data.gouv.fr/sirene/</a></p>
  
  <p>I've encountered some issues choosing the parameters of both the HTTP Client and the Hadoop FS Destination.</p>
  
  <p><a href="https://image.noelshack.com/fichiers/2017/44/2/1509457504-streamsets-f.jpg" rel="nofollow noreferrer">https://image.noelshack.com/fichiers/2017/44/2/1509457504-streamsets-f.jpg</a></p>
  
  <p>I get this error : HTTP_00 - Cannot parse record: java.io.IOException: org.apache.commons.compress.archivers.ArchiveException: No Archiver found for the stream signature</p>
  
  <p>I'll show you my configuration.</p>
  
  <p><strong><em>HTTP Client :</em></strong></p>
  
  <p><strong>General</strong></p>
  
  <p>Name : HTTP Client INSEE</p>
  
  <p>Description : Client HTTP SIRENE</p>
  
  <p>On Record Error : Send to Error</p>
  
  <p><strong>HTTP</strong> </p>
  
  <p>Resource URL : <a href="http://files.data.gouv.fr/sirene/" rel="nofollow noreferrer">http://files.data.gouv.fr/sirene/</a></p>
  
  <p>Headers : sirene_ : sirene_</p>
  
  <p>Mode : Streaming</p>
  
  <p>Per-Status Actions</p>
  
  <p>HTTP Statis Code : 500 | Action for status : Retry with exponential backoff |</p>
  
  <p>Base Backoff Interval (ms) : 1000 | Max Retries : 10</p>
  
  <p>HTTP Method : GET</p>
  
  <p>Body Time Zone : UTC (UTC)</p>
  
  <p>Request Transfert Encoding : BUFFERED</p>
  
  <p>HTTP Compression : None</p>
  
  <p>Connect Timeout : 0</p>
  
  <p>Read Timeout : 0</p>
  
  <p>Authentication Type : None</p>
  
  <p>Use OAuth 2</p>
  
  <p>Use Proxy</p>
  
  <p>Max Batch Size (records) : 1000</p>
  
  <p>Batch Wait Time (ms) : 2000</p>
  
  <p><strong>Pagination</strong></p>
  
  <p>Pagination Mode : None</p>
  
  <p><strong>TLS</strong></p>
  
  <p>UseTLS</p>
  
  <p><strong>Timeout Handling</strong></p>
  
  <p>Action for timeout : Retry immediately</p>
  
  <p>Max Retries : 10</p>
  
  <p><strong>Data Format</strong></p>
  
  <p>Date Format : Delimited</p>
  
  <p>Compression Format : Archive</p>
  
  <p>File Name Pattern within Compressed Directory : *.csv</p>
  
  <p>Delimiter Format Type : Custom</p>
  
  <p>Header Line : With Header Line</p>
  
  <p>Max Record Length (chars) : 1024</p>
  
  <p>Allow Extra Columns</p>
  
  <p>Delimiter Character : Semicolon</p>
  
  <p>Escape Character : Other \</p>
  
  <p>Quote Character : Other "</p>
  
  <p>Root Field Type : List-Map</p>
  
  <p>Lines to Skip : 0</p>
  
  <p>Parse NULLs</p>
  
  <p>Charset : UTF-8</p>
  
  <p>Ignore Control Characters</p>
  
  <p><strong><em>Hadoop FS Destination :</em></strong></p>
  
  <p><strong>General</strong></p>
  
  <p>Name : Hadoop FS 1</p>
  
  <p>Description : Writing into HDFS</p>
  
  <p>Stage Library : CDH 5.7.6</p>
  
  <p>Produce Events </p>
  
  <p>Required Fields</p>
  
  <p>Preconditions</p>
  
  <p>On Record Error : Send to Error</p>
  
  <p><strong>Output Files</strong></p>
  
  <p>File Type : Whole File</p>
  
  <p>Files Prefix </p>
  
  <p>Directory in Header</p>
  
  <p>Directory Template : /user/pap/StreamSets/sirene/</p>
  
  <p>Data Time Zone : UTC (UTC)</p>
  
  <p>Time Basis : ${time:now()}</p>
  
  <p>Use Roll Attribute</p>
  
  <p>Validate HDFS Permissions : ON</p>
  
  <p>Skip file recovery : ON</p>
  
  <p><strong>Late Records</strong></p>
  
  <p>Late Record Time Limit (secs) : ${1 * HOURS}</p>
  
  <p>Late Record Handling : Send to error</p>
  
  <p><strong>Data Format</strong> </p>
  
  <p>Data Format : whole File</p>
  
  <p>File Name Expression : ${record:value('/fileInfo/filename')}</p>
  
  <p>Permissions Expression : 777</p>
  
  <p>File Exists : Overwrite</p>
  
  <p>Include Checksum in Events </p>
  
  <p>... so what am I doing wrong ? :(</p>