<p>Try downloading <code>s3cmd</code> version <code>1.6.0</code>: <a href="http://sourceforge.net/projects/s3tools/files/s3cmd/" rel="noreferrer">http://sourceforge.net/projects/s3tools/files/s3cmd/</a></p>

<pre><code>$ s3cmd --configure
</code></pre>

<p>Enter your credentials found in the account management tab of the Amazon AWS website interface.</p>

<pre><code>$ s3cmd get --recursive --skip-existing s3://arxiv/src/ --requester-pays
</code></pre>
How to download data from Amazon's requester pay buckets?
<p>I have been struggling for about a week to download arXiv articles as mentioned here: <a href="http://arxiv.org/help/bulk_data_s3#src" rel="noreferrer">http://arxiv.org/help/bulk_data_s3#src</a>.</p>

<p>I have tried lots of things: <code>s3Browser</code>, <code>s3cmd</code>. I am able to login to my buckets but I am unable to download data from arXiv bucket. </p>

<p>I tried:</p>

<ol>
<li><code>s3cmd get s3://arxiv/pdf/arXiv_pdf_1001_001.tar</code></li>
</ol>

<p>See:</p>

<pre><code>$ s3cmd get s3://arxiv/pdf/arXiv_pdf_1001_001.tar


s3://arxiv/pdf/arXiv_pdf_1001_001.tar -&gt; ./arXiv_pdf_1001_001.tar  [1 of 1]
s3://arxiv/pdf/arXiv_pdf_1001_001.tar -&gt; ./arXiv_pdf_1001_001.tar  [1 of 1]
ERROR: S3 error: Unknown error
</code></pre>

<ol start="2">
<li><code>s3cmd get</code> with <code>x-amz-request-payer:requester</code></li>
</ol>

<p>It gave me same error again:</p>

<pre><code>$ s3cmd get --add-header="x-amz-request-payer:requester" s3://arxiv/pdf/arXiv_pdf_manifest.xml
s3://arxiv/pdf/arXiv_pdf_manifest.xml -&gt; ./arXiv_pdf_manifest.xml  [1 of 1]
s3://arxiv/pdf/arXiv_pdf_manifest.xml -&gt; ./arXiv_pdf_manifest.xml  [1 of 1]
ERROR: S3 error: Unknown error
</code></pre>

<ol start="3">
<li>Copying</li>
</ol>

<p>I have tried copying files from that folder too.</p>

<pre><code>$ aws s3 cp s3://arxiv/pdf/arXiv_pdf_1001_001.tar .

A client error (403) occurred when calling the HeadObject operation: Forbidden
Completed 1 part(s) with ... file(s) remaining
</code></pre>

<p>This probably means that I made a mistake. The problem is I don't know how and what to add that will convey my permission to pay for download.</p>

<p>I am unable to figure out what should I do for downloading data from S3. I have been reading a lot on AWS sites, but nowhere I can get pinpoint solution to my problem.</p>

<p>How can I bulk download the arXiv data?</p>