<p><strong>Requester Pays</strong> is a feature on Amazon S3 buckets that requires the user of the bucket to pay Data Transfer costs associated with accessing data.</p>

<p>Normally, the owner of an S3 bucket pays Data Transfer costs, but this can be expensive for free / Open Source projects. Thus, the bucket owner can activated Requester Pays to reduce the portion of costs they will be charged.</p>

<p>Therefore, when accessing a Requester Pays bucket, you will need to authenticate yourself so that S3 knows whom to charge.</p>

<p>I recommend using the official <strong><a href="http://aws.amazon.com/cli/" rel="nofollow noreferrer">AWS Command-Line Interface (CLI)</a></strong> to access AWS services. You can provide your credentials via:</p>

<pre><code>aws configure
</code></pre>

<p>and then view the bucket via:</p>

<pre><code>aws s3 ls s3://arxiv/pdf/
</code></pre>

<p>and download via:</p>

<pre><code>aws s3 cp s3://arxiv/pdf/arXiv_pdf_1001_001.tar .
</code></pre>

<p><strong>UPDATE:</strong> I just tried the above myself, and received <code>Access Denied</code> error messages (both on the bucket listing and the download command). When using <code>s3cmd</code>, it says <code>ERROR: S3 error: Access Denied</code>. <strong>It would appear that the permissions on the bucket no longer permit access.</strong> You should contact the owners of the bucket to request access.</p>
How to download data from Amazon's requester pay buckets?
<p>I have been struggling for about a week to download arXiv articles as mentioned here: <a href="http://arxiv.org/help/bulk_data_s3#src" rel="noreferrer">http://arxiv.org/help/bulk_data_s3#src</a>.</p>

<p>I have tried lots of things: <code>s3Browser</code>, <code>s3cmd</code>. I am able to login to my buckets but I am unable to download data from arXiv bucket. </p>

<p>I tried:</p>

<ol>
<li><code>s3cmd get s3://arxiv/pdf/arXiv_pdf_1001_001.tar</code></li>
</ol>

<p>See:</p>

<pre><code>$ s3cmd get s3://arxiv/pdf/arXiv_pdf_1001_001.tar


s3://arxiv/pdf/arXiv_pdf_1001_001.tar -&gt; ./arXiv_pdf_1001_001.tar  [1 of 1]
s3://arxiv/pdf/arXiv_pdf_1001_001.tar -&gt; ./arXiv_pdf_1001_001.tar  [1 of 1]
ERROR: S3 error: Unknown error
</code></pre>

<ol start="2">
<li><code>s3cmd get</code> with <code>x-amz-request-payer:requester</code></li>
</ol>

<p>It gave me same error again:</p>

<pre><code>$ s3cmd get --add-header="x-amz-request-payer:requester" s3://arxiv/pdf/arXiv_pdf_manifest.xml
s3://arxiv/pdf/arXiv_pdf_manifest.xml -&gt; ./arXiv_pdf_manifest.xml  [1 of 1]
s3://arxiv/pdf/arXiv_pdf_manifest.xml -&gt; ./arXiv_pdf_manifest.xml  [1 of 1]
ERROR: S3 error: Unknown error
</code></pre>

<ol start="3">
<li>Copying</li>
</ol>

<p>I have tried copying files from that folder too.</p>

<pre><code>$ aws s3 cp s3://arxiv/pdf/arXiv_pdf_1001_001.tar .

A client error (403) occurred when calling the HeadObject operation: Forbidden
Completed 1 part(s) with ... file(s) remaining
</code></pre>

<p>This probably means that I made a mistake. The problem is I don't know how and what to add that will convey my permission to pay for download.</p>

<p>I am unable to figure out what should I do for downloading data from S3. I have been reading a lot on AWS sites, but nowhere I can get pinpoint solution to my problem.</p>

<p>How can I bulk download the arXiv data?</p>