<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto"><code class="notranslate">torch.hub.load_state_dict_from_url</code> errors when downloading a valid <code class="notranslate">state_dict</code>.</p>
<h2 dir="auto">To Reproduce</h2>
<ol dir="auto">
<li>
<p dir="auto">Create a <code class="notranslate">state_dict</code></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from torch import hub, nn
import hashlib


def sha256(file: str, chunk_size: int = 4096) -&gt; str:
    hasher = hashlib.sha256()
    with open(file, &quot;rb&quot;) as fh:
        for chunk in iter(lambda: fh.read(chunk_size), b&quot;&quot;):
            hasher.update(chunk)
    return hasher.hexdigest()


file = &quot;conv2d.pth&quot;
model = nn.Conv2d(3, 3, 1)
state_dict = model.state_dict()
torch.save(state_dict, file)

print(sha256(file)[:8])"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">torch</span> <span class="pl-k">import</span> <span class="pl-s1">hub</span>, <span class="pl-s1">nn</span>
<span class="pl-k">import</span> <span class="pl-s1">hashlib</span>


<span class="pl-k">def</span> <span class="pl-en">sha256</span>(<span class="pl-s1">file</span>: <span class="pl-s1">str</span>, <span class="pl-s1">chunk_size</span>: <span class="pl-s1">int</span> <span class="pl-c1">=</span> <span class="pl-c1">4096</span>) <span class="pl-c1">-&gt;</span> <span class="pl-s1">str</span>:
    <span class="pl-s1">hasher</span> <span class="pl-c1">=</span> <span class="pl-s1">hashlib</span>.<span class="pl-en">sha256</span>()
    <span class="pl-k">with</span> <span class="pl-en">open</span>(<span class="pl-s1">file</span>, <span class="pl-s">"rb"</span>) <span class="pl-k">as</span> <span class="pl-s1">fh</span>:
        <span class="pl-k">for</span> <span class="pl-s1">chunk</span> <span class="pl-c1">in</span> <span class="pl-en">iter</span>(<span class="pl-k">lambda</span>: <span class="pl-s1">fh</span>.<span class="pl-en">read</span>(<span class="pl-s1">chunk_size</span>), <span class="pl-s">b""</span>):
            <span class="pl-s1">hasher</span>.<span class="pl-en">update</span>(<span class="pl-s1">chunk</span>)
    <span class="pl-k">return</span> <span class="pl-s1">hasher</span>.<span class="pl-en">hexdigest</span>()


<span class="pl-s1">file</span> <span class="pl-c1">=</span> <span class="pl-s">"conv2d.pth"</span>
<span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-s1">nn</span>.<span class="pl-v">Conv2d</span>(<span class="pl-c1">3</span>, <span class="pl-c1">3</span>, <span class="pl-c1">1</span>)
<span class="pl-s1">state_dict</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-en">state_dict</span>()
<span class="pl-s1">torch</span>.<span class="pl-en">save</span>(<span class="pl-s1">state_dict</span>, <span class="pl-s1">file</span>)

<span class="pl-en">print</span>(<span class="pl-en">sha256</span>(<span class="pl-s1">file</span>)[:<span class="pl-c1">8</span>])</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="96603ba9"><pre class="notranslate"><code class="notranslate">96603ba9
</code></pre></div>
</li>
<li>
<p dir="auto">Upload it (not shown) and make sure its uploaded correctly</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="url = f&quot;https://download.pystiche.org/models/{file}&quot;

downloaded_file = f&quot;downloaded_{file}&quot;
hub.download_url_to_file(url, downloaded_file)
print(sha256(downloaded_file)[:8])"><pre class="notranslate"><span class="pl-s1">url</span> <span class="pl-c1">=</span> <span class="pl-s">f"https://download.pystiche.org/models/<span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">file</span><span class="pl-kos">}</span></span>"</span>

<span class="pl-s1">downloaded_file</span> <span class="pl-c1">=</span> <span class="pl-s">f"downloaded_<span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">file</span><span class="pl-kos">}</span></span>"</span>
<span class="pl-s1">hub</span>.<span class="pl-en">download_url_to_file</span>(<span class="pl-s1">url</span>, <span class="pl-s1">downloaded_file</span>)
<span class="pl-en">print</span>(<span class="pl-en">sha256</span>(<span class="pl-s1">downloaded_file</span>)[:<span class="pl-c1">8</span>])</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="96603ba9"><pre class="notranslate"><code class="notranslate">96603ba9
</code></pre></div>
</li>
<li>
<p dir="auto">Try to download it directly with <code class="notranslate">torch.hub.download_state_dict_from_url</code></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="state_dict = hub.load_state_dict_from_url(url)"><pre class="notranslate"><span class="pl-s1">state_dict</span> <span class="pl-c1">=</span> <span class="pl-s1">hub</span>.<span class="pl-en">load_state_dict_from_url</span>(<span class="pl-s1">url</span>)</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="RuntimeError: Only one file(not dir) is allowed in the zipfile"><pre class="notranslate"><code class="notranslate">RuntimeError: Only one file(not dir) is allowed in the zipfile
</code></pre></div>
</li>
</ol>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">Load the <code class="notranslate">state_dict</code> without error.</p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="PyTorch version: 1.6.0
Is debug build: No
CUDA used to build PyTorch: 10.2

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: Could not collect

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 1080
Nvidia driver version: 440.33.01
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.6.0
[pip3] torchvision==0.7.0
[conda] Could not collect"><pre class="notranslate"><code class="notranslate">PyTorch version: 1.6.0
Is debug build: No
CUDA used to build PyTorch: 10.2

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: Could not collect

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 1080
Nvidia driver version: 440.33.01
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.6.0
[pip3] torchvision==0.7.0
[conda] Could not collect
</code></pre></div>
<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">If you call <code class="notranslate">torch.hub.load_state_dict_from_url()</code> on a file saved in the new zipfile serialization format, it raises <code class="notranslate">RuntimeError('Only one file(not dir) is allowed in the zipfile')</code>.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Upload a state dictionary file saved using the vanilla <code class="notranslate">torch.save()</code> to a server.</li>
<li>Call <code class="notranslate">torch.hub.load_state_dict_from_url()</code> to that url.</li>
<li>You'll see the <code class="notranslate">RuntimeError</code> raised.</li>
</ol>
<p dir="auto">We discovered this as part of the effort to support pytorch 1.6 with pytorch lightning. Here's one of the failed test runs: <a href="https://github.com/PyTorchLightning/pytorch-lightning/runs/920684391">https://github.com/PyTorchLightning/pytorch-lightning/runs/920684391</a></p>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto"><code class="notranslate">torch.hub.load_state_dict_from_url()</code> should load the new zipfile serialization files without any errors.</p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Collecting environment information...
PyTorch version: 1.6.0
Is debug build: No
CUDA used to build PyTorch: None

OS: Mac OSX 10.14.6
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.7
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] pytorch-lightning==0.9.0rc3
[pip3] torch==1.6.0
[pip3] torchtext==0.6.0
[pip3] torchvision==0.7.0
[conda] Could not collect"><pre class="notranslate"><code class="notranslate">Collecting environment information...
PyTorch version: 1.6.0
Is debug build: No
CUDA used to build PyTorch: None

OS: Mac OSX 10.14.6
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.7
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] pytorch-lightning==0.9.0rc3
[pip3] torch==1.6.0
[pip3] torchtext==0.6.0
[pip3] torchvision==0.7.0
[conda] Could not collect
</code></pre></div>
<h2 dir="auto">Additional context</h2>
<p dir="auto">We use <code class="notranslate">torch.hub.load_state_dict_from_url()</code> to load remote PyTorch Lightning checkpoints.</p>
<p dir="auto">CC <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/williamFalcon/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/williamFalcon">@williamFalcon</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Borda/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Borda">@Borda</a></p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ailzhang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ailzhang">@ailzhang</a></p>