{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-hep/uproot/issues/190","repository_url":"https://api.github.com/repos/scikit-hep/uproot","labels_url":"https://api.github.com/repos/scikit-hep/uproot/issues/190/labels{/name}","comments_url":"https://api.github.com/repos/scikit-hep/uproot/issues/190/comments","events_url":"https://api.github.com/repos/scikit-hep/uproot/issues/190/events","html_url":"https://github.com/scikit-hep/uproot/issues/190","id":390049446,"node_id":"MDU6SXNzdWUzOTAwNDk0NDY=","number":190,"title":"Weird error in HTTP/Chunked sources","user":{"login":"PerilousApricot","id":93354,"node_id":"MDQ6VXNlcjkzMzU0","avatar_url":"https://avatars0.githubusercontent.com/u/93354?v=4","gravatar_id":"","url":"https://api.github.com/users/PerilousApricot","html_url":"https://github.com/PerilousApricot","followers_url":"https://api.github.com/users/PerilousApricot/followers","following_url":"https://api.github.com/users/PerilousApricot/following{/other_user}","gists_url":"https://api.github.com/users/PerilousApricot/gists{/gist_id}","starred_url":"https://api.github.com/users/PerilousApricot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/PerilousApricot/subscriptions","organizations_url":"https://api.github.com/users/PerilousApricot/orgs","repos_url":"https://api.github.com/users/PerilousApricot/repos","events_url":"https://api.github.com/users/PerilousApricot/events{/privacy}","received_events_url":"https://api.github.com/users/PerilousApricot/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-12-12T03:25:21Z","updated_at":"2018-12-12T08:19:32Z","closed_at":null,"author_association":"NONE","body":"Hello,\r\n\r\nI'm working on some code to read xrootd files via HTTP(s) and have hit an error I don't get. With the following test script\r\n\r\n```python\r\n#!/usr/bin/env python\r\n\r\nimport uproot\r\nf = uproot.open(\"https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root\")\r\n```\r\n\r\nwith the following patch applied (to relax the SSL settings)\r\n```diff\r\ndiff --git a/uproot/source/http.py b/uproot/source/http.py\r\nindex 4a1564a..bcc71a1 100644\r\n--- a/uproot/source/http.py\r\n+++ b/uproot/source/http.py\r\n@@ -30,6 +30,7 @@\r\n \r\n import os.path\r\n import re\r\n+import ssl\r\n try:\r\n     from urllib.request import urlopen, Request\r\n except ImportError:\r\n@@ -46,6 +47,13 @@ class HTTPSource(uproot.source.chunked.ChunkedSource):\r\n     def __init__(self, path, *args, **kwds):\r\n         super(HTTPSource, self).__init__(path, *args, **kwds)\r\n         self._size = None\r\n+        if path.startswith(\"http://\"):\r\n+            self._sslcontext = None\r\n+        else:\r\n+            # Temporarily disable server cert verfication while I work out how\r\n+            # to always find the grid CA bundle\r\n+            self._sslcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23)\r\n+            self._sslcontext.verify_mode = ssl.CERT_NONE\r\n \r\n     defaults = {\"chunkbytes\": 16*1024, \"limitbytes\": 16*1024**2}\r\n \r\n@@ -59,7 +67,7 @@ class HTTPSource(uproot.source.chunked.ChunkedSource):\r\n \r\n     def _read(self, chunkindex):\r\n         request = Request(self.path, headers={\"Range\": \"bytes={0}-{1}\".format(chunkindex * self._chunkbytes, (chun\r\n-        handle = urlopen(request)\r\n+        handle = urlopen(request, context=self._sslcontext)\r\n         data = handle.read()\r\n         if self._size is None:\r\n             m = self._contentrange.match(handle.headers.get(\"content-range\", \"\"))\r\n```\r\n\r\nI get the error\r\n```\r\nTraceback (most recent call last):\r\n  File \"./test.py\", line 4, in <module>\r\n    f = uproot.open(\"https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root\")\r\n  File \"/home/meloam/projects/uproot/uproot/rootio.py\", line 80, in open\r\n    return http(path, httpsource)\r\n  File \"/home/meloam/projects/uproot/uproot/rootio.py\", line 101, in http\r\n    return ROOTDirectory.read(openfcn(path), **options)\r\n  File \"/home/meloam/projects/uproot/uproot/rootio.py\", line 164, in read\r\n    streamerkey = TKey.read(source, Cursor(fSeekInfo), streamercontext, None)\r\n  File \"/home/meloam/projects/uproot/uproot/rootio.py\", line 842, in read\r\n    out = cls._readinto(out, source, cursor, context, parent)\r\n  File \"/home/meloam/projects/uproot/uproot/rootio.py\", line 864, in _readinto\r\n    self._fNbytes, self._fVersion, self._fObjlen, self._fDatime, self._fKeylen, self._fCycle, self._fSeekKey, self._fSeekPdir = cursor.fields(source, self._format_small)\r\n  File \"/home/meloam/projects/uproot/uproot/source/cursor.py\", line 70, in fields\r\n    return format.unpack(source.data(start, stop))\r\n  File \"/home/meloam/projects/uproot/uproot/source/chunked.py\", line 102, in data\r\n    raise IndexError(\"indexes {0}:{1} are beyond the end of data source {2}\".format(gstart + len(chunk), stop, repr(self.path)))\r\nIndexError: indexes 29630001:29630027 are beyond the end of data source 'https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root'\r\n```\r\n\r\nThough the file contains `29635611` bytes. I can verify that's what seen by HTTP as well:\r\n```\r\n$ curl -L -k -vvv https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root -o out.txt\r\n$ ls -l out.txt \r\n-rw-r--r--. 1 meloam cms 29635611 Dec 11 21:08 out.txt\r\n```\r\n\r\nI'm not exactly sure of what's going on -- the same file opens fine from the local filesystem -- is it possible there's a silly bug in the chunked source I'm missing? I'm gonna leave this as a placeholder while I try to solve it"},"comment":{"url":"https://api.github.com/repos/scikit-hep/uproot/issues/comments/446500836","html_url":"https://github.com/scikit-hep/uproot/issues/190#issuecomment-446500836","issue_url":"https://api.github.com/repos/scikit-hep/uproot/issues/190","id":446500836,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NjUwMDgzNg==","user":{"login":"sbinet","id":1070920,"node_id":"MDQ6VXNlcjEwNzA5MjA=","avatar_url":"https://avatars2.githubusercontent.com/u/1070920?v=4","gravatar_id":"","url":"https://api.github.com/users/sbinet","html_url":"https://github.com/sbinet","followers_url":"https://api.github.com/users/sbinet/followers","following_url":"https://api.github.com/users/sbinet/following{/other_user}","gists_url":"https://api.github.com/users/sbinet/gists{/gist_id}","starred_url":"https://api.github.com/users/sbinet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sbinet/subscriptions","organizations_url":"https://api.github.com/users/sbinet/orgs","repos_url":"https://api.github.com/users/sbinet/repos","events_url":"https://api.github.com/users/sbinet/events{/privacy}","received_events_url":"https://api.github.com/users/sbinet/received_events","type":"User","site_admin":false},"created_at":"2018-12-12T08:19:32Z","updated_at":"2018-12-12T08:19:32Z","author_association":"NONE","body":"but... shouldn't it be:\r\n```python\r\nimport uproot\r\nf = uproot.open(\"root://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root\")\r\n```\r\n\r\n_i.e._ explicitly use the `xrootd` protocol?"}}
{"actor":{"display_login":"sbinet"}}