{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/12970","repository_url":"https://api.github.com/repos/pydata/pandas","labels_url":"https://api.github.com/repos/pydata/pandas/issues/12970/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/12970/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/12970/events","html_url":"https://github.com/pydata/pandas/issues/12970","id":150585350,"number":12970,"title":"url scraping error","user":{"login":"csayantan","id":18216368,"avatar_url":"https://avatars.githubusercontent.com/u/18216368?v=3","gravatar_id":"","url":"https://api.github.com/users/csayantan","html_url":"https://github.com/csayantan","followers_url":"https://api.github.com/users/csayantan/followers","following_url":"https://api.github.com/users/csayantan/following{/other_user}","gists_url":"https://api.github.com/users/csayantan/gists{/gist_id}","starred_url":"https://api.github.com/users/csayantan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/csayantan/subscriptions","organizations_url":"https://api.github.com/users/csayantan/orgs","repos_url":"https://api.github.com/users/csayantan/repos","events_url":"https://api.github.com/users/csayantan/events{/privacy}","received_events_url":"https://api.github.com/users/csayantan/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/IO%20HTML","name":"IO HTML","color":"006b75"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Usage%20Question","name":"Usage Question","color":"0052cc"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-04-23T19:03:29Z","updated_at":"2016-04-24T04:41:13Z","closed_at":"2016-04-23T19:06:23Z","body":"```\r\npd.show_versions()\r\n\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 2.7.10.final.0\r\npython-bits: 64\r\nOS: Linux\r\nOS-release: 4.2.0-34-generic\r\nmachine: x86_64\r\nprocessor: x86_64\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: en_IN\r\n\r\npandas: 0.18.0\r\nnose: None\r\npip: 8.1.1\r\nsetuptools: 20.3.1\r\nCython: 0.23.5\r\nnumpy: 1.11.0\r\nscipy: 0.17.0\r\nstatsmodels: 0.6.1\r\nxarray: None\r\nIPython: 4.1.2\r\nsphinx: None\r\npatsy: 0.4.1\r\ndateutil: 2.5.2\r\npytz: 2016.3\r\nblosc: None\r\nbottleneck: 1.0.0\r\ntables: None\r\nnumexpr: 2.5.1\r\nmatplotlib: 1.5.1\r\nopenpyxl: None\r\nxlrd: None\r\nxlwt: None\r\nxlsxwriter: None\r\nlxml: None\r\nbs4: 4.4.1\r\nhtml5lib: 0.999\r\nhttplib2: 0.9\r\napiclient: None\r\nsqlalchemy: 1.0.12\r\npymysql: None\r\npsycopg2: None\r\njinja2: 2.8\r\nboto: None\r\n```\r\n\r\n\r\n\r\n```\r\nfrom nsepy import get_history\r\nfrom datetime import date\r\nimport pandas as pd\r\nimport requests\r\nfrom io import BytesIO \r\nimport certifi\r\nfrom dateutil.relativedelta import relativedelta\r\n#import numpy as np\r\n#import matplotlib.pyplot as plt\r\nimport datetime\r\nimport numpy as np\r\nimport matplotlib.colors as colors\r\nimport matplotlib.finance as finance\r\nimport matplotlib.dates as mdates\r\nimport matplotlib.ticker as mticker\r\nimport matplotlib.mlab as mlab\r\nimport matplotlib.pyplot as plt\r\nimport matplotlib.font_manager as font_manager\r\nimport requests\r\nfrom nsepy.archives import html_to_rows\r\nfrom bs4 import BeautifulSoup\r\nimport pandas as pd\r\nimport numpy as np\r\nimport matplotlib as plt\r\n'''\r\nimport sys\r\n\r\n\r\nreload(sys)  \r\nsys.setdefaultencoding('Cp1252')\r\n'''\r\nurl1 = 'http://www.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp?segmentLink=17&instrument=OPTIDX&symbol=NIFTY&date=26MAY2016'\r\ndef dataFromHtmlInPy(url):\r\n\r\n\theaders = { 'Accept' : '*/*',\r\n\t            'User-Agent' : 'Mozilla/5.0',\r\n\t            'Refers' : 'http://www.nseindia.com',\r\n\t            'Connection' : 'keep-alive'\r\n\t          }\r\n            \r\n\tgetContents = requests.get(url,headers=headers).text\r\n\t#dataFromHtm=pd.read_html(BytesIO(getContents))\r\n        dataFromHtm = BeautifulSoup(getContents, \"lxml\")\r\n        return dataFromHtm\r\n\r\ndata=pd.DataFrame(dataFromHtmlInPy(url=url1))\r\nprint(data.head())\r\n```\r\nI am new to panda and python.however I need dataframe as the same in the url1.\r\nfew info i need to scrapped from there. i need to calculate daily PCR. \r\ni encounter an error like:\r\n```\r\n\r\nTraceback (most recent call last):\r\n  File \"dataFromNsePage.py\", line 46, in <module>\r\n    data=pd.DataFrame(dataFromHtmlInPy(url=url1))\r\n  File \"/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py\", line 282, in __init__\r\n    raise TypeError(\"data argument can't be an iterator\")\r\nTypeError: data argument can't be an iterator \r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/213888062","html_url":"https://github.com/pydata/pandas/issues/12970#issuecomment-213888062","issue_url":"https://api.github.com/repos/pydata/pandas/issues/12970","id":213888062,"user":{"login":"csayantan","id":18216368,"avatar_url":"https://avatars.githubusercontent.com/u/18216368?v=3","gravatar_id":"","url":"https://api.github.com/users/csayantan","html_url":"https://github.com/csayantan","followers_url":"https://api.github.com/users/csayantan/followers","following_url":"https://api.github.com/users/csayantan/following{/other_user}","gists_url":"https://api.github.com/users/csayantan/gists{/gist_id}","starred_url":"https://api.github.com/users/csayantan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/csayantan/subscriptions","organizations_url":"https://api.github.com/users/csayantan/orgs","repos_url":"https://api.github.com/users/csayantan/repos","events_url":"https://api.github.com/users/csayantan/events{/privacy}","received_events_url":"https://api.github.com/users/csayantan/received_events","type":"User","site_admin":false},"created_at":"2016-04-24T04:41:13Z","updated_at":"2016-04-24T04:41:13Z","body":"```\r\nfrom nsepy import get_history\r\nfrom datetime import date\r\nimport pandas as pd\r\nimport requests\r\nfrom io import BytesIO \r\nimport certifi\r\nfrom dateutil.relativedelta import relativedelta\r\n#import numpy as np\r\n#import matplotlib.pyplot as plt\r\nimport datetime\r\nimport numpy as np\r\nimport matplotlib.colors as colors\r\nimport matplotlib.finance as finance\r\nimport matplotlib.dates as mdates\r\nimport matplotlib.ticker as mticker\r\nimport matplotlib.mlab as mlab\r\nimport matplotlib.pyplot as plt\r\nimport matplotlib.font_manager as font_manager\r\nimport requests\r\nfrom nsepy.archives import html_to_rows\r\nfrom bs4 import BeautifulSoup\r\nimport pandas as pd\r\nimport numpy as np\r\nimport matplotlib as plt\r\n\r\nimport sys\r\n\r\n\r\nreload(sys)  \r\nsys.setdefaultencoding('Cp1252')\r\n\r\nurl1 = 'http://www.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp?segmentLink=17&instrument=OPTIDX&symbol=NIFTY&date=26MAY2016'\r\ndef dataFromHtmlInPy(url):\r\n\r\n\theaders = { 'Accept' : '*/*',\r\n\t            'User-Agent' : 'Mozilla/5.0',\r\n\t            'Refers' : 'http://www.nseindia.com',\r\n\t            'Connection' : 'keep-alive'\r\n\t          }\r\n            \r\n\tgetContents = requests.get(url,headers=headers).content\r\n\tdataFromHtm=pd.read_html(BytesIO(getContents))\r\n        #dataFromHtm = BeautifulSoup(getContents, \"lxml\")\r\n        return dataFromHtm\r\n\r\ndata=pd.DataFrame(dataFromHtmlInPy(url=url1))\r\nprint(data.to_html())\r\n```\r\n\r\nthis code returned  data like this.but i am missing the table of content of it. please suggest \r\n```\r\n<table border=\"1\" class=\"dataframe\">\r\n  <thead>\r\n    <tr style=\"text-align: right;\">\r\n      <th></th>\r\n      <th>0</th>\r\n    </tr>\r\n  </thead>\r\n  <tbody>\r\n    <tr>\r\n      <th>0</th>\r\n      <td>0  \\\r\n0  Opt...</td>\r\n    </tr>\r\n    <tr>\r\n      <th>1</th>\r\n      <td>CALLS         Â     PUTS     Chart     OI ...</td>\r\n    </tr>\r\n    <tr>\r\n      <th>2</th>\r\n      <td>0\r\n0  Top</td>\r\n    </tr>\r\n  </tbody>\r\n</table>\r\n\r\n```\r\n\r\n"}}