{"action":"created","issue":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/14628","repository_url":"https://api.github.com/repos/pandas-dev/pandas","labels_url":"https://api.github.com/repos/pandas-dev/pandas/issues/14628/labels{/name}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/14628/comments","events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/14628/events","html_url":"https://github.com/pandas-dev/pandas/issues/14628","id":188388847,"number":14628,"title":"to_csv execution unreliable in combination with for loop","user":{"login":"Void2258","id":6126362,"avatar_url":"https://avatars.githubusercontent.com/u/6126362?v=3","gravatar_id":"","url":"https://api.github.com/users/Void2258","html_url":"https://github.com/Void2258","followers_url":"https://api.github.com/users/Void2258/followers","following_url":"https://api.github.com/users/Void2258/following{/other_user}","gists_url":"https://api.github.com/users/Void2258/gists{/gist_id}","starred_url":"https://api.github.com/users/Void2258/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Void2258/subscriptions","organizations_url":"https://api.github.com/users/Void2258/orgs","repos_url":"https://api.github.com/users/Void2258/repos","events_url":"https://api.github.com/users/Void2258/events{/privacy}","received_events_url":"https://api.github.com/users/Void2258/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-11-10T00:30:51Z","updated_at":"2016-11-10T03:43:28Z","closed_at":null,"body":"I am outputting a single column from a dataframe to a csv. The data, however, is too long for some older downstream applications, so I have the csv add line breaks every `size` items so that individual rows in the output aren't too long.\r\n\r\n#### Example\r\n```python\r\nfrom tkinter import Tk\r\nfrom tkinter import filedialog\r\nimport pandas as pd\r\nimport numpy as np\r\n\r\nTk().withdraw() # we don't want a full GUI, so keep the root window from appearing\r\ninfilename = filedialog.askopenfilename()# show an \"Open\" dialog box and return the path to the selected file\r\ndata = pd.read_csv(infilename, header=None) #usecols=[0], only get 1st column, specify no header\r\noutfilename = filedialog.asksaveasfile() #get save location\r\n\r\nsize=50 #number of items per line\r\ncol=0\r\nindexes = np.arange(0,len(data),size)#have to use numpy since range is now an immutable type in python 3\r\nindexes = np.append(indexes,[len(data)]) #add the uneven final index\r\nfor i in range(len(indexes)-1):\r\n    holder = pd.DataFrame(data.iloc[indexes[i]:indexes[i+1],col]).T\r\n    holder.to_csv(outfilename, index=False, header=False)\r\n```\r\n\r\n#### Expected Output\r\nfor input: \r\nA,B,C,D,E,F,G,H,I,J,K  #this is actually a column in the input, rendered horizontal for space\r\n\r\nBecomes in the file (size=3):\r\nA,B,C\r\nD,E,F\r\nG,H,I\r\nJ,K\r\n\r\nDespite not throwing any errors, the final loop (with the uneven final index) does not write to the file, even though the information is assigned to `holder` without issue. Also, despite my not setting the `mode` param, acts as `w` and overwrites file if existing on the first loop, then acts as `a` and appends to the file on subsequent loops, indicating some possible statefullness which may be relevant. Since no errors are thrown, I cannot figure out why the final information is not being written. \r\n\r\n#### Output of ``pd.show_versions()``\r\n\r\n<details>\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 3.5.2.final.0\r\npython-bits: 64\r\nOS: Windows\r\nOS-release: 10\r\nmachine: AMD64\r\nprocessor: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: None\r\n\r\npandas: 0.18.1\r\nnose: 1.3.7\r\npip: 8.1.2\r\nsetuptools: 23.0.0\r\nCython: 0.24\r\nnumpy: 1.11.1\r\nscipy: 0.17.1\r\nstatsmodels: 0.6.1\r\nxarray: None\r\nIPython: 4.2.0\r\nsphinx: 1.3.1\r\npatsy: 0.4.1\r\ndateutil: 2.5.3\r\npytz: 2016.4\r\nblosc: None\r\nbottleneck: 1.1.0\r\ntables: 3.2.2\r\nnumexpr: 2.6.0\r\nmatplotlib: 1.5.1\r\nopenpyxl: 2.3.2\r\nxlrd: 1.0.0\r\nxlwt: 1.1.2\r\nxlsxwriter: 0.9.2\r\nlxml: 3.6.0\r\nbs4: 4.4.1\r\nhtml5lib: None\r\nhttplib2: None\r\napiclient: None\r\nsqlalchemy: 1.0.13\r\npymysql: None\r\npsycopg2: None\r\njinja2: 2.8\r\nboto: 2.40.0\r\npandas_datareader: None\r\n\r\n</details>\r\n"},"comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments/259594714","html_url":"https://github.com/pandas-dev/pandas/issues/14628#issuecomment-259594714","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/14628","id":259594714,"user":{"login":"Void2258","id":6126362,"avatar_url":"https://avatars.githubusercontent.com/u/6126362?v=3","gravatar_id":"","url":"https://api.github.com/users/Void2258","html_url":"https://github.com/Void2258","followers_url":"https://api.github.com/users/Void2258/followers","following_url":"https://api.github.com/users/Void2258/following{/other_user}","gists_url":"https://api.github.com/users/Void2258/gists{/gist_id}","starred_url":"https://api.github.com/users/Void2258/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Void2258/subscriptions","organizations_url":"https://api.github.com/users/Void2258/orgs","repos_url":"https://api.github.com/users/Void2258/repos","events_url":"https://api.github.com/users/Void2258/events{/privacy}","received_events_url":"https://api.github.com/users/Void2258/received_events","type":"User","site_admin":false},"created_at":"2016-11-10T03:43:28Z","updated_at":"2016-11-10T03:43:28Z","body":"OK, I just tried hardcoding the paths and removing all tkinter. Now, it writes the data perfectly BUT it no longer shows the variable `mode` behavior; it repeatedly overwrites the single line of data, but it writes the last set successfully. The example below includes a modified for loop to account for this.  \r\n\r\n```python\r\nimport pandas as pd\r\nimport numpy as np\r\n\r\ninfilename = 'C:\\\\Users\\\\...infile.csv'\r\ndata = pd.read_csv(infilename, header=None) #usecols=[0], only get 1st column, specify no header\r\noutfilename = 'C:\\\\Users\\\\...test.txt'\r\n\r\nsize=50 #number of items per line\r\ncol=0\r\nindexes = np.arange(0,len(data),size)#have to use numpy since range is now an immutable type in python 3\r\nindexes = np.append(indexes,[len(data)]) #add the uneven final index\r\nfor i in range(len(indexes)-1):\r\n    holder = pd.DataFrame(data.iloc[indexes[i]:indexes[i+1],col]).T\r\n    if i ==0:\r\n        holder.to_csv(outfilename, index=False, header=False)\r\n    else:\r\n        holder.to_csv(outfilename, index=False, header=False, mode='a')\r\n```"}}
{"actor":{"display_login":"Void2258"}}