{"action":"opened","issue":{"url":"https://api.github.com/repos/cylc/cylc/issues/2912","repository_url":"https://api.github.com/repos/cylc/cylc","labels_url":"https://api.github.com/repos/cylc/cylc/issues/2912/labels{/name}","comments_url":"https://api.github.com/repos/cylc/cylc/issues/2912/comments","events_url":"https://api.github.com/repos/cylc/cylc/issues/2912/events","html_url":"https://github.com/cylc/cylc/issues/2912","id":395796611,"node_id":"MDU6SXNzdWUzOTU3OTY2MTE=","number":2912,"title":"Use json instead of pickle for broadcast_states in cylc-cat-state?","user":{"login":"kinow","id":304786,"node_id":"MDQ6VXNlcjMwNDc4Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/304786?v=4","gravatar_id":"","url":"https://api.github.com/users/kinow","html_url":"https://github.com/kinow","followers_url":"https://api.github.com/users/kinow/followers","following_url":"https://api.github.com/users/kinow/following{/other_user}","gists_url":"https://api.github.com/users/kinow/gists{/gist_id}","starred_url":"https://api.github.com/users/kinow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kinow/subscriptions","organizations_url":"https://api.github.com/users/kinow/orgs","repos_url":"https://api.github.com/users/kinow/repos","events_url":"https://api.github.com/users/kinow/events{/privacy}","received_events_url":"https://api.github.com/users/kinow/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-01-04T03:15:44Z","updated_at":"2019-01-04T03:15:44Z","closed_at":null,"author_association":"MEMBER","body":"Hi,\r\n\r\nWhile testing the changes for #2815 I wanted to confirm the task states, so used `cylc cat-state` as below.\r\n\r\n```bash\r\ntestuser@cylc:~/suites$ cylc cat-state pbs2\r\nrun mode : live\r\ntime : 2019-01-04T03:00:46Z (1546570846)\r\ninitial cycle : 20190104T0251Z\r\nfinal cycle : None\r\n(dp0\r\n.\r\nBegin task states\r\n```\r\n\r\nAnd that `(dp0` caught my eye. A quick look at the code, showed me that it was the [`broadcast_states`](https://github.com/cylc/cylc/blob/c43c0c3009732358b64114fc4846e33b70989d13/bin/cylc-cat-state#L107) empty dictionary being dumped with `pickle`.\r\n\r\nThen I tried an example from the CUG to send a broadcast to the `pbs2` running suite.\r\n\r\n```bash\r\ntestuser@cylc:~/suites$ cylc broadcast -s \" [ environment ] VERSE = the quick brown fox \" pbs2\r\nBroadcast set:\r\n+ [root.*] [environment]VERSE=the quick brown fox\r\n```\r\n\r\nRunning `cat-state` again gives an ugly - and not so useful - output:\r\n\r\n```bash\r\ntestuser@cylc:~/suites$ cylc cat-state pbs2\r\nrun mode : live\r\ntime : 2019-01-04T03:07:41Z (1546571261)\r\ninitial cycle : 20190104T0251Z\r\nfinal cycle : None\r\n(dp0\r\nS'*'\r\np1\r\n(dp2\r\nS'root'\r\np3\r\n(dp4\r\nS'environment'\r\np5\r\n(dp6\r\nS'VERSE'\r\np7\r\nS'the quick brown fox'\r\np8\r\nssss.\r\nBegin task states\r\nt1.20200313T0251Z : status=ready, spawned=False\r\n```\r\n\r\nInstead of using `pickle`, if we use the `json` module, I think the output is clearer.\r\n\r\n```\r\ntestuser@cylc:~/suites$ cylc cat-state pbs2\r\nrun mode : live\r\ntime : 2019-01-04T03:07:05Z (1546571225)\r\ninitial cycle : 20190104T0251Z\r\nfinal cycle : None\r\n{\"*\": {\"root\": {\"environment\": {\"VERSE\": \"the quick brown fox\"}}}}\r\nBegin task states\r\nt1.20200228T0251Z : status=ready, spawned=False\r\n```\r\n\r\nHere's what I used for this test:\r\n\r\n```diff\r\nkinow@kinow-VirtualBox:~/Development/python/workspace/cylc$ git diff\r\ndiff --git a/bin/cylc-cat-state b/bin/cylc-cat-state\r\nindex 0dffa5507..6543e2e6f 100755\r\n--- a/bin/cylc-cat-state\r\n+++ b/bin/cylc-cat-state\r\n@@ -27,7 +27,7 @@ if remrun():\r\n     sys.exit(0)\r\n \r\n import os\r\n-import pickle\r\n+import json\r\n import re\r\n import sqlite3\r\n import traceback\r\n@@ -104,7 +104,7 @@ def _get_state_lines(suite):\r\n         r\"time : %(time_str)s (%(time_since_epoch)s)\" % data,\r\n         r\"initial cycle : %(initial_point)s\" % data,\r\n         r\"final cycle : %(final_point)s\" % data]\r\n-    lines += pickle.dumps(data[\"broadcast_states\"]).splitlines()\r\n+    lines += json.dumps(data[\"broadcast_states\"]).splitlines()\r\n     lines.append(r\"Begin task states\")\r\n     for item in data[\"task_pool\"]:\r\n         lines.append(\r\n```\r\n\r\nJust would have to update some tests, but I think this is doable. Would it be useful, and also would it be a good idea or is it going to disappear with the migration to the web (I don't think so, but just in case). Not sure if the `pickle` output is actually used as-is for troubleshooting/operations...\r\n\r\nCheers\r\nBruno\r\n"}}
{"actor":{"display_login":"kinow"}}