{"action":"created","issue":{"url":"https://api.github.com/repos/ESMValGroup/ESMValTool/issues/665","repository_url":"https://api.github.com/repos/ESMValGroup/ESMValTool","labels_url":"https://api.github.com/repos/ESMValGroup/ESMValTool/issues/665/labels{/name}","comments_url":"https://api.github.com/repos/ESMValGroup/ESMValTool/issues/665/comments","events_url":"https://api.github.com/repos/ESMValGroup/ESMValTool/issues/665/events","html_url":"https://github.com/ESMValGroup/ESMValTool/issues/665","id":370969876,"node_id":"MDU6SXNzdWUzNzA5Njk4NzY=","number":665,"title":"Tool fails because 'institute' key is not found","user":{"login":"schlunma","id":32543114,"node_id":"MDQ6VXNlcjMyNTQzMTE0","avatar_url":"https://avatars3.githubusercontent.com/u/32543114?v=4","gravatar_id":"","url":"https://api.github.com/users/schlunma","html_url":"https://github.com/schlunma","followers_url":"https://api.github.com/users/schlunma/followers","following_url":"https://api.github.com/users/schlunma/following{/other_user}","gists_url":"https://api.github.com/users/schlunma/gists{/gist_id}","starred_url":"https://api.github.com/users/schlunma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schlunma/subscriptions","organizations_url":"https://api.github.com/users/schlunma/orgs","repos_url":"https://api.github.com/users/schlunma/repos","events_url":"https://api.github.com/users/schlunma/events{/privacy}","received_events_url":"https://api.github.com/users/schlunma/received_events","type":"User","site_admin":false},"labels":[{"id":526450342,"node_id":"MDU6TGFiZWw1MjY0NTAzNDI=","url":"https://api.github.com/repos/ESMValGroup/ESMValTool/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"open","locked":false,"assignee":{"login":"jvegasbsc","id":13482714,"node_id":"MDQ6VXNlcjEzNDgyNzE0","avatar_url":"https://avatars2.githubusercontent.com/u/13482714?v=4","gravatar_id":"","url":"https://api.github.com/users/jvegasbsc","html_url":"https://github.com/jvegasbsc","followers_url":"https://api.github.com/users/jvegasbsc/followers","following_url":"https://api.github.com/users/jvegasbsc/following{/other_user}","gists_url":"https://api.github.com/users/jvegasbsc/gists{/gist_id}","starred_url":"https://api.github.com/users/jvegasbsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jvegasbsc/subscriptions","organizations_url":"https://api.github.com/users/jvegasbsc/orgs","repos_url":"https://api.github.com/users/jvegasbsc/repos","events_url":"https://api.github.com/users/jvegasbsc/events{/privacy}","received_events_url":"https://api.github.com/users/jvegasbsc/received_events","type":"User","site_admin":false},"assignees":[{"login":"jvegasbsc","id":13482714,"node_id":"MDQ6VXNlcjEzNDgyNzE0","avatar_url":"https://avatars2.githubusercontent.com/u/13482714?v=4","gravatar_id":"","url":"https://api.github.com/users/jvegasbsc","html_url":"https://github.com/jvegasbsc","followers_url":"https://api.github.com/users/jvegasbsc/followers","following_url":"https://api.github.com/users/jvegasbsc/following{/other_user}","gists_url":"https://api.github.com/users/jvegasbsc/gists{/gist_id}","starred_url":"https://api.github.com/users/jvegasbsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jvegasbsc/subscriptions","organizations_url":"https://api.github.com/users/jvegasbsc/orgs","repos_url":"https://api.github.com/users/jvegasbsc/repos","events_url":"https://api.github.com/users/jvegasbsc/events{/privacy}","received_events_url":"https://api.github.com/users/jvegasbsc/received_events","type":"User","site_admin":false}],"milestone":null,"comments":10,"created_at":"2018-10-17T09:03:40Z","updated_at":"2018-10-17T10:43:25Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"After the latest PR (#632), the tool fails for every recipe (I tested all examples) with the following error:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"~/ESMValTool/esmvaltool/_main.py\", line 215, in run\r\n    conf = main(args)\r\n  File \"~/ESMValTool/esmvaltool/_main.py\", line 143, in main\r\n    process_recipe(recipe_file=recipe, config_user=cfg)\r\n  File \"~/ESMValTool/esmvaltool/_main.py\", line 189, in process_recipe\r\n    recipe = read_recipe_file(recipe_file, config_user)\r\n  File \"~/ESMValTool/esmvaltool/_recipe.py\", line 56, in read_recipe_file\r\n    raw_recipe, config_user, initialize_tasks, recipe_file=filename)\r\n  File \"~/ESMValTool/esmvaltool/_recipe.py\", line 803, in __init__\r\n    raw_recipe['diagnostics'], raw_recipe.get('datasets', []))\r\n  File \"~/ESMValTool/esmvaltool/_recipe.py\", line 834, in _initialize_diagnostics\r\n    raw_diagnostic.get('additional_datasets', []))\r\n  File \"~/ESMValTool/esmvaltool/_recipe.py\", line 915, in _initialize_preprocessor_output\r\n    self._initialize_variables(raw_variable, raw_datasets)\r\n  File \"~/ESMValTool/esmvaltool/_recipe.py\", line 893, in _initialize_variables\r\n    drs=self._cfg['drs'])\r\n  File \"~/ESMValTool/esmvaltool/_data_finder.py\", line 260, in get_input_fx_filelist\r\n    files = _find_input_files(var, rootpath, drs, fx_var)\r\n  File \"~/ESMValTool/esmvaltool/_data_finder.py\", line 235, in _find_input_files\r\n    input_dirs = _find_input_dirs(variable, rootpath, drs, fx_var)\r\n  File \"~/ESMValTool/esmvaltool/_data_finder.py\", line 209, in _find_input_dirs\r\n    for dirname_template in _replace_tags(path_template, variable, fx_var):\r\n  File \"~/ESMValTool/esmvaltool/_data_finder.py\", line 116, in _replace_tags\r\n    \"your recipe entry\".format(tag, variable))\r\nKeyError: \"Dataset key institute must be specified for {'preprocessor': 'preprocessor_1', 'field': 'T3M', 'fx_files': ['sftlf'], 'short_name': 'ta', 'diagnostic': 'ta_diagnostics', 'dataset': 'bcc-csm1-1', 'project': 'CMIP5', 'mip': 'Amon', 'exp': 'historical', 'ensemble': 'r1i1p1', 'start_year': 2000, 'end_year': 2002, 'cmor_table': 'CMIP5', 'filename': '/scratch/b/b309141/work_v2/recipe_ncl_20181017_085921/preproc/ta_diagnostics_preprocessor_1_ta/CMIP5_bcc-csm1-1_Amon_historical_r1i1p1_T3M_ta_2000-2002.nc', 'frequency': 'mon', 'modeling_realm': ['atmos']}, check your recipe entry\r\n```\r\n\r\nI used my old `config-user.yml` and a new one with a list as rootpaths; both fail. The old version (8e39c7ce) still works."},"comment":{"url":"https://api.github.com/repos/ESMValGroup/ESMValTool/issues/comments/430580177","html_url":"https://github.com/ESMValGroup/ESMValTool/issues/665#issuecomment-430580177","issue_url":"https://api.github.com/repos/ESMValGroup/ESMValTool/issues/665","id":430580177,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMDU4MDE3Nw==","user":{"login":"mattiarighi","id":25367743,"node_id":"MDQ6VXNlcjI1MzY3NzQz","avatar_url":"https://avatars1.githubusercontent.com/u/25367743?v=4","gravatar_id":"","url":"https://api.github.com/users/mattiarighi","html_url":"https://github.com/mattiarighi","followers_url":"https://api.github.com/users/mattiarighi/followers","following_url":"https://api.github.com/users/mattiarighi/following{/other_user}","gists_url":"https://api.github.com/users/mattiarighi/gists{/gist_id}","starred_url":"https://api.github.com/users/mattiarighi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattiarighi/subscriptions","organizations_url":"https://api.github.com/users/mattiarighi/orgs","repos_url":"https://api.github.com/users/mattiarighi/repos","events_url":"https://api.github.com/users/mattiarighi/events{/privacy}","received_events_url":"https://api.github.com/users/mattiarighi/received_events","type":"User","site_admin":false},"created_at":"2018-10-17T10:43:25Z","updated_at":"2018-10-17T10:43:25Z","author_association":"CONTRIBUTOR","body":"branch: version2_development\r\ncommit: c39788\r\nmd5sum: 4ac58a9aa6166e3c4bd9c0f71a3d53e3  _recipe.py\r\n_recipe.py:880 `institute = get_institutes(variable)`"}}
{"actor":{"display_login":"mattiarighi"}}