{"action":"created","issue":{"url":"https://api.github.com/repos/benchflow/data-transformers/issues/86","repository_url":"https://api.github.com/repos/benchflow/data-transformers","labels_url":"https://api.github.com/repos/benchflow/data-transformers/issues/86/labels{/name}","comments_url":"https://api.github.com/repos/benchflow/data-transformers/issues/86/comments","events_url":"https://api.github.com/repos/benchflow/data-transformers/issues/86/events","html_url":"https://github.com/benchflow/data-transformers/issues/86","id":172329215,"number":86,"title":"Mysql Transformer Error In Retrieving Configuration File","user":{"login":"VincenzoFerme","id":7163641,"avatar_url":"https://avatars.githubusercontent.com/u/7163641?v=3","gravatar_id":"","url":"https://api.github.com/users/VincenzoFerme","html_url":"https://github.com/VincenzoFerme","followers_url":"https://api.github.com/users/VincenzoFerme/followers","following_url":"https://api.github.com/users/VincenzoFerme/following{/other_user}","gists_url":"https://api.github.com/users/VincenzoFerme/gists{/gist_id}","starred_url":"https://api.github.com/users/VincenzoFerme/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VincenzoFerme/subscriptions","organizations_url":"https://api.github.com/users/VincenzoFerme/orgs","repos_url":"https://api.github.com/users/VincenzoFerme/repos","events_url":"https://api.github.com/users/VincenzoFerme/events{/privacy}","received_events_url":"https://api.github.com/users/VincenzoFerme/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/benchflow/data-transformers/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"Cerfoglg","id":8819678,"avatar_url":"https://avatars.githubusercontent.com/u/8819678?v=3","gravatar_id":"","url":"https://api.github.com/users/Cerfoglg","html_url":"https://github.com/Cerfoglg","followers_url":"https://api.github.com/users/Cerfoglg/followers","following_url":"https://api.github.com/users/Cerfoglg/following{/other_user}","gists_url":"https://api.github.com/users/Cerfoglg/gists{/gist_id}","starred_url":"https://api.github.com/users/Cerfoglg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Cerfoglg/subscriptions","organizations_url":"https://api.github.com/users/Cerfoglg/orgs","repos_url":"https://api.github.com/users/Cerfoglg/repos","events_url":"https://api.github.com/users/Cerfoglg/events{/privacy}","received_events_url":"https://api.github.com/users/Cerfoglg/received_events","type":"User","site_admin":false},"assignees":[{"login":"Cerfoglg","id":8819678,"avatar_url":"https://avatars.githubusercontent.com/u/8819678?v=3","gravatar_id":"","url":"https://api.github.com/users/Cerfoglg","html_url":"https://github.com/Cerfoglg","followers_url":"https://api.github.com/users/Cerfoglg/followers","following_url":"https://api.github.com/users/Cerfoglg/following{/other_user}","gists_url":"https://api.github.com/users/Cerfoglg/gists{/gist_id}","starred_url":"https://api.github.com/users/Cerfoglg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Cerfoglg/subscriptions","organizations_url":"https://api.github.com/users/Cerfoglg/orgs","repos_url":"https://api.github.com/users/Cerfoglg/repos","events_url":"https://api.github.com/users/Cerfoglg/events{/privacy}","received_events_url":"https://api.github.com/users/Cerfoglg/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/benchflow/data-transformers/milestones/1","html_url":"https://github.com/benchflow/data-transformers/milestones/Priorities","labels_url":"https://api.github.com/repos/benchflow/data-transformers/milestones/1/labels","id":1366601,"number":1,"title":"Priorities","description":"Stuff we must do soon. We never change or rename this milestone.","creator":{"login":"VincenzoFerme","id":7163641,"avatar_url":"https://avatars.githubusercontent.com/u/7163641?v=3","gravatar_id":"","url":"https://api.github.com/users/VincenzoFerme","html_url":"https://github.com/VincenzoFerme","followers_url":"https://api.github.com/users/VincenzoFerme/followers","following_url":"https://api.github.com/users/VincenzoFerme/following{/other_user}","gists_url":"https://api.github.com/users/VincenzoFerme/gists{/gist_id}","starred_url":"https://api.github.com/users/VincenzoFerme/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VincenzoFerme/subscriptions","organizations_url":"https://api.github.com/users/VincenzoFerme/orgs","repos_url":"https://api.github.com/users/VincenzoFerme/repos","events_url":"https://api.github.com/users/VincenzoFerme/events{/privacy}","received_events_url":"https://api.github.com/users/VincenzoFerme/received_events","type":"User","site_admin":false},"open_issues":4,"closed_issues":25,"state":"open","created_at":"2015-10-20T16:20:08Z","updated_at":"2016-08-21T16:43:55Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2016-08-21T16:43:55Z","updated_at":"2016-08-21T17:11:00Z","closed_at":null,"body":"An execution log reporting the error as in the title. Moreover in case of an error, the computation must not be marked as succeeded as it seems to be marked from the following logs.\r\n\r\n```\r\n8/21/2016 6:11:18 PMReceived message: {\"minio_key\":\"eaa2/BenchFlow.wfmsTest.1.1/BenchFlow.wfmsTest.1.1.1/benchflow.collector.mysql.db_BenchFlow.wfmsTest.1.1.1/mysql/mysql/process-engine\",\"trial_id\":\"BenchFlow.wfmsTest.1.1.1\",\"experiment_id\":\"BenchFlow.wfmsTest.1.1\",\"container_id\":\"mysql\",\"container_name\":\"mysql\",\"host_id\":\"host\",\"collector_name\":\"mysql\"}\r\n8/21/2016 6:11:18 PMmysql topic, submitting script data-transformers/transformers/mysqlTransformer.py, minio location: eaa2/BenchFlow.wfmsTest.1.1/BenchFlow.wfmsTest.1.1.1/benchflow.collector.mysql.db_BenchFlow.wfmsTest.1.1.1/mysql/mysql/process-engine, trial id: BenchFlow.wfmsTest.1.1.1\r\n8/21/2016 6:11:18 PMTransformer work request queued for script mysql, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial\r\n8/21/2016 6:11:18 PMReceived work request for script mysql, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial\r\n8/21/2016 6:11:18 PMDispatching work request for script mysql, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial\r\n8/21/2016 6:11:18 PMTransformer worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/data-transformers/commons/commons.py,/app/data-transformers/transformations/dataTransformations.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/data-transformers/transformers/mysqlTransformer.py {\"cassandra_keyspace\":\"benchflow\",\"minio_host\":\"minio\",\"minio_port\":\"9000\",\"minio_access_key\":\"CYNQML6R7V12MTT32W6P\",\"minio_secret_key\":\"SQ96V5pg02Z3kZ/0ViF9YY6GwWzZvoBmElpzEEjn\",\"file_bucket\":\"runs\",\"file_path\":\"eaa2/BenchFlow.wfmsTest.1.1/BenchFlow.wfmsTest.1.1.1/benchflow.collector.mysql.db_BenchFlow.wfmsTest.1.1.1/mysql/mysql/process-engine\",\"trial_id\":\"BenchFlow.wfmsTest.1.1.1\",\"experiment_id\":\"BenchFlow.wfmsTest.1.1\",\"config_file\":\"data-transformers.configuration.yml\",\"container_id\":\"mysql\",\"host_id\":\"host\"}]\r\n8/21/2016 6:11:18 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin\r\n8/21/2016 6:11:22 PMTraceback (most recent call last):\r\n8/21/2016 6:11:22 PM  File \"/app/data-transformers/transformers/mysqlTransformer.py\", line 196, in <module>\r\n8/21/2016 6:11:22 PM    main()\r\n8/21/2016 6:11:22 PM  File \"/app/data-transformers/transformers/mysqlTransformer.py\", line 140, in main\r\n8/21/2016 6:11:22 PM    with open(confPath) as f:\r\n8/21/2016 6:11:22 PMIOError: [Errno 2] No such file or directory: u'/tmp/spark-1f5c7d2d-5bd1-4645-accf-6511a3dddd34/userFiles-47d4f8d2-27ee-4f7e-88d5-66cd1f15a41f/data-transformers.configuration.yml'\r\n8/21/2016 6:11:22 PM\r\n8/21/2016 6:11:22 PMScript data-transformers/transformers/mysqlTransformer.py processed\r\n8/21/2016 6:11:22 PMTransformer script completed successfully: mysql, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial\r\n8/21/2016 6:11:22 PMAll requirements met for: mysql\r\n```\r\n\r\nPlease test also all the other transformers again in the current state of the system. And add test cases for all the encountered problems."},"comment":{"url":"https://api.github.com/repos/benchflow/data-transformers/issues/comments/241269385","html_url":"https://github.com/benchflow/data-transformers/issues/86#issuecomment-241269385","issue_url":"https://api.github.com/repos/benchflow/data-transformers/issues/86","id":241269385,"user":{"login":"simonedavico","id":4446853,"avatar_url":"https://avatars.githubusercontent.com/u/4446853?v=3","gravatar_id":"","url":"https://api.github.com/users/simonedavico","html_url":"https://github.com/simonedavico","followers_url":"https://api.github.com/users/simonedavico/followers","following_url":"https://api.github.com/users/simonedavico/following{/other_user}","gists_url":"https://api.github.com/users/simonedavico/gists{/gist_id}","starred_url":"https://api.github.com/users/simonedavico/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simonedavico/subscriptions","organizations_url":"https://api.github.com/users/simonedavico/orgs","repos_url":"https://api.github.com/users/simonedavico/repos","events_url":"https://api.github.com/users/simonedavico/events{/privacy}","received_events_url":"https://api.github.com/users/simonedavico/received_events","type":"User","site_admin":false},"created_at":"2016-08-21T17:11:00Z","updated_at":"2016-08-21T17:11:00Z","body":"@Cerfoglg in the configuration file capitalisation doesn't matter.\r\nOne could write all possible lowercase/uppercase combinations of the letters w,f,m,s.\r\nThis is because the correct way to shorten \"Workflow Management Systems\" is WfMS in the literature :)\r\n\r\nIt's better if you don't assume lowercase or uppercase when you read field values from test configuration files. For sanity, when you read a field, convert it to all lowercase first (should be just a method call), and then compare its value. "}}
{"actor":{"display_login":"simonedavico"}}