{"action":"created","issue":{"url":"https://api.github.com/repos/expfactory/expfactory-docker/issues/66","repository_url":"https://api.github.com/repos/expfactory/expfactory-docker","labels_url":"https://api.github.com/repos/expfactory/expfactory-docker/issues/66/labels{/name}","comments_url":"https://api.github.com/repos/expfactory/expfactory-docker/issues/66/comments","events_url":"https://api.github.com/repos/expfactory/expfactory-docker/issues/66/events","html_url":"https://github.com/expfactory/expfactory-docker/issues/66","id":140436791,"number":66,"title":"Discussion over current setup for Experiment Factory and Mturk","user":{"login":"IanEisenberg","id":9325440,"avatar_url":"https://avatars.githubusercontent.com/u/9325440?v=3","gravatar_id":"","url":"https://api.github.com/users/IanEisenberg","html_url":"https://github.com/IanEisenberg","followers_url":"https://api.github.com/users/IanEisenberg/followers","following_url":"https://api.github.com/users/IanEisenberg/following{/other_user}","gists_url":"https://api.github.com/users/IanEisenberg/gists{/gist_id}","starred_url":"https://api.github.com/users/IanEisenberg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/IanEisenberg/subscriptions","organizations_url":"https://api.github.com/users/IanEisenberg/orgs","repos_url":"https://api.github.com/users/IanEisenberg/repos","events_url":"https://api.github.com/users/IanEisenberg/events{/privacy}","received_events_url":"https://api.github.com/users/IanEisenberg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-03-13T02:12:26Z","updated_at":"2016-03-13T03:31:12Z","closed_at":null,"body":"Referencing @poldrack, @bissettp, and @zenkavi, along with @vsoch for this discussion.\r\n\r\n@vosch recently updated the experiment factory structure quite a bit. Now clicking on \"start experiment\" at the end of consent/instructions in the Mturk iframe will open a new page which will either say \"You have to be in this experiment to proceed\", \"We have no more experiments for you at this time\" or it shows you the next experiment. You can do multiple experiments in a row, but if you leave and want to start again, you go to the Mturk page again, go through consent/instructions, click \"start experiment\" and go.\r\n\r\nBecause this is a fundamental way we deploy experiments I thought it'd be good to have everyone get an update. I think some choices @vsoch made are great beyond our current plans. \r\n\r\n**(1)** Each time a person returns they are shown ad/consent/instructions again. While potentially annoying to workers, I like that people are reminded about \"what's going on\" before each \"session\" of experiments. It does mean that people who do the session in more batches will see instructions/consent more often. Is that an inadvertent manipulation we'd like to avoid? I think the benefits of reminding people each session are good, and none of the instructions mention anything about our DVs (e.g. \"we are looking at self regulation\").\r\n\r\n**(2)** At the end of the tasks the HIT is not returned. This means that every person's HIT is returned at expiration regardless of when they actually complete the HIT. This has some Pros and Cons. \r\n\r\n**Cons:** a little weird - turkers are used to having their HIT returned at completion of work. Their pay will be delayed, etc. They may see that they still have an outstanding HIT, try to continue, and come back to \"no more experiments\" and get annoyed, and give us bad reviews.\r\n\r\n **Pros** (and here I think there is potential in the future), this setup allows us to _phase_ deployment. Basically we can give a set of people a battery with 12 tasks in it, wait until they complete all of them, and then add more tasks. The HIT is still alive so we just need some way to let them know to check it again. Not sure when we'd use this, but considering that right now the infrastructure is built to randomly deploy experiments it may be a useful workaround to control order (though maybe we want another way to control presentation order in the future), or create a forced delay between tasks. Additionally I imagine this approach is simpler, which will be helpful (especially when @vosch doesn't have as much time to maintain this infrastructure!)\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/expfactory/expfactory-docker/issues/comments/195862339","html_url":"https://github.com/expfactory/expfactory-docker/issues/66#issuecomment-195862339","issue_url":"https://api.github.com/repos/expfactory/expfactory-docker/issues/66","id":195862339,"user":{"login":"vsoch","id":814322,"avatar_url":"https://avatars.githubusercontent.com/u/814322?v=3","gravatar_id":"","url":"https://api.github.com/users/vsoch","html_url":"https://github.com/vsoch","followers_url":"https://api.github.com/users/vsoch/followers","following_url":"https://api.github.com/users/vsoch/following{/other_user}","gists_url":"https://api.github.com/users/vsoch/gists{/gist_id}","starred_url":"https://api.github.com/users/vsoch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vsoch/subscriptions","organizations_url":"https://api.github.com/users/vsoch/orgs","repos_url":"https://api.github.com/users/vsoch/repos","events_url":"https://api.github.com/users/vsoch/events{/privacy}","received_events_url":"https://api.github.com/users/vsoch/received_events","type":"User","site_admin":false},"created_at":"2016-03-13T03:31:12Z","updated_at":"2016-03-13T03:31:12Z","body":"Hmm, definitely not desired behavior. It's hard to tell what is limited to sandbox vs. bug. I will test this tomorrow."}}