{"action":"created","issue":{"url":"https://api.github.com/repos/moorepants/eme185/issues/145","repository_url":"https://api.github.com/repos/moorepants/eme185","labels_url":"https://api.github.com/repos/moorepants/eme185/issues/145/labels{/name}","comments_url":"https://api.github.com/repos/moorepants/eme185/issues/145/comments","events_url":"https://api.github.com/repos/moorepants/eme185/issues/145/events","html_url":"https://github.com/moorepants/eme185/issues/145","id":293604400,"number":145,"title":"Tuning up the rubric design","user":{"login":"moorepants","id":276007,"avatar_url":"https://avatars0.githubusercontent.com/u/276007?v=4","gravatar_id":"","url":"https://api.github.com/users/moorepants","html_url":"https://github.com/moorepants","followers_url":"https://api.github.com/users/moorepants/followers","following_url":"https://api.github.com/users/moorepants/following{/other_user}","gists_url":"https://api.github.com/users/moorepants/gists{/gist_id}","starred_url":"https://api.github.com/users/moorepants/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moorepants/subscriptions","organizations_url":"https://api.github.com/users/moorepants/orgs","repos_url":"https://api.github.com/users/moorepants/repos","events_url":"https://api.github.com/users/moorepants/events{/privacy}","received_events_url":"https://api.github.com/users/moorepants/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-02-01T16:51:50Z","updated_at":"2018-02-01T19:45:53Z","closed_at":null,"author_association":"OWNER","body":"Rubric goals:\r\n\r\n- Provide students with concrete/clear explanations of what they are expected to include.\r\n- Ensure that grading is consistent among many graders.\r\n- Ensure that the grade for each assignment is on a 100 pt scale and that the distribution of grades has a spread around a mean of about 80. The +3*sigma should be at 100 and -3*sigma at 60 (occasionally there may be a grade in the 50-60 range). This will ensure clear numerical feedback to the students and reduce the likelihood that curving the final grade is needed. It is also better to have to curve up a bit than down.\r\n- Interpretation of the rubrics by the graders should be simple, simple numbers, simple scales, simple wording.\r\n- The rubrics should give feedback on the particular assignment, e.g. \"The group can develop a well designed memo.\" But a student can collect thorough data and present beautifully, but miss the point and the data that the memo is based on is in the wrong direction or flawed. In this case, we also need a way to give feedback to the group that tells them that they are not on a good design track. Another example may be that they present a design perfectly, but the design is just a bad design. We may be able to design rubrics that produce a grade that reflects both getting the presentation of the work correct and simultaneously grades whether the work is good or poor, but I haven't had success with that so far and am not sure what to do. We could talk with the UCD assessment coordinator to get help with this.\r\n- Students should interpret the grading scale correctly. There was at least one instance in 2017 where a team assumed that the progress section accounted for 50 points, so they ignored or downplayed the content and presentation sections.\r\n- Rubric rows should tie into ABET criteria.\r\n\r\nCurrent Design:\r\n\r\n- Rubrics have two sections: content and presentation.\r\n- The point layout ensures scores lie between 40 and 100, but due to the fact that few teams ever get lots of zeros, the distribution lies more in the 60 to 100 range.\r\n- The rubrics have lots of rows and are quite detailed (maybe too much so).\r\n- There is a progress row worth 10 of 100 points, that provides the grader flexibility on adjusting the grade wrt to anything that the other rubric rows don't specifically address. Docking this grade should always come with a written comment as to why. It was intended to let the grader subjectively think about whether the team is on track wrt to the other teams in the class and provide a mechanism for feedback about that (we had no way to give this feedback in 2016 and grades were quite inflated and down curving was necessary on the final grade).\r\n- Presentation scores are lower percentage than the content percentage.\r\n- The point ranges for each ruric row are often 0 to 1.25, 0 to 2.5, and 0 to 5. This is a bit annoying for the grader because you have to calculate different intermediary values for each row. The original spreadsheet rubrics allowed the grader to select 1, 2, 3, 4 or 5 for each row, just whole numbers.\r\n\r\nThings to potentially fix:\r\n\r\n- Simplify the rubrics.\r\n- Add in ABET rows.\r\n- Make numerical values simpler: whole number choices preferably.\r\n- Try to design each cell with a positive statement instead of mix of positive/negative."},"comment":{"url":"https://api.github.com/repos/moorepants/eme185/issues/comments/362380278","html_url":"https://github.com/moorepants/eme185/issues/145#issuecomment-362380278","issue_url":"https://api.github.com/repos/moorepants/eme185/issues/145","id":362380278,"user":{"login":"moorepants","id":276007,"avatar_url":"https://avatars0.githubusercontent.com/u/276007?v=4","gravatar_id":"","url":"https://api.github.com/users/moorepants","html_url":"https://github.com/moorepants","followers_url":"https://api.github.com/users/moorepants/followers","following_url":"https://api.github.com/users/moorepants/following{/other_user}","gists_url":"https://api.github.com/users/moorepants/gists{/gist_id}","starred_url":"https://api.github.com/users/moorepants/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moorepants/subscriptions","organizations_url":"https://api.github.com/users/moorepants/orgs","repos_url":"https://api.github.com/users/moorepants/repos","events_url":"https://api.github.com/users/moorepants/events{/privacy}","received_events_url":"https://api.github.com/users/moorepants/received_events","type":"User","site_admin":false},"created_at":"2018-02-01T19:45:53Z","updated_at":"2018-02-01T19:45:53Z","author_association":"OWNER","body":"We could redistribute points in the rubrics to cover 0-100 points and them make each row go from some min value to the max where the min is nonzero to get the same effect. But, edit the rubrics in Canvas is quite the painful experience. You can see some of the complaints I've filled here: https://github.com/moorepants/eng-edu/issues/16 wrt to that. I pretty much loathe trying edit them, especially the points. That being said, I'm reluctant to spend a lot of time adjusting them if it is for little gain. Not sure redistributing the points gains us a lot."}}
{"actor":{"display_login":"moorepants"}}