Adding HendrycksTest dataset
Adding Hendrycks test from https://arxiv.org/abs/2009.03300.
I'm having a bit of trouble with dummy data creation because some lines in the csv files aren't being loaded properly (only the first entry loaded in a row of length 6). The dataset is loading just fine. Hope you can kindly help!
Thank you!
@lhoestq Thank you for the review. I've made the suggested changes. There still might be some problems with dummy data though due to some csv loading issues (which I haven't found the cause to). I took a look at the dummy data and some csv lines were cropped. I fixed them :) @andyzoujm Any reason why this dataset scrip was called "hendrycks_test" instead of "mmlu"?

We are thinking of renaming it... That's because we didn't call it MMLU in the paper (the shorthand didn't
emerge until over a year later), and people at OpenAI were calling it that.

Andy

On Wed, Apr 26, 2023 at 8:44 AM Albert Villanova del Moral <
***@***.***> wrote:

> @andyzoujm <https://github.com/andyzoujm> Any reason why this dataset
> scrip was called "hendrycks_test" instead of "mmlu"?
>
> We are thinking of renaming it...
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/pull/2370#issuecomment-1523358110>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKLQJMZZFZBGJTBOIFWJ5KDXDEKB5ANCNFSM45BAOSIQ>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
 Thanks for your reply. Just for the records: we have renamed it to "cais/mmlu": https://huggingface.co/datasets/cais/mmlu