Download datasets from a private hub
For reference:
https://github.com/huggingface/transformers/issues/15514
https://github.com/huggingface/huggingface_hub/issues/650 Hi ! For information one can set the environment variable `HF_ENDPOINT` (default is `https://huggingface.co`) if they want to use a private hub.

We may need to coordinate with the other libraries to have a consistent way of changing the hub endpoint Yes, I tested it successfully this morning. Thanks.
In the context of a private hub deployment, customers would like to use load_dataset() to load datasets from their hub, not from the public hub. This doesn't seem to be configurable at the moment and it would be nice to add this feature.

The obvious workaround is to clone the repo first and then load it from local storage, but this adds an extra step. It'd be great to have the same experience regardless of where the hub is hosted.

The same issue exists with the transformers library and the CLI. I'm going to create issues there as well, and I'll reference them below.