Download only split data
Hi @capsabogdan! Unfortunately, it's hard to implement because quite often datasets data is being hosted in a single archive for all splits :( So we have to download the whole archive to split it into splits. This is the case for CommonVoice too. 

However, for cases when data is distributed in separate archives ащк different splits I suppose it can (and will) be implemented someday. 


Btw for quick check of the dataset you can use [streaming](https://huggingface.co/docs/datasets/stream):
```python
cv = load_dataset("mozilla-foundation/common_voice_11_0", "en", split="test", streaming=True)
cv = iter(cv)
print(next(cv))

>> {'client_id': 'a07b17f8234ded5e847443ea6f423cef745cbbc7537fb637d58326000aa751e829a21c4fd0a35fc17fb833aa7e95ebafce5efd19beeb8d843887b85e4eb35f5b',
>>  'path': None,
>>  'audio': {'path': 'cv-corpus-11.0-2022-09-21/en/clips/common_voice_en_100363.mp3',
>>  'array': array([ 0.0000000e+00,  1.1748125e-14,  1.5450088e-14, ...,
>>          1.3011958e-06, -6.3548953e-08, -9.9098514e-08], dtype=float32),
>> ...}

``` thank you for the answer but am not sure if this will not be helpful, as we
need maybe just 10% of the datasets for some experiment

can we get just a portion of the dataset with stream?


is there really no solution? :(

Am Di., 15. Nov. 2022 um 16:55 Uhr schrieb Polina Kazakova <
***@***.***>:

> Hi @capsabogdan <https://github.com/capsabogdan>! Unfortunately, it's
> hard to implement because quite often datasets data is being hosted in a
> single archive for all splits :( So we have to download the whole archive
> to split it into splits. This is the case for CommonVoice too.
>
> However, for cases when data is distributed in separate archives in
> different splits I suppose it can be implemented someday.
>
> Btw for quick check of the dataset you can use streaming
> <https://huggingface.co/docs/datasets/stream>:
>
> cv = load_dataset("mozilla-foundation/common_voice_11_0", "en", split="test", streaming=True)cv = iter(cv)print(next(cv))
> >> {'client_id': 'a07b17f8234ded5e847443ea6f423cef745cbbc7537fb637d58326000aa751e829a21c4fd0a35fc17fb833aa7e95ebafce5efd19beeb8d843887b85e4eb35f5b',>>  'path': None,>>  'audio': {'path': 'cv-corpus-11.0-2022-09-21/en/clips/common_voice_en_100363.mp3',>>  'array': array([ 0.0000000e+00,  1.1748125e-14,  1.5450088e-14, ...,>>          1.3011958e-06, -6.3548953e-08, -9.9098514e-08], dtype=float32),>> ...}
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/5243#issuecomment-1315512887>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ALSIFOC3JYRCTH54OBRUJULWIOW6PANCNFSM6AAAAAASAYO2LY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
 maybe it would be nice if you guys ould do some sort of shard before
loading the dataset, so users can download just chunks of data :)

I think this would be very helpful

Am Di., 15. Nov. 2022 um 19:24 Uhr schrieb Bogdan Capsa <
***@***.***>:

> thank you for the answer but am not sure if this will not be helpful, as
> we need maybe just 10% of the datasets for some experiment
>
> can we get just a portion of the dataset with stream?
>
>
> is there really no solution? :(
>
> Am Di., 15. Nov. 2022 um 16:55 Uhr schrieb Polina Kazakova <
> ***@***.***>:
>
>> Hi @capsabogdan <https://github.com/capsabogdan>! Unfortunately, it's
>> hard to implement because quite often datasets data is being hosted in a
>> single archive for all splits :( So we have to download the whole archive
>> to split it into splits. This is the case for CommonVoice too.
>>
>> However, for cases when data is distributed in separate archives in
>> different splits I suppose it can be implemented someday.
>>
>> Btw for quick check of the dataset you can use streaming
>> <https://huggingface.co/docs/datasets/stream>:
>>
>> cv = load_dataset("mozilla-foundation/common_voice_11_0", "en", split="test", streaming=True)cv = iter(cv)print(next(cv))
>> >> {'client_id': 'a07b17f8234ded5e847443ea6f423cef745cbbc7537fb637d58326000aa751e829a21c4fd0a35fc17fb833aa7e95ebafce5efd19beeb8d843887b85e4eb35f5b',>>  'path': None,>>  'audio': {'path': 'cv-corpus-11.0-2022-09-21/en/clips/common_voice_en_100363.mp3',>>  'array': array([ 0.0000000e+00,  1.1748125e-14,  1.5450088e-14, ...,>>          1.3011958e-06, -6.3548953e-08, -9.9098514e-08], dtype=float32),>> ...}
>>
>> —
>> Reply to this email directly, view it on GitHub
>> <https://github.com/huggingface/datasets/issues/5243#issuecomment-1315512887>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/ALSIFOC3JYRCTH54OBRUJULWIOW6PANCNFSM6AAAAAASAYO2LY>
>> .
>> You are receiving this because you were mentioned.Message ID:
>> ***@***.***>
>>
>

### Feature request

Is it possible to download only the data that I am requesting and not the entire dataset? I run out of disk spaceas it seems to download the entire dataset, instead of only the part needed.

common_voice["test"] = load_dataset("mozilla-foundation/common_voice_11_0", "en", split="test", 
                                    cache_dir="cache/path...",
                                    use_auth_token=True,
                                    download_config=DownloadConfig(delete_extracted='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')
                                    )


### Motivation

efficiency improvement

### Your contribution

n/a