Improve default patterns resolution
Separate the default patterns that match directories from the ones matching files and ensure directories are checked first (reverts the change from https://github.com/huggingface/datasets/pull/6244, which merged these patterns). Also, ensure that the glob patterns do not overlap to avoid duplicates in the result. 

Additionally, replace `get_fs_token_paths` with `url_to_fs` to avoid [unnecessary glob calls](https://github.com/fsspec/filesystem_spec/blob/14dce8ca78f7aa509a20edb263bff83a7760c24d/fsspec/core.py#L655-L656).

fix https://github.com/huggingface/datasets/issues/6259
fix https://github.com/huggingface/datasets/issues/6272