HF DOWNLOAD

Functions

datasets.utils.hf_download.download_dataset_file(repo_id, local_dir, filename, revision='main')

Download a single file from a HF dataset repo into local_dir.

Return type:

str

datasets.utils.hf_download.download_dataset_snapshot(repo_id, local_dir, revision='main')

Download a Hugging Face dataset repository snapshot into local_dir.

datasets.utils.hf_download.download_dataset_snapshot_with_patterns(repo_id, local_dir, revision='main', allow_patterns=None, ignore_patterns=None)

Download a filtered HF dataset snapshot into local_dir.

datasets.utils.hf_download.ensure_required_files_from_hf(local_dir, required_relpaths, repo_id, revision='main')

Ensure required local files exist, downloading the HF snapshot if needed.

datasets.utils.hf_download.download_dataset_file(repo_id, local_dir, filename, revision='main')

Download a single file from a HF dataset repo into local_dir.

Return type:

str

datasets.utils.hf_download.download_dataset_snapshot(repo_id, local_dir, revision='main')

Download a Hugging Face dataset repository snapshot into local_dir.

datasets.utils.hf_download.download_dataset_snapshot_with_patterns(repo_id, local_dir, revision='main', allow_patterns=None, ignore_patterns=None)

Download a filtered HF dataset snapshot into local_dir.

datasets.utils.hf_download.ensure_required_files_from_hf(local_dir, required_relpaths, repo_id, revision='main')

Ensure required local files exist, downloading the HF snapshot if needed.