CONTINUAL DATASET

Classes

class datasets.utils.continual_dataset.ContinualDataset(args)

Bases: object

A base class for defining continual learning datasets.

Data is divided into tasks and loaded only when the get_data_loaders method is called.

NAME

the name of the dataset

Type:

str

SETTING

the setting of the dataset

Type:

str

N_CLASSES_PER_TASK

the number of classes per task

Type:

int

N_TASKS

the number of tasks

Type:

int

N_CLASSES

the number of classes

Type:

int

SIZE

the size of the dataset

Type:

Tuple[int]

AVAIL_SCHEDS

the available schedulers

Type:

List[str]

class_names

list of the class names of the dataset (should be populated by get_class_names)

Type:

List[str]

train_loader

the training loader

Type:

DataLoader

test_loaders

the test loaders

Type:

List[DataLoader]

i

the current task

Type:

int

c_task

the current task

Type:

int

args

the arguments which contains the hyperparameters

Type:

Namespace

AVAIL_SCHEDS = ['multisteplr']

NAME: str

N_CLASSES: int

N_CLASSES_PER_TASK: int

N_TASKS: int

SETTING: str

SIZE: Tuple[int]

base_fields = ('SETTING', 'N_CLASSES_PER_TASK', 'N_TASKS', 'SIZE', 'N_CLASSES', 'AVAIL_SCHEDS')

class_names: List[str] = None

composed_fields = {'TEST_TRANSFORM': <function build_torchvision_transform>, 'TRANSFORM': <function build_torchvision_transform>}

get_backbone()

Returns the name of the backbone to be used for the current dataset. This can be changes using the –backbone argument or by setting it in the dataset_config.

Return type:

str

get_batch_size()

Returns the batch size to be used for the current dataset.

get_class_names()

Returns the class names for the current dataset.

Return type:

List[str]

get_data_loaders()

Creates and returns the training and test loaders for the current task. The current training loader and all test loaders are stored in self.

Returns:

the current training and test loaders

Return type:

Tuple[DataLoader, DataLoader]

static get_denormalization_transform()

Returns the transform used for denormalizing the current dataset.

Return type:

Module

get_epochs()

Returns the number of epochs to be used for the current dataset.

get_iters()

Returns the number of iterations to be used for the current dataset.

static get_loss()

Returns the loss to be used for the current dataset.

Return type:

Module

get_minibatch_size()

Returns the minibatch size to be used for the current dataset.

static get_normalization_transform()

Returns the transform used for normalizing the current dataset.

Return type:

Module

get_offsets(task_idx=None)

Compute the start and end class index for the current task.

Parameters:

task_idx (int) – the task index

Returns:

the start and end class index for the current task

Return type:

tuple

get_prompt_templates()

Returns the prompt templates for the current dataset. By default, it returns the ImageNet prompt templates.

Return type:

List[str]

static get_transform()

Returns the transform to be used for the current dataset.

Return type:

Module

optional_fields = ('MEAN', 'STD')

classmethod set_default_from_config(config, parser)

Sets the default arguments from the configuration file. The default values will be set in the class attributes and will be available for all instances of the class.

The arguments that are related to the dataset (i.e., are in the ‘base_fields’, ‘optional_fields’, or ‘composed_fields’) will be removed from the config dictionary to avoid conflicts with the command line arguments.

Parameters:

• config (dict) – the configuration file

• parser (ArgumentParser) – the argument parser to set the default values

Returns:

the configuration file without the dataset-related arguments

Return type:

dict

class datasets.utils.continual_dataset.MammothDatasetWrapper(ext_dataset, train=False)

Bases: Dataset, object

Wraps the datasets used inside the ContinualDataset class to allow for a more flexible retrieval of the data.

add_extra_return_field(field_name, field_value)

Adds an extra field to the dataset.

Parameters:

• field_name (str) – the name of the field

• field_value – the value of the field

data: ndarray

extend_return_items(ret_tuple, index)

Extends the return tuple with the extra fields defined in extra_return_fields.

Parameters:

ret_tuple (Tuple[torch.Tensor, int, torch.Tensor, Optional[torch.Tensor]]) – the current return tuple

Returns:

the extended return tuple

Return type:

Tuple[torch.Tensor, int, Optional[torch.Tensor], Sequence[Optional[torch.Tensor]]]

extra_return_fields: Tuple[str] = ()

indexes: ndarray

is_init: bool = False

required_fields = ('data', 'targets')

targets: ndarray

Functions

datasets.utils.continual_dataset.fix_class_names_order(class_names, args)

Permutes the order of the class names according to the class order specified in the arguments. The order reflects that of store_masked_loaders.

Parameters:

• class_names (List[str]) – the list of class names. This should contain all classes in the dataset (not just the current task’s ones).

• args (Namespace) – the command line arguments

Returns:

the class names in the correct order

Return type:

List[str]

datasets.utils.continual_dataset.store_masked_loaders(train_dataset, test_dataset, setting)

Divides the dataset into tasks.

datasets.utils.continual_dataset.train_dataset

the training dataset

Type:

Dataset

datasets.utils.continual_dataset.test_dataset

the test dataset

Type:

Dataset

datasets.utils.continual_dataset.setting

the setting of the dataset

Type:

ContinualDataset

Returns:

the training and test loaders

Return type:

Tuple[DataLoader, DataLoader]