SECOND STAGE STARPROMPT

Arguments

Options

--keys_ckpt_pathstr

Help: Path for first-stage keys. The keys can be saved by runninng first_stage_starprompt with –save_first_stage_keys=1.This can be:- A path to a checkpoint file (.pt) containing ONLY THE FIRST STAGE KEYS.- A path to the checkpoint made by first_stage_starprompt- The job-id (conf_jobnum) of the first_stage_starprompt run that made the keys.- A JSON file containing the job-id (conf_jobnum) of the first_stage_starprompt run that made the keys.The JSON is expected to contain an entry for each dataset and seed: {dataset: {seed: job-id}}.

• Default: None

Frozen hyperparameters

--virtual_bs_nint

Help: virtual batch size iterations

• Default: 1

--enable_data_aug_query0|1|True|False -> bool

Help: Use default transform with data aug to generate the CLIP’s response?

• Default: 1

--use_clip_preprocess_eval0|1|True|False -> bool

Help: Use CLIP’s transform during eval instead of the default test transform?

• Default: 0

--ortho_split_valint

Help: None

• Default: 0

--gr_mog_n_iters_second_stageint

Help: Number of EM iterations during fit for GR with MOG.

• Default: 500

--gr_mog_n_componentsint

Help: Number of components for GR with MOG.

• Default: 5

--batch_size_grint

Help: Batch size for Generative Replay.

• Default: 128

--num_samples_grint

Help: Number of samples for Generative Replay.

• Default: 256

--prefix_tuning_prompt_lenint

Help: Prompt length for prefix tuning. Used only if –prompt_mode==concat.

• Default: 5

Ablations hyperparameters

--gr_modelstr

Help: Type of distribution model for Generative Replay. - mog: Mixture of Gaussian. - gaussian: Single Gaussian distribution.

• Default: mog

• Choices: mog, gaussian

--enable_gr0|1|True|False -> bool

Help: Enable Generative Replay.

• Default: 1

--statc_keys_use_templates0|1|True|False -> bool

Help: Use templates for the second stage if no keys are loaded.

• Default: 1

--prompt_modestr

Help: Prompt type for the second stage. - residual: STAR-Prompt style prompting. - concat: Prefix-Tuning style prompting.

• Default: residual

• Choices: residual, concat

--enable_confidence_modulation0|1|True|False -> bool

Help: Enable confidence modulation with CLIP similarities (Eq. 5 of the main paper)?

• Default: 1

Tunable hyperparameters

--lambda_ortho_second_stagefloat

Help: orthogonality loss coefficient

• Default: 10

--num_monte_carlo_gr_second_stageint

Help: how many times to sample from the dataset for alignment

• Default: 1

--num_epochs_gr_second_stageint

Help: Num. of epochs for GR.

• Default: 10

--learning_rate_gr_second_stagefloat

Help: Learning rate for GR.

• Default: 0.001

Classes

class models.second_stage_starprompt.SecondStageStarprompt(backbone, loss, args, transform, dataset=None)

Bases: ContinualModel

Second-stage of StarPrompt. Requires the keys saved from the first stage.

COMPATIBILITY: List[str] = ['class-il', 'domain-il', 'task-il', 'general-continual']

NAME: str = 'second_stage_starprompt'

align()

backup()

begin_task(dataset)

create_features_dataset()

end_task(dataset)

forward(x)

get_parameters()

static get_parser(parser)

Return type:

ArgumentParser

get_scheduler()

net: Model

norm(t)

observe(inputs, labels, not_aug_inputs, epoch=None)

recall()

train_alignment_epoch(classifier, optim, epoch)

update_statistics(dataset)