L2P#

Arguments#

Options

--prompt_poolbool

Help: None

  • Default: True

--pool_size_l2pint

Help: number of prompts (M in paper)

  • Default: 10

--lengthint

Help: length of prompt (L_p in paper)

  • Default: 5

--top_kint

Help: top k prompts to use (N in paper)

  • Default: 5

--prompt_keybool

Help: Use learnable prompt key

  • Default: True

--prompt_key_initstr

Help: initialization type for key’s prompts

  • Default: uniform

--use_prompt_maskbool

Help: None

  • Default: False

--batchwise_promptbool

Help: None

  • Default: True

--embedding_keystr

Help: None

  • Default: cls

--predefined_keystr

Help: None

--pull_constraintunknown

Help: None

  • Default: True

--pull_constraint_coefffloat

Help: None

  • Default: 0.1

--global_poolstr

Help: type of global pooling for final sequence

  • Default: token

  • Choices: token, avg

--head_typestr

Help: input type of classification head

  • Default: prompt

  • Choices: token, gap, prompt, token+prompt

--freezelist

Help: freeze part in backbone model

  • Default: ['blocks', 'patch_embed', 'cls_token', 'norm', 'pos_embed']

--schedstr

Help: LR scheduler (default: “constant”

  • Default: constant

--lr_noisefloat

Help: learning rate noise on/off epoch percentages

  • Default: None

--lr_noise_pctfloat

Help: learning rate noise limit percent (default: 0.67)

  • Default: 0.67

--lr_noise_stdfloat

Help: learning rate noise std-dev (default: 1.0)

  • Default: 1.0

--warmup_lrfloat

Help: warmup learning rate (default: 1e-6)

  • Default: 1e-06

--min_lrfloat

Help: lower lr bound for cyclic schedulers that hit 0 (1e-5)

  • Default: 1e-05

--decay_epochsfloat

Help: epoch interval to decay LR

  • Default: 30

--warmup_epochsint

Help: epochs to warmup LR, if scheduler supports

  • Default: 5

--cooldown_epochsint

Help: epochs to cooldown LR at min_lr, after cyclic schedule ends

  • Default: 10

--patience_epochsint

Help: patience epochs for Plateau LR scheduler (default: 10

  • Default: 10

--decay_ratefloat

Help: LR decay rate (default: 0.1)

  • Default: 0.1

--unscale_lrbool

Help: scaling lr by batch size (default: True)

  • Default: True

--clip_gradfloat

Help: Clip gradient norm

  • Default: 1

L2P: Learning to Prompt for Continual Learning

Note

L2P USES A CUSTOM BACKBONE: vit_base_patch16_224. The backbone is a ViT-B/16 pretrained on Imagenet 21k and finetuned on ImageNet 1k.

Classes#

class models.l2p.L2P(backbone, loss, args, transform, dataset=None)[source]#

Bases: ContinualModel

Learning to Prompt (L2P).

COMPATIBILITY: List[str] = ['class-il', 'domain-il', 'task-il', 'general-continual']#
NAME: str = 'l2p'#
begin_task(dataset)[source]#
forward(x)[source]#
get_parameters()[source]#
static get_parser(parser)[source]#
Return type:

ArgumentParser

observe(inputs, labels, not_aug_inputs, epoch=None)[source]#