ADAPTER

Classes

class models.moe_adapters_utils.adapter.Adapter(d_model=None, bottleneck=None, dropout=0.0, init_option='lora', adapter_scalar='1.0', adapter_layernorm_option='in')

Bases: Module

forward(x, add_residual=True, residual=None)