GENERATIVE REPLAY

Classes

class models.lora_prototype_utils.generative_replay.FeaturesDataset(X, y)

Bases: Dataset

class models.lora_prototype_utils.generative_replay.GaussianMixture(n_components, n_features, covariance_type='full', eps=1e-06, init_params='kmeans', mu_init=None, var_init=None)

Bases: Module

Fits a mixture of k=1,..,K Gaussians to the input data (K is supplied via n_components). Input tensors are expected to be flat with dimensions (n: number of samples, d: number of features). The model then extends them to (n, 1, d). The model parametrization (mu, sigma) is stored as (1, k, d), probabilities are shaped (n, k, 1) if they relate to an individual sample, or (1, k, 1) if they assign membership probabilities to one of the mixture components.

bic(x)

Bayesian information criterion for a batch of samples. :param x: torch.Tensor (n, d) or (n, 1, d)

Returns:

float

Return type:

bic

check_size(x)

fit(x, delta=0.001, n_iter=100, warm_start=False)

Fits model to the data. :param x: torch.Tensor (n, d) or (n, k, d)

options:

delta: float n_iter: int warm_start: bool

get_kmeans_mu(x, n_centers, init_times=50, min_delta=0.001)

Find an initial value for the mean. Requires a threshold min_delta for the k-means algorithm to stop iterating. The algorithm is repeated init_times often, after which the best centerpoint is returned. :param x: torch.FloatTensor (n, d) or (n, 1, d) :param init_times: init :param min_delta: int

predict(x, probs=False)

Assigns input data to one of the mixture components by evaluating the likelihood under each. If probs=True returns normalized probabilities of class membership. :param x: torch.Tensor (n, d) or (n, 1, d) :param probs: bool

Returns:

torch.Tensor (n, k) (or) y: torch.LongTensor (n)

Return type:

p_k

predict_proba(x)

Returns normalized probabilities of class membership. :param x: torch.Tensor (n, d) or (n, 1, d)

Returns:

torch.LongTensor (n)

Return type:

y

sample(n)

Samples from the model. :param n: int

Returns:

torch.Tensor (n, d) y: torch.Tensor (n)

Return type:

x

score_samples(x)

Computes log-likelihood of samples under the current model. :param x: torch.Tensor (n, d) or (n, 1, d)

Returns:

torch.LongTensor (n)

Return type:

score

class models.lora_prototype_utils.generative_replay.MixtureOfGaussians(embed_dim, n_components=3, n_iters=100)

Bases: Module

fit(x)

forward(n_sample, scale_mean=1.0)

sample(n_sample)

Functions

models.lora_prototype_utils.generative_replay.calculate_matmul(mat_a, mat_b)

Calculate matrix product of two matrics with mat_a[0] >= mat_b[0]. Bypasses torch.matmul to reduce memory footprint. :param mat_a: torch.Tensor (n, k, 1, d) :param mat_b: torch.Tensor (n, k, d, 1)

models.lora_prototype_utils.generative_replay.calculate_matmul_n_times(n_components, mat_a, mat_b)

Calculate matrix product of two matrics with mat_a[0] >= mat_b[0]. Bypasses torch.matmul to reduce memory footprint. :param mat_a: torch.Tensor (n, k, 1, d) :param mat_b: torch.Tensor (1, k, d, d)