secure_learning.regularizers module

Contains penalty functions.

class secure_learning.regularizers.BaseRegularizer

Bases: object

Base class for regularizations.

class secure_learning.regularizers.DifferentiableRegularizer

Bases: ABC, BaseRegularizer

Differentiable regularizations can be included via their gradient.

abstractmethod __call__(weights)

Evaluate the regularization gradient function.

Parameters:

weights (List[SecureFixedPoint]) – Weight vector

Return type:

List[SecureFixedPoint]

Returns:

Value of regularization gradient function evaluated with the provided parameters.

class secure_learning.regularizers.L1Regularizer(alpha)

Bases: NonDifferentiableRegularizer

Implementation for L1 regularization: $f(w) = ||w||_1$.

__call__(weights, eta)

Apply the proximal function for the L1 regularizer.

This proximal function is more commonly known as the soft-thresholding algorithm. The soft-thresholding algorithm pulls every element of $w$ (weights vector) closer to zero. It does so in a component-wise fashion. More specifically: $$ textrm{new_}w_i = left{ begin{array}{cl} w_i - nu & : w_i > nu \ 0 & : -nu < w_i < nu \ w_i + nu & : -nu < w_i end{array} right. $$

Here, $nu$ is a value that depends on eta and the regularization constant $alpha$.

Parameters:

• weights (List[SecureFixedPoint]) – Weight vector.

• eta (Union[float, SecureFixedPoint]) – Learning rate.

Return type:

List[SecureFixedPoint]

Returns:

Value of proximal function evaluated with the provided parameters.

__init__(alpha)

Constructor method.

Parameters:

alpha (float) – Regularization parameter

class secure_learning.regularizers.L2Regularizer(alpha)

Bases: DifferentiableRegularizer

Implementation for L2 regularization: $$f(w) = frac{alpha}{2} times ||w||^2_2`.$$

__call__(weights)

Apply the initialized L2 regularization.

Parameters:

weights (List[SecureFixedPoint]) – Weight vector to be regularized.

Return type:

List[SecureFixedPoint]

Returns:

Value of regularization gradient evaluated with the provided parameters.

__init__(alpha)

Constructor method.

Parameters:

alpha (float) – Regularization parameter

class secure_learning.regularizers.NonDifferentiableRegularizer

Bases: ABC, BaseRegularizer

Non-differential regularization can be included via a proximal method.

abstractmethod __call__(weights, eta)

Apply the proximal function for this regularizer.

Parameters:

• weights (List[SecureFixedPoint]) – Weight vector.

• eta (Union[float, SecureFixedPoint]) – Learning rate.

Return type:

List[SecureFixedPoint]

Returns:

Value of proximal function evaluated with the provided parameters.