Linear¶

class continual.Linear(in_features, out_features, bias=True, device=None, dtype=None, channel_dim=- 1)[source]¶

Applies a linear transformation to a dimension of the incoming data: $y = xA^T + b$ .

This module supports TensorFloat32.

Parameters:

in_features (int) – size of each input sample
out_features (int) – size of each output sample
bias (bool) – If set to False, the layer will not learn an additive bias. Default: True
channel_dim – Channel dimension index over which to perform linear projection. Default: -1.

Shape:

Input: $(B, C_{in}, T, *)$ where $*$ means any number of additional dimensions and $C_{in} = \text{in\_features}$ if channel_dim = 2. If channel_dim = -1, the order of input dimensions is $(*, C_{in})$ .
Output: $(B, C_{out}, T, *)$ where all but the last dimension are the same shape as the input and $C_{out} = \text{out\_features}$ if channel_dim = 2. If channel_dim = -1, the order of input dimensions is $(*, C_{in})$ .

Variables:

weight (torch.Tensor) – the learnable weights of the module of shape $(\text{out\_features}, \text{in\_features})$ . The values are initialized from $\mathcal{U}(-\sqrt{k}, \sqrt{k})$ , where $k = \frac{1}{\text{in\_features}}$
bias – the learnable bias of the module of shape $(\text{out\_features})$ . If bias is True, the values are initialized from $\mathcal{U}(-\sqrt{k}, \sqrt{k})$ where $k = \frac{1}{\text{in\_features}}$

Examples:

# Use like torch.nn.Linear
m = co.Linear(20, 30)
input = torch.randn(128, 20)
output = m(input)
assert output.size() == torch.Size([128, 30])

# Or in conjunction with other continual modules
#                   B  C  T   H    W
input = torch.randn(1, 3, 16, 128, 128)
net = co.Sequential(
    co.Conv3d(3, 32, 3),
    co.AdaptiveAvgPool3d((1, 1, 1), 32),
    co.Linear(32, 10, channel_dim=1),
)
output = net(input)
assert output.size() == torch.Size([1, 10, 1, 1, 1])