Changelog¶
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog.
This project adheres to Semantic Versioning, with the exception that v0.X updates include backwards-incompatible API changes. From v1.0.0 and on, the project will adherence strictly to Semantic Versioning.
Unpublished¶
[1.2.3] - 2023-06-16¶
Fixed¶
Ensure state_index remains on the same device after clean_state.
[1.2.2] - 2023-05-24¶
Fixed¶
Option to use strings to specify transformer activation.
[1.2.1] - 2023-03-24¶
### Added
Onnx as dev requirement.
Changed¶
Allow torch>=2.0.
[1.2.0] - 2023-03-16¶
### Added
Skip
module.“leading” mode in
Residual
.
[1.1.3] - 2023-03-15¶
### Added
Description of state handling to README.
Fixed¶
Documentation formatting for
co.Identity()
examples.Horovod check for newer pytorch lightning versions.
[1.1.2] - 2023-01-13¶
### Added
query_index
argument toSingleOutputTransformerEncoderLayer
.
Fixed¶
Residual
centred residual andDelay
auto_delay forward_step.
[1.1.1] - 2023-01-10¶
### Added
Support for
GroupNorm
andInstanceNorm
[1.1.0] - 2022-12-19¶
### Added
append
function toco.Sequential
.Production-ready docstrings for public functions.
reduce_max to
Reduce
.
### Changed
Rename
Unity
toIdentity
to followtorch.nn
.Major overhaul of README, improving descriptions and adding benchmark.
Major overhaul of docs, improving descriptions and adding benchmark.
MHA warnings to only log once.
Removed¶
Unused parameters
batch_first
andbidirectional
for RNN, GRU, and LSTM.
[1.0.4] - 2022-12-07¶
Fixed¶
co.Conditional
onnx support for single-option config.
[1.0.3] - 2022-12-07¶
Fixed¶
co.Conditional
onnx support.
[1.0.2] - 2022-12-06¶
Fixed¶
co.Conv
onnx export for kernel_size=1.
[1.0.1] - 2022-12-02¶
Added¶
Ability to access onnx from root, i.e.
co.onnx
.
[1.0.0] - 2022-12-02¶
Added¶
ONNX compatibility to all library modules!
co.onnx.export
function mirroringtorch.onnx.export
.purely functional
_forward_step
function to all modules._state_shape
and_dynamic_state_inds
properties to modules.Add about info to package root.
Changed¶
Change call_mode internally from enum to tensor.
Change internal state_indexes to tensors.
Change stride to tuple.
Change padding to tuple.
### Fixed
Fix assertion bug in co.Lambda.
### Removed
TensorPlaceholder in favour of
None
.
[0.17.1] - 2022-06-02¶
Added¶
Missing part on Continual Transformers in README.
Removed¶
Conv cpp impl.
[0.17.0] - 2022-05-12¶
Added¶
Citations for Continual Inference lib paper.
Docs.
Automatic conversion for RNN modules.
Continual Transformer modules, including:
RecyclingPositionalEncoding
RetroactiveMultiheadAttention
SingleOutputMultiheadAttention
SingleOutputTransformerEncoderLayer
RetroactiveTransformerEncoderLayer
TransformerEncoderLayerFactory
TransformerEncoder
[0.16.0] - 2022-04-04¶
### Added
“lagging” option for shrink in
co.Delay
andco.Residual
.co.RNN
.co.LSTM
.co.GRU
.
Changed¶
phantom_padding
renamed toresidual_shrink
.
[0.15.6] - 2022-03-18¶
Fixed¶
Missing cpp file in package.
[0.15.5] - 2022-03-05¶
Added¶
CoConv step impl in C++.
[0.15.4] - 2022-01-28¶
Fixed¶
FLOPs module registration compatibility with ptflops >=v0.6.8.
[0.15.3] - 2021-12-13¶
Added¶
Call-mode specific functions in
co.Lambda
[0.15.2] - 2021-12-11¶
Added¶
Support for functor in
co.Lambda
Removed¶
nn.LayerNorm
from automatically convertible modules
[0.15.1] - 2021-12-10¶
Added¶
nn.LayerNorm
to automatically convertible modules
[0.15.0] - 2021-10-29¶
Added¶
ParallelDispatch
module.Conditional
predicate print in__repr__
.
Fixed¶
Sequential
padding
computation.Lambda
__repr__
function prints.
Removed¶
CI testing for python v3.6.
[0.14.0] - 2021-09-20¶
Added¶
Added
phantom_padding
toResidual
.Added
receptive_field
property.Added
Reshape
module.
Changed¶
Rename
forward_shrink
argument toauto_shrink
inDelay
.Torch requirement to v1.9.
Replace
Lambda
unsqueeze_step with takes_time and new default to False.
Fixed¶
padding
property in sequence.delay
property in sequence.strict
mode inload_state_dict
.
Removed¶
Assertion error in
BroadcastReduce
for modules with different delays.
[0.13.0] - 2021-09-14¶
Added¶
Add
forward_shrink
option toDelay
andResidual
.
[0.12.0] - 2021-09-14¶
Added¶
Add
Constant
.Add
Zero
.Add
One
.
[0.11.4] - 2021-09-08¶
Fixed¶
co.ConvXd
cuda compatibility.
[0.11.3] - 2021-09-08¶
Added¶
Add
flatten_state_dict
state variable.
Removed¶
Debug message for Convolutions with padding.
[0.11.2] - 2021-09-08¶
Fixed¶
call_mode
forLinear
.
[0.11.1] - 2021-09-06¶
Added¶
Add
call_mode
.Add
warm_up
.
Changed¶
Container implementations to use
__call__
with alternating call_modes. This change was necessary to properly trigger the torch hooks needed inptflops
.
Fixed¶
ptflops
compatibility.
[0.11.0] - 2021-08-31¶
Added¶
co.Linear
module.Improved repr in
co.Lambda
.Option to skip unsqueeze in
co.Lambda.forward_step
.
[0.10.0] - 2021-08-27¶
Changed¶
Renamed
co.Parallel
toco.BroadcastReduce
.
Added¶
co.Broadcast
module.new
co.Parallel
module.co.Reduce
module.Automatic inference of
co.Broadcast.num_streams
inco.Sequential
.
[0.9.0] - 2021-08-26¶
Added¶
co.Lambda
module.co.Add
module.co.Multiply
module.co.Unity
module.co.Conditional
module.
[0.8.1] - 2021-08-26¶
Fixed¶
Bug in
forward_stepping
.Bug in
clean_state
.
[0.8.0] - 2021-08-24¶
Fixed¶
Bugs in
forward_step(s)
withupdate_state=False
.
Changed¶
forward_steps
interface to always includepad_end
argument.Name of “interface.py” to “module.py”.
Implementations of
forward_step(s)
to be consolidated in CoModule.
Removed¶
Padded
interface.
[0.7.0] - 2021-08-24¶
Added¶
Independent state_dict and load_state_dict functions.
Added nonempty check for aggregation functions in Parallel.
update_state
argument to allforward_step(s)
methods.Additional tests for edge-cases
Changed¶
Changed default pad_end value to False.
Fixed¶
Continual interface and conversion to support both class and module.
Replicate padding in
co._ConvNd
[0.6.1] - 2021-08-23¶
Changed¶
co.Residual
modules to be unnamed. This allows the module state dicts to be flattened.
[0.6.0] - 2021-08-23¶
Added¶
Flattened state dict export and loading via a
flatten
argument. This feature improves interoperability complex modules, that were not originally constructed with theco.Sequential
andco.Parallel
building blocks.Context manager for triggering flattened state_dict export and loading.
[0.5.0] - 2021-08-20¶
Added¶
Support for zero-delay in
co.Delay
Support for broadcasting in
co.Parallel
Mul (hadamard product) aggregation in
co.Parallel
Example of Squeeze and Excitation block
Changed¶
co._PoolNd
attribute naming: “temporal_*” removed as prefix for kernel_size, stride, dilation, and padding.
[0.4.0] - 2021-08-19¶
Added¶
co.Delay
handling for padding.Handling of initialization and strides in containers
Changed¶
co.Conv
build_from
behavior to not change dilation and stride. Argument overload supported instead.pad_start
andpad_end
args to convolution and pooling modulesforward_steps
.Behavior of modules while they initialize. Now, a TensorPlaceholder is passed for initializing steps.
Removed¶
Automatic unsqueeze in pooling.
[0.3.1] - 2021-08-18¶
Added¶
Support for dropout.
[0.3.0] - 2021-08-18¶
Added¶
Support for dilation and stride in pooling.
Changed¶
Pooling API to match torch.nn better.
_ConvCoNd.forward_steps
doesn’t invokeclean_state
anymore.
[0.2.2] - 2021-08-17¶
Added¶
Automatic conversion of batch normalization and activation functions.
Fixed¶
Separate dilation and stride in pool.
Changed¶
Conv forward to use temporal padding like (like nn.Conv).
Removed¶
co.BatchNorm2d
[0.2.1] - 2021-08-17¶
Changed¶
Renamed
unsqueezed
toforward_stepping
.
Removed¶
Unused utility
Zeros
[0.2.0] - 2021-08-16¶
Changed¶
Naming to match
torch.nn
. This lets the continual modules be used as drop-in replacements fortorch.nn
modules.Renamed
forward_regular_unrolled
toforward
,forward_regular
toforward_steps
, andforward
forforward_step
.Renamed
from_regular
tobuild_from
.Renamed
continual
tounsqueezed
.
Added¶
Sequential
wrapper for sequential application of modulesParallel
wrapper for parallel application and aggregation of inputsResidual
wrapper for adding a unity residual to a modulecontinual
conversion functionregister
function for 3rd party modules to register their conversionAdditional tests
[0.1.2] - 2021-08-1¶
Added¶
Pooling modules:
MaxPool1d
,AvgPool3d
,MaxPool3d
,AdaptiveAvgPool3d
,AdaptiveMaxPool3d
.
[0.1.1] - 2021-08-10¶
Added¶
Updated README.
[0.1.0] - 2021-08-10¶
Added¶
Initial publicly available implementation of the library.