What’s new

• “Enhancements” for new features

• “Bugs” for bug fixes

• “API changes” for backward-incompatible changes

Develop branch

Enhancements

• Split Conv2dGrowingModule into two subclass FullConv2dGrowingModule`(that does the same as the previous class) and `RestrictedConv2dGrowingModule (that compute only the best 1x1 convolution as the second layer at growth time) (#92 by Théo Rudkiewicz).

• Code factorization of methods compute_optimal_added_parameters and compute_optimal_delta that are now abstracted in the GrowingModule class. (#87 by Théo Rudkiewicz).

• Stops automatically computing parameter update in Conv2dGrowingModule.compute_optimal_added_parameters`to be consistent with `LinearGrowingModule.compute_optimal_added_parameters (#87 by Théo Rudkiewicz) .

• Adds a generic GrowingContainer to simplify model management along with unit testing. Propagates modifications to models. (#77 by Stéphane Rivaud)

• Refactor and simplify repo structure (#72 and #73 by Stella Douka)

• Simplify global device handling (#72 by Stella Douka)

• Integrate an MLP Mixer (#70 by Stéphane Rivaud)

• Integrate a Residual MLP (#69 by Stéphane Rivaud)

• Option to restrict action space (#60 by Stella Douka)

• Add support for Conv2d layers in the sequential case (#34 by Théo Rudkiewicz)

• Replaced the assert statements with self.assert* methods in the unit tests (#50 by Théo Rudkiewicz)

• Reduce unit tests computational load, add parametrized unit tests (#46 by Sylvain Chevallier)

• Add the possibly to separate S for natural gradient and S for new weights (#33 by Théo Rudkiewicz)

• Added GPU tracking (#16 by Stella Douka)

• Added Bayesian Information Criterion for selecting network expansion (#16 by Stella Douka)

• Unified documentation style (#14 by Stella Douka)

• Updated Unit Tests (#14 by Stella Douka)

• Option to disable logging (#14 by Stella Douka)

• Add CI (#2 by Sylvain Chevallier)

• Modify LinearGrowingModule to operate on the last dimension of an input tensor with arbitrary shape (#54 by Stéphane Rivaud)

Bugs

• Delete leftover activity tensors (#78 by Stella Douka)

• Fix inconsistency with torch.empty not creating empty tensors (#78 by Stella Douka)

• Expansion of existing nodes not executed in GrowingDAG (#78 by Stella Douka)

• Fix the computation of optimal added neurons without natural gradient step (#74 by Stéphane Rivaud)

• Fix the data type management for growth related computations. (#79 by Stéphane Rivaud)

• Revert global state changes, solve test issues (#70 by Stella Douka)

• Fix the data augmentation bug in get_dataset (#58 by Stéphane Rivaud)

• Use a different scaling factor for input and output extensions. In addition, apply_change and extended_forward have now compatible behavior in terms of scaling factor. (#48 by Théo Rudkiewicz)

• Fix the change application when updating the previous layer (#48 by Théo Rudkiewicz)

• Fix the sub-selection of added neurons in the sequential case (#41 by Théo Rudkiewicz)

• Correct codecov upload (#49 by Sylvain Chevallier)

• Fix dataset input_shape: remove the flattening in data augmentation (#56 by Stéphane Rivaud)

• Fix memory leak from issue #96 (#97 by Théo Rudkiewicz)

API changes

• Renamed AdditionGrowingModule to MergeGrowingModule for clarity (#84 by Stella Douka)

• Added support for configuration files that override default class arguments (#38 by Stella Douka)