How to grow?

Table 3 How to grow across algorithms

Algorithm	How to grow
AutoGrow	Add blocks with random initialization.
Firefly	Gradient-based splits and additions.
GradMax	\(\boldsymbol{\Psi}=0\); maximize gradient norm.
NeST	Sparse neuron or edge addition.
Net2WiderNet	Function-preserving neuron splitting.
Net2DeeperNet	Function-preserving identity layer insertion.
Network Morphism	Function-preserving morphism.
NORTH	Add orthogonal neurons.
SENN	\(\boldsymbol{\Omega}=0\); maximize natural-gradient objective.
Splitting	Split along the most unstable direction.
Tiny	Low-rank residual-gradient matching.
Variance Transfer	Variance-preserving widening with stage-wise learning rates.