VainF
What exactly does this method prune?
After reading the paper it never made sense that structured pruning was pruning what, parameters, weights? I think it's a bit abstract, please answer my questions, thanks!
Weights are pruned in a structured manner that is considered "physical." In this context, "physical" pruning refers to the actual removal of tensor entries rather than the common approach of masking with zeros or keeping track of indices, as often seen in unstructured pruning methods.
To illustrate, imagine a weight tensor for a convolutional layer with specifications (output channels=265, input channels=64, kernel size=3x3). When channels are pruned, it means removing them from the tensor (128, 64, 3, 3). This direct elimination of weights contributes to a more straightforward and resource-efficient model compared to methods involving masked values or preserved indices.
