An efficient way to refine DenseNet

DenseNet features dense connections between layers. Such an architecture is elegant but suffers memory-hungry and time-consuming. In this paper, we explore the relation between density of connections and performance of DenseNet (Huang et al., in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017). We find that sometimes even just preserving 25% connections does not harm the performance but get a little promotion. We aim to provide users a trade-off between performance and efficiency. We analyze the relation in two connection-trimming ways. One is preserving connection proportionally as a given rate and the other as a given quantity of connection. We evaluate the performance and efficiency between all the architectures on the competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). Experimental results demonstrate that moderate connection trimming achieves the significant performance for DenseNet, but requires almost less than half of the GPU memories, i.e., 40% fewer parameters and about 40% less time for prediction.