| Abstract: | Deep neural networks (DNNs) are resource-intensive and call for efficient compression methods to reduce the resource cost. For a composite DNN with various modules, the optimal resource allocation among these modules remains a question. To address this issue, we propose a novel unified compression framework that compress the whole network in an end-to-end way, without any multi-stage heuristics nor expensive hyper-parameters tuning. We demonstrate the generality of this framework by showing its superior performance in compressing a recommendation system and a vision transformer. Furthermore, we discuss to what extent the optimizers learnt by learning to optimize(L2O) technique can be adapted to a special class of functions and outperform general-purpose optimizers for the minimax objective. The electronic version of this dissertation is accessible from https://hdl.handle.net/1969.1/198649 |
|---|