Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

filter approximation

Filter approximation is a model compression and acceleration technique in deep learning that replaces high-dimensional convolutional filters with computationally efficient, lower-rank representations while preserving feature extraction capabilities. The approach exploits redundancies across spatial dimensions and channel distributions by decomposing dense filter tensors into simpler components, such as low-rank matrix or tensor expansions and separable filter bases. By transforming heavy multidimensional convolution operations into sequences of smaller, lower-cost operations, filter approximation significantly reduces parameter counts and floating-point operations, enabling faster inference and lower memory usage during deployment with minimal loss in model accuracy.

1 item

Speeding up Convolutional Neural Networks with Low Rank Expansions

Speeding up Convolutional Neural Networks with Low Rank Expansions

Max Jaderberg, Andrea Vedaldi, Andrew Zisserman

OrganizationsUniversity of Oxford

Why you should read this

Presents low-rank filter decomposition techniques that accelerate convolutional neural network computation by up to 4.5x on standard hardware with negligible loss in accuracy.

The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability. Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain. Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition, showing a possible 2.5x speedup with no loss in accuracy, and 4.5x speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.

Added

2026-09-24