Efficient Deep Learning: Post-Training Weight Compression of Neural Networks via Low-Rank Singular Value Decomposition

Madhav Garg

Efficient Deep Learning: Post-Training Weight Compression of Neural Networks via Low-Rank Singular Value Decomposition

Author(s)	Madhav Garg
Country	India
Abstract	Modern deep neural networks (DNNs) achieve state-of-the-art accuracy at the cost of large computational and memory footprints, which makes them difficult to deploy on resource-constrained edge devices such as smartphones, IoT sensors, and medical implants. This paper studies low-rank approximation via Singular Value Decomposition (SVD) as a deterministic, post-training compression technique that reduces parameter count without any retraining. We derive, from first principles, the relationship between the truncation rank k, the resulting compression ratio (CR), and the reconstruction error under the Frobenius norm, grounding the analysis in the Eckart–Young–Mirsky theorem. We then validate the theory on a simulated 512×512 weight matrix with an exponentially decaying singular-value spectrum — a common signature of trained neural network layers — and report fully reproducible, code-verified results (fixed random seed = 42). The experiment traces a complete Pareto frontier: an aggressive rank of k = 10 compresses the matrix 25.6× but destroys 82.8% of its structure, whereas a conservative rank of k = 300 preserves 99.6% fidelity but costs more to store than the original matrix. Between these extremes we identify k = 164 as the empirical “elbow” of the trade-off curve, where the layer shrinks by 35.9% while its reconstruction error stays just under a 5% fidelity target. We discuss why this frontier exists, its practical limits, and how it could guide automatic, layer-by-layer compression of real, pretrained networks.
Keywords	neural network compression, singular value decomposition, low-rank approximation, edge AI, model efficiency.
Published In	Volume 7, Issue 9, September 2026
Published On	2026-09-28

View / Download PDF File

About IJLRP Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics	Join as a Reviewer Editors & Reviewers Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijlrp.com

International Journal of Leading Research Publication

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Monthly Scholarly International Journal

Efficient Deep Learning: Post-Training Weight Compression of Neural Networks via Low-Rank Singular Value Decomposition

Share this