Pruning in Deep Learning: Structured vs Unstructured

Source: Dev.to

Introduction

Deep learning models are becoming larger and more powerful every year. From mobile vision systems to large language models, the number of parameters has exploded. But do we really need all those parameters?
Pruning is a model compression technique that removes unnecessary parameters from neural networks while maintaining performance. It helps reduce model size, improve inference speed, and lower computational cost.

In this article we’ll explore:

• What is pruning?
• Why pruning is needed
• Structured vs. unstructured pruning
• Practical trade‑offs

🚀 Why Do We Need Pruning?

Modern neural networks:

• Require high memory
• Consume more power
• Have slower inference on edge devices
• Are expensive to deploy

Typical scenarios:

• Mobile apps need lightweight models
• Embedded systems have limited RAM
• Edge AI requires fast inference

Pruning addresses these issues by removing redundant weights.

🌳 What Is Model Pruning?

Model pruning is the process of removing parameters (weights, neurons, filters, or even layers) from a trained neural network to make it smaller and faster.

Key idea: Many weights in a trained network contribute very little to the final prediction, so they can be removed.

Typical workflow

1. Train the full model
2. Remove less important weights
3. Fine‑tune the pruned model

🔹 1. Unstructured Pruning

📌 What Is Unstructured Pruning?

Unstructured pruning removes individual weights from the network based on an importance criterion (usually small‑magnitude weights). The result is a sparse matrix where many entries are zero.

How It Works

1. Calculate the magnitude of each weight.
2. Zero out the weights with the smallest magnitudes.

Advantages

• Can achieve very high compression rates.

Disadvantages

• Sparse matrices are not always hardware‑friendly, which may limit speed gains on some devices.

Example

If a layer has 1,000 weights and 70 % are pruned, only 300 non‑zero weights remain.

🔹 2. Structured Pruning

What Is Structured Pruning?

Structured pruning removes entire neurons, channels, filters, or layers instead of individual weights. Rather than creating sparsity, it changes the network architecture.

How It Works

1. Evaluate the importance of filters, neurons, or channels.
2. Remove the least important ones entirely.

Advantages

• Hardware‑friendly; leads to actual reductions in computation and memory usage.

Disadvantages

• May cause a slightly higher accuracy drop if pruning is aggressive.

Example

If a CNN layer has 64 filters and 20 are removed, the layer now has 44 filters, reducing both parameters and FLOPs.

When to Use Which?

Use Structured Pruning When

• Deploying to real‑world applications with strict latency or memory constraints (e.g., MobileNet optimization).
• You need hardware‑friendly speedups.

Use Unstructured Pruning When

• Maximum compression is the primary goal and the target hardware can exploit sparsity.

Final Thoughts

Pruning is not just about reducing size—it’s about making AI practical. As models grow larger, efficiency techniques like pruning become essential. Structured pruning is generally more suitable for deployment, while unstructured pruning offers the highest possible compression. The future of AI lies in smarter, leaner models rather than ever‑bigger ones.