The Art of Small Images: Practical Techniques for Shaving Hundreds of MB Off AI and Java Containers

Source: Dev.to

Why Containers Grow Large

Most teams know the feeling: the container finally works, models load, the JVM starts, and endpoints respond. But then someone points out the image size—sometimes eight hundred megabytes or more. While this isn’t surprising, the large size still causes problems. It slows local development, puts pressure on CI pipelines, and quietly shapes how systems evolve.

AI and Java containers often grow large for understandable reasons. Machine‑learning stacks need native libraries, Python wheels with compiled extensions, CUDA dependencies, and tools for testing. Java images might include full JDKs, debugging tools, and leftover build artifacts that were once helpful but never cleaned up.

Common Patterns

• Base image choices – Teams often default to Ubuntu or other full distributions because they feel safe and familiar. The base image you choose affects everything that follows.
• Forgotten dependencies – Many containers still include libraries that were useful during testing but were never removed. In Python, transitive dependencies can quietly increase the image size. In Java, unused modules often stay on the classpath.
• Layer ordering – Treat container layers as a story. Each layer should answer: Why is this here? When was it needed? Is it still needed?

Practical Techniques

Reducing image size usually comes from many small changes rather than a single big fix:

1. Clean package caches – Remove apt/yum caches, pip wheels, and other temporary files after installation.
2. Reorder layers for reuse – Place rarely‑changing layers (e.g., base OS, language runtimes) early so they can be cached across builds.
3. Strip symbols – Use strip on binaries and libraries to remove debugging symbols.
4. Use runtime‑only images – For Java, switch from a full JDK to a JRE or a minimal runtime like jlink‑generated images. For Python, use slim base images and install only runtime dependencies.
5. Multi‑stage builds – Compile code and build assets in an intermediate stage, then copy only the final artifacts into a minimal final image.
6. Explicitly exclude build artifacts – Add .dockerignore entries for source files, test suites, and documentation that aren’t needed at runtime.
7. Leverage language‑specific tools – For Java, use tools like jlink or jpackage to create custom runtimes. For Python, use pip install --no-cache-dir and pip freeze to lock only required packages.

Benefits of Smaller Images

• Faster development cycles – Pulling, building, and pushing images becomes quicker.
• Reduced CI load – Smaller layers mean less storage and bandwidth consumption.
• Clearer assumptions – Minimal images make it obvious what is required at runtime, encouraging curiosity and intentional design.
• Improved security – Fewer packages reduce the attack surface and simplify vulnerability scanning.
• Better resource utilization – Smaller images consume less disk space on nodes and can start faster in orchestration platforms.

Cultivating a Design Habit

Cutting hundreds of megabytes from AI and Java containers isn’t just about performance; it’s a design habit that values patience, curiosity, and the willingness to rethink old decisions that no longer fit. The real skill lies in knowing when it’s time to review what already seems good enough and applying these incremental improvements consistently.