Linux File System Architecture: A Deep Dive into VFS, Inodes, and Storage

Source: Dev.to

Virtual File System (VFS)

The idea that “everything is a file” is made possible by a technology called VFS (Virtual File System).

VFS acts as an abstraction layer. It provides a unified input/output interface for all computer resources, including data and devices. Thanks to VFS, the user (and applications) can treat everything as a “file” without worrying about the underlying physical medium.

If VFS didn’t exist, you would have to be conscious of the disk format type every time you saved a file.

The Variety of File Systems

While VFS provides a unified interface, the actual data lives on various types of media. It’s no longer just about hard drives.

Exploring Filesystems with Docker

Let’s look at what filesystems actually exist in a modern environment. We’ll use a Docker container (Rocky Linux 9) because it’s the easiest way to see a mix of different filesystem types.

# We use --privileged to allow mount operations for demonstration
docker run -it --privileged --name fs-lab rockylinux:9 /bin/bash

Check the disk usage and types

# -T: Print file system type (ext4, xfs, tmpfs, etc.)
# -h: Human‑readable sizes
[root@container /]# df -Th
Filesystem     Type     Size  Used Avail Use% Mounted on
overlay        overlay   59G   13G   44G  23% /
tmpfs          tmpfs     64M     0   64M   0% /dev
shm            tmpfs     64M     0   64M   0% /dev/shm
/dev/vda1      ext4      59G   13G   44G  23% /etc/hosts

We can categorize these into four main types:

Type 1 – Disk‑based Filesystems (Persistent)

• Examples: ext4, xfs, btrfs
• Role: Storing OS core files, user data (/home), and logs (/var).
• Note: ext4 is simply the fourth generation of the extended filesystem.

Type 2 – Memory‑based Filesystems (Volatile)

• Examples: tmpfs, ramfs
• Role: Temporary files (/tmp, /run).
• Why? /run stores PID files and sockets that are only valid for the current session. Writing these to a physical disk would be unnecessary overhead.

Type 3 – Pseudo Filesystems (Virtual)

These files do not exist on any persistent storage. They are generated on‑the‑fly by the kernel when you access them. They look like files, but they are actually windows into the kernel.

• Examples: /proc, /sys, /sys/fs/cgroup, /dev/pts

Type 4 – Layered Filesystems (Union Mount)

This is the magic behind Docker. These filesystems allow you to stack multiple directories (layers) and present them as a single unified filesystem.

• Examples: overlay, aufs
• Role: Merging a read‑only base layer (OS image) with a writable upper layer (container changes).

Deep Dive: OverlayFS

OverlayFS uses a specific structure to achieve deduplication and fast startup.

• Lower Dir (Base): The Docker image. Read‑only. Shared among all containers.
• Upper Dir: The writable layer for your specific container. Starts empty. Changes are copied here (Copy‑on‑Write).
• Merged Dir: The view you see. The upper layer “overlays” the lower layer.

What Is a “File” Really? (The Inode)

Filenames like example.txt are just for humans. Linux identifies files by inodes.

Structure of an Inode

# Create a dummy file
echo "dummy" > example.txt

# 'stat' shows the inode metadata
stat example.txt

Sample output

  File: example.txt
  Size: 6           Blocks: 8          IO Block: 4096   regular file
Device: f5h/245d  Inode: 1714604     Links: 1
...

Viewing an Inode

ls -i example.txt
# Output: 1714604 example.txt

Note: The filename is not stored in the inode. It lives in the directory entry that points to the inode number.

When Does an Inode Change?

Understanding inode changes helps you debug issues such as “why did my log monitoring stop working?”

cp vs. mv

# cp changes the inode
ls -i example.txt      # 1714604
cp example.txt copy.txt
ls -i copy.txt         # 1714664 (NEW)

# mv keeps the inode
mv copy.txt moved.txt
ls -i moved.txt        # 1714664 (SAME)

The sed -i Trap

Tools like sed -i (edit‑in‑place) or editors such as Vim often use a safe‑save strategy:

1. Create a temporary file (new inode).
2. Write the changes to the temporary file.
3. Rename the temporary file over the original.

# Create file
echo "hello" > config.txt
ls -i config.txt       # 1714736

# Edit with sed
sed -i 's/hello/world/' config.txt

# Inode changed!
ls -i config.txt       # 1714737

Why does this matter?
If you monitor a log file by its inode (e.g., with inotify), a rotation or an in‑place edit will give the monitor a different inode, causing it to lose track of the file.

Appending vs. Editing: The Vim Behavior

1. Appending Data (>>)

Appending adds data to existing blocks, so the inode stays the same.

# Create a file
touch target.txt
ls -i target.txt       # 1714736

# Append data
echo "append" >> target.txt

# Inode remains unchanged
ls -i target.txt       # 1714736

2. Editing with Vim

Vim (by default) writes to a new temporary file and then renames it, which changes the inode.

# Edit with Vim and save (:wq)
vi target.txt

# Inode changed!
ls -i target.txt       # 1714739

Warning: Tools like tail -F or inotify must be able to handle this “inode rotation,” otherwise they will continue watching the deleted file.

Hard Links vs. Symbolic Links

Verification

echo "Hello" > original.txt
ln original.txt hard.txt      # Hard link
ln -s original.txt sym.txt    # Symbolic link

ls -li
# Example output:
# 1714669 -rw-r--r-- 2 user group 6 Jan 10 12:00 hard.txt
# 1714669 -rw-r--r-- 2 user group 6 Jan 10 12:00 original.txt  # Same inode!
# 1714670 lrwxrwxrwx 1 user group 12 Jan 10 12:00 sym.txt -> original.txt  # Different inode

Process View: File Descriptors

Developers usually interact with files via file descriptors (FDs) rather than directly with inodes.

When a process opens a file, the kernel assigns a non‑negative integer (the FD) that indexes the process’s table of open files.

Example in Go

package main

import (
	"fmt"
	"os"
)

func main() {
	// 1. Open a file (system call to the kernel)
	file, err := os.Open("example.txt")
	if err != nil {
		panic(err)
	}
	defer file.Close() // Always close when done

	// 2. Retrieve the file descriptor
	fd := file.Fd()

	fmt.Printf("File Name: %s\n", file.Name())
	fmt.Printf("File Descriptor: %d\n", fd)
}

Result

File Name: example.txt
File Descriptor: 3

Why “3”?

Linux processes start with three standard file descriptors already open:

The kernel always assigns the lowest available number, so the first user‑opened file gets descriptor 3.

File Descriptor Exhaustion

A common production nightmare is the “too many open files” error. Because everything in Unix is a file (sockets, pipes, device nodes, etc.), many resources consume file descriptors:

• Database connections
• HTTP requests (TCP sockets)
• Log files

If a high‑traffic service opens more descriptors than the limit (ulimit -n), it will start failing to accept new connections or write to logs.

Mitigation Tips

1. Raise the limit (ulimit -n or edit /etc/security/limits.conf).
2. Close unused descriptors promptly.
3. Use connection pooling for databases and HTTP clients.
4. Monitor descriptor usage (lsof -p <pid> or cat /proc/<pid>/fd).

Solutions

• Fix Leaks: Always ensure Close() is called (use defer in Go).
• Tune Limits: The default limit (often 1024) is too low for servers. Increase it in /etc/security/limits.conf.

Conclusion

By peeling back the layers of the Linux filesystem, we can see it’s a beautifully orchestrated architecture:

• User Space: We handle filenames and file descriptors.
• VFS Layer: The kernel abstracts the differences between hardware.
• Physical Layer: Data lives on disks (e.g., ext4), RAM (tmpfs), or is calculated on the fly (/proc).

Understanding these layers is critical when debugging performance issues (e.g., “Is it disk I/O or a pseudo‑fs bottleneck?”) or working with containers (e.g., “How does my image stay small?”).