Arch Linux AUR Dependency Hell: Debugging the ICU 76 78 Migration

Source: Dev.to

The Problem

I recently encountered a frustrating issue when trying to launch Notepadqq on my Arch‑based system (EndeavourOS):

$ notepadqq
/usr/bin/../lib/notepadqq/notepadqq-bin: error while loading shared libraries:
libicui18n.so.76: cannot open shared object file: No such file or directory

This is a classic shared‑library dependency problem that’s particularly common on rolling‑release distributions like Arch Linux.

Understanding the Root Cause

What is ICU?

ICU (International Components for Unicode) is a mature, widely used set of C/C++ libraries providing Unicode and globalization support. Many applications depend on it, including Qt’s WebEngine component.

The Rolling Release Challenge

The issue stemmed from a critical version mismatch between official Arch repositories and AUR packages:

Core problem: Notepadqq hasn’t been actively maintained since 2018. As an AUR package, it doesn’t get automatically rebuilt when Arch updates system libraries like ICU. This created a desynchronisation between:

• Official repository packages (ICU 78)
• Outdated AUR binaries (expecting ICU 76)
• Orphaned repository packages (Qt5WebEngine with ICU 76 linkage)

This cascading dependency mismatch is a common pitfall when mixing official packages with unmaintained AUR software.

Diagnostic Steps

1. Check Current ICU Version

$ ls /usr/lib/libicui18n.so*
/usr/lib/libicui18n.so
/usr/lib/libicui18n.so.78
/usr/lib/libicui18n.so.78.1

This confirmed I had ICU 78, while the application expected version 76.

2. Identify Package Origin

$ pacman -Ss notepadqq
# (no output)

$ yay notepadqq
1 aur/notepadqq 2.0.0beta-3 (+1 0.32) (Installed)

Notepadqq is installed from the AUR, so it won’t be rebuilt automatically when system libraries change.

3. Attempt to Rebuild

$ yay -S notepadqq --rebuild

Resulted in linker errors:

/usr/bin/ld: /usr/lib/libQt5WebEngineCore.so: undefined reference to `icu_76::Normalizer::normalize(...)'
/usr/bin/ld: /usr/lib/libQt5WebEngineCore.so: undefined reference to `ucnv_open_76'

4. Discover the Real Culprit

$ pacman -Qo /usr/lib/libQt5WebEngineCore.so
/usr/lib/libQt5WebEngineCore.so is owned by qt5-webengine 5.15.19-4

qt5-webengine itself was compiled against the old ICU version.

5. Repository Discovery

$ sudo pacman -Syu qt5-webengine
error: target not found: qt5-webengine

Critical finding: qt5-webengine had been removed from the official Arch repositories but remained installed on the system as an orphaned package.

Solution

1. Remove conflicting packages

sudo pacman -Rns qt5-webengine notepadqq

Removed packages

• qt5-webengine (~ 159 MiB)
• notepadqq (~ 5.9 MiB)
• Related dependencies (qt5-location, qt5-webchannel, qt5-websockets)

Total freed: ≈ 174.9 MiB

2. Rebuild qt5-webengine from AUR

yay -S qt5-webengine

The package is compiled from source and linked against the current ICU 78 library.

3. Rebuild notepadqq

yay -S notepadqq

With qt5-webengine now correctly linked to ICU 78, Notepadqq builds without errors.

4. Verify the linkage

ldd /usr/bin/notepadqq | grep icu

Output:

libicui18n.so.78 => /usr/lib/libicui18n.so.78
libicuuc.so.78   => /usr/lib/libicuuc.so.78

The application now links against the proper ICU version. 🎉

Key Lessons Learned

1. AUR packages don’t auto‑update with system libraries

   • When Arch updates a core library (e.g., ICU), official packages are rebuilt automatically.
   • AUR packages are not; they must be rebuilt manually by users or maintainers.
   • Notepadqq’s last upstream update was in 2018, so it never received rebuilds for ICU 76, 77, or 78, breaking the package each time ICU changed.

2. Partial upgrades are dangerous

   • Even with frequent system updates, binary incompatibilities can linger on disk.
   • Rebooting alone won’t fix ABI mismatches.

3. Rolling release requires package consistency

   • Libraries like ICU update frequently; all dependent packages must be rebuilt or updated in sync.
   • Orphaned packages from removed repositories can cause hidden issues.

4. ABI compatibility matters

   • The version number in the shared‑library name (libicui18n.so.76 vs. libicui18n.so.78) indicates an ABI break.
   • Symlinking one version to another is unsafe and can lead to crashes or undefined behaviour.

Analyzing an ABI Break

How did I know this wasn’t just a missing symlink issue?

1. Version in library filename

   libicui18n.so.76  vs  libicui18n.so.78

   • The .76 and .78 are SONAME major versions.
   • Different major versions = intentional ABI incompatibility.

2. Version‑ed symbols in linker errors

   icu_76::Normalizer::normalize   ← version baked into symbol name
   ucnv_open_76                   ← function name includes version

   • The library exposes icu_78:: symbols, but the binary needs icu_76::.
   • No amount of symlinking can resolve this mismatch.

3. Why symlinking would fail

   Even if we created a symlink:

   sudo ln -s libicui18n.so.78 libicui18n.so.76

   • The binary would load, but it would try to call icu_76:: functions.
   • The library only provides icu_78:: functions.
   • Result: immediate crash or undefined behaviour.

Dependency Chains Run Deep

What appeared to be a simple Notepadqq issue actually required:

• Diagnosing the ICU mismatch.
• Discovering the qt5‑webengine dependency.
• Identifying the orphaned repository package.
• Rebuilding the entire dependency chain.

What NOT to Do

❌ Creating Symlinks

# DON'T DO THIS
sudo ln -s /usr/lib/libicui18n.so.78 /usr/lib/libicui18n.so.76

While this might allow the program to start, it can cause:

• Runtime crashes
• Undefined behaviour
• Data corruption
• Difficult‑to‑debug issues

❌ Ignoring Update Errors

Running pacman -Syu regularly is good, but you must also:

• Check for orphaned packages: pacman -Qtdq
• Verify AUR packages after system‑library updates
• Review failed transaction logs

Prevention Strategies

1. Regular Orphan Package Cleanup

pacman -Qtdq | pacman -Rns -

2. Monitor AUR Package Build Failures

When a system library updates, rebuild the affected AUR packages immediately:

yay -S --rebuild

Important: Unmaintained AUR packages (e.g., Notepadqq, last updated 2018) will always require manual rebuilds after system‑library updates. Consider:

• Finding actively maintained alternatives.
• Maintaining the AUR package yourself.
• Using AppImage/Flatpak versions that bundle their dependencies.

3. Use Binary‑Package Alternatives

For stability‑critical applications, consider -bin packages from the AUR that bundle their dependencies (at the cost of increased disk usage).

4. Check Library Dependencies

After major library updates, verify that no required libraries are missing:

ldd /usr/bin/<binary> | grep "not found"

Replace <binary> with the executable you want to inspect.

Conclusion

This issue perfectly demonstrates why Arch Linux (and its derivatives) is considered an “advanced” distribution. The rolling‑release model provides cutting‑edge packages but requires users to understand:

• Shared‑library versioning (ABI compatibility)
• Dependency chains
• The relationship between official repositories and the AUR
• When to rebuild vs. reinstall packages

The solution wasn’t a quick fix but a systematic diagnosis and rebuild of the dependency chain. While frustrating in the moment, issues like this deepen understanding of how Linux systems work at a fundamental level.

Have you encountered similar dependency issues on Arch?
What’s your approach to handling library version mismatches? Share your experiences in the comments!

System Info

• Distribution: EndeavourOS (Arch‑based)
• Package Manager: pacman + yay
• Affected Versions: ICU 76 → 78, Qt5WebEngine 5.15.19‑4