Creating your own Remote Cache Server for Nx and Lerna with Cacheiro
Source: Dev.to
🌐 Este artigo também está disponível em Português. If you work with monorepos using Nx or Lerna, you already know that remote caching is practically a superpower. The ability to run a build, test, or lint in your CI pipeline (or on a colleague’s machine) and share that result instantly with the rest of the team saves precious hours of pipeline and processing time. To get this benefit, the default and official solution is Nx Cloud. It is a fantastic service, but it can weigh on the budget of smaller teams or run into strict data security policies at companies that require strict self-hosted solutions. If you have tried to bypass Nx Cloud costs recently, you most likely faced a chaotic scenario. The history of hosting your own cache in Nx feels like a soap opera. In the beginning, the community relied on third-party tools. Then, Nx (formerly NRWL) removed free open-source support, centralizing everything into their paid tier. Later, they backtracked and released official, free packages for self-hosting (featuring plugins for File System, AWS S3, Google Cloud Storage, and Azure). And then came the recent cold shower: Nx abruptly deprecated all official self-hosted cache packages due to security concerns (you can check the details in the official Nx deprecation documentation). The remote caching capability still natively exists within the Nx core, but the community was left stranded without official, secure tools to bridge the connection to storage providers. To deeply understand this rollercoaster, Emily Xiong’s Medium article does an incredible job of detailing this history of caching solutions. It was precisely while dealing with this pain in my own work that I decided to build a solution to fill this gap. Cacheiro
The Cacheiro project was born to give the community back the freedom to host their own remote cache securely, modernly, and completely for free. And before you ask: yes, the name is a play on words! It joins the technical root Cache with the Portuguese suffix -eiro (which denotes a profession or someone who does a specific job, like a fazendeiro / farmer or pedreiro / bricklayer). “Cacheiro” is the one who takes care of your cache. Unlike other opinionated and rigid alternatives, Cacheiro was designed as a modular library, published in individual packages. You pick and plug only what you actually need. It is: 100% Free and Open Source. Highly extensible: you can easily customize the logic according to your infrastructure needs. Ready for the current ecosystem: It already ships with production-ready plugins for File System (FS) and Amazon S3. Aligned Roadmap: Coming soon, we will release plugins for Google Cloud Storage (GCS) and Azure, covering 100% of the gap left by the deprecated packages. 🔒 Security First: Since the deprecation of the official Nx plugins happened due to security reasons, Cacheiro was built with double the attention to this aspect. The project strictly utilizes the latest versions of its underlying libraries (free of known vulnerabilities), underwent code auditing to mitigate common loopholes, and implements a robust encryption layer resilient against injection or artifact leaks. Even though Cacheiro is a library meant for building your custom server, the main repository already comes with a fully working example ready to run locally at 127.0.0.1:3000. If you want to see the magic happen right now on your machine using the File System (FS) plugin, just follow these quick steps: Clone the repository, install the dependencies at the root, configure the local runner environment, and start the development server: git clone https://github.com/rerodrigues/nx-remote-cache-server.git cd nx-remote-cache-server npm install
Navigate to the runner package and set up the environment
cd packages/cacheiro-runner cp config/local.json.example config/local.json
Start the server
npm run dev
That’s it! Your remote cache server is up and running at http://127.0.0.1:3000. Nx can identify and consume a remote cache directly via native environment variables. For a production or daily development setup, the ideal approach is to save these variables in your terminal configuration file (like .bashrc or .zshrc) or inject them directly into your CI secrets / environment variables (GitHub Actions, GitLab CI, etc.). For our local example, configure it using the default token (change-me): export NX_REMOTE_CACHE_URL=“http://127.0.0.1:3000” export NX_REMOTE_CACHE_TOKEN=“change-me”
With these variables defined in your monorepo’s terminal, just run your project’s standard commands:
If you are using Nx:
npx nx run-many -t build
If you are using Lerna:
npx lerna run build
Run the command once to generate and upload the cache to your local server. Run it a second time and witness the performance boost with the cache being served remotely by Cacheiro. Simple as that. This is just the first article in a series where we will explore the full potential of Cacheiro. In the upcoming posts, we will go through step-by-step tutorials on how to deploy this to production for real: Part 2: How to build and customize your cache server using the File System (FS) plugin. Part 3: How to deploy Cacheiro to AWS integrating with Amazon S3. If your team is facing this issue today, make sure to check out the official project repository, leave a star, and of course, contribute with feedback or Pull Requests! Access the GitHub Repository: rerodrigues/nx-remote-cache-server How do you handle Nx caching today? Are you using Nx Cloud, did you migrate to another alternative, or were you stuck without knowing what to do after the official packages were deprecated? Leave a comment below, share your experience, and let’s talk about it! See you in the next post!