Notes on Tabby: Llama.cpp, Model Caching, and Access Tokens
Source: Dev.to
Tabby uses llama.cpp internally
One notable point is that Tabby uses llama.cpp under the hood. In practice, this means Tabby can leverage the lightweight, local‑inference approach that llama.cpp is known for, which is often used to run LLMs efficiently on local machines.
Model cache location: TABBY_MODEL_CACHE_ROOT
Tabby supports configuring where it stores cached model files. The environment variable TABBY_MODEL_CACHE_ROOT is used to control the root directory for Tabby’s model cache. Setting this variable is a straightforward way to manage disk usage, place models on a faster drive, or standardize storage paths across multiple environments.
Registry reference: registry-tabby
Tabby’s model registry can be found here. This repository is a useful reference point for understanding what models are available or how Tabby’s registry entries are organized.
How to check your token
To view the token you need for authenticated access:
- Access the service via a web browser.
- Create an account and log in.
- After logging in, locate and confirm your token from the account or user settings area.
The token can be verified after a one‑time browser‑based account setup and login.