How does file upload work in Immich codebase - Part 2.

Source: Dev.to

In this article we review the file‑upload mechanism in the Immich codebase. We will look at:

• fileUploadHandler
• uploadExecutionQueue
• fileUploader

In part 1 we learned what Immich is, located the upload button in the codebase, and reviewed the utility function openFileUploadDialog. In this part 2 we dive into the actual uploading process.

---

fileUploadHandler

fileUploadHandler is defined as shown below:

export const fileUploadHandler = async ({
  files,
  albumId,
  isLockedAssets = false,
}: FileUploadHandlerParams): Promise<string[]> => {
  const extensions = uploadManager.getExtensions();
  const promises = [];

  for (const file of files) {
    const name = file.name.toLowerCase();

    if (extensions.some((extension) => name.endsWith(extension))) {
      const deviceAssetId = getDeviceAssetId(file);
      uploadAssetsStore.addItem({ id: deviceAssetId, file, albumId });

      promises.push(
        uploadExecutionQueue.addTask(() =>
          fileUploader({ assetFile: file, deviceAssetId, albumId, isLockedAssets })
        )
      );
    }
  }

  const results = await Promise.all(promises);
  return results.filter((result): result is string => !!result);
};

We know extensions is an array of supported types from part 1. getDeviceAssetId simply returns a dynamic string:

function getDeviceAssetId(asset: File): string {
  return `web-${asset.name}-${asset.lastModified}`;
}

uploadAssetsStore is imported as follows:

import { uploadAssetsStore } from '$lib/stores/upload';

Store implementation

In the stores/upload file you will see:

import { UploadState, type UploadAsset } from '$lib/types';
import { derived, writable } from 'svelte/store';

function createUploadStore() {
  const uploadAssets = writable<UploadAsset[]>([]);
  const stats = writable({
    errors: 0,
    duplicates: 0,
    success: 0,
    total: 0,
  });
  // ...

  const addItem = (newAsset: UploadAsset) => {
    uploadAssets.update(($assets) => {
      const duplicate = $assets.find((asset) => asset.id === newAsset.id);
      if (duplicate) {
        return $assets.map((asset) => (asset.id === newAsset.id ? newAsset : asset));
      }

      stats.update((stats) => {
        stats.total++;
        return stats;
      });

      $assets.push({
        ...newAsset,
        speed: 0,
        state: UploadState.PENDING,
        progress: 0,
        eta: 0,
      });

      return $assets;
    });
  };
  // ...
}

export const uploadAssetsStore = createUploadStore();

uploadExecutionQueue

At line 44, uploadExecutionQueue is initialized:

export const uploadExecutionQueue = new ExecutorQueue({ concurrency: 2 });

ExecutorQueue

ExecutorQueue is an algorithm that uses an internal array to hold queued items and respects a concurrency limit.

import { handlePromiseError } from '$lib/utils';

interface Options {
  concurrency: number;
}

type Runnable = () => Promise<unknown>;

export class ExecutorQueue {
  private queue: Runnable[] = [];
  private running = 0;
  private _concurrency: number;

  constructor(options?: Options) {
    this._concurrency = options?.concurrency || 2;
  }

  get concurrency() {
    return this._concurrency;
  }

  set concurrency(concurrency: number) {
    if (concurrency > 0) {
      this._concurrency = concurrency;
      this.tryRun();
    }
  }

  addTask(task: Runnable): Promise<unknown> {
    return new Promise((resolve, reject) => {
      // Wrap the original task
      this.queue.push(async () => {
        try {
          this.running++;
          const result = await task();
          resolve(result);
        } catch (error) {
          reject(error);
        } finally {
          this.taskFinished();
        }
      });

      // Attempt to run it immediately if possible
      this.tryRun();
    });
  }

  private taskFinished(): void {
    this.running--;
    this.tryRun();
  }

  private tryRun() {
    if (this.running >= this.concurrency) return;

    const runnable = this.queue.shift();
    if (!runnable) return;

    handlePromiseError(runnable());
  }
}

The queue.shift() call removes the first item from the queue and runs it via handlePromiseError(runnable()). This simple approach keeps the concurrency logic clean.

fileUploader

The fileUploader function is the actual task that uploads files. It is queued like this:

uploadExecutionQueue.addTask(() =>
  fileUploader({ assetFile: file, deviceAssetId, albumId, isLockedAssets })
);

The function itself is fairly large; you can explore its full implementation in the Immich repository here.

End of part 2.

checkBulkUpload

Before the actual upload, a check is performed. For example, the demo URL of Immich rejects uploads because it is a demo.

try {
  const bytes = await assetFile.arrayBuffer();
  const hash = await crypto.subtle.digest('SHA-1', bytes);
  const checksum = Array.from(new Uint8Array(hash))
    .map((b) => b.toString(16).padStart(2, '0'))
    .join('');

  const {
    results: [checkUploadResult],
  } = await checkBulkUpload({
    assetBulkUploadCheckDto: { assets: [{ id: assetFile.name, checksum }] },
  });

  if (checkUploadResult.action === Action.Reject && checkUploadResult.assetId) {
    responseData = {
      status: AssetMediaStatus.Duplicate,
      id: checkUploadResult.assetId,
      isTrashed: checkUploadResult.isTrashed,
    };
  }
} catch (error) {
  console.error(`Error calculating sha1 file=${assetFile.name})`, error);
}

uploadRequest

If the checks pass, uploadRequest is invoked.

if (!responseData) {
  const queryParams = asQueryString(authManager.params);

  uploadAssetsStore.updateItem(deviceAssetId, { message: $t('asset_uploading') });
  const response = await uploadRequest({
    url: getBaseUrl() + '/assets' + (queryParams ? `?${queryParams}` : ''),
    data: formData,
    onUploadProgress: (event) =>
      uploadAssetsStore.updateProgress(deviceAssetId, event.loaded, event.total),
  });

  if (![200, 201].includes(response.status)) {
    throw new Error($t('errors.unable_to_upload_file'));
  }

  responseData = response.data;
}

uploadRequest is defined in lib/utils.ts:

export const uploadRequest = async (options: UploadRequestOptions): Promise<any> => {
  const { onUploadProgress: onProgress, data, url, method } = options;

  return new Promise((resolve, reject) => {
    const xhr = new XMLHttpRequest();

    xhr.addEventListener('error', (error) => reject(error));
    xhr.addEventListener('load', () => {
      if (xhr.readyState === 4 && xhr.status >= 200 && xhr.status < 300) {
        resolve(xhr.response);
      } else {
        reject(new Error(`Upload failed with status ${xhr.status}`));
      }
    });
    xhr.upload.addEventListener('progress', (event) => {
      if (event.lengthComputable && onProgress) onProgress(event);
    });

    xhr.open(method || 'POST', url);
    xhr.responseType = 'json';
    xhr.send(data);
  });
};

Note: Immich uses XMLHttpRequest for uploads, not axios or fetch.

About me

Hey, my name is Ramu Narasinga. I study code‑base architecture in large open‑source projects.

• Email: ramu.narasinga@gmail.com
• I’ve spent 200+ hours analyzing Supabase, shadcn/ui, LobeChat, and identifying patterns that separate AI‑generated code from production‑grade code.
• Check out production‑grade projects at thinkthroo.com.

References

• file-uploader.ts – line 85
• upload.ts – line 24
• executor-queue.ts – line 9
• utils.ts – line 81 (uploadRequest definition)