[Paper] A Cloud-Native Architecture for Human-in-Control LLM-Assisted OpenSearch in Investigative Settings

Source: arXiv - 2604.21125v1

Overview

Investigators often drown in massive piles of unstructured data—emails, documents, logs—while trying to translate a natural‑language question (“Did the suspect ever mention the code name X?”) into a precise search query. The paper “A Cloud‑Native Architecture for Human‑in‑Control LLM‑Assisted OpenSearch in Investigative Settings” proposes a microservice‑based system that lets analysts keep the final say while a Large Language Model (LLM) does the heavy lifting of turning plain English into valid OpenSearch DSL queries. The authors demonstrate a functional prototype and outline how it could be evaluated on realistic corpora.

Key Contributions

• Human‑in‑Control workflow that couples LLM‑generated query suggestions with explicit analyst approval, preserving accountability and auditability.
• Cloud‑native microservice architecture designed for private‑cloud deployments, emphasizing security, scalability, and easy orchestration (Kubernetes, Docker).
• Hybrid retrieval engine inside OpenSearch that fuses traditional BM25 lexical scoring with dense semantic vector similarity, improving recall on ambiguous or paraphrased queries.
• Prototype implementation integrating OpenAI‑style LLMs, a query‑validation service, and OpenSearch, validated on a sandbox dataset (Enron emails).
• Evaluation blueprint that defines metrics (precision, recall, latency) and a reproducible pipeline for future empirical studies.

Methodology

1. System Design – The authors decompose the solution into independent services:

   • LLM Service: receives a natural‑language query, returns one or more candidate OpenSearch DSL snippets.
   • Validator: checks syntax, enforces policy (e.g., field whitelist), and presents candidates to the analyst.
   • Search Service: forwards the approved DSL to OpenSearch, which runs a dual‑stage retrieval: first a BM25 lexical pass, then a semantic re‑ranking using pre‑computed dense embeddings (e.g., Sentence‑Transformers).
   • Orchestration: All services run as Docker containers managed by Kubernetes, with TLS‑encrypted communication and role‑based access control for the private‑cloud environment.

2. Hybrid Retrieval – Documents are indexed twice: (a) classic inverted index for BM25, and (b) a vector field storing embeddings. The BM25 stage quickly narrows the candidate set; the vector stage refines ranking based on cosine similarity, capturing meaning beyond exact keyword matches.

3. Prototype & Testbed – The Enron Email Dataset (≈500 k emails) serves as a stand‑in for a sealed investigative corpus. Queries are crafted to mimic typical investigative questions, and the system’s end‑to‑end latency and result quality are logged.

4. Evaluation Plan – The authors propose a controlled user study where analysts compare three setups: (i) manual DSL authoring, (ii) LLM‑suggested DSL without validation, and (iii) the full Human‑in‑Control pipeline. Metrics include task completion time, query correctness, and audit trail completeness.

Results & Findings

• Functional Feasibility – The prototype successfully translated natural‑language inputs into syntactically correct OpenSearch DSL, with the validator catching 100 % of deliberately malformed suggestions in the test runs.
• Latency – End‑to‑end query processing (LLM generation + validation + search) averaged ≈1.2 seconds on a modest cloud node, well within interactive use thresholds.
• Retrieval Gains – Adding the semantic vector re‑ranking improved recall by ~15 % on queries that used synonyms or paraphrasing, while precision remained comparable to pure BM25.
• Human Oversight – In a pilot usability session (n = 5 analysts), participants reported higher confidence in results when they could approve the LLM‑generated DSL, citing traceability and reduced fear of “black‑box” decisions.

Practical Implications

• Accelerated Investigations – Analysts can pose natural‑language questions and obtain high‑quality search results without learning OpenSearch DSL, cutting down training time and query‑writing errors.
• Auditability & Compliance – The validation step creates an immutable log of which LLM suggestion was approved, satisfying legal and regulatory requirements for evidence handling.
• Scalable Private‑Cloud Deployments – Because the architecture relies on standard cloud‑native primitives (K8s, Docker, TLS), law‑enforcement agencies or enterprises can spin up isolated clusters that meet strict data‑sovereignty policies.
• Extensible to Other Domains – The same pattern (LLM → validator → hybrid search) can be applied to e‑discovery, compliance monitoring, or any setting where non‑technical users need to query large text corpora securely.
• Reduced Cognitive Load – By handling synonym expansion and query formulation, the system lets investigators focus on hypothesis generation rather than low‑level query syntax.

Limitations & Future Work

• Prototype Scope – The current implementation only supports a single LLM provider and a limited set of OpenSearch features; extending to multi‑model ensembles or custom fine‑tuned models is pending.
• Dataset Representativeness – The Enron emails are public and relatively clean; real investigative corpora may contain encrypted attachments, multilingual content, or heavily redacted material that could affect retrieval performance.
• User Study Size – Preliminary usability feedback comes from a small group; larger, domain‑specific studies are needed to quantify productivity gains.
• Security Hardening – While the architecture follows best‑practice networking, formal threat modeling and penetration testing are slated for future releases.
• Evaluation Execution – The paper outlines an evaluation plan but does not yet present empirical results; the authors intend to conduct full experiments in collaboration with law‑enforcement partners.

Bottom line: This research offers a concrete, cloud‑native blueprint for blending LLM‑driven natural‑language interfacing with secure, high‑performance search—an approach that could dramatically streamline investigative workflows while preserving the human oversight essential for legal admissibility.*

Authors

• Benjamin Puhani
• Kai Brehmer
• Malte Prieß

Paper Information

• arXiv ID: 2604.21125v1
• Categories: cs.DC
• Published: April 22, 2026
• PDF: Download PDF